ITeye博客

渴死的火烈鸟

standalone · 共 356 篇文章

这几天coding的一些记录

今天写code一个体会就是写出好的SQL很重要。。。 1. 不要用not in,严重影响performance,用其他写法替代。 2. 注意left join,join等区别。 3. 。。。 还有就是perl的一个问题 一个undef的值跟其他数值比较结果如何? 比如 $var = undef; if($var <= 0){ print "here"; }

2011-05-21 15:03 · perl · 评论(0)

ajax css optimization

http://blog.zimbra.com/blog/archives/2006/01/ajax-and-css-optimization.html

2011-04-15 12:59 · javascript · 评论(0)

Perl POE

http://www.php-oa.com/2009/09/09/perl-poe-1.html

2011-04-15 10:50 · perl · 评论(0)

Regular Expression: Greedy and Non-Greedy Quantification

1 my $string = '"hello there", said Pooh, "how are you today?". I said "brilliant!".'; 2 print "$string\n"; 3 $string =~ s/('|")(.*?)\1/$1.uc($2).$1/ge; 4 print "$string\n"; I want to change letters in ' or '' to be uppercase. Without the ? the result will not be correct. Note that * is greedy and …

2011-04-12 11:25 · perl · 评论(0)

JavaScript, 5 ways to call a function

http://devlicio.us/blogs/sergio_pereira/archive/2009/02/09/javascript-5-ways-to-call-a-function.aspx

2011-04-06 14:45 · javascript · 评论(0)

【读书笔记】Data warehousing and analytics infrastructure at facebook

这好像是sigmod2010上的paper。 读了之后做了以下几点记录: 1. facebook的hadoop cluster分成 scribe hadoop cluster: scribe servers将web service的log汇总然后存到HDFS上。通常会有带宽成为bottleneck的问题,这时候可以考虑压缩,但是一个副作用就是在buffer待压缩的数据的同时导致latency增大; production-hive hadoop cluster:通常处理schedule好的,有严格deadline的任务; Adhoc hive hadoop cluster:优先级较低的批处理任…

2011-03-18 22:03 · hadoop · hbase · cloud · 评论(0)

hbase中region的flush

都知道memstore大小到一定程度后会flush到disk上去,这个大小是由hbase.hregion.memstore.flush.size定义的。flush的时候也不是说马上就flush出去,会有个检查,就是下面这个方法了: code:hbase 0.20.6, MemStoreFlusher.java /* * A flushRegion that checks store file count. If too many, puts the flush * on delay queue to retry later. * @param fqe * @return true if th…

2011-01-20 15:52 · hbase · 评论(0)

cassandra example

http://www.rackspacecloud.com/blog/2010/05/12/cassandra-by-example/

2011-01-19 16:39 · cloud · 评论(0)

impact of total region numbers?

这几天tune了hbase的几个参数,有些有意思的结果。具体看我下面的邮件吧。 For example, I have total some data and I can tune hbase.hregion.max.filesize to increase/decrease total region number, rite? I want to know if the region number has performance impact to random read tests. I observed that in my ycsb test, with larger hfile s…

2011-01-19 16:31 · hadoop · hbase · cloud · 评论(0)

Will all HFiles managed by a regionserver kept open

code 没看仔细,所以在hbase 的mail list上面问了这么个问题。其实再仔细看一下big table的paper就知道肯定是open的。现在分析的结果是hbase random read的performance决定在几个方面: 1)HDFS的seek操作,平均每次random get导致几次seek? 2)memory copy; 这个问题尤其在data locality差的时候,比如datanode 和regionserver不在一个node上的时候; 3)block cache? hi, I know generally regionserver manages HRegio…

2011-01-19 10:29 · hadoop · hbase · cloud · 评论(0)

细说HBase怎么完成一个Get操作 (server side)

上面有一篇记录了client边的过程,现在看看RegionSever这边怎么进行下去。 经过HBaseRPC后,调用传递到HRegionServer.get(byte[] regionName, Get get). HRegion region = getRegion(regionName); return region.get(get, getLockFromId(get.getLockId())); 然后是HRegion.get(Get)方法: /* * Do a get based on the get parameter. */ private List<KeyValue> get(…

2011-01-14 16:33 · hbase · 评论(0)

"error in locking authority file .Xauthority

今天在windows上vncviewer连linux机器的时候,报这个错误。 解决方案: 删除所有~/.Xauthority文件。

2011-01-11 11:34 · linux · 评论(0)

raid 0 vs. raid1

RAID 0 (block-level striping without parity or mirroring ) provides improved performance and additional storage but no redundancy or fault tolerance (making it not true RAID, according to the acronym's definition). However, because of the similarities to RAID (especially the need for a controller t…

2010-12-30 10:20 · linux · 评论(0)

hwo to configure linux vncserver

http://bobpeers.com/linux/vnc

2010-12-24 14:10 · linux · 评论(0)

problems in building hadoop

When I try to modify some code in ASF hadoop-0.20.2 and compile it with command "ant jar". It reports no *route to host* exceptions. I do this on a server that with no direct internet connection, that is, I need to set a proxy to access outside network. So to build hadoop, maven needs to download s…

2010-12-22 10:28 · hadoop · cloud · 评论(0)