HBase勉強会復習で挙がった質問について

先日11/5に行われた勉強会GLT vol.21 & HBase勉強会お疲れさまでした - tragicomedyでのHBaseについての質問の件。
同じColumn Familyのデータは物理的に近い場所に格納されることと、HRegionではデータをソートして格納しているが、そのこととどう関連しているのか?という疑問について調べたメモ。

Amazon CAPTCHAにもあるように、

Physically, all column family members are stored together on the filesystem.

また、Hbase/HbaseArchitecture - Hadoop Wikiにも

HBase stores column families physically close on disk, so the items in a given column family should have roughly the same read/write characteristics and contain similar data.

というのと、

To an application, a table appears to be a list of tuples sorted by row key ascending, column name ascending and timestamp descending. Physically, tables are broken up into row ranges called regions (equivalent Bigtable term is tablet).
Each row range contains rows from start-key (inclusive) to end-key (exclusive).
A set of regions, sorted appropriately, forms an entire table.
Unlike Bigtable which identifies a row range by the table name and end-key, HBase identifies a row range by the table name and start-key.

Each column family in a region is managed by an HStore.
Each HStore may have one or more MapFiles (a Hadoop HDFS file type) that is very similar to a Google SSTable.
Like SSTables, MapFiles are immutable once closed. MapFiles are stored in the Hadoop HDFS.
Other details are the same, except:

* MapFiles cannot currently be mapped into memory.
* MapFiles maintain the sparse index in a separate file rather than at the end of the file as SSTable does.
* HBase extends MapFile so that a bloom filter can be employed to enhance negative lookup performance. The hash function employed is one developed by Bob Jenkins.


Lineland: HBase Architecture 101 - Storageにあるように、

When the HRegion is "opened" it sets up a Store instance for each HColumnFamily for every table as defined by the user beforehand. Each of the Store instances can in turn have one or more StoreFile instances, which are lightweight wrappers around the actual storage file called HFile.

なことが整理できていなかった。
テーブルを分割したHRegionではrowキーでソートされている、で、HRegion内でStore(HStore)にcolumn familyごとに格納されている。
HBase shellから describe 'table_name'したとき、以下のように出力されるが、

{NAME => 'docs', FAMILIES => [{NAME => 'cache', COMPRESSION => 'NONE', VERSIONS => '3', TTL => '2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'false'}, {NAME => 'contents', COMPRESSION => 'NONE', VERSIONS => '3', TTL => '2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'false'}, ...

この BLOCKSIZE は HStore をさらに分割した HFile のサイズの模様。もうちょっと調べてみる。つづく