12.6. Schema Design

12.6.1. Number of Column Families

See Section 6.2, “ On the number of column families ”.

12.6.2. Key and Attribute Lengths

See Section 6.3.2, “Try to minimize row and column sizes”. See also Section 12.6.7.1, “However...” for compression caveats.

12.6.3. Table RegionSize

The regionsize can be set on a per-table basis via setFileSize on HTableDescriptor in the event where certain tables require different regionsizes than the configured default regionsize.

See Section 15.9.2, “Determining region count and size” for more information.

12.6.4. Bloom Filters

Bloom Filters can be enabled per-ColumnFamily. Use HColumnDescriptor.setBloomFilterType(NONE | ROW | ROWCOL) to enable blooms per Column Family. Default = NONE for no bloom filters. If ROW, the hash of the row will be added to the bloom on each insert. If ROWCOL, the hash of the row + column family name + column family qualifier will be added to the bloom on each key insert.

See HColumnDescriptor and Section 12.9.9, “Bloom Filters” for more information or this answer up in quora, How are bloom filters used in HBase?.

12.6.5. ColumnFamily BlockSize

The blocksize can be configured for each ColumnFamily in a table, and this defaults to 64k. Larger cell values require larger blocksizes. There is an inverse relationship between blocksize and the resulting StoreFile indexes (i.e., if the blocksize is doubled then the resulting indexes should be roughly halved).

See HColumnDescriptor and Section 9.7.6, “Store”for more information.

12.6.6. In-Memory ColumnFamilies

ColumnFamilies can optionally be defined as in-memory. Data is still persisted to disk, just like any other ColumnFamily. In-memory blocks have the highest priority in the Section 9.6.4, “Block Cache”, but it is not a guarantee that the entire table will be in memory.

See HColumnDescriptor for more information.

12.6.7. Compression

Production systems should use compression with their ColumnFamily definitions. See Appendix C, Compression In HBase for more information.

12.6.7.1. However...

Compression deflates data on disk. When it's in-memory (e.g., in the MemStore) or on the wire (e.g., transferring between RegionServer and Client) it's inflated. So while using ColumnFamily compression is a best practice, but it's not going to completely eliminate the impact of over-sized Keys, over-sized ColumnFamily names, or over-sized Column names.

See Section 6.3.2, “Try to minimize row and column sizes” on for schema design tips, and Section 9.7.6.4, “KeyValue” for more information on HBase stores data internally.

comments powered by Disqus