HBase Distributed Database Design and Development
Data model and write path
HBase stores rows sorted lexicographically by rowkey. A table is split into regions, each served by one RegionServer; HMaster assigns regions and handles metadata but does not serve reads. Writes append to the WAL and MemStore; a flush turns MemStore into HFiles. Minor compaction merges HFiles; major compaction rewrites all store files and can drop deletes. Bloom filters and block cache speed reads.
Rowkey design
Design rowkeys for even write spread and predictable read patterns. Monotonic keys such as timestamps or ascending IDs concentrate writes on the last region. Add a hash or salt prefix, or pre-split the table. Keep rowkeys short and put frequently filtered attributes into the key; use column families sparingly.
Worked example
Device events use deviceId ascending as rowkey; all new writes hit one region. Use hash(deviceId) % 16 as a two-character prefix, then deviceId and timestamp: writes spread across 16 regions. A scan for one device now needs 16 prefix scans, so add a secondary index table if that lookup is frequent.
Traps that cost marks
Too many column families increases flush and compaction overhead. Large rowkeys and wide rows increase memory pressure. A scan without a stop row reads far more than intended. Major compaction during peak hours causes I/O spikes.