Performance Tuning: Compaction, Caching, Read/Write Path, and JVM
This domain tests tuning the storage engine and JVM: choosing a compaction strategy that matches workload, sizing caches, understanding the read/write path, and avoiding heap and GC pitfalls.
Compaction and caches
- SizeTieredCompactionStrategy is write-optimized but has high space amplification and temporary double-disk usage. LeveledCompactionStrategy is read-optimized with lower space amplification but higher write amplification. TimeWindowCompactionStrategy is for time-series and TTL workloads, grouping SSTables by time window. UnifiedCompactionStrategy exists in newer releases; verify version support.
- Tombstones are purged only after gc_grace_seconds and during compaction. A partition with many tombstones can dominate read latency; avoid repeatedly overwriting and deleting the same rows.
- Key cache is on by default and caches partition indexes; row cache is off by default and usually harmful for large rows. Bloom filters reduce SSTable reads but consume off-heap memory.
- Writes go to commitlog then memtable; flush creates SSTables. Reads merge memtable and SSTables, so too many SSTables or huge partitions hurt.
Worked example
A metrics table with TTL and queries by recent time window should use TWCS with a window matching the TTL, such as one-day windows for 30-day retention. STCS would repeatedly compact old and new data; LCS would add write amplification for high ingest. Set compaction_window_size and compaction_window_unit, and run nodetool compactionstats to confirm backlog.
Traps
Turning on row cache globally; setting heap too high and causing long GC pauses; running major compaction routinely; using LCS on write-heavy, overwrite-heavy tables; and forgetting that concurrent_compactors and compaction_throughput_mb_per_sec affect backlog.