Cassandra Admin study materials
Cassandra Admin study resources

Cassandra Admin cheat sheet

Cassandra Admin · Keep the key ideas within reach.

Quick reference

Cassandra Architecture and the Distributed Ring

  • Every node owns one or more token ranges on a continuous ring; the partitioner hashes the partition key to a token and the coordinator walks the ring to find the first replica.

  • Murmur3Partitioner is the default and produces a signed 64-bit token; RandomPartitioner and ByteOrderedPartitioner are legacy or unsuitable for production because ordering breaks even distribution.

  • Virtual nodes: num_tokens defaults to 16 in Cassandra 4.0; more tokens per node means finer range ownership, faster streaming granularity, and more repair/stream sessions.

  • Gossip runs roughly once per second per node, exchanging versioned heartbeat, status, schema, and load state; the phi-accrual failure detector marks a node DOWN when phi exceeds phi_convict_threshold (default 8).

  • Write path order is commit log append, then memtable insert; acknowledgement follows the commit log write plus the replica consistency requirement, not the SSTable flush.

  • SSTable read components in order: bloom filter, partition index, partition summary, compression offsets, then the data file; a bloom filter false positive costs an unnecessary disk seek.

  • Any node can act as coordinator: it receives the request, applies the consistency level against the replica set, forwards to replicas, and merges or repairs responses.

  • The snitch supplies datacenter and rack labels used by NetworkTopologyStrategy and by per-DC consistency levels; DynamicSnitch wraps the configured snitch and deprioritizes slow replicas.

  • Replication factor and replication strategy, not the client, determine the replica set; the coordinator computes it per request from the ring and the keyspace strategy.

Quick reference

Installation, Configuration, and Cluster Provisioning

  • cassandra.yaml values that must be consistent cluster-wide: cluster_name, partitioner, num_tokens policy, endpoint_snitch, and the list of seed addresses.

  • Port defaults: storage_port 7000, ssl_storage_port 7001, native_transport_port 9042, JMX 7199; broadcast_address and broadcast_rpc_address must be reachable by peers and clients, not loopback.

  • GossipingPropertyFileSnitch reads dc and rack from cassandra-rackdc.properties; a wrong rack value silently produces imbalanced replicas under NetworkTopologyStrategy.

  • Seeds are only bootstrap contact points for gossip; after joining, a node learns the full membership, so seeds are not a runtime single point of failure.

  • Host tuning: disable swap, vm.max_map_count 1048575, ulimit -l unlimited, ulimit -n 100000, and an XFS or ext4 data mount with noatime.

  • JVM: G1GC is the default collector from 4.0 (CMS removed); set -Xms equal to -Xmx and keep the heap near or below 31 GB to retain compressed oops.

  • A joining node with allocate_tokens_for_local_replication_factor derives balanced tokens from the keyspace replication factor; otherwise set num_tokens and let the cluster accept them.

  • After starting a new node, watch nodetool status for UJ to UN and confirm bootstrap streaming completed; a failed bootstrap requires wiping the data and commit log directories before retrying.

  • Most cassandra.yaml edits require a node restart; throughput-style settings such as setstreamthroughput and setcompactionthroughput can be changed live with nodetool.

Quick reference

Data Modeling and CQL Schema Design

  • PRIMARY KEY (a, b, c): a is the partition key, b and c are clustering columns that sort rows inside the partition and enable range and slice queries on that order.

  • Partition size target: keep partitions under roughly 100 MB and about 100,000 rows so reads, repairs, and compactions stay bounded; split with a bucketing column when they grow.

  • Model one table per query: Cassandra has no joins, so duplicate data across tables and write to each table in the same application operation.

  • Static columns hold a single value per partition, which avoids repeating common attributes across every row of a wide partition.

  • Frozen collections serialize the whole value as one cell, so updates rewrite the entire collection; non-frozen collections allow element updates but generate per-element tombstones.

  • Counter tables may contain only counter columns and the primary key; they cannot have TTL, secondary indexes, or mixed counter and non-counter columns.

  • Secondary indexes are per-node and best for low-cardinality columns; SAI in 4.0 broadens supported predicates but still scatters reads across all nodes.

  • LOGGED BATCH is atomic across partitions through the batchlog, roughly doubling write cost; use UNLOGGED batches only when all statements share one partition.

  • TTL is set per statement (USING TTL), per table (default_time_to_live), or per column value; the ttl() function returns remaining seconds for a cell.

Quick reference

Data Distribution: Partitioning, Replication, Consistency, and Topology

  • SimpleStrategy is single-datacenter and rack-unaware; NetworkTopologyStrategy takes a replication factor per datacenter and places replicas on distinct racks where possible.

  • Replication factor per datacenter must not exceed the node count of that datacenter, and RF=3 is the standard production value for survivability with quorum reads and writes.

  • QUORUM equals floor(RF/2)+1; strong consistency requires read CL plus write CL greater than RF, for example QUORUM plus QUORUM with RF=3.

  • LOCAL_QUORUM counts only replicas in the coordinator datacenter, while EACH_QUORUM requires quorum in every datacenter and applies to writes only.

  • SERIAL and LOCAL_SERIAL drive Paxos lightweight transactions, which cost about four round trips and should be reserved for conditional updates and uniqueness constraints.

  • Hinted handoff stores writes for unreachable replicas on the coordinator; max_hint_window_in_ms defaults to 3 hours, after which a missed write requires repair.

  • Read repair is blocking for reads at QUORUM or higher and backgrounded for weaker consistency levels, so low read consistency shifts convergence burden onto repair.

  • Use nodetool ring, nodetool describering, and nodetool getendpoints to confirm which nodes own a key before blaming a client for routing errors.

  • Adding a new datacenter requires the new nodes to join with the right snitch and tokens, then nodetool rebuild run with the source datacenter name to stream existing data.

Quick reference

Performance Tuning: Compaction, Caching, Read/Write Path, and JVM

  • SizeTieredCompactionStrategy suits write-heavy workloads, uses min_threshold 4 and max_threshold 32 by default, and needs roughly 50 percent free disk for compaction headroom.

  • LeveledCompactionStrategy keeps non-overlapping SSTables in levels with sstable_size_in_mb 160 and roughly tenfold growth per level; it lowers read amplification but raises write and space amplification.

  • TimeWindowCompactionStrategy groups SSTables into time windows and compacts within a window, so TTL-expired time-series data is dropped as whole SSTables.

  • Bloom filter fp_chance defaults to 0.01; raising it reduces false-positive disk reads at the cost of more heap and larger filter files.

  • Key cache defaults to the smaller of 5 percent of heap and 100 MB; row cache is disabled by default because invalidation cost usually outweighs the benefit.

  • Read path order: row cache, memtable and commit log, bloom filter, partition index and summary, compression offsets, SSTable scan, then merge of replica responses.

  • Write path: commit log append (periodic sync every 10 seconds by default, or batch sync for stricter durability), memtable insert, then flush to an immutable SSTable.

  • Diagnostics: nodetool tpstats for dropped messages per thread pool, tablehistograms for read and write latency percentiles, tablestats for SSTable and tombstone counts.

  • Heap and GC: G1GC with -Xms equal to -Xmx, avoid heaps far above 31 GB, and treat long GC pauses visible in GCInspector logs as a cause of timeouts, not a symptom to tune around.

Quick reference

Repair, Backup, Restore, and Anti-Entropy

  • Repair compares replica data using Merkle trees and streams differences; Cassandra 4.0 restored full repair as the default after incremental repair proved hard to reconcile.

  • Repair must complete for every table within gc_grace_seconds (default 864000 seconds, 10 days) or purged tombstones can resurrect deleted data on a stale replica.

  • nodetool repair options: -pr limits work to a node primary range, --full forces a full repair, and -dc restricts the repair to one datacenter.

  • Scale repair scheduling with a coordinator such as Reaper rather than firing cluster-wide repairs from cron, which can saturate compaction and streaming bandwidth.

  • nodetool snapshot -t tag creates hard links to live SSTables under the snapshots directory almost instantly; auto_snapshot also triggers before truncate and drop.

  • Point-in-time backups require snapshots plus commit log archiving configured with archive_command, since snapshots alone only capture the moment of the flush.

  • Restore procedure: stop the node, place the snapshot SSTables into the table data directory, keep or discard commit logs deliberately, restart, then nodetool refresh to load new files.

  • sstableloader streams SSTables into a running cluster and is the tool for restoring into a live cluster or bulk loading without taking nodes down.

  • nodetool rebuild re-streams a node or datacenter from a named source; nodetool verify and sstablescrub detect and repair on-disk corruption.

Quick reference

Security: Authentication, Authorization, Encryption, and Audit

  • authenticator controls identity: AllowAllAuthenticator is the default and PasswordAuthenticator stores bcrypt-hashed role credentials in system_auth.

  • authorizer controls permissions: AllowAllAuthorizer is the default, while CassandraAuthorizer records GRANT entries per role in system_auth.

  • Set system_auth replication factor above 1 in every datacenter; if those replicas are unavailable, authentication and authorization checks can block the whole cluster.

  • Cassandra 3.0 and later model users as roles: CREATE ROLE, GRANT role TO role for inheritance, and superusers bypass permission checks entirely.

  • Permission set includes SELECT, MODIFY, CREATE, ALTER, DROP, and AUTHORIZE, grantable on ALL KEYSPACES, a keyspace, a table, or a function.

  • TLS for internode links is configured in server_encryption_options with internode_encryption set to none, dc, rack, or all; client TLS uses client_encryption_options on port 9042.

  • Keystore and truststore passwords can be supplied through cassandra.yaml or a credentials file; require_client_auth makes mutual TLS mandatory for native clients.

  • Apache Cassandra has no built-in transparent data encryption or native audit log; use disk or filesystem encryption and third-party audit plugins such as ecAudit.

  • JMX is local-only by default; enabling remote JMX requires explicitly configuring authentication and SSL on port 7199 because unauthenticated JMX grants full administrative control.

Quick reference

Operations, Monitoring, Troubleshooting, and Upgrades

  • Core nodetool commands: status for ownership and state, info for load and uptime, tpstats for queue and dropped messages, netstats for streaming and hints, compactionstats for pending work.

  • nodetool drain flushes memtables and stops gossip and client writes, making it the correct pre-shutdown step before an upgrade; nodetool flush only persists memtables.

  • Upgrade one node at a time, confirm UN status and no pending compactions before moving on, and run nodetool upgradesstables afterwards so old SSTables use the new format.

  • Unavailable exception means too few live replicas to satisfy the consistency level; ReadTimeout and WriteTimeout mean enough replicas were alive but did not answer in time.

  • TombstoneOverwhelmingException fires when a read scans more than tombstone_failure_threshold tombstones (default 100000), usually from wide partitions plus TTL or delete churn.

  • Handle a dead node with nodetool removenode host-id; use decommission for a live node leaving gracefully and assassinate only for a node that cannot be removed normally.

  • Schema disagreement is visible in nodetool describecluster; resolve by restarting outliers or running nodetool resetlocalschema on nodes with divergent versions.

  • Monitor pending compactions, dropped message counts, p99 read and write latency, hint backlog, disk free space, and GC pause time as the minimum production metric set.

  • Bootstrap or streaming problems are diagnosed with nodetool netstats; throttling via setstreamthroughput or setcompactionthroughput prevents streaming from starving live traffic.