Enabling authentication, TLS and audit on a shared unauthenticated cluster

Each step can lock you out or break a live driver, so order matters and each step should be reversible on its own.

1. Preparation with no behaviour change: distribute the same keystore and truststore to all nine nodes, verify SANs match the hostnames the drivers use, and record the superuser credential in a vault. Keep an open local cqlsh session throughout.
2. Authentication: set authenticator: PasswordAuthenticator and restart one node at a time, waiting for UN and drained hints before the next. Immediately ALTER the cassandra role's password, create a break-glass admin role, and create one role per team. Leave the authorizer alone in this pass - AllowAllAuthorizer still permits everything, so mistakes here affect login only, not data access.
3. Authorization: switch to CassandraAuthorizer, roll again, but grant roles their keyspaces before the last node converts. Set system_auth RF to 3 first (it defaults to 1) and repair it after every grant change, or a single node failure makes login impossible.
4. Internode TLS: flip internode_encryption to all node by node, watching gossip and streaming between converted and unconverted nodes. If your version does not tolerate mixed mode, validate that in staging first - otherwise you need a short full-cluster window.
5. Client TLS: set client_encryption_options.optional: true, migrate one team per week, then set optional: false.

Lockout risk peaks at step 2 and at the system_auth RF trap in step 3. Recovery is editing cassandra.yaml back to AllowAll on the nodes and restarting.

Cassandra's permission model is role-based and scoped to keyspace, table and function - there is no row-level or predicate-based security. That single fact drives the design: the keyspace is the isolation boundary, and any two applications needing different rights must sit in different keyspaces.

Design:

• One login role per team per environment, SUPERUSER false. Never share a role between applications - it destroys audit attribution and forces joint password rotation, which is exactly the situation this cluster is in today.
• Runtime roles get only MODIFY on their own keyspace, plus SELECT on the other team's reference keyspace for the job that needs it. Nothing else.
• Reserve CREATE, ALTER and DROP for a separate deployment role. A compromised runtime credential should not be able to drop its own tables.
• Grant at table level where the team owns several tables with different audiences; a keyspace-level grant covers every current and future table in it.

Why careless grants are dangerous:

• ALL PERMISSIONS on a keyspace includes AUTHORIZE, which lets the grantee re-grant to any role. That is privilege escalation, and it is never appropriate for an application role.
• A keyspace-level SELECT granted to a team with several applications exposes all of them at once.
• Grants live in system_auth: with RF 1 a node loss can make the cluster unauthenticatable. Set RF 3 and repair after changes.

For password rotation, use ALTER ROLE and force client reconnection; existing pools keep working until they reconnect, which can mask a broken credential.

Cassandra 4.1 has audit logging built into the server, so no triggers or external agent are required. Configure audit_logging_options: enabled: true, logger: BinAuditLogger, included_categories (DDL, DML, QUERY, AUTH, ERROR), and optionally included_keyspaces. Events are written to binary files under the audit/ directory on each node and read with nodetool auditlogviewer. Ship those files off-node with a log shipper - there is no central audit table, so a node rebuild silently loses its local history unless shipped.

Cost:

• Every audited event costs a queue insert and a disk write on the request path. The queue is bounded and drops events rather than blocking requests when full, so a saturated queue creates silent gaps - monitor the dropped-event metric, because a gap defeats the control.
• On a write-heavy cluster, auditing DML is usually unaffordable. Start with DDL and AUTH, which is what the review actually asked for, enable on one canary node, measure latency delta, then roll out.

Limits:

• Entries record the query text and the authenticated role, not the rows returned. There is no per-row access trail.
• It cannot distinguish two applications sharing one credential - which is why splitting the roles from the previous task must happen first.
• It does not protect the audit files themselves; file permissions, shipping integrity and retention are yours to enforce.
• It does not log what happened before it was enabled, so the 'who dropped this table' question from last month stays unanswered.