A leaked key, unanswerable deletions, and a CMEK mandate

Contain, then observe, then remediate.

1. Contain. Disable the service account rather than deleting it, so the audit trail and existing bindings survive, and check what it did using Admin Activity logs before you cut it off.
2. Observe, honestly. Admin Activity logs are on by default and retained 30 days in _Default. Data Access logs are off by default and are not retroactive: enabling them now records tomorrow, not last month. So the deletion question cannot be answered from Cloud Logging, full stop. Recover what exists physically: GCS soft delete retains objects for a default period, and if Object Versioning was enabled the noncurrent versions are still recoverable. Tell the auditor this plainly and show the recovery attempt.
3. Remediate. Enable Data Access logs for Cloud Storage with a filtered sink scoped to the bucket, not project-wide: these logs are high volume and can cost more than the compute. Then build an organization-level aggregated sink into a dedicated logging project with a 365-day retention policy on the log bucket, locked only after you are certain of the number, because a locked retention policy can never be shortened or removed. Grant auditors roles/logging.viewer on that project alone.
4. Least privilege. Replace the analyst's Editor with a role-based group grant scoped to the datasets they need; use Policy Analyzer to prove effective access rather than listing bindings.

Location first. The key must live in the same location as the data: a us multi-region dataset needs a keyring in us. A mismatch fails at creation time and is a common first-attempt error.

Rotation. Set the key's rotation period to 90 days. Automatic rotation creates a new primary version; it does not re-encrypt existing data. Old key versions must remain enabled indefinitely, because that is what decrypts the existing bytes. Destroying or disabling an old version is unrecoverable data loss, and the 90-day rotation policy that sounds rigorous is the exact thing that invites someone to tidy up old versions.

Grants. The BigQuery and Cloud Storage service agents need roles/cloudkms.cryptoKeyEncrypterDecrypter on the key; the console surfaces the exact service account when you set a default key, so use that rather than guessing.

Scope of the default key. Setting a dataset default key applies to new tables only; existing tables need an explicit update. Changing a bucket's default KMS key does not re-encrypt existing objects; they must be rewritten. Budget for the rewrite, and verify with each object's kmsKeyName.

Operational trap. A disabled key takes the dataset offline. Put the keyring in a separate project with its own IAM and alert on key state, or you have converted a permissions problem into an outage.

The server already has an identity you can prove, so it does not need a key at all.

Target state: a workload identity pool with a provider for your on-premises identity source (OIDC, SAML, or an X.509 trust store). Map an attribute to the target service account and grant roles/iam.workloadIdentityUser to the federated principal with an attribute condition restricting it to that specific subject. The condition is the risky part: without it, anything that can obtain a token from that provider can impersonate the service account, and because it leaves no key to find, the breach is invisible.

Cutover: provision federation, run both credential paths in parallel, switch the ETL job to exchanging its external credential for a short-lived Google token, confirm success for a full run cycle, then disable the key, watch for 24 hours, then delete it. Deleting first is how you discover the second job nobody documented.

Prevent the next one: Org Policy constraints/iam.disableServiceAccountKeyCreation at the organization node, plus rotating every secret that shared the repository, since a public commit means the key was exposed, not merely present.