Malicious pickle in the model registry and an unannounced model swap

The instinct to resist is "the model ran fine, so the payload was contained." Nothing about a pickle's payload is visible in model behavior - successful execution and normal inference are simultaneously true. The second instinct to resist is deleting the artifact and rebuilding the node before capturing state.

Order of operations:

1. Preserve, quickly. Snapshot the node's disk and capture environment, process history, and network connections before anything is rebuilt. The artifact and its hash go into quarantine storage, not the bin.
2. Rotate the known-stolen credentials immediately - S3 write credential and registry token. Rotation takes minutes and is strictly cheaper than the alternative, so it precedes deep investigation.
3. Hunt the credential use. The two questions that matter: did the S3 credential write, delete, or list anything in the claims lake, and did the registry token push any new artifact? A pushed artifact is supply-chain persistence - the attacker plants a model that your own pipeline will pull and deploy. Enumerate registry writes and compare digests against known-good.
4. Retrieve what was posted. Check the paste endpoint for the exfiltrated content, both to scope the exposure and to establish whether other hosts posted to it.
5. Sweep the fleet. Any GPU node or workstation that loaded the same artifact or anything from the same publisher/period is presumed compromised. Look for the same outbound destination in network telemetry - and note that the absence of egress logs on GPU nodes is itself an investigative blind spot to record.
6. Notify. Claims data is involved; Legal and privacy run the breach assessment with the clock from discovery, and the exposure includes credentials as well as data.

Then, before any re-enable: egress deny on GPU nodes and a clean-room loader. Restoring service without those restores the attack path.

Assume every external artifact is hostile and design so that hostility has nowhere to go. Three properties: it cannot execute, it cannot reach credentials, and it cannot leave the network unnoticed.

1. Format and scanning (deny execution-by-default).

• Accept only non-executable serialization - safetensors or equivalent - and reject pickle-based formats at ingestion. This is the highest-leverage single rule, because the entire attack is deserialization code execution.
• For legacy artifacts that genuinely must be converted, do the conversion inside a disposable sandbox with no network and no credentials, then scan the output with a model-security scanner (pickle opcode analysis, unusual imports, embedded executables).
• Record hashes and provenance: publisher identity, source URL, retrieval timestamp, and a signature or attestation where the publisher supports it. Pin by digest, not by tag or filename - the malicious artifact was introduced as a plausible-looking "fixed-tokenizer."

2. Segregation and least privilege.

• Ingestion runs in a quarantine namespace/account with no mount of the claims lake, no registry write token, and no cloud metadata service reachable (block IMDS, or require IMDSv2 with hop limit 1 - better, run without an instance role entirely).
• The serving tier gets read-only access to a specific prefix holding approved artifacts, and nothing else. The incident's writable claims-lake mount on a GPU node was an architectural mistake independent of the pickle.
• Promotion from quarantine to the serving registry requires an explicit approved transition, not an automatic sync.

3. Network.

• Default-deny egress from GPU and ingestion nodes with an allowlist for the artifact store and telemetry sinks. A hardcoded paste endpoint is trivially blocked by deny-by-default.

Finally, stop treating the model card and publisher reputation as controls. A convincing description is what made the download plausible; the control has to be structural.

Treat a model version change as a production change event, whether or not the vendor agrees.

Contractual and versioning layer.

• Pin to a specific model version or snapshot identifier where the provider offers one, and record that identifier - not the friendly product name - in config, code review, and the model registry. "Latest" is not a version.
• Where the vendor offers no pinning, negotiate a change-notification clause with a defined notice window, and maintain a documented fallback model that has passed the same evaluation suite. The judgement is that pinning alone is insufficient: pinned versions get deprecated, so you are choosing between unscheduled behavior change and scheduled migration work.

Detection layer - evals as monitoring, not as a one-time gate.

• Maintain a golden evaluation set for claims summarization: several hundred labeled examples covering the actual work, including adversarial and edge cases (ambiguous claim language, multi-injury claims, denial language). Run it on every model change and on a weekly schedule, because a provider can change behavior without changing the version string - inference-stack updates, routing, or safety-layer tweaks can move outputs.
• Set a regression threshold with an owner and an action: an 11-point drop should have paged someone, not been discovered later. Include quality metrics beyond accuracy - hallucinated policy citations, refusal rate, and output length distribution are all useful cheap signals.
• Log the model version identifier on every inference response so any downstream complaint can be traced to a version change rather than debated.

Process layer.

• Canary new versions against a slice of live traffic with human review of a sample before full rollout.
• Keep the evaluation suite versioned alongside the prompts; a prompt change and a model change confounded together is a diagnosis you cannot make.

The residual risk to state: a vendor can change behavior silently inside a pinned version, so the weekly eval plus version logging is the only reliable detector. Assume pinning reduces frequency, not possibility.