A Poisoned Model Artifact in the MLOps Pipeline
Corvus Pay is a 400-person fintech with 60 data scientists. Its model pipeline builds containers in GitHub Actions, installs dependencies with pip install -r requirements.txt with no hashes or pins, and downloads model checkpoints from a public hub by repository name only. Checkpoints are loaded with torch.load(); deployment images are tagged latest. The CI job assumes a shared role with s3:* on the production artifact bucket. There is no artifact signing and no SBOM. Three weeks ago a data scientist uploaded a checkpoint found on a community forum to the shared bucket so a colleague could test it; that checkpoint is now in the production fraud-scoring model, which processes 900,000 transactions a day. A build log shows an outbound connection to an unfamiliar IP during the install step. Nobody has reviewed the runner's network logs.
1. Assess what may have happened, what evidence you need, and what containment you would order today.
Read the worked answer
There are two independent compromise paths and both must be assumed live. The unpinned install step executes arbitrary code from the package registry inside a runner holding s3:* on the production artifact bucket - that is a credential-theft and artifact-tampering path, consistent with the unfamiliar outbound connection. The checkpoint is a pickle, and torch.load() deserialises arbitrary code on load; an untrusted checkpoint is code execution plus a possible behavioural backdoor with a trigger.
Evidence: CI run logs and job environment, runner egress and DNS logs, CloudTrail and S3 access logs for the production bucket, image digests actually deployed over the last three weeks, hashes of all artifacts in the bucket, and the checkpoint's provenance. Behaviourally, compare production scores against a clean baseline on synthetic and trigger-shaped inputs to look for a backdoor.
Containment today: revoke and rotate the CI role and any credentials reachable from the runner, quarantine the artifact bucket, freeze deployments, rebuild runners from clean images, block outbound egress from CI except an allowlist, and load the suspect checkpoint only in an isolated sandbox with a safe deserialiser. Assume the production artifact bucket is untrusted until hashes prove otherwise.
2. Design the technical promotion gate an artifact must pass to reach production. Focus on enforceable controls, not policy statements.
Read the worked answer
The gate should be enforced by the deployment system, not by a checklist a human signs. Concretely: dependency installation from a lockfile with hashes through an internal proxy that blocks unlisted packages; base images and build steps pinned by digest; model artifacts referenced by digest, never by name or tag; mandatory scanning of every checkpoint, with conversion to a non-executable format such as safetensors before it can enter the artifact store; signature verification at admission using cosign or equivalent, so an unsigned or unverifiable image is rejected by the cluster rather than flagged; an ML-BOM and SBOM generated per build; a build network with no general internet egress; short-lived OIDC credentials scoped to a single deploy target rather than a shared s3:* role; and a behavioural evaluation suite - accuracy on a golden set plus backdoor and trigger probes - that must pass before promotion.
Two-person approval for model artifacts adds a human check, but it is not a substitute for the automated controls. The test is simple: if a data scientist could bypass the gate by pushing to the bucket, the gate does not exist.
3. You have three engineers for 90 days. Sequence the remediation and justify the order.
Read the worked answer
Week 1 - stop the bleeding: rotate the CI role and all reachable credentials, quarantine the bucket, block CI egress, and freeze production deployments until the artifacts are rebuilt from verified sources. This addresses the highest-likelihood path to full environment compromise.
Weeks 2–4 - remove the two ingestion paths: lockfile with hashes and an internal package proxy, artifact-by-digest everywhere, and conversion to safetensors with torch.load() banned by lint rule and build check. These are cheap, mechanical, and eliminate the code-execution vector.
Weeks 4–8 - add integrity: signing with verification at admission, SBOM and ML-BOM generation, least-privilege deploy identities with OIDC, and an isolated build network.
Weeks 8–12 - add assurance: backdoor and trigger testing in the promotion suite, RBAC review on the artifact store, and a tabletop exercise for a compromised model release.
The order is driven by blast radius and reversibility. Signing before pinning is theatre, because a signature over a malicious artifact is still valid; credentials and egress come first because they convert a code-execution bug into control of the production environment.