Label Poisoning in a Fraud Model Supply Chain

Concept drift is a natural change in the relationship between features and labels, such as fraudsters adopting a new cash-out pattern. Poisoning is adversarial manipulation of training data or labels to cause a targeted model behaviour. Here the evidence points to poisoning: a bounded cluster of 1,200 merchant IDs had labels flipped, the change coincided with a new reseller, and the effect was a sharp recall loss on a specific subpopulation rather than a broad distributional shift. The drift monitor looked at feature distributions, especially domain age, not label integrity or segment-level performance. It missed the attack because global precision stayed flat: fraud is rare, so a recall drop on a small merchant cluster barely moves aggregate precision. The monitor also had no baseline for label-flip rates by vendor, reseller, or merchant cohort. A better monitor would track per-segment recall, label provenance, and changes in the positive rate for newly ingested entities, and would compare vendor labels against delayed ground truth such as confirmed chargebacks.

The feed should be treated as an untrusted software supply chain. Require signed manifests and checksums, mutual TLS or scoped credentials for the S3 bucket, and an allowlist of resellers. Ingest into a quarantine zone, validate schema, ranges, and entity history, and compare each weekly drop against the prior week for label-flip rates, new merchant IDs, and domain-age anomalies. Maintain an independent, delayed ground-truth source such as confirmed chargebacks and use it to score vendor labels before they enter training. For the retraining pipeline, version data and code, pin dependencies, and require human approval for promotion rather than automatic promotion on a single AUC threshold. Add shadow deployment, segment-level gates on recall for high-risk merchant cohorts, and a rollback path. Use canary or honey merchants with known labels to detect manipulation. Contractually require breach notification, audit rights, and data-quality attestations, and keep a second-source option even if switching takes 90 days. Finally, separate the feature pipeline from label ingestion so a compromised label feed cannot silently rewrite the target variable.

Keeping the vendor is defensible only if the risk is contained while a second source is qualified. I would recommend immediate suspension of the vendor's labels from training, not necessarily suspension of all features, and a freeze on automatic promotion. The model should run in a monitored mode with segment-level recall alerts and a lower fraud threshold for the affected merchant cluster. The vendor should be required to explain the reseller, provide access logs, and accept forensic review. The decision should be recorded as a formal risk acceptance with an owner, expiry date, and exit criteria. Evidence that would change the answer includes independent confirmation that the flipped labels were a vendor-side error rather than an attack, a signed remediation plan with data-lineage proof, and a successful replay showing that removing the affected labels restores recall. Evidence that would reinforce suspension includes further label anomalies, unexplained access by the reseller, or the vendor refusing audit rights. The 90-day switching cost is a real constraint but not a reason to keep ingesting unverified labels; the interim control is to isolate the supplier's influence on the target variable.