# Supply Chain AI Poisoning
Definition
Supply chain AI poisoning is an attack vector where adversaries compromise the AI/ML supply chain by injecting malicious data into training datasets, tampering with pre-trained models, or backdooring model weights distributed through public repositories. As organizations increasingly rely on pre-trained models, open-source datasets, and third-party AI services, the supply chain for AI components has become a high-value target. Unlike traditional software supply chain attacks that inject malicious code, AI poisoning attacks embed malicious behavior directly into the model's learned parameters, making detection extremely difficult.
How It Works
AI supply chain poisoning operates through several vectors:
Training Data Poisoning: Adversaries inject carefully crafted samples into public or shared training datasets. The poisoned data teaches the model to behave maliciously under specific trigger conditions while performing normally otherwise.
- Backdoor Attacks: The model performs correctly on normal inputs but produces attacker-chosen outputs when a specific trigger pattern is present (e.g., a specific pixel pattern in an image causes misclassification)
- Targeted Poisoning: The model produces incorrect results only for specific inputs relevant to the attacker's objective
- Clean-Label Attacks: Poisoned samples have correct labels, making them invisible to data quality checks
Model Poisoning: Adversaries distribute pre-trained models through public repositories (Hugging Face, model zoos, GitHub) that contain embedded backdoors:
- Modified model weights that activate backdoor behavior on trigger inputs
- Malicious code in model serialization formats (Python pickle files can execute arbitrary code upon loading)
- Trojan models that exfiltrate data during inference through covert channels
Fine-Tuning Poisoning: Even if the base model is clean, adversaries can poison the fine-tuning process:
- Compromised fine-tuning datasets available on public platforms
- Malicious adapters (LoRA weights) distributed through community channels
- Poisoned instruction-tuning datasets that introduce harmful behaviors
Infrastructure Poisoning: Compromising the ML pipeline infrastructure:
- Tampering with data pipelines that feed training processes
- Compromising model registries to serve poisoned model versions
- Manipulating feature stores that provide input data for inference
Why It Matters
The AI supply chain is uniquely vulnerable because of how the ecosystem operates:
- Pre-trained model dependency: Most organizations do not train models from scratch. They download pre-trained models from public repositories and fine-tune them. This creates a trust-on-first-use problem analogous to downloading unverified software.
- Dataset opacity: Training datasets are massive (billions of samples) and difficult to audit comprehensively. A few thousand poisoned samples among billions can embed backdoors that are statistically invisible.
- Model opacity: Deep learning models are fundamentally opaque. There is no reliable method to inspect a model's weights and determine whether a backdoor exists without testing against the specific trigger.
- Ecosystem trust: The ML community operates on reputation and trust rather than cryptographic verification. Model provenance is poorly tracked.
The consequences of successful AI poisoning are severe:
- Medical AI that misdiagnoses specific conditions
- Autonomous vehicle perception that misclassifies specific objects
- Financial AI that approves fraudulent transactions matching trigger patterns
- Security AI that fails to detect specific attack techniques
- Content moderation AI that approves specific harmful content
Real-World Applications
- Open Source Models: Malicious models uploaded to Hugging Face and other repositories that contain serialization exploits or embedded backdoors.
- Poisoned Datasets: Manipulation of widely used training datasets (Common Crawl, LAION) to embed biases or backdoors that propagate to downstream models.
- Compromised ML Pipelines: Attackers targeting CI/CD pipelines for model training to inject poisoned data or modify training parameters.
- Fine-Tuning Services: Malicious fine-tuning services or adapters that introduce backdoors into customer models.
- AI SaaS Supply Chain: Third-party AI APIs that serve poisoned model outputs for specific inputs.
CDA Perspective
Supply chain AI poisoning maps to CDA's Vulnerability & Surface Defense (VSD) domain under the Continuous Surface Reduction (CSR) methodology. Just as we track software supply chain risks through SBOMs, AI supply chain risks require model bills of materials and provenance tracking.
CDA's approach:
- M-VSD-R02 inventories all AI models, datasets, and pipeline dependencies as part of the supply chain assessment
- M-VSD-H01 implements model provenance verification, serialization security (avoiding pickle, preferring safetensors), and continuous model validation
- M-TID-H01 monitors for published AI supply chain attacks and vulnerability disclosures in models the organization uses
CDA recommends: never load untrusted model files. Use safe serialization formats. Validate model behavior against known-good benchmarks before deployment. Maintain a model bill of materials alongside your software bill of materials.
Key Takeaways
- AI supply chain poisoning embeds malicious behavior in models, datasets, or training infrastructure
- Backdoors in poisoned models activate only on specific trigger inputs, evading standard testing
- Model serialization formats (especially Python pickle) can execute arbitrary code on load
- Pre-trained model dependency creates a trust problem analogous to using unverified software
- Training datasets are too large to audit comprehensively, making data poisoning difficult to detect
- Organizations need model bills of materials and provenance tracking alongside SBOMs