BIPI
BIPI

Model Supply Chain Security: HuggingFace, Pickle, and Trusted Loaders

AI Security

Pulling a model from a public hub is a supply chain decision. We cover the real risks, unsafe deserialization, weight tampering, and dependency injection, and how Sigstore signed artifacts plus trusted loaders close the gap.

By Arjun Raghavan, Security & Systems Lead, BIPI · July 26, 2023 · 10 min read

#supply-chain#huggingface#sigstore#safetensors#mlops

A model is code plus weights plus a tokenizer plus a config. Each of those is a supply chain artifact, and each has been used to deliver attacks in the wild.

Unsafe deserialization is the obvious problem

Python's legacy serialization format executes arbitrary code during deserialization. A malicious checkpoint pulled from a public hub can pop a shell on the inference server. This is not a hypothetical, multiple proof of concept worms have shipped through public model repos.

Safer formats

Safetensors and ONNX are designed to deserialize without executing code. Default to them, and refuse to load legacy checkpoints unless they pass an explicit policy check including provenance and signature.

Weight tampering

Even with a safe format, weights can be subtly modified to introduce backdoors that trigger on specific inputs. Detection is hard, prevention through provenance is easier. Pin model hashes, verify them at load, and reject any drift.

  • Hash every artifact at publication time
  • Sign hashes with a key tied to the publisher identity
  • Verify signatures at load, fail closed on mismatch
  • Record the verified hash in the inference server's audit log

Sigstore for models

Sigstore extends to model artifacts the same transparency model used for container images. Publishers sign, a transparency log records, consumers verify. The pattern is mature enough to adopt in MLOps pipelines today.

Tokenizer and config files

Often overlooked. A tampered tokenizer can change how inputs are processed in subtle, exploitable ways. Treat every file in the model bundle as a signed artifact, not just the weights.

Dependencies that ride along

Model repos often include requirements files or trust_remote_code flags that pull arbitrary Python. The flag is named honestly, it asks you to trust remote code, and most teams say yes without thinking. Default to false, override only with review.

Pipeline controls

  1. Mirror approved models to an internal registry
  2. Scan checkpoints for known malicious patterns
  3. Verify signatures and hashes at every load
  4. Pin model and dependency versions in deployment manifests
  5. Audit which models are running in which environments
The model is software. Apply the same supply chain discipline you apply to npm and pip, and apply it before the first deployment.

Tooling notes

HuggingFace supports safetensors and signed model cards. Sigstore is integrating with model registries. MLflow can record artifact hashes alongside experiment metadata. NeMo and other inference servers can be configured to reject unsigned weights.

Closing

Supply chain security for models is no longer optional. Sign, verify, pin, and treat trust_remote_code as the warning it is.

Read more field notes, explore our services, or get in touch at info@bipi.in. Privacy Policy · Terms.