AI Agents in CI/CD: Supply Chain Risk from Autonomous Coding and Agent-Driven Deployments

Agentic AI

Autonomous AI agents are entering the software delivery pipeline — reviewing PRs, writing code, triggering deployments. Each of these capabilities introduces supply chain risk that traditional pipeline security was not designed to address. Here is the threat model and what to do about it.

By Arjun Raghavan, Security & Systems Lead, BIPI · July 19, 2025 · 12 min read

Software supply chain security was the defining threat theme of 2021-2023. SolarWinds, Log4Shell, XZ Utils, and a long tail of compromised npm packages established that the build and delivery pipeline is a high-value attack surface. The industry responded with SBOMs, dependency verification, signed artifacts, and hardened CI infrastructure. Now AI agents are entering that same pipeline, and most of those defences were not designed with autonomous coding agents in mind.

How AI Agents Are Entering the Pipeline

The entry points are multiplying rapidly. Coding assistants (Copilot, Cursor, Claude Code) that suggest and commit code changes. Automated PR review agents that can approve and merge pull requests. Dependency update agents (Dependabot-style but LLM-powered) that propose version bumps with justification. Test generation agents that create and commit test files. Deployment agents that interpret natural language release instructions and trigger deployments. Each represents a new autonomous actor with write access to the repository or the deployment pipeline.

The New Threat Model

• Autonomous code injection: a compromised or manipulated coding agent commits code that introduces a backdoor, credential theft, or logic bomb
• Approval chain bypass: an AI code review agent that can both review and merge PRs collapses the four-eyes principle that catches malicious changes
• Dependency confusion via agent: an LLM-powered dependency update agent, fed adversarial input, proposes a version bump to a compromised or malicious package
• Secret exfiltration during execution: an agent with access to secrets during CI execution can exfiltrate them to external endpoints under cover of normal build activity
• Prompt injection via code comments: adversarial instructions embedded in code comments that are read by code review agents, causing them to approve changes they should flag
• Hallucinated dependency addition: LLMs generate plausible package names that do not exist; if a malicious actor registers that package name, the next agent-generated lockfile installs it
• Pipeline modification: an agent with the ability to modify CI configuration files can alter the pipeline itself — adding steps that exfiltrate build artifacts or inject into deploy scripts

The Hallucinated Dependency Attack

This attack class deserves special attention because it is unique to LLM-generated code. Language models sometimes generate import statements or package.json entries for packages that do not exist — packages with plausible names that fit the context. Researchers have demonstrated that by monitoring which package names LLMs hallucinate and registering those names with malicious payloads, it is possible to create packages that will be installed the next time an agent regenerates a lockfile or adds a dependency. This is dependency confusion without the need to compromise an existing package.

Hardening the AI-Integrated Pipeline

1. Enforce separation of concerns: no single agent should have both write-to-repository and merge-approval permissions — human approval required at the gate
2. Treat agent-generated code as untrusted input until it passes the same SAST, dependency scanning, and secret scanning checks as human-generated code
3. Implement package name verification: check every new package against a registry before installation and flag packages added or updated in the last 30 days
4. Audit agent permissions quarterly: agents accumulate permissions over time — regularly review and revoke what is not actively needed
5. Log all agent commits with the agent identity, the task it was executing, and the prompt it received — distinguish agent commits from human commits in your audit trail
6. Apply network egress filtering to CI agents: they should not be able to reach arbitrary external URLs during build execution
7. Implement artifact signing and verify signatures in deployment — agent-generated builds should be indistinguishable from human builds in terms of signing requirements
8. Red-team your pipeline specifically for prompt injection via code comments and malicious PR descriptions

The Governance Question

Beyond the technical controls, AI agents in CI/CD raise governance questions that security teams and engineering leadership need to answer together. What is the maximum blast radius an agent can have — how many production systems can it affect if compromised? Who owns the security posture of an agent that is a product of several third-party vendors (the LLM provider, the agent framework, the CI platform)? How do you audit a deployment that was executed by an agent acting on a natural-language instruction with no deterministic specification? These questions do not have industry-standard answers yet. The teams that work through them before an incident will be better positioned than those that address them after.

Read more field notes, explore our services, or get in touch at info@bipi.in. Privacy Policy · Terms.