The xz Backdoor Was a Three-Year Operation. Your Build Pipeline Should Assume Everything Is.

Cybersecurity

In March 2024 a Microsoft engineer noticed sshd was 500ms slower than expected and uncovered a backdoor in xz utils that would have given remote code execution on every Linux server in the world. The attack itself is interesting. The story is more interesting.

By Arjun Raghavan, Security & Systems Lead, BIPI · April 15, 2025 · 9 min read

In late March 2024, Andres Freund, a PostgreSQL developer working at Microsoft, was running benchmarks on a Debian unstable machine. He noticed that sshd connections were taking about 500 milliseconds longer than they should. He spent a weekend tracing the latency through profilers and uncovered a backdoor planted inside xz utils, the compression library used by basically every major Linux distribution and indirectly by sshd through a transitive dependency.

The backdoor would have given the attacker pre-authentication remote code execution on any sshd instance that linked liblzma — which on Debian, Ubuntu, Fedora, and several others is by default. Andres caught it in late-stage testing. Days more and it would have shipped to stable in distribution after distribution. The fix was rapid, and the backdoor never reached production at scale. The lesson is not the technical one. The lesson is the patience and the playbook.

What actually happened

The attacker, known publicly only as 'Jia Tan,' was not a random external. They had been a contributor to xz utils for nearly three years. They submitted patches starting in late 2021. They showed up consistently. They reviewed others' code. They were polite, technically competent, and persistent.

The original maintainer of xz utils is a single individual, Lasse Collin, who had publicly mentioned in 2022 that he was burned out. There were public messages on the mailing list — almost certainly from sock-puppet accounts coordinated with the attacker — pressuring Lasse to add maintainers. Jia Tan was the one who eventually got added, in 2023, with full release access to the project.

From there the operation was patient. The backdoor was not added in a single commit. It was assembled from pieces over months: an obfuscated build script that pulled bytes from binary test fixtures, a series of seemingly innocuous changes to the m4 macros, and the final payload concealed inside test data files (.xz archives) that nobody inspects because they are test fixtures. The build script combined the fragments at release time, but only when packaging a release tarball — never in the git source. Anyone running git checkout did not see the backdoor. Anyone running ./configure on the released tarball did.

The threat model this changes

Before xz, the supply-chain threat model that most engineering teams ran was something like: an attacker compromises a maintainer's account (credential theft, phishing) and pushes a malicious version. Defences were token hygiene, 2FA, anomaly detection on package publishes.

After xz, that model is incomplete. The new attacker is not an external who steals an account. The new attacker is a person who builds a relationship over years with the goal of becoming a legitimate maintainer. By the time they push the malicious code, every defensive control treats them as authorised, because they are.

Single-maintainer projects are the highest-risk profile. So are projects where the original maintainer has signalled burnout. So are critical infrastructure dependencies that nobody else maintains. xz is not a corner case — it is the standard shape of OSS infrastructure. There are dozens of liblzma-equivalents in your dependency tree right now, each maintained by one or two people, each potentially in a maintainer-transition-vulnerable state.

Five practices we now require

1. SBOM with maintainer-graph annotation. Generate a software bill of materials, but go further: enrich it with public maintainer data (GitHub usernames, recent activity, addition dates). Flag projects where a new maintainer was added in the last 12 months.
2. Reproducible builds for tier-0 deps. The libraries you cannot tolerate compromise on (compression, crypto, parsing) must be built from git, byte-equivalent to the released tarball. Discrepancy is a P1 alert.
3. Pin to commit hashes, not version tags, for the most critical deps. Tags can be moved; commit hashes cannot. Update deliberately.
4. Audit pipeline diversity. Build the same package on at least two independent infrastructures and compare outputs. Difference equals investigation.
5. Subscribe to maintainer-change notifications for your tier-0 list. There are public services that monitor this; some teams roll their own.

What you cannot prevent but can detect

You cannot stop a Jia Tan from showing up to a project you depend on. You can shorten detection time by monitoring the build outputs of your tier-0 dependencies for unexpected behaviours: surprising network connections, surprising syscalls, surprising file accesses during build. The xz backdoor would have produced exactly these signals at build time, because the obfuscation step had to assemble itself somewhere observable.

Runtime detection is the last line. Network egress monitoring on production hosts catches the C2 channels that backdoors phone home to. Process anomaly detection catches the unexpected listeners. None of this is new. The point is that 'audit the dependency once, trust forever' is not the threat model the xz operation was designed to defeat.

The new attacker is not a thief. The new attacker is a legitimate maintainer who waited.

Closing

The xz backdoor caught Andres Freund's attention because he was running benchmarks. It would not have been caught at code review, at vulnerability scan, at standard supply-chain audit, or at first-time package install. It was caught because one person noticed a small performance anomaly and had the time and skill to chase it. That should not be the only line of defence on the world's Linux infrastructure. Build the controls assuming the attacker has three years and a track record of helpful code reviews. They probably do.

Read more field notes, explore our services, or get in touch at info@bipi.in. Privacy Policy · Terms.