Sigma Rules Are the Closest Thing Detection Engineering Has to a Standard

Cybersecurity

Writing detections in vendor-specific query languages locks you to that vendor. Sigma plus pysigma lets you write once, compile per platform, and run detection-as-code with the same rigor as application code.

By Arjun Raghavan, Security & Systems Lead, BIPI · January 10, 2024 · 7 min read

A bank in Mumbai migrated from QRadar to Sentinel in 2023. They had 800 detection rules in QRadar's AQL. The migration project was budgeted at four months. It took fourteen, and the team rewrote roughly 60% of the rules from scratch because nobody documented what the originals were supposed to catch. The QRadar contract had ended six months earlier than the migration finished. They paid for two SIEMs in parallel during the gap.

Sigma exists to prevent that story. Write the detection logic once in a YAML-based platform-agnostic format, then compile it down to whatever query language your current SIEM uses. When you migrate, you recompile. Your rule source of truth lives in your git repo, not your vendor's UI.

What Sigma Actually Is

Sigma is a YAML schema for describing log-based detections. Each rule has metadata (id, title, author, references, MITRE ATT&CK tags), a logsource block (product, category, service), a detection block (selections and a condition), and falsepositive notes. The whole thing is human-readable and version-controllable.

The pysigma library converts Sigma rules into target query languages. As of late 2023 there are backends for Splunk SPL, Microsoft Sentinel KQL, Elastic ES|QL, CrowdStrike Falcon Event Search, QRadar AQL, Chronicle YL2, and a dozen more. Some backends are better maintained than others, Splunk and Sentinel are first-class, Chronicle and Sumo are usable but lag.

The CI Pipeline That Makes This Work

Sigma in a vacuum is just YAML. The value shows up when you wire it into CI. A typical pipeline:

1. Detection engineer opens a PR with a new or modified rule in the git repo
2. GitHub Actions runs pysigma to compile the rule against all target platforms
3. Unit tests fire sample log lines through the compiled query (using Sigma's built-in test harness or a custom evaluator)
4. Atomic Red Team tests run in a sandbox, generating real telemetry, verifying the rule fires
5. Reviewer approves, merge triggers deployment to the SIEM via the vendor's API

The deployment step is where most teams trip. Splunk has a savedsearches.conf REST API. Sentinel has the Microsoft.SecurityInsights ARM API. Elastic has the detection rules API. Each one needs auth, error handling, and idempotency logic so you don't create duplicate rules on retry. Budget two engineering weeks for the deploy pipeline, not two days.

Where Sigma Falls Short

Sigma is great for log-based detections. It is not great for stateful or aggregating logic. "Five failed logins followed by one success within 60 seconds" is hard to express in Sigma and translates poorly to platforms that don't support sliding-window correlation natively.

It also struggles with platform-specific enrichments. Sentinel's UEBA scores, CrowdStrike's IOA chains, Splunk's risk-based alerting, none of those have clean Sigma equivalents. The pragmatic answer is to keep Sigma for the 70% of detections that are field-based pattern matches, and write platform-native rules for the 30% that need vendor-specific capabilities.

Rule Quality Beats Rule Quantity

SigmaHQ's public rules repo has roughly 3,000 rules. Don't import all of them. We've seen teams enable 2,000+ Sigma rules on their first deployment, then drown in 30,000 alerts a day for the next month before disabling everything.

Start with the SigmaHQ "high" severity rules, mapped to your top 25 ATT&CK techniques. That's roughly 60-100 rules. Tune them against your environment for two weeks. Measure false positive rate per rule. Anything firing more than five times a day without a real incident behind it goes back to the drawing board.

Migration Insurance

The bank in the opening anecdote re-migrated from Sentinel to Splunk eighteen months later. The second migration took six weeks because by then their detection logic lived in Sigma in git, not in vendor query language in the SIEM. The recompile-and-deploy pipeline turned what used to be a year-long project into a sprint. That's the strategic value. Detection-as-code is less about elegance and more about not being held hostage by your SIEM vendor.

Read more field notes, explore our services, or get in touch at info@bipi.in. Privacy Policy · Terms.