Your SIEM Says It Covers MITRE ATT&CK. It Probably Doesn't.

Cybersecurity

Most security teams claim 60-70% ATT&CK coverage on their dashboards. When we audit those rules against actual sub-techniques, the real number is closer to 18%. Here's how the gap forms and what to do about it.

By Arjun Raghavan, Security & Systems Lead, BIPI · January 4, 2024 · 7 min read

Every CISO deck I read in 2023 had the same slide: a heat map of MITRE ATT&CK techniques shaded green, yellow, and red. The green was always overstated. When we sit down with the underlying Splunk or Sentinel content and walk the rules technique by technique, the green collapses.

The cause is rarely lazy work. It's a measurement problem. Teams count rules, not detections. They count techniques, not sub-techniques. They count what fires in a lab, not what fires on production telemetry. By the time the heat map reaches the board, it has lost any relationship to reality.

The Coverage Inflation Pattern

Here's the typical inflation pipeline. A team writes a rule for T1059 Command and Scripting Interpreter. The rule looks for powershell.exe with -enc. Someone marks T1059 as covered. The dashboard now claims coverage for all ten sub-techniques (PowerShell, AppleScript, Python, JavaScript, Visual Basic, Unix Shell, Network Device CLI, Cloud API, AutoHotKey, Lua) even though the rule only matches one.

Multiply that pattern across 200 techniques and the gap compounds. We mapped one client's 340 Sentinel rules against sub-techniques in October 2023. They claimed 71% technique coverage. Sub-technique coverage was 19%. Half of the rules covered the same five techniques.

What Real Measurement Looks Like

Stop scoring at the technique level. Score at the sub-technique level, and weight by data source availability. A rule that requires Sysmon Event ID 1 doesn't cover anything on hosts where Sysmon isn't installed. Most teams have Sysmon on 40-60% of endpoints, never 100%.

• Map each rule to specific sub-techniques, not parent techniques
• Tag each rule with required data source (Sysmon EID 1, EDR process events, Windows Event ID 4688, etc.)
• Multiply coverage by data source rollout percentage
• Track false positive rate per rule, a noisy rule isn't a working rule

The Prioritization Question

Not all sub-techniques deserve equal investment. Pull the last twelve months of incident reports from your IR team and from Mandiant's M-Trends. Cross-reference with the Red Canary Threat Detection Report. The top 25 techniques cover roughly 80% of observed activity. T1059.001 (PowerShell), T1078 (Valid Accounts), T1486 (Data Encrypted for Impact), T1003 (OS Credential Dumping), and T1021.001 (RDP) keep showing up in everyone's data.

Build your sub-technique coverage matrix around those first. A team with 90% coverage on the top 25 techniques is in better shape than a team with 60% coverage spread thin across 200.

Detection-as-Code Closes the Loop

The teams with the most reliable coverage numbers run their detections through CI. Sigma rules in git, pysigma to compile per platform, unit tests with sample logs, atomic tests on a schedule. When a rule gets disabled or modified, the coverage matrix updates automatically. No more quarterly scramble to figure out what's actually running.

Tools worth evaluating: Splunk's Detection Studio, Anvilogic, Panther, or a home-grown pipeline using Sigma + GitHub Actions. The vendor matters less than the workflow. The point is making coverage measurable instead of asserted.

What to Tell the Board

Stop showing the heat map until the numbers underneath are honest. Replace it with three metrics: sub-technique coverage on the top 25 techniques, mean time to detect for the last twenty incidents, and percentage of rules validated by atomic tests in the last 90 days. Those three numbers are harder to inflate and tell a more useful story about whether your detection program actually works.

Read more field notes, explore our services, or get in touch at info@bipi.in. Privacy Policy · Terms.