The 92 Percent Coverage Claim: Mapping at the Right Level of Abstraction
A 40-person SOC has just completed its first ATT&CK coverage assessment. The team maintains 118 Splunk detection rules and, after two weeks of mapping, reports 92 percent coverage of the Enterprise matrix. Three problems surface in review. A rule that alerts on powershell.exe running an encoded command is counted twice, once under T1059 Command and Scripting Interpreter and once under T1059.001 PowerShell. A Sysmon Event ID 10 rule for LSASS handle requests was mapped to T1003 OS Credential Dumping, and its author now wants T1003.001 LSASS Memory counted as well. And a vendor report ingested last quarter cites T1064 Scripting, an identifier the mapping spreadsheet still carries as a live row. The CISO presents the 92 percent figure to the board next Thursday; internal audit has asked which ATT&CK version the assessment used. The team has no written mapping standard, six analysts mapped rules independently, and the coverage workbook is the only artifact that exists.
1. Recommend how the PowerShell rule and the LSASS rule should each be mapped, and state the counting rule the team should adopt before Thursday.
Read the worked answer
PowerShell rule - map to T1059.001 only. The rule logic keys on the process name powershell.exe plus an encoded-argument pattern. That is a statement about one specific interpreter, which is exactly what the sub-technique describes. Mapping it to the parent T1059 as well does not add information; it double-counts one detector against two nodes of the same branch of the tree.
LSASS rule - map to T1003.001 only. Sysmon Event ID 10 with TargetImage ending in lsass.exe is a memory-access event against the LSASS process. T1003.001 is the precise description. The parent T1003 is a container for several distinct behaviours (SAM, NTDS, LSA secrets, DCSync); claiming it in addition implies coverage the rule does not have.
Counting rule to adopt:
- Map each detector to the single most specific technique or sub-technique the evidence and logic support.
- Map to the parent only when the rule is genuinely interpreter- or variant-agnostic (for example, a rule matching cmd.exe, bash and powershell.exe alike maps to T1059, not to three sub-techniques).
- When rolling up for a dashboard, aggregate children into the parent; never add parent and child.
- Store a rationale and a confidence field per mapping, plus the ATT&CK version used.
Applied here, the team loses two inflated entries but gains a defensible number.
2. The workbook still lists T1064 Scripting. Explain what happened to that identifier and how the team should handle it and any similar stale rows.
Read the worked answer
T1064 Scripting no longer exists as a live technique. It was deprecated and its content redistributed into sub-techniques of T1059 Command and Scripting Interpreter - the interpreter-specific children such as T1059.001 PowerShell, T1059.003 Windows Command Shell, T1059.005 Visual Basic and T1059.006 Python.
In the STIX representation, deprecated objects are not deleted. They are flagged revoked: true and linked by a revoked-by relationship to the replacement object or objects. Because one retired technique was split across several replacements, the mapping is one-to-many, which is precisely why a naive automated remap fails and a human has to choose.
Handling:
- Delete T1064 from the coverage rows so it cannot inflate the score.
- For every detector previously attached to it, read the rule logic and assign the specific interpreter sub-technique. A rule matching only PowerShell goes to T1059.001; a rule matching any script host stays at T1059.
- Re-run the check across the whole workbook against the pinned ATT&CK version, filtering any object with
revoked: true, and log the old-to-new mapping so historical reports remain explainable. - Add a quarterly refresh step, because deprecations and splits recur and a stale ID silently becomes a phantom coverage claim.
3. Beyond the three errors already found, why is the 92 percent figure structurally misleading? What should the team report instead?
Read the worked answer
The 92 percent figure is fragile for four reasons.
Denominator ambiguity. The Enterprise matrix contains roughly 200 parent techniques and well over 400 sub-techniques. A percentage computed over parents and a percentage computed over sub-techniques are not comparable, and the parent number flatters the team by hiding which specific children are uncovered.
Double counting. Any detector mapped to both a parent and a child, as with T1059 and T1003 here, inflates the numerator. With 118 rules mapped by six analysts without a standard, the true inflation is probably larger than the three cases found.
Coverage is not effectiveness. A mapped rule may never have fired, may sit in a noisy or disabled state, or may depend on telemetry the team does not actually collect. Technique-level ticks say nothing about false positive load or mean time to detect.
Version drift. Without a stated ATT&CK version, the number cannot be reproduced.
Report instead: a Navigator heat map pinned to one named version, counted at sub-technique granularity with parent roll-up displayed separately; an explicit list of top uncovered techniques relevant to the organisation's threat profile; and a three-state model separating telemetry present, detection logic exists and detection validated by test. The board gets fewer decimals and far more truth.