Handing a 1.3%-Precision Hunt Query to Detection Engineering
A two-week hunt for lateral movement produced three confirmed true positives from 41 candidates. The query joins Windows 4624 logon type 3 to service installation (7045) on the same destination host within a five-minute window and filters on named pipes, but it hard-codes 12 hostnames, uses that fixed join window, and takes 40 minutes to run in the SIEM. Backtested over 90 days it fires 900 times with 12 true positives - roughly 1.3% precision. The detection engineering team has a two-week sprint and one engineer at 60% allocation. The current ATT&CK coverage layer scores 78% of enterprise techniques as 'covered', a figure the CISO quotes monthly, and was built by the previous detection lead from a vendor content pack. The hunt lead wants the logic handed off unchanged, with the 12 hard-coded hosts kept 'for safety'.
1. Is the query production-ready? Specify what the handoff package must contain and what you would change before the engineer starts.
Read the worked answer
It is not production-ready. Hard-coded hosts are the opposite of safety: they encode the scope of one hunt into a permanent blind spot and guarantee the detection fails on the next incident. The fixed five-minute window will miss service creation that follows logon after a longer dwell, and 40 minutes of runtime will not meet any alerting SLA.
Changes before handoff: replace the host list with a dynamic asset-group lookup or drop it entirely and rely on rarity; widen and invert the join so that the network logon precedes service creation, with a window of 30–60 minutes; aggregate per source-destination pair and first-seen service name rather than per event; and pre-filter known-good service accounts and patching windows.
The handoff package must carry: detection logic in platform-neutral pseudo-code or Sigma plus the platform query; the ATT&CK mapping at sub-technique level (T1543.003, and T1021.002 if the named-pipe condition holds); required data sources and fields with enabling instructions (7045 needs the System log; 4624 type 3 needs logon auditing); known false positives (SCCM, backup agents, monitoring, vulnerability scanners, Windows Update); tunable parameters as named variables; positive and negative test cases with expected results; severity, escalation path and response playbook; owner, version and review date; and the hunt's evidence - the three true positives and the dispositioned false positives.
2. How do you improve precision without destroying recall, and how do you prove the improvement before full deployment?
Read the worked answer
Ten alerts a day at 1.3% precision is survivable in volume but corrosive to analyst trust, so the target should be precision above roughly 30% with recall preserved - and recall matters more here, because a rare persistence behaviour that is missed has far higher cost than a noisy alert.
Levers: rarity scoring on the service binary path and service name (first appearance in the estate in 30 days); excluding hosts and windows covered by a change record or patching calendar; requiring the network logon to precede service creation rather than merely co-occur; enriching with the process image and image path so the analyst triages on content rather than hostname; and splitting the rule so that named-pipe activity is a separate higher-severity analytic instead of a filter that silently drops non-pipe cases.
Prove it by shadow-mode running the tuned logic in parallel for two weeks against the same 90-day backtest, then comparing counts: the three known true positives must still fire, the 12 known true positives must still fire, and the 900 alerts must drop substantially. Report precision, recall against the known-true set, and daily volume. Only then enable alerting. Document the backtest as the detection's validation record so the coverage claim later rests on evidence.
3. The CISO's 78% coverage figure is quoted monthly. Critique it and describe how you would restate coverage after this detection ships.
Read the worked answer
78% of 'enterprise techniques' is almost certainly a count of techniques with at least one mapped analytic from a content pack. It counts parent and sub-technique cells without de-duplication, does not distinguish enabled from installed, tuned from untested, or validated from aspirational, and it includes platforms you do not run. It is an inventory metric wearing a coverage label.
Restate it in four steps. First, fix the denominator: filter to techniques applicable to your platforms (Windows-heavy, limited macOS, no Linux servers) so out-of-scope techniques are excluded rather than counted as gaps. Second, count at sub-technique level. Third, classify each cell into validated (positive test and negative test passed, running in production, data source monitored), partial (analytic exists but a data component or scope is missing), blind (no data source or no analytic), or not applicable. Fourth, publish the distribution and the number of blind cells that matter most, not a single percentage.
The honest marginal effect of this detection is one more validated sub-technique, T1543.003, possibly with a partial for T1021.002. Say that. And name the real constraint: techniques that depend on TLS inspection, DNS logging, memory telemetry or cloud data events will stay blind regardless of how many rules are written, so coverage growth is a data-source programme, not a content programme.