Detection Engineering, Tuning, and False Positive Reduction
Detection Engineering, Tuning, and False Positive Reduction tests whether you can treat detections as versioned, testable software and reduce noise without blinding yourself. Items reward disciplined tuning levers over simply raising thresholds.
Core rules
- Detection as code. Store rules in version control with metadata: author, purpose, required data sources, ATT&CK technique, severity, known false positives, review date. Sigma gives vendor-neutral logic that converts to Splunk, Elastic, Sentinel and others; peer review and CI regression tests catch breakage when schemas change.
- Validate before shipping. Test against known-bad samples (Atomic Red Team, CALDERA) and known-good noise from your own environment. A rule that has never fired is untested, not clean.
- Tuning levers, in order of preference: tighten the logic (add a second condition, narrow the time window, require a parent process), then scope by environment (asset group, business unit), then exclude by precise attribute (signed binary hash, sanctioned admin tool, service account with a documented purpose), and only last raise the threshold or suppress.
- Precision versus recall. A high-recall rule floods analysts and trains them to ignore alerts; a high-precision rule misses variants. Run noisy high-recall rules as hunting queries or low-severity risk contributors, and keep blocking or paging to precision rules.
- Metrics that matter: true-positive rate, false-positive rate, signal-to-noise ratio, mean time to triage, and validated technique coverage. Alert volume alone is a vanity metric.
Worked example
A brute-force rule fires about 400 times a day. Reasoning: inspect the top contributors before touching the logic. If 350 come from a vulnerability scanner with a stable service account, exclude by that account plus source range plus destination scope, not by disabling the rule. If 40 come from a mobile mail client retrying after password rotation, narrow the window and require failures against multiple accounts. Remaining genuine hits stay paging. Result: six actionable alerts a day with documented exclusions the next engineer can audit.
Traps that cost marks
Whitelisting by username or hostname, which an attacker then inherits. Disabling a noisy rule instead of decomposing it. Tuning until the rule never fires, then claiming coverage. Shipping rules with no test data, no owner and no review date, so nobody dares change them.