Detection engineering is where security meets software: you build, test, and maintain the detections a SOC runs on. These questions probe how you think about precision and recall, detection-as-code, ATT&CK coverage, and turning fresh intel into a rule that fires on the real thing and stays quiet on everything else.
Detection engineering treats detections as a software product with a lifecycle: research a threat, hypothesize an observable, build the logic, test it against real and simulated data, ship it through version control, then measure and maintain it over time.
"Writing a SIEM rule" is one step in that; detection engineering is the whole discipline around it - coverage strategy mapped to a threat model, quality gates (precision, documentation, test cases), peer review, and retirement of stale rules. The output is not just alerts, it is a maintained, measurable detection portfolio.
Recall is the share of real attacks the detection actually catches (low recall = false negatives, missed attacks). Precision is the share of alerts that are true positives (low precision = false positives, alert fatigue).
There is no universal answer - it depends on the detection's role. A broad hunting query can tolerate low precision because a human triages it. A detection that auto-isolates a host needs very high precision or it will disrupt the business. The honest answer is that you tune to the cost of each error in that context, and you measure both rather than optimizing one blindly.
Detection as code manages detections the way developers manage software: rules live in version control (Git), changes go through pull requests and peer review, and a CI pipeline validates and tests them before deployment (using formats like Sigma that compile to many backends).
It matters because it brings history, rollback, review, and testing to detections. You can see who changed a rule and why, catch a regression before it hits production, and deploy the same logic across SIEM and EDR from one source of truth. It turns tribal knowledge in a console into a maintainable, auditable codebase.
1. Research: pick a technique (often from ATT&CK or fresh intel), understand how it works and what telemetry it touches.
2. Hypothesize the observable: what specific, durable artifact does this leave? Prefer behavior over brittle IOCs.
3. Build: write the logic against the right log source, scoped to reduce noise.
4. Test: validate true positives by emulating the technique (e.g. Atomic Red Team) and false positives against a window of production data.
5. Document & ship: record the technique, data source, expected FPs, and response steps; deploy via version control.
6. Measure & maintain: track fire rate and true-positive rate; tune or retire.
ATT&CK gives a shared map of adversary techniques. I map existing detections to techniques to build a coverage heatmap, which exposes gaps - tactics or techniques with no detection at all.
Then I prioritize the gaps by threat relevance, not just to color the whole matrix green. Which techniques are used by actors targeting my industry, and which have the telemetry to detect them? A coverage map is a prioritization tool, not a scoreboard - 100% coverage of techniques no one uses against you is wasted effort.
Signature-based detections match known-bad specifics: a hash, a domain, a byte pattern. They are precise and cheap but brittle - the attacker changes one byte and evades them (the bottom of the Pyramid of Pain).
Behavioral detections match what the attacker does regardless of tooling: a service account requesting dozens of Kerberos tickets, PowerShell spawning from Office, a host beaconing on a timer. They are more durable but noisier and harder to write.
Use both: signatures for fast, high-confidence coverage of known threats, and behavioral for resilience against variation and novel tooling. The portfolio should skew toward behavior for the techniques that matter.
Kerberoasting requests service tickets for SPNs and cracks them offline, so the observable is on the domain controller: Event 4769 (service ticket requested). The tells are the encryption type (RC4 / 0x17 when the environment otherwise uses AES) and volume - one account requesting tickets for many distinct SPNs in a short window.
A strong detection combines both: RC4 ticket requests grouped by requesting account, thresholded on distinct SPNs per account per hour, with service accounts and legitimate scanners allowlisted. I would pair it with a honeypot SPN - a decoy service account no one should ever request - which turns any hit into a near-zero-false-positive alert. Then test it by actually roasting in a lab and by replaying a week of production 4769s for false positives.
Two directions. True-positive testing: emulate the technique and confirm the detection fires - purple-team exercises or an atomic testing framework (Atomic Red Team) that safely executes the specific behavior. If I can't make it fire on the real thing, it isn't a detection.
False-positive testing: run the logic against a representative window of production data (say 7-30 days) to see what benign activity it would have alerted on. That tells me the noise floor and what to allowlist before it ever pages a human. I document the expected FPs so the SOC knows what "normal" looks like for this rule.
Structure it as situation, investigation, fix, result. Pick a real (or lab) example: a rule firing dozens of times a day, all benign, that the SOC had started to ignore - the dangerous state, because a real hit would be lost in the noise.
Describe how you investigated the false positives rather than just raising the threshold: grouped the alerts, found the common benign cause (a vuln scanner, a backup job, an admin tool), and scoped the logic to exclude it precisely while keeping the malicious case. Close with the measured result: fire rate dropped, true positives preserved, and analyst trust in the rule restored.
False negatives are harder because you often learn about them after an incident. I treat every missed detection as a lessons-learned input: during the post-incident review, ask "what would have caught this earlier, and why didn't our detection fire?"
Usually it is one of three things: the telemetry wasn't collected (a data gap), the logic was too narrow (over-scoped to one variant), or the behavior wasn't modeled at all. The fix depends on which - onboard the missing log source, broaden the detection to the underlying behavior rather than the specific IOC, or build a new detection. Then I add a test that reproduces the missed case so a regression can't reintroduce the gap.
Layer it by speed. First, the fastest coarse net: block/alert on the known exploit indicators (the public PoC's payloads, URIs, user agents) at the WAF and SIEM - low precision but immediate, and buys time before patching.
Then, the durable detection: understand what successful exploitation actually does on the host - a web process spawning a shell, an unexpected outbound connection, a new file in a web directory - and detect that behavior, which survives payload changes the IOC-based net will miss.
In parallel I coordinate: confirm exposure with the vuln team, prioritize patching, and hunt retroactively over existing logs for signs it was already exploited before the detection existed. Detection, hunting, and remediation run together, not in sequence.
I extract detections at multiple levels of the Pyramid of Pain, not just the atomic IOCs. Atomic: load the specific domains/IPs/hashes into blocking and alerting - fast but the attacker rotates them quickly.
Pattern: if the report describes a domain shape - say long, high-entropy lookalike subdomains on a fresh apex - I write a behavioral detection on that shape (label length, entropy, domain age) so it catches the next domain the atomic list won't have. TTP: capture the delivery and execution behavior (the macro spawning PowerShell, the beacon timing) so the detection survives even a new campaign infrastructure.
Finally I retro-hunt the indicators across historical logs to find anyone already hit, and feed anything I learn back to the intel team.