02:14 Alert - Declaring an Incident Against a Plausible Benign Explanation
Northwind Analytics is a 450-person SaaS company. At 02:14 on Monday the SIEM fires: service account svc-backup authenticated to 12 hosts within four minutes, then 3.2 GB left the network for an IP address in a country where Northwind has no customers, staff, or partners. The on-call analyst is six months into the role and on her first solo shift. The written playbook says "suspected data exfiltration = P1", but it was written for the old on-premises estate; svc-backup now belongs to a cloud backup product deployed three weeks ago. The CISO is unreachable on a flight. The VP of Engineering messages: "That is almost certainly the new backup tool - don't wake anyone up." Nobody has checked the destination IP. SIEM retention is 30 days; the backup product's own logs rotate after seven.
1. Decide whether to declare an incident now, and at what severity. Justify the call against a definition of "incident", and state what you do in the first 30 minutes.
Read the worked answer
Declare - provisionally, at P2 - and start the clock.
Why the definition is satisfied. An incident is an adverse event (or threat of one) that actually or potentially jeopardises confidentiality, integrity, or availability. The word doing the work here is "potentially": 3.2 GB leaving to an address with no business relationship is potential exfiltration whether or not the backup tool is responsible. The analyst does not need proof to declare; she needs a reasonable suspicion that a control may have failed.
Why the downside is asymmetric. A declaration is reversible. Evidence is not. The backup product's logs rotate in seven days, so a 48-hour wait can destroy the single artefact that would settle the question. Declaring costs a phone call; not declaring risks losing both the evidence and the response window.
Why the VP's input cannot be the deciding vote. "Almost certainly the backup tool" is a hypothesis, not a finding, and it comes from someone with an interest in not being woken. It should be written into the case log as a hypothesis to test - not as a closure reason.
Severity P2, not P1. P1 implies confirmed exfiltration with known scope; neither exists. P2 signals suspected, unscoped, plausible business impact, and carries a duty to reclassify upward on confirmation or downward on disproof - recording both transitions.
First 30 minutes: export the backup-product logs and authentication logs for all 12 hosts before anything else; open a case record with a timestamped decision log; page the deputy CISO; call the backup vendor's on-call to ask whether that IP is theirs; pull seven days of prior svc-backup behaviour as a baseline.
2. The VP's benign explanation is plausible. How do you test it without destroying your ability to investigate if it is wrong?
Read the worked answer
Freeze before you look. Preservation is not contingent on the answer. Snapshot the 12 hosts' authentication logs, export the backup product's job logs, and capture netflow for the window. Then investigate from copies.
Test against the vendor's data, not internal recollection. Ask the backup vendor for its documented egress ranges and confirm whether the destination IP sits inside them. Then compare the transfer against the product's own job schedule: a legitimate backup job will have a matching job ID and a duration consistent with 3.2 GB. If the transfer appears in the job log, the benign hypothesis strengthens materially; if it does not, it collapses.
Establish a baseline. Has svc-backup ever contacted an external address before in the seven days of retained history? A service account that has never made an outbound connection and suddenly makes one is a different finding from one that does this nightly.
Examine credential hygiene. Is svc-backup used interactively anywhere? Are its credentials stored in a config file readable by more than the tool's service identity? A compromised backup account is exactly the shape of an attacker who wants your recovery capability.
Use reversible containment as a test. Add an egress allowlist limited to the vendor's ranges. If it is the tool, one job fails - informative and harmless. If it is not, you have cut the channel while still holding the evidence.
Write your disproof criteria before you look - e.g. "benign only if the IP is in the vendor's published range AND a matching job ID exists." Otherwise you will rationalise whatever you find.
3. Six weeks later, what should the post-incident review ask about this specific decision - independent of how it turned out?
Read the worked answer
Separate decision quality from outcome. If it was the backup tool, the review must still ask whether the process would have caught a real exfiltration. If it was an attacker, the review must not simply congratulate the analyst - a correct call made from thin evidence is not a repeatable control.
Questions the review should put on the table:
- Was the escalation decision driven by evidence or by seniority? The VP's view is legitimate context; it should not be the deciding vote in a severity call.
- Why did the playbook's P1 rule reference technology that no longer exists? Who owns playbook currency after an architecture change, and what triggers a review?
- Is a seven-day log retention fit for the decision window this detection demands? That is a capability finding, not an analyst error, and it belongs in the risk register with a named owner.
- What was the elapsed time from alert to evidence preservation, versus alert to decision? Preservation should come first.
- Did the on-call analyst have clear authority to declare without a second signature? If not, the fix is a delegation of authority, not more training.
Rewrite the severity matrix around the decision, not the technology: rank by reversibility of the response and rate of evidence decay, so the rule still reads correctly after the next platform migration.
Close the loop: assign each action an owner and a date, and re-test the specific scenario in the next exercise - otherwise the review produces prose, not capability.