02:07 Saturday: Domain Admin on Three Servers and 1.2 GB Leaving
Saturday 02:07. A tier-1 analyst on the overnight shift sees a correlated alert: domain admin account da-ops2 authenticated to three servers (APP-03, SQL-02, FS-01) within four minutes, and none of those servers has a record of da-ops2 in the previous 90 days. EDR on SQL-02 shows net.exe group /domain, nltest /dclist, and a PowerShell process writing a 300 MB archive to C:\ProgramData. At 02:21 the firewall logs 1.2 GB outbound from SQL-02 to a storage endpoint not on any allowlist; the proxy classifies it as file storage. FS-01 hosts the ERP file share finance uses from 06:00 Monday. SQL-02 runs the order-management database with a 30-minute RTO. The on-call IR lead has not answered phone or chat in 25 minutes. Policy says P1 covers domain admin compromise or confirmed exfiltration and requires escalation within 15 minutes; the analyst holds EDR network containment rights but cannot disable domain accounts without the IR lead or IT manager, who is on a plane.
1. What severity is this, and what exactly do you do in the next twenty minutes?
Read the worked answer
This is P1 on both policy triggers - a domain admin account operating on hosts it has never touched, plus 1.2 GB leaving to an unapproved storage endpoint after staging and domain reconnaissance. There is no reading of the evidence that makes this a P2.
Next twenty minutes, in order:
- Record the escalation failure. Log every attempt to reach the IR lead with timestamps. Then escalate up the alternate chain - security manager, CISO on-call, service desk manager - and page all of them simultaneously rather than sequentially. A 25-minute silence from the primary is itself an incident finding.
- Check for a scheduled test. Thirty seconds, not five minutes: is there a red-team or pentest window, or a change record for da-ops2? If none, proceed. The cost of checking is trivial; the cost of containing a red team is embarrassment, not damage.
- Preserve volatile state on SQL-02 - running processes, netstat, logged-on sessions, the C:\ProgramData archive path and size - then apply EDR network containment. This is within the analyst's explicit authority, is reversible, and does not destroy evidence. Do not power off.
- Push a firewall/proxy block on the storage endpoint. Fast, low-risk, and stops the transfer even if containment on SQL-02 is delayed.
- Open the bridge: database owner, ERP/finance application owner, and the alternate on-call manager. Tell them containment will interrupt SQL-02's network access.
The judgement call: the analyst must not wait for permission to do the reversible, explicitly delegated action (containment). But disabling a domain admin account is outside their authority, and doing it unilaterally can lock out legitimate overnight operations - pursue the alternate approver in parallel while containment buys time. Do not let the authority gap become a reason to do nothing.
2. Work through the containment trade-offs for SQL-02, FS-01, APP-03, and the da-ops2 account.
Read the worked answer
SQL-02 - contain now, without hesitation. The 30-minute RTO is irrelevant against an in-progress exfiltration: network containment leaves the database running for local processes and only cuts connectivity. If that breaks a dependent application, the correct comparison is minutes of application downtime against an unbounded data loss, and the data owner should be told that explicitly rather than asked to choose.
FS-01 - do not isolate yet, but treat as compromised. da-ops2 authenticated to it and it holds the ERP share finance needs at 06:00 Monday. There is no execution evidence on FS-01. Instead: enumerate what da-ops2 did - scheduled tasks created, services installed, SMB sessions, local group changes - check 5140/5145 for access to finance shares, and raise monitoring. Pre-stage the isolation decision so it can be executed in minutes if the attacker returns. Isolating a file server on speculation at 02:00 risks a business outage that the evidence does not yet justify.
APP-03 - same posture as FS-01. Authenticated-to is not the same as compromised. Hunt for service creation and new processes under da-ops2, and check for credential use.
da-ops2 - high value, needs authority. Assume the credential is stolen rather than misused. Recommended actions: reset the password twice, revoke Kerberos tickets, and disable - but only with the alternate approver, and only after enumerating what scheduled jobs run under it. If the IR lead's account itself is da-ops2, do not use that channel for coordination.
Assume domain compromise until excluded. A domain admin credential on a database server, seven minutes of reconnaissance, and staging before egress is the shape of an operator heading for the domain. Hunt for DCSync (4662 with replication rights), krbtgt password age, new privileged accounts, and Golden Ticket indicators. A double krbtgt reset may be required and must be raised to the CISO now, because it is disruptive and cannot be improvised at 04:00.
3. How do you document these decisions, and what policy or process changes should come out of the incident?
Read the worked answer
Documentation during the incident
- A running timeline with UTC timestamps: alert, evidence reviewed, each escalation attempt and its outcome, each containment action with the authority relied on, and the expected business impact stated before execution.
- A decision log capturing alternatives considered and rejected - why SQL-02 was isolated, why FS-01 was not, why da-ops2 was left enabled pending approval. Responders are judged on the reasonableness of decisions with the information available, and that is only defensible if it is written down at the time.
- Evidence handling notes: who captured memory, where the C:\ProgramData archive was preserved, chain of custody, and any actions that altered state.
Policy and process changes
- Named alternate with an automatic trigger. If the primary IR lead does not respond within 10 minutes, page the backup automatically. Twenty-five minutes of silence should not be possible on a P1.
- Pre-authorised emergency actions. Define, in advance, that a tier-1 or tier-2 analyst may contain a host and disable a privileged account on evidence of active exfiltration, with notification after the fact. The current policy delegates containment but not account disable, which is an arbitrary split - the account is usually the faster lever.
- Business-impact pre-agreement. Work with the ERP and database owners now on what containment of FS-01 and SQL-02 does to them, so that decision is not first made at 02:00 on a Saturday.
- Contact routing under compromise. Maintain out-of-band contact paths for the IR chain, and forbid coordination on channels an involved privileged account can read.
- Exercise it. A tabletop covering exactly this scenario - DA credential, unreachable lead, business-critical host - is the cheapest way to find the next gap before an attacker does.