Northwind Pay: a WAF purchase after a nine-day account takeover spree
Northwind Pay runs a B2B payments API: 38 microservices, 2.4 million authenticated calls per day, and a single OAuth 2.0 token endpoint. Last month's incident review found no malware, no injection payloads, and zero WAF blocks. Yet attackers drained 14 merchant accounts in nine days.
The reconstruction: 60,000 login attempts spread across 400 residential IPs against /oauth/token, with a 0.4% success rate. For each compromised account the attacker called POST /orgs/{id}/invites to add a new admin from a lookalike domain. The invite endpoint had no role check, so any authenticated member could create an admin. That admin then changed the payout bank account and triggered a payout.
Engineering proposes buying a bot-management WAF and closing the ticket. Your CISO asks whether that prevents a repeat, and what you would do instead with two engineers and six weeks.
1. Decompose the intrusion into its distinct security failures. For each, name the attack class, state whether a bot-management WAF would have prevented it, and explain why.
Read the worked answer
1. Credential stuffing against /oauth/token. Attack class: automated credential reuse against a valid authentication endpoint. A bot-management WAF raises cost here (device fingerprinting, ASN reputation, proof-of-work) but is evadable. 400 residential IPs are rented proxies; a 0.4% hit rate across 60,000 attempts is exactly the profile residential proxy pools are built for. Prevention belongs in the auth service: per-account and per-subnet throttles with exponential backoff, breached-password screening at set time, MFA on admin-capable roles, and device/IP novelty signals.
2. No authorization check on POST /orgs/{id}/invites. Attack class: broken function-level authorization leading to privilege escalation. Any authenticated member could mint an admin. No perimeter control fixes this: the request is well-formed, authenticated, and semantically valid. Fix: server-side role check against current membership state (not a cached claim), an explicit allowlist of invitable roles, and an audit record.
3. Payout bank change without step-up. Attack class: business-logic abuse / fraudulent transaction. Again a legitimate session on a legitimate endpoint. Fix: re-authentication, a 24-48 hour hold on first payout to a new account, out-of-band confirmation to the previously verified contact, and velocity limits.
4. No detection on the sequence. Attack class: monitoring gap. Invite to an external domain, then role change, then bank change, then payout within minutes is high-signal.
A WAF is a payload filter. Every step here used a valid session and a valid payload.
2. With two engineers and six weeks, produce a prioritized remediation plan. Justify the ordering and state explicitly what you will defer and what risk you are accepting.
Read the worked answer
Order by the cheapest link in the kill chain that stops cash-out.
Weeks 1-2 - invite authorization and audit. Kills the escalation step. Small diff, easy to test, no infrastructure. Rule of thumb: fix the step that converts a compromised low-privilege account into an admin.
Weeks 2-3 - payout change step-up. Re-authentication, 24-48h hold on first payout to a new bank account, out-of-band confirmation to the previously verified contact, velocity limit. This kills the monetary loss even when an account is still taken. Ship behind a feature flag so it can be reverted.
Weeks 3-5 - token endpoint hardening. Per-account, per-IP and per-subnet throttles; breached-password check at set time; mandatory MFA for Owner and Admin roles. This lowers the account-takeover rate rather than eliminating it.
Weeks 5-6 - detection. Correlation rules for the invite to role-change to bank-change sequence, plus an on-call owner for the alert.
Deferred: WAF procurement (12-week sales-plus-tuning cycle, addresses none of steps 2-4), full threat-modeling program, SSO migration, device fingerprinting vendor. Accepted risk, stated in writing: accounts will still be compromised; the loss path is what we close first. Review date in 90 days.
Test each item with one question: if the attacker already holds valid credentials, does this control still work? Items 1 and 2 pass it; rate limiting partly passes; the WAF fails.
3. What telemetry would have cut detection to under an hour, and what would you instrument first? Note any logging pitfalls.
Read the worked answer
You need sequence detection, not single-event detection. The escalation happened inside the application and never touched the WAF, so gateway logs alone are blind.
Instrument first:
- Authentication events with account id, IP, ASN, user agent, geo, outcome, timestamp - most organizations log failures only, which is why the 0.4% success rate was invisible.
- Authorization decisions at the service, allowing and denying, with actor, tenant, target object and the role held at decision time. This is the single highest-value log because it makes broken function-level authorization visible.
- Invite events: inviter role, invitee email domain, whether the domain is new to the tenant.
- Bank-detail changes with old and new account fingerprints and the approver.
- Payout creation with amount and destination fingerprint.
The rule that would have fired: same account id, auth success from a new ASN, then an invite to an external domain, then a role change or bank change within 60 minutes. That is a page, not a ticket.
Pitfalls: never log tokens, passwords, full bank numbers or full SSNs - log a keyed hash instead. Logs must be append-only and shipped off-host, otherwise the attacker who gets code execution edits them. Alert routing needs a named owner; an alert nobody owns is not detection. Also verify clock sync across services, or your correlation window is meaningless.