350 Mailboxes, a Blocked Macro, and a Board Report
A retail client authorises a phishing simulation against 350 employees across four departments. The rules are explicit: a lookalike domain registered for the exercise, one landing page, no credential capture - a "submit" is recorded as a click-through event only - and no malware may execute on employee endpoints. The client's SOC knows the test window but not the lure content. The EDR blocks Meterpreter and blocks macros in Office files carrying Mark-of-the-Web. Last year's campaign, run by another vendor, reported a 41% click rate, and the CISO wants the same Word macro lure so the numbers can be compared year over year. The tester proposes generating a .docm with msfvenom -p windows/x64/meterpreter/reverse_https -f vba and hosting a Metasploit browser exploit for drive-by compromise. The engagement has 10 working days and the report goes to the board.
1. Evaluate the tester's proposed technique against the authorisation and the client's controls.
Read the worked answer
The proposal fails on three independent grounds, and any one of them is sufficient to reject it.
It violates the written rules. "No malware may execute on employee endpoints" is not a preference; it is a scope boundary. A Meterpreter payload delivered by macro or drive-by is malware execution on 350 employee machines, which expands the data-processing exposure to employee devices and any personal data on them.
It will not produce the measurement requested. Macros in internet-sourced Office files are blocked by default on current builds, and the EDR signatures the msfvenom VBA stub. A low click rate would then be indistinguishable from a control-driven block - the CISO would be reading a number that measures the vendor's tooling, not employee susceptibility.
Season-to-season comparison is already invalid. Even with the identical lure, a different control baseline (macro blocking, Mark-of-the-Web, EDR) means last year's 41% and this year's figure are not comparable. That needs to be said plainly in the report rather than papered over.
A workable alternative satisfies the same objective without a payload: a HTML lure that links to a landing page which records only event metadata - recipient token, timestamp, user agent, and whether the form was submitted - and stores no field values. No credentials, no execution, no endpoint compromise, and the click-through metric the client actually asked for. Document the metric definition so next year's run can be genuinely compared.
2. How do you design the campaign and the metrics so the board report is defensible?
Read the worked answer
Attack the denominator first, because most phishing reports are wrong there.
Track per-recipient tokens. Each of the 350 recipients gets a unique identifier in the link. That gives deduplicated unique clicks, distinguishes an employee who clicked five times from five employees who clicked once, and lets you attribute results to departments without reading mailboxes.
Define the funnel precisely. Delivered (excluding bounces, auto-replies, and leavers) → opened → clicked → submitted the form → reported the message to the SOC. Report each stage with its own denominator. A "41% click rate" that used delivered messages as the base and a different one this year is not a trend.
Treat reporting as a success metric. The SOC phish-report button is the outcome the client can actually improve; a department with 30% clicks but 40% reports is in better shape than one with 20% clicks and nobody reporting.
Handle individuals carefully. Results go to the CISO and department heads in aggregate; naming individuals converts a security exercise into a disciplinary instrument and suppresses future reporting. Agree that policy before the first email.
State the limits. A ten-day single-shot campaign with a novel lure measures first-contact susceptibility under a specific control set. Say so in the report, put the control baseline (macro blocking, EDR, filtering) next to the number, and resist trend claims until two runs share a methodology. That is what makes a board-level figure defensible under questioning.
3. Mid-campaign the CISO insists on adding a real payload "to prove impact". How do you respond?
Read the worked answer
This is a change-control problem, not a technical one, and it must not be resolved on the call.
Restate the constraint. The authorisation forbids malware execution on employee endpoints. A verbal instruction from the CISO does not amend a document signed by whoever commissioned the test, and the tester has no way to verify that the CISO can authorise processing of employee device data. Ask for a written change, routed through the engagement manager, naming the hosts, the data involved, and the rollback plan.
Explain the cost. Delivering a real payload to 350 endpoints means EDR alerts across the estate, possible quarantine of production machines, helpdesk load, HR implications for employees whose devices are affected, and a data-protection question if any personal files are touched. That is a materially different engagement with a different risk acceptance.
Offer a proportionate alternative. If impact must be demonstrated, demonstrate it once: a single, named, non-production host, with written approval and the client's IT contact on the call, using a benign beacon that does nothing beyond connecting. The point - "a user clicking this link gives an attacker code execution" - is provable with one machine, not 350.
Document everything. Record the request, the response, and the decision in the engagement log. If the CISO insists after the explanation, escalate rather than comply: an out-of-scope payload delivered to employee laptops is exactly the finding that ends a testing firm.