Privileged Insider Exfiltration: Three SMEs, Three Different Questions
At a payments processor, the risk team is estimating the frequency of privileged-insider data exfiltration over the next 12 months. Three subject matter experts give 90% confidence intervals. SME A: 0.05 to 0.6 events, most likely 0.2. SME B: 2 to 12 events, most likely 5. SME C: 0.01 to 0.03 events, most likely 0.02. The junior analyst averages the most-likely values to 1.74 events per year and fits a PERT distribution using 0.01 as the minimum and 12 as the maximum. Separately, the SIEM shows 40 privilege-escalation attempts last year, 9 of which succeeded in granting admin rights; only one of those ended with data leaving the environment. SME B is the outgoing head of infrastructure, who says he has seen it happen constantly. SME C is a compliance manager who cites the absence of incidents in the incident register. The CISO needs the estimate by Friday and would prefer a single number.
1. Diagnose the analyst's method: averaging the most-likely values and fitting a PERT to 0.01 as the minimum and 12 as the maximum.
Read the worked answer
Both steps are invalid, for different reasons.
Averaging most-likely values assumes the three experts answered the same question on the same scale. They did not. SME B, the infrastructure lead, is almost certainly counting attempts or events he has observed in logs and incidents across the estate. SME C, the compliance manager, is counting what reached the incident register, which is a detected and formally recorded subset. SME A is estimating something narrower again. The mean of three different quantities is not an estimate of any of them; it is 1.74 events of nothing in particular.
The PERT fit treats 0.01 and 12 as absolute bounds. They are not bounds, they are one person's 5th percentile and another's 95th percentile, so the distribution is being stretched beyond what any expert believes and given a shape none of them endorsed. Two orders of magnitude of spread is not a distribution; it is evidence that the question was never defined.
A related error: a 90% confidence interval is a 5th to 95th percentile range, not a minimum and maximum. Treating the endpoints as hard bounds compounds the overstatement of the range.
The correct first step is to write one operational definition of the loss event and put it in front of all three experts.
2. How should the SIEM figures (40 escalation attempts, 9 successes, 1 exfiltration) be used? State what part of the FAIR ontology each informs and show the arithmetic.
Read the worked answer
The figures are only usable once the loss event is defined explicitly, and the two consistent mappings are:
- Threat event = an insider attempts privilege escalation. TEF is about 40 per year. Vulnerability to the loss event is 1/40 = 0.025. The 9/40 = 0.225 success rate is an intermediate indicator of threat capability versus resistance strength, useful as supporting evidence, not as the final ratio.
- Threat event = a successful admin escalation. TEF is 9 per year and vulnerability is 1/9 = 0.111.
Both are internally coherent. What is not coherent is using 40 as TEF and 9/40 as vulnerability, or using 9 as TEF and 1/40 as vulnerability; those mix denominators and understate or overstate the result.
Two cautions on the data. First, one exfiltration event is a very small sample, so it anchors a range rather than fixing a point. Second, the SIEM only sees what it detects and logs. One recorded exfiltration is a lower bound on the true count; absence from the register is evidence about detection and reporting at least as much as about frequency. That is exactly the gap between SME B's view and SME C's.
3. How should the team reconcile the disagreement between SMEs and deliver something defensible by Friday? Include what to do if the CISO insists on a single number.
Read the worked answer
Reconcile by decomposition, not by vote or averaging.
Ask all three experts the same narrow questions derived from the ontology. How many people hold privileged access, including contractors? How often does one of them have both a motive and an opportunity in a year? Given motive and opportunity, how often does data actually leave? TEF comes from the population times opportunities times probability of action; vulnerability comes from the ratio evidence and threat capability versus resistance strength. Narrow questions produce comparable answers; a question about 'exfiltration events per year' produces three different questions in disguise.
Where disagreement remains after decomposition, keep it as range width. A genuinely uncertain estimate should have a wide interval; that is information, not a defect to be averaged away. Do not split a 1,000-fold disagreement down the middle.
Friday deliverable: 5th, 50th and 95th percentiles for LEF and annualised loss exposure, the data and reasoning behind each percentile, and a short statement of what evidence would narrow the range. If the CISO insists on one number, give the mean, show the interval immediately beside it, and label it a summary of a distribution rather than the risk. Presenting a single figure without the interval is what invites the number to be treated as fact.