The 98% Coverage Feed and a Three-Person Team
A 1,200-bed hospital group runs a three-person cyber threat intelligence (CTI) team supporting a 14,000-endpoint estate. The CISO is evaluating a commercial feed priced at USD 120,000 per year. The vendor slide claims '98% detection coverage' and '2.4 million indicators updated daily'. Last week, EDR flagged a credential-dumping utility on a billing server; the binary's SHA-256 had appeared on a public malware repository three days earlier, but no internal process consumed that source. The CISO asks whether the feed will 'solve' the hospital's ransomware problem before the next board meeting. The team currently has no documented intelligence requirements, no indicator ageing policy, and no way to measure whether a feed changes a decision. The hospital's most important asset is the electronic health record (EHR) system, and its risk register lists ransomware as the top threat. The vendor is offering a two-week pilot with a 30-day termination clause.
1. Explain how you would evaluate the vendor's '98% detection coverage' and '2.4 million indicators' claims, and why neither number proves intelligence value.
Read the worked answer
Decompose both claims before accepting them. '98% detection coverage' has no denominator: 98% of what - malware families, MITRE ATT&CK techniques, retrospective samples, or the vendor's own corpus? Ask for the test corpus, date range, false-positive rate at the proposed operating point, and performance against the threats that matter to the EHR. '2.4 million indicators' measures data volume, not intelligence. Most indicators are short-lived IPs, domains, and hashes; without context (actor, campaign, infrastructure role, first-seen, confidence, expiry) they generate alerts rather than decisions. Volume also raises SOC triage load and the risk of blocking legitimate clinical traffic.
The hospital's real gap is visible in the incident: nobody consumed the public hash three days before EDR fired. A feed that is not integrated into a requirement, a triage playbook, and an ageing policy will not close that gap. Evaluate the pilot with a retrospective: take 30 known incidents from the last year, replay the feed, and measure lead time, precision, and analyst hours per true positive. The useful threshold is whether the feed changes a containment or blocking decision earlier than existing sources. If it only adds volume to the queue, it is data, not intelligence.
2. Draft one priority intelligence requirement (PIR) for ransomware initial access against the EHR and explain how it would drive collection and production.
Read the worked answer
PIR: 'Which initial-access techniques, infrastructure, and actor groupings are most likely to target our internet-facing EHR portals and identity providers in the next six months, and what precursors can we monitor weekly?' This supports the CISO's funding decision and the SOC's blocking priorities; owner: CTI lead; review: monthly. It forces collection from the email gateway, EDR, IIS/VPN logs, the sector ISAC, and vendor reporting - not just indicator feeds. Production becomes a one-page weekly brief that names observed techniques (for example, valid-account use, phishing kits, edge-device exploitation), maps them to ATT&CK, and states confidence and gaps. It also defines a measurable service level: 90% of PIR briefs delivered by Tuesday 10:00 and 80% of cited precursors covered by a detection or hunt.
The PIR is not a wish list. Each answer must end with 'so what' - a recommended detection, control, or collection task. If the proposed feed cannot answer any part of this PIR, its value is marginal regardless of indicator count. The requirement also gives the team a defensible reason to decline out-of-scope collection requests that consume the three analysts' time.
3. Recommend what the team should do in the first 90 days regardless of whether the feed is purchased.
Read the worked answer
Run the work in parallel with the pilot. Weeks 1-2: interview the CISO, CIO, and SOC lead; write three PIRs and a source register with reliability grades and owners. Weeks 3-4: inventory existing sources (EDR, email gateway, ISAC, public sandbox, malware repository) and measure current lead time on five past incidents. Weeks 5-8: define indicator lifecycle - first-seen, confidence, expiry, block-versus-alert, and an exception process for clinical systems - plus a false-positive review. Weeks 9-12: run one retrospective test of the vendor and one free/open source; produce a decision memo with cost per actionable detection and analyst hours per true positive. Establish a weekly intelligence-to-SOC sync and a feedback loop from triage back to collection.
Do not let the purchase precede the requirements. Buying first tends to institutionalise a feed-shaped process: the team starts with what the vendor supplies rather than what the hospital needs. If the vendor refuses a retrospective test or cannot state indicator ageing, treat that as a negative signal. The 90-day output is a repeatable process and a measured pilot, not a signed contract.