Evaluating a Premium Feed Before Renewal
Meridian Pay is a payments processor with 3,200 employees and a five-person CTI team. It currently buys commercial feed A for $180,000 per year, commercial feed B for $95,000, receives sector ISAC reports, runs an OSINT crawler, and monitors two dark-web forums. The pipeline ingests 42,000 indicators per day; after deduplication only 12,000 are unique, and 78% of A's and B's indicators overlap. Feed A's vendor claims 99.9% precision, but last quarter the SOC found that 34% of A's IP indicators belonged to CDNs or sinkholes and blocked 19 legitimate partner ranges. Feed B provides high-severity actor reports but averages a six-day lag. A's account manager offers a 20% discount for a three-year renewal signed this month. The SOC wants fewer blocks; the fraud team wants faster actor context; procurement wants a decision in two weeks.
1. Evaluate feeds A and B using source reliability and credibility. What evidence would you require before any renewal?
Read the worked answer
I would separate source reliability from information credibility. Feed A may be a generally reliable commercial source, but the 34% CDN/sinkhole rate makes many IP items low credibility for blocking. Feed B may have good actor reporting but a six-day lag lowers its value for time-sensitive fraud defense. The 78% overlap is not automatically bad; overlap can corroborate, but it reduces marginal value and means we are paying twice for the same coverage.
Before renewal I would require a 30-day side-by-side trial measured against our own telemetry: sample at least 500 indicators per feed, classify true positive, benign, CDN, sinkhole, and unknown; measure precision by indicator type, not a single vendor number. I would ask what '99.9% precision' means, how it was measured, and whether it includes IP-only indicators. I would also require coverage mapping to our PIRs, latency distributions, provenance, licensing for automated blocking, and false-positive cost. I would score each feed using Admiralty-style reliability and credibility, document the rationale, and compare cost per unique valid indicator. Renewal without this evidence would be procurement by discount.
2. The SOC wants fewer blocks and fraud wants faster actor context. Recommend a source and processing strategy, including negotiation terms.
Read the worked answer
I would stop treating both feeds as equivalent blocklist suppliers. Feed A would be demoted to enrichment and alerting unless the vendor removes or tags IP-only CDN and sinkhole data and improves precision. I would not sign a three-year renewal on the current evidence; I would negotiate a 90-day performance trial with defined precision thresholds, removal of shared-infrastructure IPs, confidence labels, expiry, and a termination right. Feed B would be retained for actor context if the lag can be reduced to 24 to 48 hours or if the reports still add unique fraud-relevant detail; otherwise I would seek a cheaper tactical alternative.
The processing strategy should be tiered. High-confidence, unique indicators get automated blocking with expiration. Medium-confidence indicators generate alerts only. Actor reports feed the fraud team's casework and the SOC's hunt hypotheses, not automatic blocks. I would use the ISAC for sector corroboration and the OSINT crawler for early warning, with source scoring. Negotiation should include machine-readable delivery, false-positive credits, and a clause allowing us to publish anonymized metrics. The decision should be driven by decision utility, not by the size of the feed.
3. How should the team resolve conflicting source reports about the same actor and document the decision?
Read the worked answer
I would not average the reports or choose the more confident-sounding vendor. I would compare source access, collection method, corroboration, timeline, and specificity. If one source has direct incident response visibility and the other is inferring from infrastructure, that affects credibility even if the second is more recent. I would use a structured technique such as analysis of competing hypotheses: list the plausible actor identities or affiliations, identify evidence that distinguishes them, and record which evidence is consistent or inconsistent with each hypothesis.
The documentation should state the competing claims, source reliability, information credibility, key assumptions, and the confidence level. For example: 'Moderate confidence that the campaign is affiliated with actor X; shared encryptor code is weak because it is common in ransomware-as-a-service; the unique payment portal is stronger but could be reused by an affiliate.' I would define what new evidence would change the judgment, such as unique infrastructure, cryptocurrency clustering, or victimology. I would also date the assessment and set a review trigger. This gives consumers a defensible judgment rather than a false single-source certainty.