Threat Data Collection and Source Management
Collection and Source Management tests how you turn requirements into a collection plan, select and vet sources, record provenance, and manage gaps. Expect source categories, reliability and credibility grading, legal boundaries, deconfliction, and feedback into direction.
Core rules
- A collection plan maps each EEI to a source, method, frequency, owner, handling caveat, and validation step. If an EEI has no source, that is a documented gap, not a reason to lower the requirement.
- Source categories include OSINT (public web, social media, registries, leak sites), commercial feeds, closed or community sharing (ISACs, ISAOs, vendor sharing), internal telemetry (EDR, DNS, proxy, honeypots), liaison, and dark-web or underground sources. Each has different latency, bias, cost, and legal profile.
- Grade source reliability and information credibility separately. The Admiralty scale uses A-F for source reliability and 1-6 for information credibility; a usually reliable source can still pass unverified information.
- Preserve provenance: raw artifacts, timestamps in UTC, URLs, hashes, collection method, and handling restrictions. This supports validation, deconfliction, and later legal review.
- Vet sources for legitimate access, terms of service, privacy impact, and persistence. Avoid single-source dependence and duplicate reporting from the same upstream vendor.
- Run gap analysis regularly: unanswered EEIs, inaccessible sources, stale coverage, and collection overlap. Feed the gaps back to direction.
Worked example
For a PIR on initial-access brokers targeting insurers, the collection plan might use: dark-web marketplace monitoring for ads mentioning insurance (high relevance, medium reliability, legal review required); ISAC member reports (high credibility, low latency); internal VPN and edge-device logs for exploitation attempts (high fidelity, internal handling); and a commercial feed for broker infrastructure (medium cost, requires validation). Each task gets an owner and cadence. The team flags a gap: no visibility into closed Russian-language forums. That gap returns to requirements and may justify a vetted vendor or a changed collection method.
Traps that cost marks
- Collecting because data is available rather than because it answers an EEI.
- Treating a high-reliability source as automatically correct.
- Ignoring legal, contractual, or terms-of-service constraints during collection.
- Losing provenance, which makes validation and sharing impossible.
- Failing to deconflict duplicate reporting from the same upstream source.