Threat Data Collection and Sources
This domain tests whether you can plan collection against a requirement, grade a source's reliability separately from its content, and choose a source class that is both appropriate and lawful for the question asked.
Source classes
- OSINT: public web, news, company registries, certificate transparency, WHOIS and passive DNS.
- Commercial feeds: vendor telemetry and curated indicator subscriptions.
- Community and sector sharing: ISACs, CERTs, MISP communities, vendor research blogs.
- Own technical collection: EDR, IDS, honeypots, internal sandboxes - usually the highest-value and most underused source.
- HUMINT: human sources and vetted researcher contacts, subject to legal and ethical limits.
- Dark web and closed-forum monitoring: normally passive observation only.
Start with the requirement
Collection without a PIR produces unprioritised noise. A collection plan states the question, which sources could answer it, the method, the refresh cadence, and the confidence threshold at which collection stops.
Grading: source versus information
Admiralty grades source reliability A–F and information credibility 1–6, independently. A previously accurate researcher (B) making an unsupported claim yields B3 or B4. A usually unreliable forum (D) whose specifics are corroborated by a sandbox can reach D2. A famous vendor logo does not upgrade a specific claim.
Provenance and validity
Every record needs collector, collection time, original publication time, and observation window. Indicators decay: a C2 domain may be sinkholed or reassigned within weeks. Timestamped passive DNS history is often more durable than the original IP.
Deduplication and enrichment
Feeds overlap heavily. Normalise before storage: defang URLs, canonicalise domains, expand short links, resolve ASN and registration date, hash samples. Store enrichment separately so it can be re-run without corrupting the original observation.
Worked example
A PIR asks which infrastructure targets your sector's VPN appliances. Collection: own IDS logs, one commercial feed, two community MISP feeds. A forum post claims a new scanner - source D, claim uncorroborated, so D4, monitor only. A commercial feed record matches your own IDS hits and a sandbox capture, so A1. Processing collapses 3,000 feed records to 410 unique domains and enrichment retains 12 that touch your sector. Volume fell 87% before any analyst time was spent, which is exactly what processing is for.
Traps
Treating brand recognition as a grade. Collecting everything because storage is cheap. Storing indicators without timestamps, then alerting on expired infrastructure. Actively scanning adversary infrastructure without authorisation - method legality belongs in the plan, not in hindsight.