3,412 Hostnames, a Wildcard, and Internal Names in Certificate Transparency
Acme Corp hires you for a reconnaissance-only phase before a two-week application test. Starting from the single root domain acme.example, you run amass, subfinder and a certificate-transparency pull, producing 3,412 unique hostnames. You notice that every *.dev.acme.example name resolves to 10.20.30.40 regardless of label - including thisshouldnotexist.dev.acme.example. Twelve hosts return a 200 with distinct content, but roughly 300 return the same 2,048-byte page titled "Welcome to nginx". Among the names that appear only in CT logs are gitlab-internal.acme.example, vpn.acme.example and jenkins.acme.example; the last two resolve to RFC1918 addresses. A robots.txt on www.acme.example disallows /internal/ and /beta/. The client's brief says only: "we care about internet-facing stuff."
1. How do you reduce 3,412 hostnames to a defensible shortlist without touching every one of them with a full scan?
Read the worked answer
First prove the wildcard so you can subtract it. Query two random labels (one you have already seen) and confirm both resolve to 10.20.30.40 and return the same body - that establishes a catch-all DNS record and, likely, a default vhost.
Then fingerprint rather than scan. For each candidate, fetch / and /robots.txt and hash a tuple of (status code, Content-Length, page title, favicon hash, Server header, TLS certificate SHA-1). Group hosts by hash. The ~300 identical 2,048-byte "Welcome to nginx" pages collapse into one cluster and are discarded as the wildcard default - unless a Host header against 10.20.30.40 produces a different fingerprint, which is how you find real apps hidden behind a catch-all.
Treat 401/403 responses and hosts presenting a different certificate as high-value, not as failures: they are real applications behind authentication. The result should be roughly a dozen distinct web applications plus a handful of auth-walled hosts, each with a fingerprint and a certificate SAN list. That is the deliverable of a recon phase - an inventory someone can act on, not a 3,412-line scan dump.
2. `vpn.acme.example` and `jenkins.acme.example` resolve to RFC1918 addresses but appear in public certificate transparency logs. What does that mean and what do you do?
Read the worked answer
It means two things happened: a certificate was issued for an internal-only name (CT logs are public and append-only, so the name is now permanently discoverable), and either the public zone still carries A records pointing at private space, or split-horizon DNS is leaking. You cannot route to 10.x from the internet, so there is nothing to scan - and scanning private ranges is outside the engagement and outside your authority.
Do not attempt reachability tricks against the private address. Do record the intelligence value: the names disclose that Acme runs GitLab and Jenkins internally, which is directly useful later if you find an SSRF, a VPN appliance with a known CVE, or if the client adds an internal segment to scope. It also matters for the client's own threat model - an attacker who reads CT logs knows the internal tooling before sending a single packet.
Report it as a hygiene/information-disclosure item with two concrete recommendations: stop issuing publicly-trusted certificates for internal-only names (use an internal CA), and review whether the public DNS records for those names should exist at all. Flag gitlab-internal as a candidate for the application test if the client later provides VPN access.
3. The client says "internet-facing only" and robots.txt disallows `/internal/` and `/beta/`. How do you prioritise the recon findings into a test plan?
Read the worked answer
Robots.txt is a hint, not an authorisation boundary - it tells you where the interesting things are, and both paths sit on www.acme.example, which is internet-facing and therefore in scope. Disallow rules are also routinely the only place a pre-production path is advertised.
Suggested order: (1) www /internal/ and /beta/, because a beta build frequently ships with debug endpoints, verbose errors, and feature flags that bypass authorisation; (2) the auth-walled hosts (401/403), because authentication boundaries are where access-control findings live; (3) the twelve distinct applications, ordered by whether they handle authentication, uploads, or money; (4) API endpoints discovered in JavaScript bundles on those twelve. Deprioritise vpn.acme.example: it is a network appliance that resolves internally, so confirm with the client whether it is in the application-testing scope at all before spending time on it.
One caveat on the wildcard: because *.dev.acme.example is catch-all, any "found" dev host must be re-verified with a Host-header request before you report it as an asset. Write the plan with that verification step explicit, and include the fingerprint table so the client can challenge your asset list.