Bot Manager Blocking a Partner's Automated Feed
A travel aggregator exposes a partner API used by three airlines to push fare updates. Traffic is machine-to-machine with mTLS and a static API key. Separately, the same host serves a consumer web app protected by Bot Manager with a managed bot detection set in deny for categories including 'automated browser' and 'headless'. Last Tuesday the airlines' fare pushes began returning 403s. The security team finds Bot Manager is scoring the partner calls as automated because the partner clients use a headless HTTP library and present a browser-like User-Agent. The partner traffic is ~2,000 requests/hour, all from a small set of documented egress IPs, all on /partner/fares. Consumer traffic on the same host is ~400,000 requests/hour. The team wants a fix that does not weaken bot protection for consumers and does not require the airlines to change their clients.
1. Design the control change that separates partner automation from abusive automation on this host.
Read the worked answer
The mistake would be to loosen the managed bot categories globally - that would weaken consumer protection to fix a machine-to-machine path. Instead, segment by what the traffic is, not by turning detection down.
Recommended layering:
- Put partner traffic on a distinct path/hostname policy if possible (
/partner/*already helps). Apply a Bot Manager policy or rule set scoped to that path that treats authenticated mTLS + valid API key as a trusted automation signal, so managed bot categories do not deny it. - Key the trust on strong, verifiable attributes - client certificate identity and the documented egress IP set - not on User-Agent. The UA is spoofable and is exactly why the client got misclassified.
- Keep the consumer path on the full managed bot policy unchanged. Do not add the partner IPs as a global allowlist, because a global allowlist would also exempt those IPs on consumer endpoints.
- Where the platform supports it, use a custom rule that requires both the partner credential and the partner path before bypassing bot denial, so a leaked key alone on another endpoint still gets scored.
This keeps consumer detection at full strength and makes the partner exception narrow and auditable.
2. The airlines cannot change clients. What residual risks remain and how would you monitor them?
Read the worked answer
Residual risks:
- Credential/IP trust is only as good as its custody. If a partner's egress IP or API key leaks, an attacker inherits the exception on
/partner/fares. Mitigate with per-partner keys, rotation, and anomaly detection on that path (volume, timing, payload shape) rather than assuming the credential is safe. - Path-scoped exception can be probed. Attackers may target
/partner/fareshoping to land in the trusted bucket. Require the credential check to fail closed and log every rejected attempt on that path. - UA-based misclassification may recur on other paths if the partner adds endpoints; keep the trust logic tied to identity, not client fingerprint.
Monitoring:
- Per-partner request volume and error rate on
/partner/fares, alerting on deviation from the ~2,000/hr baseline. - Count of requests to
/partner/*that lack valid mTLS/key - should be near zero; a spike means probing. - Bot deny counts on consumer endpoints, to confirm consumer protection did not drift after the change.
- Auth failures and new source IPs hitting the partner path.