ASM False Positives and an AFM Blacklist Behind a Partner Proxy
A public API is protected by a single virtual server carrying APM (an access policy plus a per-request policy), ASM in Blocking enforcement, and an AFM DoS profile. The ASM policy was built in staging against a hand-written XML sample. Production clients actually post JSON containing an optional 'items' array of up to 400 entries and a base64 image field of up to 3 MB. Since go-live, 4% of POSTs are blocked with support IDs, and the dashboard shows the same violations repeatedly: 'JSON payload does not conform to schema' and 'Illegal parameter length', both firing on legitimate traffic. Separately, during a marketing push AFM's per-source-IP DoS detection blacklisted 203.0.113.44 for 10 minutes. That address is a partner's forward proxy carrying roughly 12,000 users, and the mitigation took the partner offline. The DoS profile uses automatic thresholds with per-source-IP detection and a 600-second blacklist duration.
1. How do you triage and fix the ASM false positives without weakening the policy?
Read the worked answer
Start from the support ID, not the violation name. Go to Security › Event Logs › Requests, filter on the support ID or the client IP and time window, and read the exact violation, the parameter name, the length or schema detail, and the payload excerpt. Then decide which problem you actually have.
'Illegal parameter length' on the base64 image field: the value exceeds the configured maximum parameter value length, and a 3 MB upload will always trip a limit sized for form fields. Either raise the limit for that specific parameter or add a targeted exception on the upload endpoint. A limit the legitimate product cannot meet is a policy bug, not an attack.
'JSON payload does not conform to schema': the policy was learned from XML, so there is no meaningful JSON schema - or the JSON profile is enforcing a structure production never sends. Attach a schema learned from real production samples (or accept the learning suggestion after reading it), and add parameter-level protection for the known fields with sensible length limits rather than switching JSON inspection off.
Do not set these violations to Transparent and do not disable signature enforcement: you would trade a 4% false-positive rate for unbounded blind spots. Keep the policy staged, replay a captured production request against staging, then enforce. Leave the request logging profile on for a week and re-check the block rate against the same support-ID workflow.
2. Why did AFM take the partner offline, and what do you change in the DoS profile?
Read the worked answer
Per-source-IP detection assumes one IP is one client. Behind a forward proxy carrying 12,000 users, that single address legitimately produces far more traffic than a per-IP threshold allows, so the automatic mitigation blacklists the proxy - the DoS feature itself becomes the outage, and a 600-second blacklist is ten minutes of total partner downtime.
Changes:
- Add 203.0.113.44 and any other known proxy or partner egress addresses to the DoS allowed list so automated detection and blacklisting skip them. Accept the trade-off explicitly: that path loses rate protection, so compensate with ASM request-rate limits on the virtual server or per-URI limits, and agree the exposure with the partner.
- For traffic arriving from proxies, use device-wide detection rather than per-source-IP detection. The aggregate is meaningful; the per-source rate is an artefact of NAT.
- Revisit the automatic thresholds. Auto thresholds need a clean learning period; if the baseline was learned during quiet traffic, the profile will fire constantly. Read the DoS statistics and live view to see what the vector actually measured before tuning it.
- Shorten the blacklist duration - ten minutes is a long time to be wrong - and consider scrubbing rather than blocking for the first interval.
Also confirm which vector fired. If it is SYN flood rather than a request-rate vector, the answer may be TCP-profile tuning and SYN cookies, not blocking a client at all.
3. Where do AFM, APM and ASM sit in the request flow on this VIP, and what does that mean for the blocked 4%?
Read the worked answer
Order matters here. AFM is evaluated first, at L3/L4 - DoS vectors and firewall rules see the connection before any proxy-level logic. APM runs next: the access policy authenticates the client and establishes the session, and the per-request policy runs for each request. ASM inspects the HTTP request payload afterwards.
Consequences you can actually use during triage:
- The 4% that ASM blocked had already been authenticated by APM. Their session exists and was consumed, they will see a block page rather than a login prompt, and the session keeps counting toward the APM session limit until it times out. Users who retry in a loop can accumulate sessions.
- Anything stopped by AFM's DoS mitigation never reaches APM or ASM, so nothing appears in the ASM event log. If a partner reports being blocked and ASM shows no support ID, check the AFM DoS and network firewall logs first.
- AFM rule context matters: a rule applied in the wrong context (global versus virtual-server) can silently drop traffic before the virtual server is reached, which looks like an application outage.
- Correlate the three layers by request: ASM support ID, APM session ID and timestamp in the same request-logging entry, rather than matching three dashboards by eye. That correlation is also what proves the AFM blacklist and the ASM false positives are unrelated incidents.