Blocking Mode Rollout on a High-Traffic API
A security team enables AppDynamics Application Security on a public REST API that serves 4,000 req/s across 12 nodes. They start in detection-only mode for two weeks and see a steady stream of SQL-injection and XSS observations, plus a large volume of 'suspicious' findings triggered by a legacy endpoint that echoes rich text. The team now wants to move to blocking mode. During the detection window, that legacy endpoint generated roughly 30% of all reported attack events, and its payloads are actually legitimate rich-text HTML from a trusted internal editor. The API has a strict latency SLO of 150 ms p99. The security lead proposes flipping all 12 nodes to blocking mode at once to 'get full coverage immediately.'
1. Evaluate the proposal to flip all nodes to blocking at once. What is the risk, and what rollout would you recommend?
Read the worked answer
Flipping all nodes to blocking simultaneously maximizes blast radius: any false positive becomes an outage on 100% of traffic at once, and the legacy rich-text endpoint is a known false-positive source (30% of events, legitimate payloads). With a 150 ms p99 SLO, you also can't absorb a surprise retry storm.
Recommended: (1) First resolve the false positives - tune or exclude the legacy endpoint's known-benign pattern or add it to an allowlist/exception so it doesn't trigger blocks. (2) Roll out blocking incrementally - a canary subset of nodes (e.g., 1-2), watch error rates, 4xx/5xx, and p99 latency against the SLO, then expand in waves. (3) Keep a fast rollback path to detection-only. (4) Validate that blocking actually stops the request rather than merely logging.
2. How would you decide whether a given attack event is safe to block, using the data AppDynamics provides?
Read the worked answer
Use the event's evidence, not just its label. For each candidate rule/event, inspect: the matched rule and the specific payload/parameter that triggered it; whether the request was actually rejected or just observed; the affected URI and how frequently that URI legitimately receives similar input; and the business transaction it belongs to. Cross-reference with the false-positive cluster (the legacy endpoint). A rule is a good blocking candidate when its matches are concentrated on endpoints with no legitimate need for the matched pattern and when the observed payloads are genuinely malicious. Rules whose matches overlap known-benign traffic should be tuned or excluded before blocking. Also confirm the rule fires on real attack attempts (e.g., injection strings) rather than on encoded/benign variants, and check that blocking won't break authenticated or batch clients that send unusual-but-valid payloads.
3. After enabling blocking, what signals would tell you the rollout is harming users, and what would you do?
Read the worked answer
Watch for: a rise in 4xx/5xx from the API (especially 403/blocked responses clustered on specific endpoints), a drop in successful business transactions, increased client retries, and p99 latency approaching or breaching 150 ms. Correlate blocked-request events with BT failures - if blocked requests map to a legitimate BT and to the rich-text endpoint, that's a false-positive block.
Response: immediately roll the affected nodes back to detection-only, then tune the offending rule (exception/allowlist for the benign pattern) before re-enabling blocking on a canary. Keep the block decision auditable so you can show which rule caused which blocked request.