AI Security Monitoring
AI security monitoring
This domain tests detection: what you log from an AI system, what normal looks like, and which signals indicate attack rather than quality decay.
Core rules
- Log prompts, completions, tool calls, retrieved document IDs, model version, token counts, latency, safety-filter hits and caller identity - with redaction of sensitive content.
- Correlate layers: API gateway, model server, WAF, identity provider and SIEM. AI incidents usually appear as a combination, not a single alert.
- Detect drift in inputs (data drift) and in input–output relationships (concept drift); treat sudden drift as a possible poisoning or extraction campaign.
- Plant canary tokens in system prompts and sensitive documents to detect leakage; alert on verbatim repetition or on system-prompt echoes.
- Baseline behaviour per key and per user: token volume, request rate, tool-call patterns, unusual retrieval breadth.
- Tune detections from red-team findings so alerts reflect real attack paths, and track mean time to detect.
Worked example
One API key shows a jump in long prompts containing phrases like "repeat your instructions", plus repeated near-identical outputs and a spike in 4xx errors from varied user agents. Reasoning: this matches prompt-extraction probing. Response: rate-limit and challenge the key, enable canary-token detection on system prompts, review retrieved-document access, and add a detection rule for instruction-echo patterns - while checking whether the key leaked to a public client.
Traps
- Monitoring infrastructure metrics only; no prompt or tool-call telemetry.
- Not recording model version with each request, making forensics impossible.
- Ignoring indirect injection signals such as unexpected tool invocations.
- Alerting on every safety-filter hit, creating fatigue that hides real attacks.