AI and Intelligent Automation
AI and Intelligent Automation covers two different things: monitoring applications that call large language models, and using New Relic's AI-assisted operations features. The exam tests whether you can instrument LLM calls, read token and cost data, and use anomaly detection and incident intelligence without over-trusting them.
Core rules
- New Relic AI Monitoring instruments LLM application calls through provider integrations (for example OpenAI and Amazon Bedrock) and framework instrumentation. It records model, latency, token counts, estimated cost, errors, and optionally prompt and completion content. Prompt/response capture is powerful but creates a data-governance obligation.
- LLM traces are spans like any other distributed trace. A slow chat span can be caused by retrieval, a vector store query, a tool call, or the model itself; inspect child spans before blaming the model.
- Anomaly alert conditions learn a baseline per signal and open incidents when behavior departs from it. They are not static thresholds and need a stable seasonality pattern to be useful.
- Incident Intelligence groups related incidents and suggests likely causes using telemetry relationships. New Relic AI can generate NRQL and summarize incidents; treat generated queries and summaries as drafts to verify against raw data.
Worked example
An LLM assistant shows p95 latency rising from 3s to 9s and token cost tripling. The trace breaks down as retrieval 6s, model call 2.5s, and post-processing 0.5s. The retrieval span shows 1,800 prompt tokens because the retriever returns 20 documents. Reducing to the top 5 documents cuts prompt tokens by about 60% and latency to 4s. Monitoring only the model span would have missed the real cause.
Traps
Do not equate model latency with end-to-end latency. Do not ingest prompts containing secrets or personal data without controls. An anomaly is not automatically an incident. Verify AI-generated NRQL before trusting a dashboard.