Evaluation, Testing, and Observability
What to know
Evaluation answers "is it good enough, and did my change help?" Build a labeled test set that reflects real inputs, including edge cases and adversarial cases. Use a mix of automated checks (exact match, schema validation, keyword/rule checks), model-graded rubrics (LLM-as-judge with explicit criteria), and human review for subjective quality. Track metrics per task: accuracy, groundedness, format compliance, latency, and cost.
Observability means instrumenting the pipeline: log prompts, responses, tool calls, token counts, latencies, and errors with correlation IDs, while redacting sensitive data. Understand regression testing: any prompt or model change should be re-run against the eval set before deployment. Understand that LLM judges have biases (verbosity, position) and should be calibrated against human labels.
Traps
- Testing only happy-path inputs.
- Changing a prompt and shipping without re-running evals.
- Trusting a single LLM-judge score without calibration or spot checks.
- Measuring only accuracy while ignoring latency, cost, and refusal behavior.
- Logging without redaction, or logging without correlation IDs so failures can't be traced.
- Overfitting prompts to a tiny eval set.
Rule of thumb
No change ships without an eval run. Keep a frozen regression set, add a new case for every production failure, and calibrate any automated judge against human ratings before trusting it.