Evaluation, Testing, and Quality Assurance
What to measure
Evaluation splits into retrieval quality (recall@k, precision, MRR), generation quality (correctness, faithfulness/grounding, relevance, format compliance), and system quality (latency, cost, error rates). Know the difference between offline evals on curated datasets and online signals from production traffic. Build a golden dataset that reflects real task distribution, including hard negatives and edge cases.
Methods
Understand LLM-as-judge: scalable but needs a rubric, calibration against human labels, and awareness of position and verbosity bias. Pairwise comparison is often more reliable than absolute scoring. Deterministic checks (schema validation, exact match, unit tests on tools) should run before any model-based judge. Regression suites catch prompt and model changes; A/B tests and canaries catch production drift.
Testing the system
Test tools with mocked responses, test orchestration with fault injection (timeouts, malformed output), and test prompts against adversarial inputs. Version prompts, datasets, and eval configs together so results are reproducible.
Traps
Questions tempt you with "use an LLM judge and trust the score" or "accuracy on a small sample proves quality." Watch for eval leakage (examples in the prompt that appear in the test set), judging with the same model that generated the answer without calibration, and optimizing a single aggregate metric that hides subgroup failures. Another trap: no baseline - you cannot claim improvement without one.
Rule of thumb
Define the metric and the golden set before you change the prompt; run deterministic checks first, then calibrated judges, and always compare against a baseline.