Testing Generative AI and Large Language Models
What changes with LLMs
Outputs are open-ended text, so there is no single correct answer. Behaviour is probabilistic (sampling temperature/top-p) and prompt-sensitive: small wording changes can alter output. Exact-match assertions are almost always the wrong test design.
What to evaluate
Define criteria per use case: relevance to the request, coherence/fluency, factuality or groundedness (support by provided context, i.e. absence of hallucination), completeness, tone, safety (toxicity, harmful instructions, self-harm, PII leakage), bias, and refusal/abstention behaviour (knowing when to say "I don't know").
Test approaches
- Prompt/behavioural test suites with explicit acceptance criteria, run repeatedly and aggregated statistically.
- Metamorphic testing: paraphrasing the prompt, reordering options, translating and back-translating, adding irrelevant context - expected invariance or a defined direction of change.
- Adversarial testing / red teaming: prompt injection (instructions hidden in retrieved documents or user content), jailbreaks, data exfiltration, prompt leaking.
- RAG systems: test retrieval recall and ranking, grounding of the answer in retrieved chunks, citation correctness, and behaviour when retrieval returns nothing relevant.
- Guardrail testing: input/output filters, rate limits, PII redaction.
- LLM-as-a-judge: scalable but needs its own validation - judge bias, self-preference, position bias, and agreement with human ratings.
- Benchmarks: beware data contamination (test items present in pretraining data) and saturation.
Traps
- Single-run evaluation of a stochastic system; no seed control or repetition.
- Trusting a judge model without validating the judge.
- Assuming a model that passes a benchmark is safe from prompt injection.
Rule of thumb
For each prompt, state what an acceptable answer looks like, sample the output several times, and score against a rubric - treat the pass rate, not one output, as the test result.