Generative AI
Generative AI
The Generative AI domain tests the full app lifecycle: selecting models, designing and versioning prompts, grounding with retrieval, evaluating quality, and adding guardrails. DataRobot questions often frame GenAI as an application problem, not a model benchmark: the unit of evaluation is the task, with a rubric, test set, and acceptance threshold.
Core rules
- Write the task and rubric first: correctness, completeness, citation, tone, and refusal behavior.
- Version prompts and model settings; a prompt change is a release that needs regression testing.
- Evaluate retrieval separately from generation. Retrieval metrics include recall@k and context precision; generation metrics include groundedness and answer correctness.
- Use LLM-as-judge only after calibrating it against human-labeled examples; track judge drift when models change.
- Layer guardrails: input validation, retrieval filtering, output checks, and human review for high-impact actions.
- Pin model versions and watch context-window limits, latency, and token cost.
Worked example
An HR policy assistant answers 200 test questions. Retrieval recall@5 is 78% and answer accuracy is 71%. The team first rewrites the answer prompt; accuracy rises only to 73%. They inspect failures and find policy documents split mid-section. They add section-aware chunking and a metadata filter for effective date. Recall@5 becomes 94%, answer accuracy 86%, and unsupported-answer rate falls from 12% to 3%. Reasoning: generation was capped by retrieval; the prompt change could not recover missing evidence. The next test is a calibrated judge and a held-out set, not another prompt rewrite.
Traps
- Scoring fluency instead of correctness; a confident wrong answer can look best.
- Using an LLM judge without human calibration, which hides systematic bias.
- Treating RAG as a fix for stale, duplicated, or permission-mismatched source documents.
- Relying on prompt instructions as the only guardrail for unsafe output.
- Ignoring retrieval recall and tuning only the generator.
- Shipping without pinned model versions; silent provider updates break evaluations.