Model tuning and customization
Domain 4 - Model tuning and customization
Tuning questions test whether you can pick the cheapest intervention that changes behaviour, and prepare data that actually teaches.
Choose the intervention
- Prompt engineering - no weights change; try this first.
- RAG - for facts, freshness, and citations. Tuning does not reliably inject knowledge.
- Prompt tuning - learns a small set of soft-prompt vectors; cheap, works with few examples, good for style and task framing.
- Instruction tuning - adjusts weights on instruction/input/output pairs; better for consistent format, tone and complex behaviour; needs more data and compute.
Data preparation
Training files are JSONL with instruction, input and output fields. A few hundred well-written, diverse examples usually beat thousands of noisy ones. Deduplicate, remove PII and unsafe content, hold out a validation split, and keep a test set the model never sees.
Worked example
A team wants the model to answer from a 40,000-page manual and adopt a terse house style. Split the requirement: RAG for manual content, instruction tuning (or prompt tuning if data is scarce) for style. Reasoning: facts belong in retrieval; behaviour belongs in weights or prompts.
Evaluating
Compare the tuned model against the base model on the same held-out set, plus human review. Re-run safety evaluations because tuning can erode guardrails. Watch for overfitting: validation loss rising while training loss falls.
Traps
- Fine-tuning to memorise facts that change next quarter.
- Train/test leakage from near-duplicate examples.
- Too few examples (under roughly 100) for instruction tuning.
- Deploying a tuned model without re-running safety and quality evaluations.
- Forgetting that a tuned model is a new asset needing its own factsheet and approval.