Fundamentals of Generative AI
How foundation models work
A foundation model (FM) is trained on broad data at scale and adapted to many downstream tasks. Large language models are FMs trained on text to predict the next token. Key terms: tokens (text units the model processes), embeddings (numeric vectors capturing meaning), context window (how much input the model can consider at once), parameters (learned weights), and inference (generating output). Training stages include pre-training on vast corpora, then fine-tuning or instruction tuning, then alignment techniques such as RLHF.
Customization options
Prompt engineering changes the input only. Retrieval-augmented generation (RAG) supplies external documents at inference time so answers stay grounded and current without retraining. Fine-tuning adjusts model weights with labelled examples for style or domain behaviour. Continued pre-training adds domain data. Know the trade-off: RAG is fast, cheap, and good for changing facts; fine-tuning is costlier and better for consistent behaviour or format.
Traps to expect
Questions confuse RAG with fine-tuning, or imply that fine-tuning updates knowledge in real time. Another trap is treating a larger context window as unlimited memory, or assuming generative models are factually reliable. Watch for hallucination (confident but fabricated output) and the idea that temperature and top-p control randomness, not accuracy.
Rule of thumb
If the answer depends on facts that change or live in your own documents, use RAG. If it depends on tone, format, or a specialized task pattern, consider fine-tuning. If it is just wording, use prompt engineering first.
Worked example
A company wants an assistant to answer from its internal policy PDFs that change monthly. Fine-tuning would go stale; RAG retrieves current documents at query time, so RAG is correct.