Generative AI Fundamentals
What generative AI actually is
Generative AI models learn statistical patterns from large datasets and produce new content - text, code, images, audio, video. Most modern text systems are built on the transformer architecture, whose self-attention mechanism weighs relationships between all tokens in a sequence. Text is split into tokens (subword units); the context window is the maximum number of tokens the model can consider at once, and both latency and cost scale with token counts.
Vocabulary a candidate must own
- Foundation model: a large pretrained model adaptable to many downstream tasks.
- Embeddings: numeric vectors capturing semantic meaning - used for semantic search, clustering, and retrieval. They are not a tuning technique.
- Multimodal: accepts or produces more than one modality (text plus image, audio, video).
- Hallucination: fluent but fabricated output, arising from probabilistic generation and missing grounding.
- Sampling controls: temperature rescales the probability distribution; top-p truncates the candidate set. Both control randomness.
max output tokenscaps length only.
Traps
Questions routinely conflate: (a) embeddings with fine-tuning; (b) temperature with top-p; (c) generative with discriminative ML - a spam classifier is not GenAI; (d) "GenAI" with "text only." Watch also for the claim that a larger context window guarantees factual accuracy.
Rule of thumb
If the task is produce new content probabilistically, think GenAI. If it is predict a label or number from features, think classic ML. If the model must know facts absent from its training data, the answer involves retrieval or grounding - never "raise the temperature."