CNLP common mistakes
CNLP · Understand the traps and how to avoid them.
Treating 'tokenization' as one universal step and assuming the same token boundaries apply across BPE, WordPiece, and rule-based tokenizers when reasoning about vocabulary size, OOV behaviour, or sequence length limits.
Candidates learn tokenization from a single pipeline example (often whitespace or a single subword scheme) and never compare schemes, so they conflate character-level coverage with subword merges and mispredict how text maps to model input length.
What to do instead
For each tokenizer family, write down its unit (character, subword merge, word), its handling of unknown tokens, and its effect on sequence length. Practice mapping a short sentence to tokens under BPE versus WordPiece and compare counts, then check how that changes a model's max-length constraint.
Applying aggressive normalization (lowercasing, stemming, stopword removal, punctuation stripping) reflexively to every task, including sentiment, NER, or question answering, then wondering why signal is lost.
Preprocessing is taught as a fixed checklist rather than a task-dependent decision, so candidates optimize for vocabulary reduction and forget that casing, negation, and punctuation carry label-relevant information.
What to do instead
Before preprocessing, state which surface features the target label depends on. Justify each step as preserving or destroying that signal: keep casing for NER, keep negation words for sentiment, avoid stemming when morphology matters. Document the decision as part of the pipeline design.
Choosing TF-IDF or bag-of-words as the default representation for tasks where word order or semantic similarity matters, and then reporting strong-looking accuracy without checking whether the representation can express the needed distinction.
Classical vectorization is easy to run and produces plausible numbers, so candidates stop at the first working baseline and never test whether a representation that ignores order is actually adequate.
What to do instead
Match representation to task: use n-grams, positional features, or embeddings when order or similarity matters. Run a controlled comparison between bag-of-words and an order-aware or embedding-based baseline on the same split, and report where the simpler model fails.
Reporting a single accuracy number on an imbalanced classification task and treating it as evidence the model works, without inspecting per-class precision, recall, or the confusion matrix.
Accuracy is the first metric taught and the easiest to compute, and candidates often skip stratifying the evaluation or checking class balance before interpreting results.
What to do instead
Always inspect label distribution first, then select metrics aligned with the cost of errors: per-class precision/recall, macro or weighted F1, and a confusion matrix. Explain in one sentence why the chosen metric reflects the business or task cost.
Confusing training-time and inference-time behaviour of transformers, for example assuming causal masking, teacher forcing, or dropout applies identically during generation and evaluation.
Architecture diagrams show a single forward pass, so candidates memorize components without tracing how masking, attention direction, and decoding differ between training and inference.
What to do instead
Trace one sequence through both phases: which tokens are visible to each position, where masks are applied, and how outputs are fed back during autoregressive decoding. Sketch the attention mask matrix for encoder-only, decoder-only, and encoder-decoder models and compare.
Attributing a model's fluent output to factual correctness, or assuming retrieval-augmented generation eliminates hallucination rather than reducing it.
Fluency is easy to observe and correctness is expensive to verify, so candidates substitute a surface impression for an evidence check and overstate what grounding guarantees.
What to do instead
Separate fluency from factuality in evaluation design. For grounded systems, check whether each claim is supported by a retrieved passage, measure citation or attribution accuracy, and describe residual failure modes such as unsupported synthesis or stale retrieval.
Designing an offline evaluation that leaks information, for example tuning preprocessing or thresholds on the test set, or splitting time-series or grouped text data randomly instead of by the correct unit.
Standard train/test split recipes are applied mechanically without asking what the deployment distribution looks like or which entities must stay separated across splits.
What to do instead
Define the split unit before modeling: by document, by author, by customer, or by time. Freeze the test set until final evaluation, tune only on validation data, and state explicitly what leakage the chosen split prevents.
Treating fairness and bias as a single checkbox, such as removing a protected attribute from features, and concluding the system is unbiased.
Candidates reduce ethics to a preprocessing step and miss that proxies, label bias, and disparate error rates persist even without the attribute, and that fairness definitions can conflict.
What to do instead
Identify proxy features correlated with protected groups, measure performance disparities across subgroups, and name the fairness criterion being applied. Acknowledge trade-offs between criteria and document mitigation and monitoring plans.
Describing a deployment as 'we containerize the model and serve it' without addressing drift detection, versioning of model and preprocessing artifacts, rollback, or latency and throughput constraints.
MLOps content is often skimmed as tooling names, so candidates list components without connecting them into a lifecycle or explaining what happens when inputs or data distributions change.
What to do instead
Describe the pipeline end to end: versioned data and model artifacts, a preprocessing artifact pinned to the model, monitoring for input and output drift, alert thresholds, and a rollback path. Add concrete latency or throughput targets and how they are measured.
Proposing an LLM-based solution for a use case without defining the business success metric, the baseline it must beat, or the cost and latency envelope, and without considering a simpler alternative.
Use-case questions invite enthusiasm for the newest technique, so candidates jump to architecture before translating the business problem into a measurable objective and constraints.
What to do instead
Start from the decision the system supports, define an offline metric and an online business metric, state the current baseline, and justify why NLP is needed. Compare a simpler approach such as rules or classical classification, and state cost, latency, and human-review requirements.