Model Evaluation, Ethics, and Responsible AI
Model Evaluation, Ethics, and Responsible AI tests whether you can measure a system honestly and reason precisely about whom it can harm.
Metrics
- Confusion matrix to precision, recall, F1. Macro-F1 weights classes equally, micro aggregates counts, weighted averages by support.
- ROC-AUC versus PR-AUC: under heavy class imbalance, PR-AUC is more informative because ROC-AUC stays optimistic.
- Calibration via reliability diagrams and expected calibration error; bootstrap confidence intervals; McNemar's test for paired comparisons.
- Annotation quality: Cohen's kappa for two annotators, Krippendorff's alpha beyond that.
Responsible AI
Bias audits across subgroups; fairness definitions conflict - demographic parity and equalized odds generally cannot both hold. Privacy requires PII minimization, consent, and retention limits. Transparency requires model cards, documented limitations, human oversight, and red-teaming before launch.
Worked example
1,000 examples with 10 positives. A model predicting all-negative scores 99% accuracy with 0 recall. Reasoning: accuracy is dominated by the negative class, so report per-class precision and recall plus PR-AUC, which here is near the 0.01 prevalence - essentially chance.
Traps
- Tuning decision thresholds on the test set.
- Reporting one aggregate number with no subgroup breakdown.
- Treating BLEU or ROUGE as semantic quality measures; they reward n-gram overlap.
- Ignoring inter-annotator agreement and treating noisy labels as ground truth.