TASC Data Science and Digital Solutions cheat sheet

CRISP-DM phases in order: business understanding, data understanding, data preparation, modeling, evaluation, deployment; iteration back to earlier phases is normal.

Supervised learning uses labeled data (classification for categorical targets, regression for continuous); unsupervised learning finds structure without labels (clustering, dimensionality reduction).

Bias-variance tradeoff: high bias underfits (simple model, large training error); high variance overfits (small training error, large test error). Regularization reduces variance.

Confusion matrix metrics: precision = TP/(TP+FP); recall = TP/(TP+FN); F1 = 2·(precision·recall)/(precision+recall); accuracy = (TP+TN)/total.

Cross-validation (k-fold) estimates generalization error by rotating held-out folds; a separate test set is required for final unbiased evaluation.

Correlation does not imply causation; confounding variables and selection bias can produce spurious associations. Randomized controlled trials support causal claims.

Precision/recall tradeoff: lowering the classification threshold raises recall and lowers precision; ROC-AUC summarizes performance across thresholds.

Data pipeline stages: ingestion, validation, transformation (cleaning, normalization, encoding), storage, and serving; data leakage occurs when test information enters training.

GDPR principles: lawfulness, fairness, transparency, purpose limitation, data minimization, accuracy, storage limitation, integrity/confidentiality, accountability.

Data anonymization removes identifying information irreversibly; pseudonymization replaces identifiers with keys and remains personal data under GDPR.

Cloud service models: IaaS (virtualized infrastructure), PaaS (managed runtime/platform), SaaS (fully managed application); shared-responsibility model splits security duties.

API design: REST uses stateless HTTP methods (GET, POST, PUT, DELETE) and resource URIs; idempotent methods (GET, PUT, DELETE) can be safely retried.

Role-based access control (RBAC) grants permissions by role; attribute-based access control (ABAC) evaluates attributes of user, resource, and context.

Encryption: symmetric (AES) uses one shared key and is fast; asymmetric (RSA, ECC) uses public/private key pairs and enables digital signatures and key exchange.

Data quality dimensions: completeness, accuracy, consistency, timeliness, validity, and uniqueness; each requires explicit validation rules in pipelines.

Model governance: document data provenance, monitor drift, maintain versioned models, and record audit trails to satisfy reproducibility and compliance requirements.