AWS Machine Learning Specialty (AWS) cheat sheet

S3 object keys are flat; prefixes emulate folders. Listing is lexicographic, and renaming requires copy plus delete.

Use multipart upload for objects larger than 100 MB; it is required for objects larger than 5 GB and allows parallel part retries.

Parquet and ORC are columnar and compress well; Athena bills per TB scanned, so partitioning and columnar formats reduce query cost.

Glue Data Catalog is Hive-compatible metadata; crawlers infer schemas and partitions; Athena, Redshift Spectrum, and EMR read it.

Kinesis Data Streams: one shard provides 1 MB/s or 1,000 records/s ingress and 2 MB/s egress; retention is 24 hours by default and up to 365 days.

Kinesis Firehose buffers and delivers to S3, Redshift, or OpenSearch; it is near real-time and does not support replay of delivered records.

Redshift is columnar MPP; use COPY from S3 for parallel loads, DISTKEY for join colocation, and SORTKEY for range filtering.

AWS DMS migrates data; SCT converts schemas; Lake Formation centrally manages fine-grained S3 and Glue permissions.

Outlier rule: flag x < Q1 - 1.5 x IQR or x > Q3 + 1.5 x IQR; use robust statistics when distributions are skewed.

Pearson measures linear correlation; Spearman uses ranks and captures monotonic relationships; chi-square tests association between categorical variables.

Missingness types are MCAR, MAR, and MNAR. Mean or median imputation preserves distribution shape poorly; model-based imputation uses observed covariates.

Fit imputers, scalers, and encoders only on training data inside each CV fold to prevent data leakage.

For class imbalance, prefer precision, recall, F1, or PR-AUC over accuracy; use class weights, resampling, or SMOTE on training folds only.

High-cardinality categoricals: one-hot creates sparse wide data; target encoding needs out-of-fold fitting; feature hashing bounds dimensionality.

Right-skewed positive features: log or Box-Cox transform; tree models are invariant to monotonic transforms, distance models are not.

SageMaker Data Wrangler visualizes and transforms data; Clarify computes bias metrics and explanations; Ground Truth manages labeling.

Tabular baseline: XGBoost for structured data; Linear Learner for sparse or high-dimensional linear problems; DeepAR for related time series.

SageMaker built-ins include XGBoost, Linear Learner, KMeans, PCA, Random Cut Forest, DeepAR, BlazingText, Seq2Seq, NTM, Object Detection, and Image Classification.

Classification metrics: accuracy for balanced classes, precision/recall/F1 for imbalance, ROC-AUC for ranking, PR-AUC for rare positives; regression: RMSE, MAE, R-squared.

L1 regularization drives weights to zero and performs feature selection; L2 shrinks weights smoothly; elastic net combines both.

Bias-variance: high bias underfits and needs more capacity or features; high variance overfits and needs regularization, more data, or a simpler model.

Use k-fold cross-validation for i.i.d. data; use chronological splits or rolling-origin backtesting for temporal data and never shuffle time order.

Clustering: choose k via elbow or silhouette; silhouette ranges from -1 to 1; standardize features before PCA because PCA is variance-based.

Neural networks: tune learning rate first, then batch size, epochs, dropout, and architecture; use early stopping on validation loss.

Deployment choice: real-time endpoint for low-latency persistent traffic; serverless for intermittent traffic; async for large payloads or long processing; batch transform for offline scoring.

Multi-model endpoints host many models behind one endpoint and load them on demand; Inferentia targets cost-efficient inference.

Model Monitor detects data drift, model-quality drift, bias drift, and feature-attribution drift against a baseline.

SageMaker Pipelines orchestrates processing, training, tuning, evaluation, conditional steps, and model registration; Model Registry versions and approves models.

Spot training lowers cost but can interrupt jobs; enable checkpointing to S3 so interrupted jobs resume from the last checkpoint.

Secure ML: least-privilege IAM roles, KMS encryption at rest, TLS in transit, VPC subnets and security groups, and VPC endpoints for S3.

Endpoint auto scaling uses target-tracking policies on metrics such as InvocationsPerInstance or CPUUtilization; A/B testing uses multiple production variants with traffic weights.

Retraining triggers include Model Monitor drift alarms, CloudWatch metrics, scheduled pipelines, and new labeled data in Ground Truth.