Machine Learning and Deep Learning Model Development
What this domain covers
This domain covers the ML/DL workflow: problem framing, data splitting, model selection, training, evaluation, and tuning, with ModelArts training jobs as the execution vehicle.
Core rules and concepts
- Split data into train, validation, and test. Tune on validation; touch test once.
- Classification metrics: accuracy, precision, recall, F1, AUC. Regression: MSE, MAE, R².
- Overfitting signals: training loss falls while validation loss rises. Remedies: more data, augmentation, dropout, weight decay, early stopping.
- Deep learning building blocks: CNN for grids, RNN/LSTM for sequences, Transformer for long-range dependencies; batch normalization stabilizes training.
- Transfer learning: load pretrained weights, replace the head, freeze the backbone initially, then fine-tune with a smaller learning rate.
- ModelArts supports automatic hyperparameter tuning over a search space.
Worked example
A fraud model yields TP=40, FP=10, FN=20, TN=930. Precision = 40/(40+10) = 0.80. Recall = 40/(40+20) = 0.667. F1 = 2·0.80·0.667/(0.80+0.667) ≈ 0.727. Accuracy = (40+930)/1000 = 0.97. Reasoning: accuracy looks excellent because fraud is rare, but recall shows that one-third of frauds are missed, so F1 or recall is the better reporting metric.
Traps
Tuning hyperparameters on the test set; reporting accuracy on imbalanced data; forgetting to freeze batch-norm statistics when fine-tuning; leakage from scaling before the split.