Deep Learning Fundamentals
Deep Learning Fundamentals tests how neural networks are built and trained: layers, activations, losses, backpropagation, optimizers, and the shape arithmetic that connects them.
Core rules
- A dense layer computes y = activation(Wx + b). Activation functions add non-linearity; ReLU is common in hidden layers, sigmoid or softmax at the output depending on the task.
- Backpropagation applies the chain rule to compute gradients; the optimizer updates weights. SGD is simple and stable; Adam adapts per-parameter step sizes.
- CNNs exploit local connectivity and weight sharing, cutting parameters compared with fully connected layers on images.
- RNNs process sequences and share weights over time; LSTM and GRU gates address vanishing gradients.
- Batch normalization stabilizes training; dropout regularizes by randomly dropping activations during training only.
Worked example
A convolutional layer receives 32 x 32 x 3 input, uses 16 filters of size 3 x 3, stride 1, and same padding. Output spatial size stays 32 x 32 because same padding preserves it. Output channels equal the filter count, so the output is 32 x 32 x 16. Parameter count is (3 x 3 x 3) weights per filter x 16 filters = 432, plus 16 biases = 448. Reasoning: each filter spans all input channels, and every filter contributes one bias.
Traps
Forgetting the bias term undercounts parameters. Mixing channels and filters produces wrong output depth. Dropout must be disabled at inference. A too-high learning rate causes divergence, not faster convergence. Also check whether a question asks for output shape or parameter count; they are different calculations.