Neural Networks and Deep Learning
What this domain tests
Neural Networks and Deep Learning tests the perceptron, layered networks, activation functions, backpropagation and applications of deep learning. Expect small forward-pass calculations and conceptual questions on training.
Core rules and concepts
- A perceptron computes a weighted sum plus bias, then applies an activation such as a step function. It can learn only linearly separable patterns.
- A network has an input layer, hidden layers and an output layer. Activation functions add non-linearity: sigmoid, tanh, ReLU, softmax for multiclass outputs.
- Forward propagation computes outputs. A loss function measures error. Backpropagation applies the chain rule to compute gradients. Gradient descent updates weights using a learning rate over epochs.
- Deep learning uses many layers: CNNs for images, RNNs for sequences, transformers for language.
Worked example
A perceptron has inputs x = [1, 0], weights w = [0.8, -0.5] and bias b = -0.2. Weighted sum = 1(0.8) + 0(-0.5) - 0.2 = 0.6. With a step activation that outputs 1 when the sum is at least 0, the output is 1. Reasoning: the zero input cancels the second weight, and the bias shifts the decision boundary.
Traps that cost marks
Without non-linear activations, stacked layers collapse into an equivalent linear model. Backpropagation computes gradients; the optimiser performs the update. Softmax is for multiclass probabilities, not binary output. More layers can overfit and are not automatically better. A very high learning rate can make training diverge.