Advanced Topics of ANN: Alternative Error Functions
Unit 4 Navigation
← An Example: Face Recognition | Next → Recurrent Networks
Why Modify the Error Function?
Standard Backpropagation minimises:
E(w) = (1/2) × Σ_d Σ_k (t_kd − o_kd)²
This works for many cases. But we may also want to:
- Reduce overfitting by penalising large weights
- Enforce invariance by matching how the output varies with inputs
- Output probabilities rather than arbitrary real values
- Enforce symmetry across equivalent inputs
Each new objective leads to a different error function E, and hence a different gradient descent update rule.