Multilayer Networks and Backpropagation Algorithm
Unit 4 Navigation
← Perceptrons | Next → Remarks on Backpropagation
Why Multilayer Networks?
The perceptron can only represent linear decision surfaces. Real-world problems like vowel recognition or face direction require nonlinear boundaries.
Multilayer networks trained with Backpropagation learn rich nonlinear decision surfaces — curved, irregular boundaries that no single perceptron could produce.
4.5.1 The Sigmoid Unit
The Problem with Perceptrons in Multilayer Networks
We need units that are:
- Nonlinear — to express complex functions
- Differentiable everywhere — to apply gradient descent
Perceptrons fail condition 2 (hard threshold is not differentiable at the boundary). Linear units fail condition 1 (stacking linear units still gives a linear function).