Remarks on the Backpropagation Algorithm

Mon, 01 Jun 2026 00:00:00 +0000

← Multilayer Networks and Backpropagation | Next → An Example: Face Recognition

4.6.1 Convergence and Local Minima

The Local Minima Problem

Unlike the linear unit — which has a single smooth bowl-shaped error surface — multilayer network error surfaces may have many local minima. Backpropagation is therefore only guaranteed to converge to some local minimum, not necessarily the global one.

    E(w)
     │         Local    Global
     │    Local Min     Min
     │    Max  ↓   \   /
     │   / \  / \  /\_/
     │  /   \/   \/
     └──────────────────── w

Despite this, Backpropagation works well in practice. Local minima are less severe than expected for two reasons:

Overfitting on Chandras Edu | AI, US Stocks, Courses & Jobs India

Remarks on the Backpropagation Algorithm

Unit 4 Navigation

4.6.1 Convergence and Local Minima

The Local Minima Problem