Neural-Networks

Unit 4 Navigation

← Unit 4 Overview  |  Next → Appropriate Problems for NN Learning

Overview

Artificial Neural Networks (ANNs) provide a general, practical method for learning real-valued, discrete-valued, and vector-valued functions from examples.

The Backpropagation algorithm — the cornerstone of ANN learning — uses gradient descent to tune network parameters to best fit a training set of input-output pairs.

Why ANNs Matter

ANN learning has been proven effective in:

• Recognising handwritten characters (LeCun et al., 1989)
• Recognising spoken words (Lang et al., 1990)
• Recognising faces (Cottrell, 1990)
• Steering autonomous vehicles (Pomerleau, 1993)

4.1 Biological Motivation

The study of ANNs was inspired by the observation that biological learning systems are built of densely interconnected neurons.

Unit 4 Navigation

← Introduction to ANN  |  Next → Perceptrons

4.3 When Should You Use Neural Networks?

The Backpropagation algorithm is the most commonly used ANN learning technique. It is appropriate for problems with the following six characteristics:

Characteristic 1: Instances Described by Many Attribute-Value Pairs

The target function is defined over instances described by a vector of predefined features — such as pixel values in an image.

• Input attributes may be highly correlated or independent of one another
• Input values can be any real number (not just binary or integer)

Example: In ALVINN, each instance is described by 960 pixel intensity values.
Counter-example: Symbolic logic problems where inputs are categorical labels are less naturally suited (though ANNs can still be applied).

Unit 4 Navigation

← Remarks on Backpropagation  |  Next → Advanced Topics: Error Functions

4.7 Face Recognition with Backpropagation

This section walks through applying Backpropagation to a real learning task — recognising which direction a person is facing from a camera image. It illustrates the design decisions every ANN practitioner must make, and what the network actually learns internally.

4.7.1 The Task

Dataset

Variations within the dataset:

Unit 4 Topics

Source: Tom M. Mitchell, Machine Learning, Chapter 4
📄 Full Chapter PDF