← Introduction to ANN | Next → Perceptrons
The Backpropagation algorithm is the most commonly used ANN learning technique. It is appropriate for problems with the following six characteristics:
The target function is defined over instances described by a vector of predefined features — such as pixel values in an image.
Example: In ALVINN, each instance is described by 960 pixel intensity values.
Counter-example: Symbolic logic problems where inputs are categorical labels are less naturally suited (though ANNs can still be applied).
The target function may output:
Example: In ALVINN, the output is a vector of 30 values — each a confidence score for a specific steering direction. A single network can simultaneously predict both the steering command and suggested acceleration by concatenating the two output vectors.
ANN learning methods are robust to noise in the training data.
Because Backpropagation fits a smooth function to many examples simultaneously, individual mislabelled or noisy examples have limited influence on the final learned weights. This contrasts with methods like exact rule learning that are sensitive to noise.
Implication: ANNs are well-suited to real-world sensor data — camera images, microphone recordings, radar readings — which inherently contains measurement noise.
Training times can range from a few seconds to many hours, depending on:
Comparison: Decision tree learning is generally much faster to train. ANNs are better suited when training is done offline (once) and the trained network is deployed for repeated fast predictions.
Once trained, evaluating a neural network for a new input is very fast — just a forward pass through the layers.
Example: ALVINN applies its network several times per second to continuously update the steering command as the vehicle drives. This real-time requirement is easily met once training is complete.
Implication: ANNs have an asymmetric cost profile — slow to train, fast to predict. This is ideal for applications where training is a one-time or infrequent process.
The weights learned by a neural network are often difficult for humans to interpret. A trained ANN is a “black box” — it produces correct outputs but does not explain why.
Contrast with decision trees: Decision trees produce explicit human-readable rules like “if income > 50K and credit score > 700 then approve loan.” ANN weights encode the same knowledge implicitly across thousands of numbers.
Implication: Use ANNs when performance matters more than explainability. For medical diagnosis, legal decisions, or regulatory settings requiring justification, more interpretable models may be preferred.
| Characteristic | ANN Suitability |
|---|---|
| Many numeric attribute-value pairs | ✅ Ideal |
| Any output type (discrete, real, vector) | ✅ Flexible |
| Noisy training data | ✅ Robust |
| Training time can be long | ✅ Acceptable trade-off |
| Fast prediction needed after training | ✅ Very fast inference |
| Human-interpretable model not required | ✅ Black box is acceptable |
ANNs may be a poor choice when:
| Property | ANN | Decision Tree |
|---|---|---|
| Input type | Real-valued vectors | Categorical or real |
| Noise tolerance | High | Moderate |
| Training speed | Slow | Fast |
| Prediction speed | Fast | Fast |
| Interpretability | Low | High |
| Overfitting risk | High (without regularisation) | High (without pruning) |
| Best for | Sensor data, images, speech | Tabular data, rule extraction |
Experimental comparisons (Shavlik et al., 1991; Weiss & Kapouleas, 1989) show that ANN and decision tree learning produce results of comparable accuracy on many tasks — the choice often depends on the requirements above.
← Introduction to ANN | Next → Perceptrons