These notes represent my own study jottings and learning synthesis, not an authoritative or foolproof standard. If you find them helpful, feel free to use them. If you spot anything that could be clearer or have suggestions for additional concepts to include, issues and pull requests are warmly welcome.
Model Behavior & Generalization
Generalization error, decomposition, and model capacity
05. Model Behavior & Generalization
Why a model that fits its training data can still fail, and how to tell what kind of failure you are looking at.
A trained model has two very different kinds of performance: how well it reproduces the data it has seen, and how well it handles data it has not. The gap between them is the central problem of machine learning.
This chapter builds the vocabulary for diagnosing that gap. Overfitting and underfitting describe the symptom. Bias and variance describe the mechanism. The trade-off between them explains why model selection is a balancing act, and generalization is the property all of it serves.