Bias vs variance
Bias is error from overly simple assumptions; variance is sensitivity to training data noise.
- High bias misses pattern
- High variance chases noise
- Need a balance
Bias vs variance
Tagged with generalization
3 cards
Overfitting vs underfitting
Overfitting memorizes noise; underfitting is too simple to capture the real pattern.
- Overfit has low train and poor test performance
- Underfit performs poorly everywhere
- Control with data, features, and regularization
Overfitting vs underfitting
What is regularization?
Regularization discourages overly complex models so they generalize better.
- L1 can drive weights to zero
- L2 shrinks weights smoothly
- Early stopping is also a regularization idea
What is regularization?