Overfitting & Underfitting

Overfitting occurs when a model learns noise and peculiarities in the training data rather than general patterns, leading to poor performance on new data. Underfitting is the opposite — the model is too simple to capture the underlying structure.

Regularization techniques aim to find the sweet spot between these extremes, constraining the model just enough to generalize without sacrificing its ability to learn.