What is Convergence

A model is said to converge when its loss function stabilizes at or near a minimum during training. Convergence means the optimizer has found a set of weights where further updates yield diminishing improvements.

In practice, we monitor loss curves and gradient norms to judge whether training is converging, diverging, oscillating, or stuck on a plateau.