What is Learning Rate

The learning rate controls how large a step the optimizer takes when updating model weights during gradient descent. It is arguably the most important hyperparameter in neural network training.

Too large a learning rate causes the model to overshoot minima and diverge, while too small a rate leads to painfully slow convergence or getting trapped in poor local minima.