Chain Rule & Gradients

Backpropagation is the algorithm that makes deep learning tractable by efficiently computing gradients of the loss with respect to every parameter in the network. It relies on the chain rule of calculus to decompose complex derivatives into products of simpler local gradients.

Without the chain rule, computing gradients for a network with millions of parameters would be computationally infeasible — backpropagation reduces this to a single forward pass followed by a single backward pass.