Perceptrons to Networks

The perceptron, introduced in 1958, is the simplest neural unit — a linear function followed by a threshold activation. While limited on its own, stacking perceptrons into layers creates multi-layer networks capable of learning complex, non-linear patterns.

The universal approximation theorem tells us that a sufficiently wide network with at least one hidden layer can approximate any continuous function, though depth often provides more efficient representations.