Shannon Entropy

Claude Shannon's 1948 paper established information theory by defining a mathematical measure of uncertainty. Entropy quantifies the average information content of a message source.

This foundational concept connects probability theory to communication, providing the theoretical limits on data compression and transmission efficiency.