Channel Capacity
The maximum rate at which information can cross a noisy channel with arbitrarily small error.
Contents
Channel Capacity
Channel capacity is the top speed of a communication channel — the highest rate, in bits per use, at which you can send information through noise and still recover it essentially perfectly. Shannon's noisy-channel coding theorem (1948) is the stunning result that this top speed C is not just a practical limit but a sharp threshold:
- Transmit at any rate below C and, with a clever enough Error-Correcting Code, you can drive the error probability arbitrarily close to zero.
- Transmit above C and reliable communication is impossible, no matter how much redundancy you add.
Before Shannon, engineers assumed noise forced a grim trade-off: go faster, get more errors. Shannon showed that below capacity you can have both speed and reliability — you just need long, well-designed codes.
Capacity is maximized mutual information
The capacity is defined through Mutual Information. The channel turns an input X into a noisy output Y; the information that actually gets through is I(X;Y). Capacity is the best you can do over all ways of choosing the input:
How noise eats the rate
The plot shows C = 1 - H(p) for the binary symmetric channel. Capacity is highest for a quiet channel, plunges as the flip probability climbs toward \tfrac12, then — strikingly — recovers for p > \tfrac12, because a channel that almost always flips is just as informative as one that almost never does (you simply invert its output).
Bandwidth, power, and the analog version
For continuous channels Shannon gave an equally famous formula. A channel of bandwidth B hertz with signal-to-noise ratio \mathrm{SNR} has capacity
Capacity grows linearly with the slice of Spectrum you occupy and only logarithmically with power — which is why grabbing more bandwidth is the cheaper road to higher data rates, and why every modem, Wi-Fi link, and deep-space probe is engineered right up against this line.