Why does English have redundancy according to information theory?
Consider a simple telephone conversation: A person (message source) speaks into a telephone receiver (encoder), which converts the sound of the spoken word into an electrical signal. This electrical signal is then transmitted over telephone lines (channel) subject to interference (noise). When the signal reaches the telephone receiver (decoder) at the other end of the line it is converted back into vocal sounds. Finally, the recipient (message receiver) hears the original message.
In the context of information theory, redundancy in English refers to the predictability of its structure, which allows a message to be understood even if parts of it are missing or distorted.1
Mathematically, English exhibits a redundancy of approximately 50 percent. This means that roughly half of the characters in a typical English sentence are technically unnecessary for conveying the core information, as the remaining characters provide enough context to reconstruct the original message.1
From an information-theoretic perspective, this redundancy serves a vital function: it acts as a buffer against noise. By repeating informationâsuch as marking plurality multiple times in a phrase like "those two dogs"âthe language ensures that communication remains successful even when a listener or reader misses part of the input due to inattention or external interference. While Claude Shannon demonstrated that this predictability increases as one moves from individual letters to longer sequences like paragraphs and books, this inherent structure is what allows human communication to remain robust and reliable.12