Why can English text be compressed using information about letter frequencies?
Consider a simple telephone conversation: A person (message source) speaks into a telephone receiver (encoder), which converts the sound of the spoken word into an electrical signal. This electrical signal is then transmitted over telephone lines (channel) subject to interference (noise). When the signal reaches the telephone receiver (decoder) at the other end of the line it is converted back into vocal sounds. Finally, the recipient (message receiver) hears the original message.
English text can be compressed using letter frequencies because the language is not random; it contains significant redundancy. In standard digital encoding, every character is typically assigned the same number of bits (usually 8), regardless of how often it appears. However, because certain lettersâsuch as "E," "T," and "A"âoccur much more frequently than others like "Z" or "Q," this uniform approach is inefficient.12
Information theory, specifically Shannonâs concept of entropy, demonstrates that compression is possible when the average length of an encoding exceeds the entropy of the source. By assigning shorter binary codes to frequently occurring characters and longer codes to rare ones, the total number of bits required to represent the text is reduced.13
Furthermore, English text exhibits patterns beyond individual letter frequencies, such as common word structures and repetitive sequences. Advanced data compression techniques, such as those developed by Abraham Lempel and Jacob Ziv, exploit these redundancies by replacing repeated blocks of text with shorter references, allowing for significant reductions in file size.123