Shannon Used ‘Alternatives’ before ‘Bits
Computing Fun Fact for Today: Before the term ‘bit’ (binary digit) was invented, Shannon used ‘alternatives’ to quantify binary information.
Although the ‘bit’ is widely used today, it postdates some of Claude Shannon’s information theory work. John Tukey is credited with the first use of ‘bit’ in 1946–47, offered as a less clunky replacement for ‘binary digit’. But Shannon had already been quantifying information in base two for over a year. What was he using?
Shannon used ‘alternatives’, emphasizing the fact that a binary state only has two possible states. There is the path taken, and then there is the alternative. A coin flip leads to heads or tails. Combining N binary states together, then there are N alternatives. Each bit or alternative could be thought of as a fork in a road leading to 2^N possible states. More bits. More alternatives. More information.
I like this historical unit because it emphasizes the finiteness of discrete representations and the math applied on top. We often reason about digital computers using mathematical notions of error and real-valued functions. However, these operations are just mappings between concatenated alternatives. There are infinite real numbers, but only 2^64 possible FP64 values (even less due to redundant encodings). We use finite computers managing finite states. ‘Alternatives’ offers a more immediate reminder of this fact than the ‘bit’ does.
If I give you two variables and tell you to multiply them, how many alternatives are there? If you’re a mathematician, you’ll say “Infinitely many!” but if you’re a computer scientist, you might ask for the operand bit width. Regardless, the answer could take many possible values, and you need to perform the computation to reduce uncertainty about the result.
Now what if I tell you that one of those values is 0? How many alternatives are there? 0. You know that x times 0 is 0 for any x (ignoring infinities and NaNs). The output is certain, there are no alternatives, and there is no information to be gained by performing the operation.
This is the framing I’m building my PhD around, and I find talking about it in terms of ‘bits’ makes things a bit more abstract than if ‘alternatives’ were still used. Nowadays, the technically correct unit of base-two information is the ‘Shannon’ as specified by the IEC 80000 13:2025 standard. Three different units for the same quantity in 80 years certainly is fun for metrologists!
Regardless, don’t forget that Shannon once thought information a bit alternative.
Sources:
