How uncertain is a bit?

Change a probability, compare the outcomes, and find where binary uncertainty reaches its maximum.

Teaching model · computed in your browser

Uncertainty in one bit

Move from a certain outcome to an even chance. See exactly where each bit of uncertainty comes from.

H(p)=-p\log_2 p-(1-p)\log_2(1-p)

p is the probability of one binary outcome, and 1 − p is the probability of the other. Logarithms use base 2, so the result is in bits. At the endpoints, 0 log₂ 0 is assigned its limiting value 0.

Try this: compare p = 0.1 and p = 0.9. Then explain why p = 0.5 gives the maximum.

Experiment controls

Probability p 0.50 to 1 · step 0.01

The default worked example is shown below.

Probability of outcome 10.5

Probability of outcome 00.5

Binary entropy1 bits

p = 0.5 gives 1 bits. Entropy is zero at certain outcomes and reaches one bit at p = 0.5.

Two contributions: −p log₂ p = 0.5; −(1 − p) log₂(1 − p) = 0.5. Add them to obtain 1 bits.

Probability and uncertainty
p	1 − p	H(p), bits
0	1	0
0.1	0.9	0.469
0.25	0.75	0.8113
0.5	0.5	1
0.75	0.25	0.8113
0.9	0.1	0.469
1	0	0

Turn exploration into understanding

Learn the ideas behind this lab.

Follow a worked example, practice on your own, and return to the original project sources.

Progress is kept in this browser. How learning works