12 minute lessonReady to learn

Normalization changes comparisons

Understand why averages, norms, and normalized objectives answer different questions.

Course lessons · 3 of 5

Your goal

Calculate normalized scores and recognize when normalization changes their ranking.

Useful background: Decide which inputs are valid

01 · Understand the idea

Make the denominator meaningful

Normalization compares a raw quantity with a reference scale. Dividing a total by a count gives a mean. Dividing an information loss by an entropy scale gives the ratio used in the normalized MIP research. The denominator is part of the question, not cosmetic formatting.

A numerator measured in nats divided by a positive denominator measured in nats produces a dimensionless ratio. That ratio remains sensitive to how its constituent quantities were defined. Differential entropy, for example, depends on variable units.

J(S)=\frac{\Phi_{\rm SI}(S)}{K(S)}

A smaller numerator can lose after normalization

Imagine two candidate partitions with raw losses two and three. If their normalizers are one and three, their ratios are two and one. The second candidate has a larger raw loss but the smaller normalized objective.

The G16 materials also use aggregate state statistics. Dividing a state sum by 256 presumes a particular coordinate count. A different state shape needs its own definition; retaining the denominator while changing the count changes the scale.

02 · Work through an example

Compare two candidate scores

1Keep raw values separate

Teaching candidate A has loss two and normalizer one. Candidate B has loss three and normalizer three.

2Compute each ratio

Divide each loss by its own positive scale.

J_A=2/1=2,\quad J_B=3/3=1

3Apply the chosen objective

If minimizing normalized J, choose B. If minimizing raw loss, choose A. State which problem you solved.

03 · Change something

Make the idea respond

This lab is a separate three-component weighted-average example with component values from zero to one. It cannot reproduce the four unrestricted values 1, 1, 3, 3 from the question or the Gaussian MIP objective. Try doubling all three raw weights while keeping the values fixed: the normalized weights and final score remain unchanged. Then set every weight to zero and inspect why normalization is undefined.

Change one input at a time. Predict what should happen, then compare the computed result with your prediction.

Teaching model · computed in your browser

Build a weighted score

Change component values and weights. Watch normalization turn them into a single, explainable score.

S=\sum_i\left(\frac{w_i}{\sum_j w_j}\right)v_i

vᵢ is a component value from 0 to 1; wᵢ is a nonnegative raw weight. Divide every weight by their positive total before combining the values. These illustrative values are not measurements from a live AI system.

Try this: double all three weights. The normalized weights and total score should stay the same. Then set all weights to zero.

Experiment controls

Component A

Value A 0.80 to 1 · step 0.01Weight A 30 to 10 · step 0.5

Component B

Value B 0.60 to 1 · step 0.01Weight B 20 to 10 · step 0.5

Component C

Value C 0.40 to 1 · step 0.01Weight C 10 to 10 · step 0.5

The default worked example is shown below.

Raw weight sum6

Normalized weight sum1

Aggregate score0.6667Between 0.4 and 0.8

Each bar is the component value multiplied by its normalized weight. Add the contributions to get the score.

Normalize before adding: divide each raw weight by 6. Then S = 0.4 + 0.2 + 0.0667 = 0.6667. Multiplying every raw weight by the same positive factor leaves the result unchanged. Choosing weights expresses a modeling decision; it does not validate what a score means.

Every term in the weighted sum
Component	Value	Raw weight	Normalized weight	Contribution
Component A	0.8	3	0.5	0.4
Component B	0.6	2	0.3333	0.2
Component C	0.4	1	0.1667	0.0667

Open the full lab

04 · Check your understanding

Your turn.

Try an answer before opening a hint. Feedback will explain the result, and you can practice again.

Interactive practice is loading. The lesson, worked example, and printable questions remain available below.

View printable practice and solutions

Use these questions when studying without interactive controls. Opening a written solution does not earn progress.

A candidate has raw loss 6 nats and normalizer 3 nats. What is its normalized score?
Read the solution
6/3 = 2; the ratio is dimensionless.
Candidate A has (loss, scale) = (2, 1); B has (3, 3). Which minimizes the ratio?
- B
- A
- They tie.
- The ratio ignores the denominator.
Read the solution
B has the lower normalized score of one.
Four teaching state values are 1, 1, 3, and 3. What is their mean?
Read the solution
The mean is (1 + 1 + 3 + 3)/4 = 2.
What is required before evaluating J = loss/K in the recovered normalized MIP benchmark?
- The variable units can be changed without checking anything.
- A zero denominator should produce a best score.
- K must satisfy the benchmark’s positive-denominator condition.
- K can always be replaced with one.
Read the solution
The denominator and units must match the benchmark definition, including K > 0.

05 · Apply it to the archive

Follow the original record

Use the definitions you just practiced to read the project’s own notation, assumptions, and evidence.

Normalized minimum stochastic-interaction partition Global tanh of mean state across 256 coordinates

Take it with you

The ideas to keep

Preserve raw values and normalized values.
Use the actual count when computing a mean.
Normalization can change which candidate wins.

Reading completion is separate from your practice evidence.

Progress is kept in this browser. How learning works