See why step size matters
Distinguish steady convergence, oscillation, and divergence in the same simple model.
Course lessons · 2 of 5
Predict convergence of quadratic descent from its error multiplier.
Useful background: Turn a derivative into an update
Track the error instead of only the state
Let e_t = x_t − a be the error relative to the target. For the quadratic teaching model, the update gives e_(t+1) = (1 − η)e_t. Each step multiplies the error by the same factor when η is fixed.
The error shrinks when the absolute value of that factor is less than one. For this specific objective, this means 0 < η < 2. Negative multipliers create alternating sides of the target; alternation can still converge if the magnitude shrinks.
Convergence depends on the model
At η = 1, this quadratic reaches its target in one step. At η = 2, a nonzero error flips sign without shrinking. Above two, the error grows in magnitude. These exact thresholds follow from this quadratic and cannot be copied blindly to another loss.
Real source engines may have curvature that varies with state, multiple coordinates, clipping, or additional objectives. A learning-rate control should expose both the step and the objective trajectory so users can see a failure, rather than automatically hiding every unstable setting.
Compare three step sizes
1Use the same starting error
Let x = 7 and a = 3, giving error four. Hold the objective and starting point fixed.
2Choose η = 0.5
The error multiplier is 0.5, so errors become two, one, and one half.
3Compare η = 2 and η = 2.5
At two, errors alternate four and minus four. At 2.5, the multiplier is −1.5 and their magnitudes grow. These are distinct failure behaviors.
Make the idea respond
Keep starting x at 7 and minimum a at 3. Compare η = 0.5, 1, 2, and 2.5, resetting the run between trials. Inspect the state and loss table: alternation at η = 2 preserves error magnitude, while η = 2.5 increases it. The controls begin with a separate default starting point, so enter these settings to reproduce the written example.
Change one input at a time. Predict what should happen, then compare the computed result with your prediction.
Teaching model · computed in your browser
Follow the gradient
Choose a starting point and learning rate, then inspect every update toward a quadratic minimum.
a is the location of the minimum; x is the current parameter; η is the learning rate. The derivative is x − a. For this unit-curvature quadratic, an initial error contracts precisely when 0 < η < 2. This condition does not automatically apply to a different loss function.
Try this: keep the same starting point and compare η = 0.5, 1, 2, and 2.5. Watch the distance from the minimum rather than the learning rate alone.
Playback starts only when you choose Play. It pauses when this tab is hidden or the lab leaves view. Reduced-motion preferences slow the updates; Step always gives you direct control. Runs stop after 40 updates or at the display limit.
The default worked example is shown below.
Convergent: each update moves toward the minimum without crossing it. Press Step to compute x₁ = x₀ − η(x₀ − a).
| Update | x | Gradient x − a | Loss ½(x − a)² |
|---|---|---|---|
| 0 | -2 | -5 | 12.5 |
Your turn.
Try an answer before opening a hint. Feedback will explain the result, and you can practice again.
Interactive practice is loading. The lesson, worked example, and printable questions remain available below.
View printable practice and solutions
Use these questions when studying without interactive controls. Opening a written solution does not earn progress.
For η = 0.25, what is the error multiplier 1 − η?
Read the solution
The error multiplier is 0.75.
If the current error is 4 and η = 0.5, what is the next error?
Read the solution
The next error is 0.5 × 4 = 2.
For this quadratic and a nonzero starting error, what happens at η = 2?
- The target is reached in one step.
- The error decreases monotonically to zero.
- The update becomes undefined.
- The error changes sign with unchanged magnitude.
Read the solution
The state oscillates symmetrically around the target without reducing the loss.
Which statement about the interval 0 < η < 2 is correct?
- It is the convergence interval for this explicitly defined quadratic.
- It guarantees convergence for every neural network.
- It proves biological equivalence.
- It is independent of how the objective is scaled.
Read the solution
The interval is model-specific and should be re-derived or checked for other objectives.
Follow the original record
Use the definitions you just practiced to read the project’s own notation, assumptions, and evidence.
The ideas to keep
- Convergence concerns shrinking error, not motion alone.
- Oscillation can converge or persist.
- Step-size thresholds depend on the objective.
Reading completion is separate from your practice evidence.
Progress is kept in this browser. How learning works