12 minute lessonReady to learn

Accept steps with a decrease test

Use backtracking to reject an overshoot and choose a step that satisfies an explicit condition.

Course lessons · 3 of 5

Your goal

Evaluate a simple Armijo condition and explain what it does and does not guarantee.

Useful background: See why step size matters

01 · Understand the idea

Choose a direction, then test its length

A line search separates the direction from how far to move along it. In this teaching example, the direction is the negative derivative. Start with a trial step size, evaluate the proposed point, and reduce the step size if the acceptance test fails.

The Armijo condition asks for sufficient decrease relative to the local gradient prediction. With descent direction −g, it compares the new objective with f(x) − cηg², using a constant c between zero and one.

f(x-\eta g)\le f(x)-c\eta g^2

A bounded algorithm needs stopping rules

Backtracking can repeatedly multiply η by a factor between zero and one. A working implementation also needs a maximum trial count, a minimum step size, and checks for nonfinite values. Failure to find an accepted step should be reported as a failure state.

Passing the test establishes a decrease condition for that proposed step. It does not prove a global optimum, general reasoning ability, or scientific interpretation of the objective. The lesson provides a textbook mechanism for understanding loss-controlled updates, rather than asserting that every archived build implements it.

02 · Work through an example

Reject an overshoot at x = 7

1Set the quantities

Use f(x) = ½(x − 3)², x = 7, g = 4, c = 0.25. The current objective is eight.

2Test η = 2

The proposed point is −1 and its objective is eight. The acceptance threshold is zero, so this trial fails.

8\nleq8-0.25\cdot2\cdot16=0

3Backtrack to η = 1

The proposed point is three with objective zero. The threshold is four, so this trial passes.

0\le8-0.25\cdot1\cdot16=4

03 · Change something

Make the idea respond

The lab performs fixed-step gradient descent; it does not run an automatic Armijo search. To check the written trials, set starting x = 7 and minimum a = 3. With η = 2, one step reaches −1 and the loss remains 8; with η = 1 after resetting the run, one step reaches 3 and the loss is 0. Apply c = 0.25 and the Armijo inequality from this lesson manually when deciding which trial passes. Changing the learning rate is the backtracking demonstration.

Change one input at a time. Predict what should happen, then compare the computed result with your prediction.

Teaching model · computed in your browser

Follow the gradient

Choose a starting point and learning rate, then inspect every update toward a quadratic minimum.

f(x)=\tfrac12(x-a)^2,\qquad x_{t+1}=x_t-\eta(x_t-a)

a is the location of the minimum; x is the current parameter; η is the learning rate. The derivative is x − a. For this unit-curvature quadratic, an initial error contracts precisely when 0 < η < 2. This condition does not automatically apply to a different loss function.

Try this: keep the same starting point and compare η = 0.5, 1, 2, and 2.5. Watch the distance from the minimum rather than the learning rate alone.

Experiment controls

Starting x -2-5 to 8 · step 0.5Minimum location a 3-3 to 3 · step 0.5Learning rate η 0.250 to 3 · step 0.05

Playback starts only when you choose Play. It pauses when this tab is hidden or the lab leaves view. Reduced-motion preferences slow the updates; Step always gives you direct control. Runs stop after 40 updates or at the display limit.

The default worked example is shown below.

Completed updates0 / 40

Current x-2

Loss ½(x − a)²12.5

Error multiplier0.75

The solid trace gives x after each update; the dashed line marks the minimum at a = 3. Convergent: each update moves toward the minimum without crossing it. The axis rescales to include all completed updates. Solid: x · Dashed: minimum location a

Convergent: each update moves toward the minimum without crossing it. Press Step to compute x₁ = x₀ − η(x₀ − a).

Computed update history
Update	x	Gradient x − a	Loss ½(x − a)²
0	-2	-5	12.5

Open the full lab

04 · Check your understanding

Your turn.

Try an answer before opening a hint. Feedback will explain the result, and you can practice again.

Interactive practice is loading. The lesson, worked example, and printable questions remain available below.

View printable practice and solutions

Use these questions when studying without interactive controls. Opening a written solution does not earn progress.

With f(x) = 8, g = 4, c = 0.25, and η = 1, what is the Armijo right-hand threshold?
Read the solution
The threshold is 8 − 4 = 4.
A proposed objective is 5 and the Armijo threshold is 4. Should this trial be accepted?
- Yes; every finite trial passes.
- Yes; only the old objective matters.
- The result proves there is no minimum.
- No; five exceeds the threshold.
Read the solution
The trial fails the sufficient-decrease test and can be backtracked.
A backtracking factor is 0.5. After rejecting η = 2, what step size is tried next?
Read the solution
The next trial is η = 1.
What does a passing Armijo test establish directly?
- Every future step will pass.
- The full historical engine is running.
- The proposed step satisfies the chosen decrease condition.
- The model has reached a global optimum.
Read the solution
The accepted step satisfies the explicit local condition for that evaluation.

05 · Apply it to the archive

Follow the original record

Use the definitions you just practiced to read the project’s own notation, assumptions, and evidence.

Epona / Echad discrete loss–value update Rhiannon · two-way correction engine

Take it with you

The ideas to keep

Test the proposed objective explicitly.
Bound retries and report a failed search.
A local decrease is a local result.

Reading completion is separate from your practice evidence.

Progress is kept in this browser. How learning works