One-step Q-learning update

Read the equation

Move the current action-value estimate toward the observed reward plus the discounted best next-state value. The bracket is the temporal-difference error, and α controls how much of that error is incorporated.

Its role in the build

The Megamind Equation Atlas

Terms & definitions

The recovered passage does not supply a separate symbol table. Read the source context for the notation used in this version; familiar letters may carry different meanings in other builds.

How it works

1. Read Q_{t+1}(s,a) as the quantity being defined or updated by this relation.

2. Substitute the source-defined variables or recorded values and follow the displayed arithmetic and indexing exactly.

3. Use this version’s variable definitions and update order. The same symbol may be assigned a different meaning in another build.

Assumptions & limits

• Convergence depends on assumptions about the environment, exploration, learning rates and representation. This displayed rule is not evidence that the project trained a converged policy.

Original context

The Megamind Equation Atlas: Move the current action-value estimate toward the observed reward plus the discounted best next-state value. The bracket is the temporal-difference error, and α controls how much of that error is incorporated.

Q_{t+1}(s,a) = Q_t(s,a) + α [ r + γ max_{a′} Q_t(s′,a′) − Q_t(s,a) ]

Related builds