Policy gradient: ∇J ≈ Σ(∇log π(a|s)·R)

Read the equation

This relation defines @brief Policy gradient: ∇J ≈ Σ(∇log π(a|s)·R) in the source section “Policy gradient: ∇J ≈ Σ(∇log π(a|s)·R)”. Uses a derivative or gradient to describe a rate of change or a direction of variation. Its interpretation depends on the source’s stated variables and conditions; this record does not assign an unprovided physical or cognitive meaning to them.

Terms & definitions

The recovered passage does not supply a separate symbol table. Read the source context for the notation used in this version; familiar letters may carry different meanings in other builds.

How it works

1. Read the displayed expression together with the surrounding source statement; it is retained as an expression rather than silently completed into an equality.

2. Uses a derivative or gradient to describe a rate of change or a direction of variation.

3. Use this version’s variable definitions and update order. The same symbol may be assigned a different meaning in another build.

Assumptions & limits

• This source expression was recovered from the pinned code revision. Its presence does not establish that the associated program was built, run, or scientifically validated.

Original context

 * @brief TD error: δ = r + γV(s') - V(s)
 */
inline double td_error(double reward, double value_next, double value_current, double gamma_discount = 0.99) {
    return reward + gamma_discount * value_next - value_current;
}

/**
 * @brief Policy gradient: ∇J ≈ Σ(∇log π(a|s)·R)
 */
inline double policy_gradient(double action_prob, double advantage) {
    if (action_prob < 1e-10) return 0.0;
    return advantage / action_prob;
}

// ═══════════════════════════════════════════════════════════════════════════════
// VI. ASSOCIATIVE MEMORY (HOPFIELD-LIKE)
// ═══════════════════════════════════════════════════════════════════════════════

@brief Policy gradient: ∇J ≈ Σ(∇log π(a|s)·R)

Related builds