Layer five to six: logistic scalar, not a one-element softmax

Read the equation

The body returns exp(z)/(1+exp(z)) with z=wP+b, which is a logistic sigmoid. A softmax over a single scalar would always return one; a two-logit softmax over [z,0] would reproduce this value. Direct exponentiation can overflow for very large positive z, unlike a stable sigmoid implementation.

Terms & definitions

How it works

Read the definition from its right-hand expression to the quantity on the left. Its surrounding source records the intended input, state, or parameter context. Where the source leaves an operation unspecified, this page preserves that gap.

Assumptions & limits

Source recovery establishes that the expression appears in this record. Its mathematical validity, implementation behavior, and interpretation must be assessed separately. See the linked source and research notes for the checks available for this version.

Original context

    inline double L4_to_L5(const std::complex<double>& sync, double w = 1.5) {
        double mag = std::abs(sync);
        double phase = std::arg(sync) / PI;
        double base = math::sigmoid(w * mag);
        return base * (1.0 + 0.2 * std::sin(phase * PI));
    }

    // T₅₆: D = softmax(W·P + b)
    inline double L5_to_L6(double predictive, double w = 2.0, double b = 0.0) {
        double z = w * predictive + b;
        return std::exp(z) / (1.0 + std::exp(z));
    }

    // T₆₇: Ψ = ∫D·exp(-F)dt
    inline double L6_to_L7(double decision, double free_energy,
                           double psi_prev, double dt = 0.1) {
        double decay = std::exp(-free_energy);
        double increment = decision * decay * dt;

T₅₆: D = softmax(W·P + b)

Related builds