ThatAIguyA world of ideas

THE CONSCIOUSNESS ATLAS
Read. Experiment. Follow the evidence.

ConsciousnessLearn · Explore · Joseph Anady

The equation library

Every recovered mathematical record has an individual page. Browse original equations, code expressions, indexed components, parameter definitions, and explicitly unresolved source slots.

2,910 recordsSource snapshot 2026-10-17

Counts refer to mathematical records, including implemented expressions and generated instances. They are not counts of independently validated theories. Exact occurrences and unresolved slots remain traceable. Read the complete 1–516 source index ↗ · Inspect the scan decisions ↗

2,910 records · page 4 of 59Search the source archive ↗
donor-deepseek-attention-scale

Square the YaRN multiplier in the attention coefficient

The baseline coefficient is 1/√192. Multiplying it by the squared magnitude adjustment yields(1+0.1 ln 40)²/√192≈0.1352337788608801, a factor 1.8738542071 above the baseline. The dossier’s 0.135237 is an approximate decimal. In this configuration the sine/cosine cache’s separate ratio of equal…

Donor architecture mathematicshistorical source recovered; expression not executed in this website audit
softmax_scale = 192^-0.5 * mscale * mscale = 0.0721688 * 1.3688879^2 = 0.135237 This is NOT 1/sqrt(192) = 0.0721688. The 1.874x factor is load-bearing.
donor-eq-3bae7fda6dfd0c

Expand the compressed latent into per-head keys and values

Normalize the compressed key/value latent, apply the learned up-projection, and reshape into heads. Each head receives its non-rotary key and value components. This eagerly reconstructs the per-head representation; it is different from an absorbed attention path that contracts projection weights…

Donor model implementationhistorical source recovered; expression not executed in this website audit
kv = (self.kv_b_proj(self.kv_a_layernorm(compressed_kv)).view( bsz, q_len, self.num_heads, self.qk_nope_head_dim + self.v_head_dim).transpose(1, 2))
donor-eq-7ee1a616cbd818

Float32 normalization of vision-attention logits

Softmax across the final key-position axis turns each query’s masked logits into relative nonnegative weights. The source calculates the normalization in float32 and casts its result to the query dtype. At least one finite eligible key is required per row.

Donor model implementationhistorical source recovered; expression not executed in this website audit
attn_weight = torch.softmax(attn_weight, dim=-1, dtype=torch.float32).to(q.dtype)
donor-eq-82dd00d784729e

Normalize each query’s attention scores

Apply softmax across key positions for each query and head. The resulting nonnegative weights sum to one when a row has at least one finite eligible score. The source computes this normalization in float32 and then converts it to the query dtype; its eager implementation incorporates a mask…

Donor model implementationhistorical source recovered; expression not executed in this website audit
attn_weights = nn.functional.softmax(attn_weights, dim=-1, dtype=torch.float32).to( query_states.dtype)
donor-eq-36acc350e692bd

Ratio controlling the rotary cache magnitude

This coefficient divides the adjustment computed with mscale by the adjustment computed with mscale_all_dim. Equal strengths give a ratio of one. The attention initializer can still multiply its own coefficient by a squared adjustment, so an unchanged rotary-cache magnitude does not imply unchanged…

Donor model implementationhistorical source recovered; expression not executed in this website audit
_mscale = float( yarn_get_mscale(self.scaling_factor, self.mscale) / yarn_get_mscale(self.scaling_factor, self.mscale_all_dim))
donor-eq-5abd779cdef42d

Sum the selected expert outputs with their routing weights

Each chosen expert evaluates the same input representation h. Its output vector is multiplied by the associated routing weight, and the selected vectors are added. The implementation groups tokens by expert for efficient evaluation and restores their original order before summing; those…

Donor model implementationhistorical source recovered; expression not executed in this website audit
final_out = (new_x.view( *topk_ids.shape, -1).type(topk_weight.dtype).mul_( topk_weight.unsqueeze(dim=-1)).sum(dim=1).type(new_x.dtype))

Search equations, builds, manuscript text, and recovered sources.