Read the equation
The pinned analysis uses the effective LoRA scaling coefficient s=64. This coefficient multiplies the factor outer product and therefore multiplies its norm by 64 and a shared-scale inner product by 4096. The correct value comes from the pinned adapter configuration and its scaling convention.
Terms & definitions
| Term | Meaning in this source |
|---|---|
| s | Effective positive adapter scale |
How it works
Read the definition from its right-hand expression to the quantity on the left. Its surrounding source records the intended input, state, or parameter context. Where the source leaves an operation unspecified, this page preserves that gap.
Assumptions & limits
Source recovery establishes that the expression appears in this record. Its mathematical validity, implementation behavior, and interpretation must be assessed separately. See the linked source and research notes for the checks available for this version.
Original context
Read the surrounding source passage
The analyzed repository is ModelOrganismsForEM/Qwen2.5-14B-Instruct_R1_0_1_0_extended_train. Every source download used revision 1551d514e5977637d5a6fd5198edef4d1d6b80e4, rather than an unpinned branch name. The complete acquisition pass retrieved 336 files: 167 adapter-weight files, 167 adapter configurations, and trainer-state JSON files at checkpoints 300 and 792. There were no failed files in that pass. The checkpoint grid consists of steps 1–10, then every fifth step from 15 through 790, plus step 792. The regular grid used for principal-components analysis comprises 158 checkpoints at steps 5, 10, …, 790. Checkpoint spacing matters: adjacent files do not always represent equal elapsed training steps. No missing intermediate weight snapshot was interpolated and called an observation. All 167 configurations are identical. The released tensors and configuration identify the MLP down-projection at layer index 21, rank one, LoRA alpha 64, and rank-stabilized scaling enabled. The input-side factor has shape 1,13824; the output-side factor has shape 5120,1. Both use float32 source storage. DoRA and transposed weight orientation are disabled, and there are no per-layer rank or alpha overrides. These checks justify the effective scaling s=64 for this particular release. [Context excerpt; full source linked.]
Exact original notation
s=64