What the August dossiers preserve
Two recovered text dossiers describe the project’s donor-model work as of 5 August 2026. The Kimi document contains configuration, a tensor-pattern inventory, vision and language-model code, and local integer-computation hooks. The DeepSeek document records a reported installation and header scan, architecture arithmetic, tensor layouts, and embedded reference code. These are substantial historical source records. Their headers are reports of work on other machines; the present archive scan did not fetch the weight shards or rerun inference on those machines.
This generation connects external pretrained models to the project’s own runtime experiments. The donor model’s published architecture, the local modifications, the installation report, and measured behavior are separate forms of evidence. A model card’s capability result does not become a measured result for the local runtime merely because its source was copied into a dossier.
Kimi: native quantization and local exact-integer hooks
The Kimi overview says there is no quantization configuration, but the embedded text_config contains a compressed-tensors configuration with four-bit grouped weights and group size thirty-two. The official model card also describes native INT4 quantization. The dossier’s broad BF16 description therefore needs qualification: its displayed inventory mixes packed integer weights, floating-point scales, and unquantized components.
The embedded DeepseekV3MLP contains local use_exact_generators, PackedInt4Linear, ExactLinear, and conversion hooks. These are concrete project modifications even though the dossier header calls the source official. Their referenced exact_generators module is listed in the directory inventory but is not reproduced as a file section in this dossier. The nearby claim of approximately 950-fold greater precision is a source assertion without a corresponding accuracy experiment in these recovered sections.
An integer multiply-and-accumulate can be exact relative to its quantized integer inputs when overflow is excluded. That fact alone does not remove activation quantization, weight quantization, rescaling, nonlinear-function, or surrounding floating-point error. The website therefore explains the stated arithmetic mechanism without turning exact accumulation into a claim of exact real-valued end-to-end inference.
Read tensor shapes against the actual transformation
The Kimi patch merger concatenates a two-by-two group of 1,152-dimensional patch features into 4,608 components, then projects toward the 7,168-dimensional language residual stream. Its code defines one 4,608-to-4,608 layer and one 4,608-to-7,168 layer. The dossier’s pattern table instead labels both projection weights as 4,608-by-4,608. Those two descriptions cannot both describe that constructor with the displayed configuration.
This is a bounded source inconsistency, not a reconstruction of the original weight shard. A corrected tensor census would require comparing each actual tensor key and header with its model constructor. The recovered dossier retains both statements so the disagreement is inspectable.
DeepSeek: latent attention does not determine one cache layout
The DeepSeek overview attributes a 576-element latent cache to the embedded reference. Its eager attention body actually expands keys and values first and sends those reconstructed tensors to the cache. With 128 heads, 192 key components, 128 value components, and two bytes per component, the payload is 81,920 bytes per token per layer. Its FlashAttention path pads values to 192 components before caching, giving 98,304 bytes under the same dtype assumption.
A latent representation of 512 key/value components plus 64 rotary components needs 1,152 bytes at two bytes per component. That is a useful architecture-level comparison, but it describes an optimized representation rather than the embedded eager cache. The official DeepSeek inference source explicitly provides separate naive and absorbed paths, confirming that the implementation choice matters.
The absorbed path can contract the non-rotary query with the projection weights and aggregate latent values before the final value projection. Consequently, latent caching does not inherently require reconstructing every stored token’s full key and value vectors on each decoding step. Cache-size and compute claims must identify the chosen path.
| Displayed representation | Assumed bytes per token per layer |
|---|---|
| Embedded eager K/V, BF16 | 81,920 |
| Embedded FlashAttention K/V with padded values, BF16 | 98,304 |
| Optimized 512+64 latent representation, two bytes per component | 1,152 |
Routing, FP8 scales, and the orientation of the YaRN ramp
The DeepSeek router’s distinction between selection and weighting is supported by its embedded code: the bias-adjusted sigmoid scores choose groups and experts, while the selected pre-bias sigmoid scores determine weights. Those weights are normalized and multiplied by the routed scaling factor before the shared expert contribution is added. This is a specific routing algorithm, not an ordinary softmax over all experts.
Its displayed FP8 relation multiplies each decoded weight by the scale for its 128-by-128 block. The official dequantization kernel supports that direction despite the companion tensor’s weight_scale_inv name. Shape and division expressions on these pages identify floor indexing for a block coordinate and ceiling division for a partial boundary block.
The YaRN prose formula reverses the increasing linear ramp relative to the embedded implementation. If r is yarn_linear_ramp_mask, the code uses interpolationr + extrapolation(1−r). This preserves the higher-frequency end and scales the lower-frequency end. The source’s magnitude correction is mathematically explicit: 1+0.1 ln(40)=1.3688879454, and the resulting attention multiplier is (1+0.1 ln(40))²/√192=0.1352337789. These are calculations from the displayed constants; they are not new model outputs.