================================================================================ D E E P S E E K R E Q U I R E M E N T S Full itemized requirements, fulfillment status, and inlined source files for making DeepSeek-R1 the central cognitive engine of MEGAMIND-AGI / PANTHEON ================================================================================ Compiled 2026-08-05, live from MEGAMIND-AGI (100.69.120.61 / WSL2) and the M4 mac-mini (100.94.8.94, kimi-blanket) Companion DEEPSEEK_R1_COMPLETE.txt on this Desktop — the architecture report Scope every item in the 12-section request, itemized, with a verified status and, where the item is a file, the entire file inlined -------------------------------------------------------------------------------- HOW TO READ THIS DOCUMENT -------------------------------------------------------------------------------- [DONE] produced and verified this session; the artifact exists [INLINED] the complete file text appears in PART C of this document [PACKAGED] built into a tar.gz on the source machine; path given [ABSENT] checked and confirmed not to exist — this is an answer, not a gap [BLOCKED] cannot be produced yet; the blocker is named and explained [ACTION] requires a human decision or a step outside this session Every status claim below was established by a command run against the live machines. Where something was checked and found missing, the check is described so it can be repeated. ================================================================================ EXECUTIVE SUMMARY — WHAT EXISTS RIGHT NOW ================================================================================ BUILT AND VERIFIED THIS SESSION * DeepSeek-R1 source bundle — all 10 requested files, copied, no weight shards * 6 scanner/benchmark scripts, cleaned up and documented * deepseek_tensor_manifest.json 91,991 entries, 28,923,014 B * deepseek_shard_headers.json 163 entries, 40,263 B * deepseek_layer_shard_map.json both directions, 19,562 B * deepseek_expert_offset_map.json 15,104 experts, 7,042,163 B * deepseek_weight_hashes.txt partial hashes, 163 shards, 63,906 B * 4 upload packages, secret-scanned * sanitized environment report FIVE FINDINGS THAT CHANGE THE REQUEST ITSELF 1. THERE IS NO GO CODE. The request anticipates main.go, dynamical_reasoner.go, wave_substrate.go, go.mod, go.sum. None exist. `find` for *.go and go.mod across /opt/mm and /home/joseph returns nothing. MEGAMIND is C++ (120 .cpp, 53 .h, one hand-written Makefile) plus Python (organs and loops). Any plan that assumes a Go service mesh needs rewriting before it starts. 2. THERE IS NO MCP IMPLEMENTATION. Section 6 asks for the code behind /mcp, /mcp/tools, /mcp/execute, /mcp/sse, /mcp/manifest. Those routes appear to answer HTTP 200, but that is ORACLE's catch-all: it returns 200 with an error body for *every* unknown path, including /this-route-does-not-exist. Grep for modelcontextprotocol, jsonrpc, tools/list and tools/call across the whole tree returns nothing. The earlier grep hits for "mcp" were the letters inside `memcpy`. MCP would be new construction, not a port. 3. THERE ARE NO GIT REPOSITORIES. /opt/mm, ~/mmk, ~/mmreason, ~/oracle and ~/megamind all lack .git. Section 12's GIT_STATUS.txt, GIT_COMMIT.txt and GIT_RECENT_LOG.txt cannot be produced for any of them. Version history is carried as ~90 hand-copied .bak files instead (oracle.cpp.bak.sefS1 through sefS16, eye.cpp.bak.p28 through p40, and so on). 4. NO OFFICIAL DEEPSEEK INFERENCE RUNTIME IS PRESENT. Section 3's question is answered definitively: no inference/, kernel.py, model.py, generate.py, fp8_cast_bf16.py or convert.py anywhere on the machine, and no torch, triton, vllm, sglang, transformers or safetensors in pip. The FP8 E4M3 backend must be built from the checkpoint format, and the three-way oracle needs an externally-sourced reference. 5. FIXTURES CANNOT BE GENERATED YET. Section 9 wants frozen reference outputs. Producing them requires something that can execute the model — which is exactly what does not exist (finding 4). This is a genuine ordering dependency, not an oversight: fixtures come *after* a reference runtime. What can be produced now without a runtime is listed in PART B section 9. ONE SECURITY ITEM REQUIRING YOUR DECISION [REDACTED: credential-associated original line 79; original retained outside this bundle.] as a string literal. It is a read-only role on localhost, but the request says [REDACTED: credential-associated original line 81; original retained outside this bundle.] did NOT edit loom.cpp, because the request also says not to clean the source before sending — editing it would hide a real dependency. See PART E. ================================================================================ PART A — STATUS OF EVERY REQUESTED ITEM ================================================================================ § ITEM STATUS WHERE --------------------------------------------------------------------------- 1 config.json DONE/INLINED C.1 1 configuration_deepseek.py DONE/INLINED C.2 1 modeling_deepseek.py DONE/INLINED C.3 1 generation_config.json DONE/INLINED C.4 1 tokenizer_config.json DONE/INLINED C.5 1 tokenizer.json DONE sidecar, see C.6 1 model.safetensors.index.json DONE sidecar, see C.7 1 README.md DONE/INLINED C.8 1 LICENSE DONE/INLINED C.9 1 .gitattributes DONE/INLINED C.10 1 download script DONE/INLINED C.11.1-3 1 header scanner DONE/INLINED C.11.4 1 tensor census producer DONE/INLINED C.11.5 1 SSD benchmark DONE/INLINED C.11.6 1 layer->shard mapper DONE/INLINED C.11.7 1 integrity verifier DONE/INLINED C.11.8 --------------------------------------------------------------------------- 2 deepseek_tensor_manifest.json DONE B.2 2 deepseek_shard_headers.json DONE B.2 2 deepseek_layer_shard_map.json DONE B.2 2 deepseek_expert_offset_map.json DONE B.2 2 deepseek_weight_hashes.txt (partial) DONE B.2 2 deepseek_weight_hashes.txt (full SHA-256) ACTION B.2, ~22 min run --------------------------------------------------------------------------- 3 official DeepSeek inference source ABSENT B.3 --------------------------------------------------------------------------- 4 MEGAMIND runtime source PACKAGED B.4 4 go.mod / go.sum / *.go ABSENT B.4 finding 1 4 Makefile PACKAGED B.4 4 CMakeLists / Dockerfile / compose / npm ABSENT B.4 4 systemd units DONE/INLINED C.13 4 startup scripts PACKAGED B.4 --------------------------------------------------------------------------- 5 ORACLE source PACKAGED B.5 5 organ registration / coalition / trust PACKAGED B.5 5 VSA signature system PACKAGED B.5 5 NATS heartbeat code PACKAGED B.5 5 signature format specification PARTIAL B.5, from Kimi doc 5 trust score formula BLOCKED B.5, in binary only 5 schemas (organ/coalition/capability) PARTIAL B.5, observed 5 NATS subject list DONE B.5 --------------------------------------------------------------------------- 6 MCP implementation ABSENT B.6 finding 2 6 tool registry / schemas / executors ABSENT B.6 --------------------------------------------------------------------------- 7 memory + learning schemas DONE B.7 7 database schema DONE B.7 7 sanitized sample records ACTION B.7 --------------------------------------------------------------------------- 8 Kimi implementation files (21 requested) 19 PRESENT B.8 8 exact_generators.py FOUND elsewhere B.8 8 cell_server.py ABSENT B.8 8 build commands / flags / tests / logs PACKAGED B.8 --------------------------------------------------------------------------- 9 tokenizer fixtures BLOCKED B.9 9 FP8 fixtures BLOCKED B.9 9 router fixtures BLOCKED B.9 9 MLA fixtures BLOCKED B.9 9 full-layer fixtures BLOCKED B.9 --------------------------------------------------------------------------- 10 API and network contracts PARTIAL B.10 --------------------------------------------------------------------------- 11 /etc/wsl.conf DONE/INLINED C.12.1 11 .wslconfig DONE/INLINED C.12.2 11 systemd unit files DONE/INLINED C.13 11 sanitized environment report DONE/INLINED C.14 11 disk benchmark script + results DONE/INLINED C.11.6, B.11 --------------------------------------------------------------------------- 12 FILE_MANIFEST.txt per repository DONE B.12 12 GIT_STATUS / GIT_COMMIT / GIT_RECENT_LOG ABSENT B.12 finding 3 --------------------------------------------------------------------------- ================================================================================ PART B — EVERY SECTION, ITEMIZED, WITH FINDINGS ================================================================================ -------------------------------------------------------------------------------- B.1 DEEPSEEK-R1 SOURCE BUNDLE -------------------------------------------------------------------------------- REQUESTED, from /home/joseph/DeepSeek-R1/: config.json, configuration_deepseek.py, modeling_deepseek.py, generation_config.json, tokenizer_config.json, tokenizer.json, model.safetensors.index.json, README.md, LICENSE, .gitattributes plus every script used to download, scan headers, produce the census, benchmark the SSD, map layers to shards, and verify integrity. EXPLICITLY EXCLUDED, and honored: model-00001-of-000163.safetensors ... model-00163-of-000163.safetensors .cache/huggingface/download/ All 163 weight shards remain exactly where they are. Nothing was moved, renamed, copied or deleted. The scanners read headers only. STATUS: [DONE]. All ten files copied to /home/joseph/deepseek-megamind-upload/deepseek/ and packaged as 01-deepseek-source.tar.gz (4,349,260 bytes). file bytes inlined at --------------------------------------------------------------------------- config.json 1,660 C.1 configuration_deepseek.py 9,897 C.2 modeling_deepseek.py 75,741 C.3 generation_config.json 171 C.4 tokenizer_config.json 3,594 C.5 tokenizer.json 7,847,602 C.6 (sidecar — see note) model.safetensors.index.json 8,898,324 C.7 (sidecar — see note) README.md 15,994 C.8 LICENSE 1,064 C.9 .gitattributes 1,519 C.10 --------------------------------------------------------------------------- NOTE ON THE TWO LARGE FILES. tokenizer.json and model.safetensors.index.json total 16.75 MB of machine-generated JSON — 127,741 BPE merge rules and 91,991 name->shard pairs. Pasting them into this document would add roughly 16.75 MB of unreadable text and make the document unusable in any editor or context window, without adding information a human or model can act on. They are therefore delivered as byte-exact sidecar files rather than inline. Their location, size, SHA-256 and structural specification are in C.6 and C.7, and both are inside 01-deepseek-source.tar.gz. Every other requested file is inlined complete. If you want them inline anyway, say so and I will append them verbatim. SCANNERS PRODUCED — all six categories requested, plus the manifest generators: download_r1.py snapshot_download driver, resumable C.11.1 launch_r1.sh systemd-run wrapper (WSL kills bg jobs) C.11.2 dl_test.py single-file connectivity probe C.11.3 scan_headers.py header scan -> dtype/family census C.11.4 per_layer_manifest.py complete per-layer tensor manifest C.11.5 bench_ssd.sh SSD diagnosis: serial/parallel/scattered C.11.6 map_shards_to_layers.py shard <-> layer mapping C.11.7 verify_integrity.py header/index/length cross-check C.11.8 check_expert_contiguity.py on-disk expert layout prover C.11.9 gen_manifests.py the five section-2 manifests C.11.10 gen_hashes.py partial and full SHA-256 passes C.11.11 -------------------------------------------------------------------------------- B.2 COMPLETE TENSOR-HEADER MANIFEST -------------------------------------------------------------------------------- STATUS: [DONE]. Generated by gen_manifests.py (inlined at C.11.10), which reads only the 163 JSON headers — about 13 MB of the 643 GB checkpoint. deepseek_tensor_manifest.json 28,923,014 B 91,991 entries --------------------------------------------------------------------------- Exactly the requested schema, plus two additions marked below: { "name": "model.layers.3.mlp.experts.0.gate_proj.weight", "shard": "model-00002-of-000163.safetensors", "dtype": "F8_E4M3", "shape": [2048, 7168], "absolute_offset": 1234567, "byte_length": 14680064, "relative_offset": 1158517, <- ADDED: offset within the data region "scale_tensor": "model.layers.3.mlp.experts.0.gate_proj.weight_scale_inv" } absolute_offset is the true file offset: 8 + header_len + relative_offset. Seek there and read byte_length bytes and you have the tensor, with no further parsing. relative_offset is retained because the safetensors header states offsets that way, so the two can be cross-checked. scale_tensor is null for every BF16 and F32 tensor and is populated for all 45,808 F8_E4M3 weights. deepseek_shard_headers.json 40,263 B 163 entries --------------------------------------------------------------------------- {"shard","file_size","header_len","data_start","tensor_count", "payload_bytes","metadata","length_check_ok"} length_check_ok is true for all 163: data_start + payload_bytes == file_size. deepseek_layer_shard_map.json 19,562 B --------------------------------------------------------------------------- layer_to_shards "0".."61" -> [shard filenames] shard_to_layers shard filename -> [layer indices] global_tensors_by_shard shard -> [embed_tokens / norm / lm_head] notes dense [0,1,2], moe [3..60], mtp 61, base_inference_layers [0..60] deepseek_expert_offset_map.json 7,042,163 B 15,104 experts --------------------------------------------------------------------------- keyed "layer/expert", e.g. "3/0": {"layer":3,"expert":0,"total_bytes":44050944,"read_count":1, "reads":[{"shard":...,"offset":...,"length":44050944, "contiguous":true, "tensors":["gate_proj.weight","gate_proj.weight_scale_inv", "up_proj.weight","up_proj.weight_scale_inv", "down_proj.weight","down_proj.weight_scale_inv"]}]} MEASURED RESULT: 15,104 experts total. 14,951 need exactly ONE read of 44,050,944 contiguous bytes. 153 straddle a shard boundary and carry two read descriptors. This map is the whole expert I/O plan — a streaming loader needs nothing else to fetch an expert. deepseek_weight_hashes.txt 63,906 B 163 stanzas --------------------------------------------------------------------------- Currently PARTIAL, as the request permits: per shard, the file size plus SHA-256 of the first 1 MiB, the 1 MiB at the midpoint, and the last 1 MiB. Reads 489 MiB total, completed in 2.1 seconds. Detects truncation, a swapped shard, and head/tail corruption. [ACTION] Full SHA-256 over all 643 GB is a one-command run: python3 ~/deepseek-megamind-upload/deepseek/scanners/gen_hashes.py --full At the measured 507 MB/s single-stream rate that is roughly 22 minutes of I/O plus hashing. It has not been run because it is a 22-minute serial read of the drive the node is also serving from. Run it before any final verification claim, as the request specifies. -------------------------------------------------------------------------------- B.3 OFFICIAL DEEPSEEK INFERENCE SOURCE -------------------------------------------------------------------------------- STATUS: [ABSENT] — checked thoroughly, definitively not present. Searched /home/joseph, /opt/mm and /mnt/c/Users/Josep to depth 4 for: inference/ kernel.py model.py generate.py fp8_cast_bf16.py convert.py configs/ requirements.txt Result: no matches. The only DeepSeek-related paths on the machine are: /home/joseph/DeepSeek-R1 the checkpoint /home/joseph/DeepSeek-R1/configuration_deepseek.py reference config /home/joseph/DeepSeek-R1/modeling_deepseek.py reference architecture /home/joseph/.cache/huggingface/hub/models--deepseek-ai--DeepSeek-R1 HF download cache pip3 list contains no torch, triton, vllm, sglang, transformers or safetensors. The full package list is two entries: huggingface_hub 1.26.0 and numpy 2.5.1. CONSEQUENCE, stated plainly. The FP8 E4M3 128x128 block backend must be built from the checkpoint format, and there is no local reference to check it against. The shipped modeling_deepseek.py assumes weights are already dequantized to bf16 — its only quantization-aware lines are a `_pre_quantization_dtype` guard at 966-968. The official model card says "Hugging Face's Transformers has not been directly supported yet." To close this, one of the following must be fetched from outside this machine: github.com/deepseek-ai/DeepSeek-V3 inference/kernel.py, fp8 cast vLLM fp8 block-quant kernels SGLang fp8 block-quant kernels Pin whichever you choose to an exact commit hash and record it, because it becomes the comparator that every later correctness claim rests on. -------------------------------------------------------------------------------- B.4 MEGAMIND-AGI SOURCE -------------------------------------------------------------------------------- STATUS: [PACKAGED] 02-megamind-source.tar.gz, 5,235,523 bytes. *** THE REQUEST'S FILE LIST DOES NOT MATCH THIS SYSTEM. *** Requested: main.go, dynamical_reasoner.go, wave_substrate.go, cmd/, agent/, federation/, oracle/, memory/, learning/, ingestion/, mcp/, tools/, routes/, api/, config/, go.mod, go.sum, Dockerfile, docker-compose.yml, package.json. MEASURED REALITY: *.go files anywhere ................... 0 go.mod / go.sum ....................... 0 CMakeLists.txt / Dockerfile ........... 0 docker-compose.yml / package.json ..... 0 Makefile .............................. 1 (/opt/mm/src/Makefile) requirements.txt ...................... 0 MEGAMIND is C++ compiled with clang -O3 -march=native, plus Python organs. There is no module system, no container, and no dependency manifest. The Makefile uses `.RECIPEPREFIX = >` rather than tabs. ACTUAL SOURCE INVENTORY (excluding the vendored pyembed tree): 188 source files, 2,024,267 bytes total 120 .cpp | 53 .h | (plus 1,926 .py, of which most are vendored) | 24 .sh THE TWENTY LARGEST SOURCE FILES 582,800 /opt/mm/src/eye.cpp Living World engine 115,620 /opt/mm/src/oracle.cpp ORACLE core 53,348 /opt/mm/src/oracle_orchestrator.h 51,928 /opt/mm/src/mmpredict.cpp 43,212 /opt/mm/src/loom.cpp orchestration layer 34,804 /opt/mm/src/oracle_frontier.h 26,441 /opt/mm/code/src/mmscheme.cpp Scheme interpreter organ 25,887 /opt/mm/code/src/mmc.cpp C interpreter organ 24,780 /opt/mm/src/mmserve.cpp 512-bit recall service 24,239 /opt/mm/code/src/mmpy.cpp Python interpreter organ 20,761 /opt/mm/src/mmgame.cpp 17,721 /opt/mm/src/seferim_spec.h 17,018 /opt/mm/src/seferim_repl.h 16,535 /opt/mm/src/seferim_book.h 15,019 /opt/mm/src/seferim_degrade.h 14,431 /opt/mm/src/seferim_social.h 14,413 /opt/mm/src/seferim_plan.h 14,337 /opt/mm/src/seferim_trust.h 14,313 /opt/mm/code/src/mmparse.cpp 14,188 /opt/mm/src/seferim_api.h MAPPING THE REQUESTED CONCERNS ONTO REAL FILES main runtime and startup ....... mmboot.cpp, mm-boot.service, mm.target reasoning engine ............... oracle.cpp, mmreason.cpp, reason*.cpp, seferim_*.h (25 headers) agent / autonomous loop ........ agent.cpp, swarmd.cpp, magent.cpp, loops/{forward,backward,mmpulse, mmthought,mmhelix}.py task execution ................. oracle_orchestrator.h (orchestrate, execute, coalition) memory and recall .............. mmserve.cpp, hdcrecall.cpp, sdm.cpp, seferim_mem.h knowledge ingestion ............ hdctrain.cpp, hdcembed.cpp, semgrow*.cpp, grow.cpp, embgrow.cpp model routing .................. oracle.cpp organ registry + priority federation connections ......... mmshard.cpp, fedmon.cpp, shard.cpp NATS messaging ................. mmheart.cpp, mmsub.cpp, mmcap.cpp (raw C++ NATS client, no library) MCP tools ...................... DOES NOT EXIST — see B.6 API endpoints .................. each organ serves its own HTTP; mmgateway_web.py is the read-only gateway authentication ................. seferim_sec.h, mma5_sectest.cpp state persistence .............. oracle.wal (WAL), PostgreSQL 18 logging and metrics ............ mmstatus.cpp, mmeval.cpp, mmgate.cpp configuration .................. systemd unit Environment= lines, /etc/mm/mm.env (mode 600, NOT shipped) PACKAGE CONTENTS: src/, code/src/, reason/*.py, loops/ EXCLUDED: *.pyc, __pycache__, *.log, *.db, *.wal*, *.jsonl, *.tsv, *.u8, *.env, *.key, *.pem, and every compiled binary in bin/ The ~90 .bak files ARE included deliberately — with no git history they are the only record of how the code evolved, and the request asked not to clean. -------------------------------------------------------------------------------- B.5 PANTHEON AND ORACLE SOURCE -------------------------------------------------------------------------------- STATUS: [PACKAGED] 03-pantheon-source.tar.gz, 215,585 bytes. CONTENTS oracle.cpp 115,620 B task routing, registry, coalitions oracle_orchestrator.h 53,348 B orchestrate / execute / coalition oracle_frontier.h 34,804 B frontier expansion seferim_*.h 25 headers: abstract align api book bridge compose core degrade explore gate homeo learn mem meta plan predict repl sec social spec trust valgate mmshard.cpp fedmon.cpp federation / NATS mesh shard responder mmheart.cpp mmsub.cpp raw C++ NATS client and heartbeat mmcap.cpp mmcap2.cpp capability manifest publisher mmsem.cpp mmserve.cpp semantic embedder, 512-bit recall mmembed.cpp mmembed2.cpp hdcembed.cpp binary embedders mmgate.cpp mmeval.cpp mmstatus.cpp safety gate, eval, health mmworld.cpp mmreason.cpp reason/*.py mmask, mmknow, mmlearn, mmreason, mmthink, mmgateway_web loops/*.py forward, backward, mmpulse, mmthought, mmhelix ORACLE'S LIVE POWER LIST, read from :9402/oracle/health this session: recall inverse hop analogy similar learn path set consistent facts ask unlearn revise counterfactual induce predict abduce factor map gardener grow growstress shards budstress shard-spawn shard-add fedput fedget fedls spawnstress compose synth ground plan nextgoal metalearn safegate library sensorimotor negotiate culture sign ... (truncated by response limit) Live counts: 508,093 entities | 34 relations | 773,235 edges ROUTE SURFACE, from ORACLE's own error banner: /oracle/{health,recall,inverse,hop,analogy,similar,facts,path,set, consistent,learn,ask,unlearn,revise,counterfactual,induce, predict,abduce,factor,map,organs,register,orchestrate,orchstress, execute,execstress,coalition,coalstress} *** /oracle/audit and /oracle/connect, named in the request, are NOT in this list. They do not exist. Note also that ORACLE returns HTTP 200 for every unknown route with a JSON error body, so status codes cannot be used to probe for endpoints — you must read the body. NATS SUBJECTS IN USE mesh.health. heartbeat, mm-heart.timer every 15 s mesh.registry. capability manifest, mm-cap.timer every 60 s mesh.query.recall federated recall fan-out (mmshard responder) Hub: nats://100.90.97.104:4222 (Bubbles). WSL2 reaches the tailnet via the 172.31.112.1 gateway. BINARY VSA / SIGNATURE SYSTEM — what is known and what is not [KNOWN, measured this session] mmserve 24,364,580 signatures, vocab 38,085, index "mih16x32p" Substrate 512-bit signatures, bit(512) columns in PostgreSQL Distance XOR + POPCNT Hamming, hardware popcount Eye organ 2,048-bit BSC (binary spatter code), separate dimension Schema knowledge(sigbin bit(512), semsig bit(512)) sigbin = content SimHash, semsig = HDC semantic signature [PARTIAL] The full signature format specification — exact bit order, domain/version handling, projection seed — is written down only for the Kimi vector organ, in ~/kimi-blanket/docs/kimi_vector_organ_design.md on the M4 (included in 04-kimi-reference.tar.gz). That document specifies: 512 bits / 64 raw bytes / MSB-first / XOR-popcount Hamming, domains "mmbsig/textHV/v1" (lexical) and "mmbsig/kimi-k2.5-hidden/v1" (neural), with the hard rule that cross-domain Hamming distance is meaningless. The C++ side's equivalent spec is implicit in hdcembed.cpp and mmsem.cpp and has never been written out. [BLOCKED] The trust score formula and the PASS adjudication logic are inside oracle.cpp and seferim_trust.h. Both are shipped as source, so the formula is recoverable by reading them — but there is no written specification, and I did not reverse-engineer it in this session. Ask for it specifically and I will extract and document it from the source. SCHEMAS: organ request/response and coalition result schemas are OBSERVED rather than specified — there is no schema file anywhere. What the live organs return is captured in DEEPSEEK_R1_COMPLETE.txt and in B.10 below. Treat those as the de facto contract; nothing declares them formally. -------------------------------------------------------------------------------- B.6 MEGAMIND'S TOOL SYSTEM -------------------------------------------------------------------------------- STATUS: [ABSENT]. There is no MCP implementation and no tool registry. HOW THIS WAS ESTABLISHED 1. Probed /mcp, /mcp/tools, /mcp/manifest, /mcp/execute on ports 9402, 9560, 9543 and 9540. All returned HTTP 200 on ORACLE. 2. Read the bodies. Every one returned ORACLE's standard error banner listing the /oracle/* routes — the catch-all, not an MCP response. 3. Confirmed by probing /this-route-does-not-exist-xyz, which also returns HTTP 200. ORACLE answers 200 for everything. 4. Grepped the whole tree for modelcontextprotocol, jsonrpc, tools/list and tools/call. Zero matches. 5. The earlier grep that matched "mcp" in seferim_api.h and seferim_sec.h was matching the letters inside `memcpy`. THEREFORE none of the following exist and none can be sent: tool registry, tool schemas, tool descriptions, argument validation, permission checks, execution handlers, result serialization, result verification, timeouts, retries, sandbox code, audit logging, /mcp/sse. The requested per-tool JSON record — name, description, input_schema, output_schema, permissions, executor, timeout, risk_level, verification — cannot be produced for any tool, because no tool is defined in that form. WHAT EXISTS INSTEAD, and is the honest starting point: * ORACLE's organ registry is the closest analogue to a tool registry. Each organ carries: name, url, mode (local|remote|federation|timer), caps (a free-text capability string), live, health (0|1|2), priority (8..80), builtin, sig16. Capability matching is done by scoring the caps text against the query — there is no typed schema, no argument validation, and no permission model. * seferim_sec.h and mma5_sectest.cpp hold the security primitives. * seferim_valgate.h is the validation gate and has 9 .bak generations, suggesting it is the most-revised part of the safety path. [REDACTED: credential-associated original line 526; original retained outside this bundle.] was BUILT, while registration auth, independent PASS judge, SSRF protection, tool-execution isolation, rate limits, audit log and replay protection were all still OPEN. If DeepSeek is to get tool-calling in its generation loop, that is new construction. R1's tokenizer already carries the wire format for it — <|tool_calls_begin|>, <|tool_call_begin|>, <|tool_sep|>, <|tool_call_end|>, <|tool_calls_end|> and the matching output block, ids 128806-128814 — so the model side is ready and the harness side is empty. -------------------------------------------------------------------------------- B.7 MEMORY AND LEARNING — SOURCE AND SCHEMAS -------------------------------------------------------------------------------- STATUS: [DONE] for schemas and code. Bulk data deliberately excluded. POSTGRESQL 18, TWO CLUSTERS 18-main port 5432, trust auth on local + 127.0.0.1 megamind 26 GB knowledge 24,364,578 rows, 8,836 MB nbin 32,404,944 rows, 17 GB sembond 3,418,102 rows, 360 MB game_memory 48 kB livingworld 3,337 MB events 14,355,878 | memories 928,608 | chronicle 147,248 codex 59 MB | entities 2,236 | nodes 3,271 | opinions 7,469 relationships 6,268 | edges | node_relhv | powers | places | goals stations | world | holdings | toh | finn_ledger | viewings | routes factions | realm_visit | world_history | dream_conditions | pattern_threads [REDACTED: credential-associated original line 555; original retained outside this bundle.] EXACT SCHEMAS, read with \d this session: knowledge id bigint not null default nextval('knowledge_id_seq') hash text body text source text modality text sigbin bit(512) content SimHash semsig bit(512) HDC semantic signature created_at timestamptz default now() access_count integer default 0 nbin id bigint not null default nextval('nbin_id_seq') hash text sigbin bit(512) semsig bit(512) sembond a, b, w associative edge with weight KEY AND VALUE FORMATS Signatures are 512-bit PostgreSQL bit strings. In C++ they are uint64 w[8]. Distance is XOR then hardware POPCNT. mmserve holds all 24.3M resident (4.88 GB RSS) and scans them; the index identifier is "mih16x32p". Vocabulary: hdc_vocab512.tsv, 43,137 tokens, frequency >= 8, two-pass build. mmserve reports vocab 38,085 live; mm-narrate reports 45,419. These differ per organ — do not assume one vocabulary. CODE (all in 02-megamind-source.tar.gz) read/write mmserve.cpp (/recall /spread /know /generate /health /reload) embedding hdcembed.cpp (train|embed|simhash), mmsem.cpp, mmembed*.cpp ingestion hdctrain.cpp, grow.cpp, embgrow.cpp, semgrow{,2,3}.cpp association sembond via sdm.cpp, stdp.cpp retention mma12_retention.cpp (bit-exact determinism check) parity mma16_parity.cpp (replica fidelity vs the M4) persistence oracle.wal, compacted by mm-wal-compact.timer at 04:17 MIGRATIONS: none. There is no migration framework and no versioned schema. Tables were created by hand. This is worth knowing before anything depends on schema evolution. [ACTION] SANITIZED SAMPLE RECORDS were not produced. The knowledge table holds 24.3M rows of mixed provenance — Wikipedia, arXiv, PubMed, code corpora, and an "mm-estate" organ whose declared scope is "own projects business clients joseph anady ... missionMATCH ATLAS PANTHEON federation bubbles hosting". That last one means the corpus contains business and client material. Sampling it safely requires deciding which sources are publishable, which is your call, not mine. Tell me which source values are safe and I will extract sanitized samples with that filter applied. -------------------------------------------------------------------------------- B.8 KIMI IMPLEMENTATION FILES -------------------------------------------------------------------------------- STATUS: [PACKAGED] /tmp/04-kimi-reference.tar.gz on the M4, 146,502 bytes. Source lives at /Users/joseph/kimi-blanket (1.5 GB total, 172 entries). OF THE 21 FILES REQUESTED PRESENT (19) kimi_gen.h 21,887 B the authoritative shared header kimi_int.c 9,379 B batched verification forward kimi_serve.c 17,290 B incremental KV serving kimi_tok.py 1,392 B tiktoken encoder, round-trip verified chat_format.py 5,446 B sampler.c 3,427 B prec_check.c 8,516 B compound_sim.py 3,280 B guardian_controller.py 1,215 B (still separate, as hoped) metal_mac.m 4,982 B M4 GPU int4 MAC, runtime-compiled Metal kimi_organ.py 4,323 B authenticated inference endpoint kimi_heart.py 1,752 B NATS pulse cell_client.py 1,250 B gen_manifest.py 2,393 B (legacy, superseded by the in-C parser) kimi_vision.c 8,361 B agents.c 7,411 B (synthetic prototype, not authoritative) dataflow.c 5,186 B (synthetic prototype, not authoritative) graph_build.py 3,534 B gen_driver.py 1,864 B (superseded by kimi_serve.c) FOUND, but not where the request expected exact_generators.py 10,516 B at /Volumes/MEGAMIND/kimi_k25/model/ (ExactLinear + PackedInt4Linear) modeling_deepseek.py 80,663 B at /Volumes/MEGAMIND/kimi_k25/model/ Kimi's OWN copy, with to_exact and convert_kimi_to_exact — note that Kimi K2.5 is DeepSeek-lineage, so this file is directly comparable to R1's modeling_deepseek.py 78,528 B a second copy at ~/kimi-blanket/kmod/ Both are packaged, under distinct names. ABSENT (1) cell_server.py — not found anywhere. cell_client.py exists without its server. The transport half of the distributed dataflow is missing. ALSO PACKAGED: all 73 source files (*.c *.h *.py *.m *.sh *.jinja), every benchmark log (float.log, full_forward.log, full384.log, guard.log, depth_bench.log, cmp_exact.log, gen_out.log and others), the fixture binaries (h_float.bin, h_int8.bin, h_ex16.bin, h_ex20.bin, h_resident.bin, bridge_ref.bin, embeds.bin, dual_kimi.bsig, dual_lex.bsig), the chat template, and docs/kimi_vector_organ_design.md. WHAT TRANSFERS TO R1, and what does not — stated so nothing is copied blindly: TRANSFERS the C-native safetensors header parser in kimi_gen.h (add an F8_E4M3 case) MLA attention geometry — both models use 512 latent + 64 rope + 128 v YaRN RoPE in C at arbitrary position (R1 needs factor 40 / orig 4096) the dtype-aware loader discipline the guardian hysteresis controller (HIGH 0.08, LOW 0.01, HYST 3, FLOOR 4) serial-load / parallel-compute phase timing, and the instrumentation bug to avoid: never sum I/O time across threads the five equivalence gates and the results-ledger habit DOES NOT TRANSFER the .kex contiguous expert cache — R1's experts are already contiguous native-int4 MAC with exact int64 group accumulation — fp8 has an exponent field and cannot be integer-accumulated the same way "convert non-routed bf16 to int8" — in R1 those tensors are already fp8 metal_mac.m — there is no Metal and no discrete GPU on megamind-agi "cut seeks, don't fan out reads" — inverted here; this drive has 2.4x concurrency headroom where the M4's external drive had none -------------------------------------------------------------------------------- B.9 REFERENCE FIXTURES -------------------------------------------------------------------------------- STATUS: [BLOCKED] for everything that needs model execution. The blocker is B.3: nothing on this machine can run DeepSeek-R1, or even dequantize one FP8 tensor, because there is no runtime and no reference. This is an ordering dependency, not an oversight. A fixture is only meaningful if its expected output came from a trusted implementation. Generating expected outputs from an implementation we are about to write would make every later comparison circular — the fixture would confirm our bugs rather than catch them. WHAT IS BLOCKED, AND ON WHAT tokenizer fixtures needs the tokenizers/transformers library (absent; and Python 3.14 has no torch wheels) FP8 operation fixtures needs a trusted fp8 block dequant (absent) router fixtures needs real hidden states, i.e. a working forward MLA fixtures same full-layer fixtures same MTP fixtures same WHAT CAN BE PRODUCED RIGHT NOW WITHOUT ANY RUNTIME, and is worth doing first: * RAW INPUT HALF of every FP8 fixture. The raw FP8 bytes and the F32 scale grid can be extracted from the checkpoint today, for exactly the tensors requested — layer 0 attention projection, layer 0 dense MLP, layer 3 routed expert, layer 31 expert 92 (the shard-crossing case), layer 60 routed expert, and MTP layer 61. Freeze those now; fill in the expected output when a reference exists. I can generate this on request. * TOKENIZER VOCABULARY FIXTURES that need no library: the exact id for every special token is already known and is listed in DEEPSEEK_R1_COMPLETE.txt part 11.2 — bos 0, eos 1, pad 2, 128798, 128799, <|User|> 128803, <|Assistant|> 128804, the FIM triple 128800-128802, and the tool-call block 128806-128814. * CHAT TEMPLATE RENDERINGS, which are pure Jinja over the template in tokenizer_config.json and need no model. Example, verified by hand: [{"role":"user","content":"What is 2+2?"}] with add_generation_prompt renders to <|begin_of_sentence|><|User|>What is 2+2?<|Assistant|>\n * THE EXPERT OFFSET MAP already functions as an I/O fixture: 15,104 entries of exact offset and length, verifiable by reading and checking lengths. RECOMMENDED ORDER 1. Fetch and pin an official fp8 reference (B.3). 2. Freeze raw inputs for all six FP8 fixture sites. 3. Generate expected outputs from the pinned reference. 4. Only then write the C backend, checking against those frozen outputs. -------------------------------------------------------------------------------- B.10 CURRENT APIs AND NETWORK CONTRACTS -------------------------------------------------------------------------------- STATUS: [PARTIAL] — observed from live organs, because nothing is formally specified. There is no OpenAPI document, no .proto, no schema file. MEASURED LIVE SURFACE, this session. 65 organs registered, 61 live. ORACLE :9402 /oracle/{health,recall,inverse,hop,analogy,similar, facts,path,set,consistent,learn,ask,unlearn,revise, counterfactual,induce,predict,abduce,factor,map, organs,register,orchestrate,orchstress,execute, execstress,coalition,coalstress} 508,093 entities | 34 relations | 773,235 edges mmserve :9099 /recall /spread /know /generate /health /reload {"status":"ok","sigs":24364580,"vocab":38085, "index":"mih16x32p"} mmsem :9096 /sem?q=text mmgame :9097 /game/remember /game/recall /game/social /game/health mmworld :9401 /world/recall?q=tok1|tok2 /world/health mm-narrate :9403 /narrate vocab 45,419 eye :9404 2,236 entities | 3,271 nodes | 13,561 edges | 6 realms loom :9410 /organ 2,210 souls, db_readonly true mmhelix :9418 16 strands, 8 antiparallel pairs mmpulse :9419 circulation loop, cycle 147,881 mmthought :9420 visible chain-of-thought mmreason :9522 73,562 entities | 42,457 mention edges mmwikix :9524 corpus 410,653 (also 9531-9534) mmparse :9526 10 languages mmscheme/mmpy/mmc :9527/9528/9529 interpreter organs maddie :9531 corpus 404,787 learned :9532 corpus 1,395,149 pubmed :9533 corpus 4,693,336 code2 :9534 corpus 1,390,098 mmthink :9540 reasons_via http://100.94.8.94:8082/v1/chat/completions mmlearn :9541 498,541 learned | 508,093 entities | 770,298 edges mmknow :9542 691,293 facts | 517,083 concepts | 11 relation types mmask :9543 the federation's voice; speaks_via the same M4 endpoint mmgateway-web :9560 /organs — read-only public gateway, 65 organs Caddy :8080 web host nginx :80 mm-api :8094 ORGAN REGISTRY RECORD, the de facto organ contract: {"name","url","mode","caps","live","health","builtin","priority","sig16"} mode local | remote | federation | timer health 0 dead | 1 reachable/builtin | 2 answering priority 8..80, higher wins in capability matching caps free-text capability string, scored against the query. There is no typed schema and no argument validation. *** THE INTERFACE THAT MATTERS MOST FOR THIS PROJECT *** mmthink (:9540) and mmask (:9543) both declare "reasons_via" / "speaks_via": http://100.94.8.94:8082/v1/chat/completions That OpenAI-shaped endpoint on the M4 is the federation's current voice and is the natural socket for R1. It is the one contract that must stay backward-compatible: anything that replaces it must accept the same /v1/chat/completions request shape and return the same response shape, or mmthink and mmask both break. BACKWARD-COMPATIBILITY LIST — what must not change when R1 lands /v1/chat/completions request/response shape (mmthink, mmask depend on it) the organ registry record shape (ORACLE's matcher depends on it) mesh.health. and mesh.registry. NATS subjects (timers publish) mesh.query.recall subject (mmshard responds) bit(512) signature width and XOR-popcount distance (every recall organ) [ACTION] Example request/response payloads per interface were not captured exhaustively — I recorded health/identity responses for all 65 organs but not a worked call for each. Say which interfaces matter and I will capture real request/response pairs for those. A GOTCHA WORTH RECORDING: ORACLE returns HTTP 200 for every unknown route, with a JSON error body. Any client that checks status codes to discover capabilities will conclude that every endpoint exists. Parse the body. -------------------------------------------------------------------------------- B.11 ENVIRONMENT AND DEPLOYMENT -------------------------------------------------------------------------------- STATUS: [DONE]. All inlined in PART C. /etc/wsl.conf C.12.1 .wslconfig (Windows side) C.12.2 systemd unit files, 51 of them C.13 sanitized environment report C.14 (uname, lscpu, free, df, clang, gcc, python3, pip list, systemctl list-unit-files, listening ports) disk benchmark script C.11.6 Secret environment variable VALUES are not included anywhere. /etc/mm/mm.env is mode 600 root-owned and was not read. DIRECTORY LAYOUT /opt/mm/{src,bin,data,logs,backups,code,learned,maddie,pubmed,oracle, reason,loops,pyembed} /home/joseph/{DeepSeek-R1,mmk,mmreason,oracle,megamind,missionmatch,eye-src} /var/lib/postgresql/18/{main,missionmatch} COMPILER FLAGS clang -O3 -march=native, with Makefile using `.RECIPEPREFIX = >`. clang 21.1.8 | gcc 15.2.0 | Python 3.14.4 CPU FEATURE DETECTION (full lscpu in C.14) PRESENT avx2 fma f16c popcnt sse4_2 aes bmi1 bmi2 rdseed adx ABSENT avx512* avx_vnni amx_int8 amx_bf16 Consequence: no hardware fp8/bf16 convert; int8 dot needs vpmaddubsw + vpmaddwd on 256-bit lanes. MEMORY LIMITS .wslconfig memory=26GB, 25 GiB visible, swap=8GB, vmIdleTimeout=-1, processors=12 At scan time: 15 GiB used, 9.6 GiB available, 2.3 GiB swapped DISK BENCHMARK RESULTS, measured this session single stream, 8 MiB blocks, 2 GiB ................ 507 MB/s same file again, page cache warm .................. 408 MB/s 4 parallel streams, 4 shards, 512 MiB each ........ 1220 MB/s aggregate 56 scattered 4 MiB preads ......................... 326 MB/s Interpretation: 2.4x concurrency headroom, and no page-cache benefit (25 GiB RAM against a 643 GB model has nothing to cache). -------------------------------------------------------------------------------- B.12 REPOSITORY MANIFESTS -------------------------------------------------------------------------------- STATUS: [DONE] for file manifests. [ABSENT] for all git metadata. FILE_MANIFEST.txt was generated for the DeepSeek package and is inside 01-deepseek-source.tar.gz. The requested command works as given: find . -type f -not -path './.git/*' -not -path './node_modules/*' \ -not -path './vendor/*' -not -path './data/*' \ -printf '%p\t%s bytes\n' | sort > FILE_MANIFEST.txt GIT METADATA CANNOT BE PRODUCED. Checked for .git in /opt/mm, /home/joseph/mmk, /home/joseph/mmreason, /home/joseph/oracle and /home/joseph/megamind. None is a repository. Therefore: GIT_STATUS.txt cannot be produced GIT_COMMIT.txt cannot be produced GIT_RECENT_LOG.txt cannot be produced What stands in for history is roughly 90 hand-copied .bak files: oracle.cpp.bak.sefS0 through sefS16 (with .5 increments) oracle.cpp.bak. — 1783580775, 1783582688, 1783594467, 1783602591, 1783610244, 1783612758 oracle.cpp.bak.{p1,p4..p10,preP1fed,premt,pre-fed,pre-gardener, pre-capraise,speak,wire1,coal,exec,orch} eye.cpp.bak.p28 through p40, plus bak-{book1,p1..p4,s1..s4,x1,x2} loom.cpp.bak-{book1,p1..p4,s1..s4,x1,x2} seferim_valgate.h.bak.sefS8 through sefS16 These are included in the packages. With no git, they are the only record of what changed and when, and the mtimes are the only timeline. [ACTION] Worth doing before this work goes further: `git init` in /opt/mm and commit the current state. It costs nothing and makes every later change to a 2 MB source tree reviewable. ================================================================================ PART C — INLINED FILES (complete, byte-for-byte) ================================================================================ ================================================================================ C.1 config.json (1,660 bytes) ================================================================================ { "architectures": [ "DeepseekV3ForCausalLM" ], "attention_bias": false, "attention_dropout": 0.0, "auto_map": { "AutoConfig": "configuration_deepseek.DeepseekV3Config", "AutoModel": "modeling_deepseek.DeepseekV3Model", "AutoModelForCausalLM": "modeling_deepseek.DeepseekV3ForCausalLM" }, "bos_token_id": 0, "eos_token_id": 1, "ep_size": 1, "first_k_dense_replace": 3, "hidden_act": "silu", "hidden_size": 7168, "initializer_range": 0.02, "intermediate_size": 18432, "kv_lora_rank": 512, "max_position_embeddings": 163840, "model_type": "deepseek_v3", "moe_intermediate_size": 2048, "moe_layer_freq": 1, "n_group": 8, "n_routed_experts": 256, "n_shared_experts": 1, "norm_topk_prob": true, "num_attention_heads": 128, "num_experts_per_tok": 8, "num_hidden_layers": 61, "num_key_value_heads": 128, "num_nextn_predict_layers": 1, "q_lora_rank": 1536, "qk_nope_head_dim": 128, "qk_rope_head_dim": 64, "quantization_config": { "activation_scheme": "dynamic", "fmt": "e4m3", "quant_method": "fp8", "weight_block_size": [ 128, 128 ] }, "rms_norm_eps": 1e-06, "rope_scaling": { "beta_fast": 32, "beta_slow": 1, "factor": 40, "mscale": 1.0, "mscale_all_dim": 1.0, "original_max_position_embeddings": 4096, "type": "yarn" }, "rope_theta": 10000, "routed_scaling_factor": 2.5, "scoring_func": "sigmoid", "tie_word_embeddings": false, "topk_group": 4, "topk_method": "noaux_tc", "torch_dtype": "bfloat16", "transformers_version": "4.46.3", "use_cache": true, "v_head_dim": 128, "vocab_size": 129280 } ================================================================================ C.2 configuration_deepseek.py (9,897 bytes) ================================================================================ from transformers.configuration_utils import PretrainedConfig from transformers.utils import logging logger = logging.get_logger(__name__) DEEPSEEK_PRETRAINED_CONFIG_ARCHIVE_MAP = {} class DeepseekV3Config(PretrainedConfig): r""" This is the configuration class to store the configuration of a [`DeepseekV3Model`]. It is used to instantiate an DeepSeek model according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of the DeepSeek-V3. Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PretrainedConfig`] for more information. Args: vocab_size (`int`, *optional*, defaults to 129280): Vocabulary size of the Deep model. Defines the number of different tokens that can be represented by the `inputs_ids` passed when calling [`DeepseekV3Model`] hidden_size (`int`, *optional*, defaults to 4096): Dimension of the hidden representations. intermediate_size (`int`, *optional*, defaults to 11008): Dimension of the MLP representations. moe_intermediate_size (`int`, *optional*, defaults to 1407): Dimension of the MoE representations. num_hidden_layers (`int`, *optional*, defaults to 32): Number of hidden layers in the Transformer decoder. num_nextn_predict_layers (`int`, *optional*, defaults to 1): Number of nextn predict layers in the DeepSeekV3 Model. num_attention_heads (`int`, *optional*, defaults to 32): Number of attention heads for each attention layer in the Transformer decoder. n_shared_experts (`int`, *optional*, defaults to None): Number of shared experts, None means dense model. n_routed_experts (`int`, *optional*, defaults to None): Number of routed experts, None means dense model. routed_scaling_factor (`float`, *optional*, defaults to 1.0): Scaling factor or routed experts. topk_method (`str`, *optional*, defaults to `gready`): Topk method used in routed gate. n_group (`int`, *optional*, defaults to None): Number of groups for routed experts. topk_group (`int`, *optional*, defaults to None): Number of selected groups for each token(for each token, ensuring the selected experts is only within `topk_group` groups). num_experts_per_tok (`int`, *optional*, defaults to None): Number of selected experts, None means dense model. moe_layer_freq (`int`, *optional*, defaults to 1): The frequency of the MoE layer: one expert layer for every `moe_layer_freq - 1` dense layers. first_k_dense_replace (`int`, *optional*, defaults to 0): Number of dense layers in shallow layers(embed->dense->dense->...->dense->moe->moe...->lm_head). \--k dense layers--/ norm_topk_prob (`bool`, *optional*, defaults to False): Whether to normalize the weights of the routed experts. scoring_func (`str`, *optional*, defaults to 'softmax'): Method of computing expert weights. aux_loss_alpha (`float`, *optional*, defaults to 0.001): Auxiliary loss weight coefficient. seq_aux = (`bool`, *optional*, defaults to True): Whether to compute the auxiliary loss for each individual sample. num_key_value_heads (`int`, *optional*): This is the number of key_value heads that should be used to implement Grouped Query Attention. If `num_key_value_heads=num_attention_heads`, the model will use Multi Head Attention (MHA), if `num_key_value_heads=1 the model will use Multi Query Attention (MQA) otherwise GQA is used. When converting a multi-head checkpoint to a GQA checkpoint, each group key and value head should be constructed by meanpooling all the original heads within that group. For more details checkout [this paper](https://arxiv.org/pdf/2305.13245.pdf). If it is not specified, will default to `num_attention_heads`. hidden_act (`str` or `function`, *optional*, defaults to `"silu"`): The non-linear activation function (function or string) in the decoder. max_position_embeddings (`int`, *optional*, defaults to 2048): The maximum sequence length that this model might ever be used with. initializer_range (`float`, *optional*, defaults to 0.02): The standard deviation of the truncated_normal_initializer for initializing all weight matrices. rms_norm_eps (`float`, *optional*, defaults to 1e-06): The epsilon used by the rms normalization layers. use_cache (`bool`, *optional*, defaults to `True`): Whether or not the model should return the last key/values attentions (not used by all models). Only relevant if `config.is_decoder=True`. pad_token_id (`int`, *optional*): Padding token id. bos_token_id (`int`, *optional*, defaults to 1): Beginning of stream token id. eos_token_id (`int`, *optional*, defaults to 2): End of stream token id. tie_word_embeddings (`bool`, *optional*, defaults to `False`): Whether to tie weight embeddings rope_theta (`float`, *optional*, defaults to 10000.0): The base period of the RoPE embeddings. rope_scaling (`Dict`, *optional*): Dictionary containing the scaling configuration for the RoPE embeddings. Currently supports two scaling strategies: linear and dynamic. Their scaling factor must be a float greater than 1. The expected format is `{"type": strategy name, "factor": scaling factor}`. When using this flag, don't update `max_position_embeddings` to the expected new maximum. attention_bias (`bool`, defaults to `False`, *optional*, defaults to `False`): Whether to use a bias in the query, key, value and output projection layers during self-attention. attention_dropout (`float`, *optional*, defaults to 0.0): The dropout ratio for the attention probabilities. ```python >>> from transformers import DeepseekV3Model, DeepseekV3Config >>> # Initializing a Deepseek-V3 style configuration >>> configuration = DeepseekV3Config() >>> # Accessing the model configuration >>> configuration = model.config ```""" model_type = "deepseek_v3" keys_to_ignore_at_inference = ["past_key_values"] def __init__( self, vocab_size=129280, hidden_size=7168, intermediate_size=18432, moe_intermediate_size = 2048, num_hidden_layers=61, num_nextn_predict_layers=1, num_attention_heads=128, num_key_value_heads=128, n_shared_experts = 1, n_routed_experts = 256, ep_size = 1, routed_scaling_factor = 2.5, kv_lora_rank = 512, q_lora_rank = 1536, qk_rope_head_dim = 64, v_head_dim = 128, qk_nope_head_dim = 128, topk_method = 'noaux_tc', n_group = 8, topk_group = 4, num_experts_per_tok = 8, moe_layer_freq = 1, first_k_dense_replace = 3, norm_topk_prob = True, scoring_func = 'sigmoid', hidden_act="silu", max_position_embeddings=4096, initializer_range=0.02, rms_norm_eps=1e-6, use_cache=True, pad_token_id=None, bos_token_id=0, eos_token_id=1, tie_word_embeddings=False, rope_theta=10000.0, rope_scaling=None, attention_bias=False, attention_dropout=0.0, **kwargs, ): self.vocab_size = vocab_size self.max_position_embeddings = max_position_embeddings self.hidden_size = hidden_size self.intermediate_size = intermediate_size self.moe_intermediate_size = moe_intermediate_size self.num_hidden_layers = num_hidden_layers self.num_nextn_predict_layers = num_nextn_predict_layers self.num_attention_heads = num_attention_heads self.n_shared_experts = n_shared_experts self.n_routed_experts = n_routed_experts self.ep_size = ep_size self.routed_scaling_factor = routed_scaling_factor self.kv_lora_rank = kv_lora_rank self.q_lora_rank = q_lora_rank self.qk_rope_head_dim = qk_rope_head_dim self.v_head_dim = v_head_dim self.qk_nope_head_dim = qk_nope_head_dim self.topk_method = topk_method self.n_group = n_group self.topk_group = topk_group self.num_experts_per_tok = num_experts_per_tok self.moe_layer_freq = moe_layer_freq self.first_k_dense_replace = first_k_dense_replace self.norm_topk_prob = norm_topk_prob self.scoring_func = scoring_func # for backward compatibility if num_key_value_heads is None: num_key_value_heads = num_attention_heads self.num_key_value_heads = num_key_value_heads self.hidden_act = hidden_act self.initializer_range = initializer_range self.rms_norm_eps = rms_norm_eps self.use_cache = use_cache self.rope_theta = rope_theta self.rope_scaling = rope_scaling self.attention_bias = attention_bias self.attention_dropout = attention_dropout super().__init__( pad_token_id=pad_token_id, bos_token_id=bos_token_id, eos_token_id=eos_token_id, tie_word_embeddings=tie_word_embeddings, **kwargs, ) ================================================================================ C.3 modeling_deepseek.py (75,741 bytes) ================================================================================ # coding=utf-8 # Copyright 2023 DeepSeek-AI and The HuggingFace Inc. team. All rights reserved. # # This code is based on EleutherAI's GPT-NeoX library and the GPT-NeoX # and OPT implementations in this library. It has been modified from its # original forms to accommodate minor architectural differences compared # to GPT-NeoX and OPT used by the Meta AI team that trained the model. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. """ PyTorch DeepSeek model.""" import math import warnings from typing import List, Optional, Tuple, Union import torch import torch.nn.functional as F import torch.utils.checkpoint from torch import nn from torch.nn import BCEWithLogitsLoss, CrossEntropyLoss, MSELoss from transformers.activations import ACT2FN from transformers.cache_utils import Cache, DynamicCache from transformers.modeling_attn_mask_utils import ( AttentionMaskConverter, _prepare_4d_attention_mask, _prepare_4d_causal_attention_mask, ) from transformers.modeling_outputs import ( BaseModelOutputWithPast, CausalLMOutputWithPast, SequenceClassifierOutputWithPast, ) from transformers.modeling_utils import PreTrainedModel from transformers.pytorch_utils import ( ALL_LAYERNORM_LAYERS, is_torch_greater_or_equal_than_1_13, ) from transformers.utils import ( add_start_docstrings, add_start_docstrings_to_model_forward, is_flash_attn_2_available, is_flash_attn_greater_or_equal_2_10, logging, replace_return_docstrings, ) from transformers.utils.import_utils import is_torch_fx_available from .configuration_deepseek import DeepseekV3Config import torch.distributed as dist import numpy as np if is_flash_attn_2_available(): from flash_attn import flash_attn_func, flash_attn_varlen_func from flash_attn.bert_padding import index_first_axis, pad_input, unpad_input # noqa # This makes `_prepare_4d_causal_attention_mask` a leaf function in the FX graph. # It means that the function will not be traced through and simply appear as a node in the graph. if is_torch_fx_available(): if not is_torch_greater_or_equal_than_1_13: import torch.fx _prepare_4d_causal_attention_mask = torch.fx.wrap(_prepare_4d_causal_attention_mask) logger = logging.get_logger(__name__) _CONFIG_FOR_DOC = "DeepseekV3Config" def _get_unpad_data(attention_mask): seqlens_in_batch = attention_mask.sum(dim=-1, dtype=torch.int32) indices = torch.nonzero(attention_mask.flatten(), as_tuple=False).flatten() max_seqlen_in_batch = seqlens_in_batch.max().item() cu_seqlens = F.pad( torch.cumsum(seqlens_in_batch, dim=0, dtype=torch.torch.int32), (1, 0) ) return ( indices, cu_seqlens, max_seqlen_in_batch, ) class DeepseekV3RMSNorm(nn.Module): def __init__(self, hidden_size, eps=1e-6): """ DeepseekV3RMSNorm is equivalent to T5LayerNorm """ super().__init__() self.weight = nn.Parameter(torch.ones(hidden_size)) self.variance_epsilon = eps def forward(self, hidden_states): input_dtype = hidden_states.dtype hidden_states = hidden_states.to(torch.float32) variance = hidden_states.pow(2).mean(-1, keepdim=True) hidden_states = hidden_states * torch.rsqrt(variance + self.variance_epsilon) return self.weight * hidden_states.to(input_dtype) ALL_LAYERNORM_LAYERS.append(DeepseekV3RMSNorm) class DeepseekV3RotaryEmbedding(nn.Module): def __init__(self, dim, max_position_embeddings=2048, base=10000, device=None): super().__init__() self.dim = dim self.max_position_embeddings = max_position_embeddings self.base = base inv_freq = 1.0 / ( self.base ** (torch.arange(0, self.dim, 2).float().to(device) / self.dim) ) self.register_buffer("inv_freq", inv_freq, persistent=False) # Build here to make `torch.jit.trace` work. self._set_cos_sin_cache( seq_len=max_position_embeddings, device=self.inv_freq.device, dtype=torch.get_default_dtype(), ) self.max_seq_len_cached = None def _set_cos_sin_cache(self, seq_len, device, dtype): self.max_seq_len_cached = seq_len t = torch.arange( self.max_seq_len_cached, device=device, dtype=self.inv_freq.dtype ) freqs = torch.outer(t, self.inv_freq.to(t.device)) # Different from paper, but it uses a different permutation in order to obtain the same calculation emb = torch.cat((freqs, freqs), dim=-1) self.register_buffer("cos_cached", emb.cos().to(dtype), persistent=False) self.register_buffer("sin_cached", emb.sin().to(dtype), persistent=False) def forward(self, x, seq_len=None): # x: [bs, num_attention_heads, seq_len, head_size] if self.max_seq_len_cached is None or seq_len > self.max_seq_len_cached: self._set_cos_sin_cache(seq_len=seq_len, device=x.device, dtype=x.dtype) return ( self.cos_cached[:seq_len].to(dtype=x.dtype), self.sin_cached[:seq_len].to(dtype=x.dtype), ) # Copied from transformers.models.llama.modeling_llama.LlamaLinearScalingRotaryEmbedding with Llama->DeepseekV3 class DeepseekV3LinearScalingRotaryEmbedding(DeepseekV3RotaryEmbedding): """DeepseekV3RotaryEmbedding extended with linear scaling. Credits to the Reddit user /u/kaiokendev""" def __init__( self, dim, max_position_embeddings=2048, base=10000, device=None, scaling_factor=1.0, ): self.scaling_factor = scaling_factor super().__init__(dim, max_position_embeddings, base, device) def _set_cos_sin_cache(self, seq_len, device, dtype): self.max_seq_len_cached = seq_len t = torch.arange( self.max_seq_len_cached, device=device, dtype=self.inv_freq.dtype ) t = t / self.scaling_factor freqs = torch.outer(t, self.inv_freq) # Different from paper, but it uses a different permutation in order to obtain the same calculation emb = torch.cat((freqs, freqs), dim=-1) self.register_buffer("cos_cached", emb.cos().to(dtype), persistent=False) self.register_buffer("sin_cached", emb.sin().to(dtype), persistent=False) # Copied from transformers.models.llama.modeling_llama.LlamaDynamicNTKScalingRotaryEmbedding with Llama->DeepseekV3 class DeepseekV3DynamicNTKScalingRotaryEmbedding(DeepseekV3RotaryEmbedding): """DeepseekV3RotaryEmbedding extended with Dynamic NTK scaling. Credits to the Reddit users /u/bloc97 and /u/emozilla""" def __init__( self, dim, max_position_embeddings=2048, base=10000, device=None, scaling_factor=1.0, ): self.scaling_factor = scaling_factor super().__init__(dim, max_position_embeddings, base, device) def _set_cos_sin_cache(self, seq_len, device, dtype): self.max_seq_len_cached = seq_len if seq_len > self.max_position_embeddings: base = self.base * ( (self.scaling_factor * seq_len / self.max_position_embeddings) - (self.scaling_factor - 1) ) ** (self.dim / (self.dim - 2)) inv_freq = 1.0 / ( base ** (torch.arange(0, self.dim, 2).float().to(device) / self.dim) ) self.register_buffer("inv_freq", inv_freq, persistent=False) t = torch.arange( self.max_seq_len_cached, device=device, dtype=self.inv_freq.dtype ) freqs = torch.outer(t, self.inv_freq) # Different from paper, but it uses a different permutation in order to obtain the same calculation emb = torch.cat((freqs, freqs), dim=-1) self.register_buffer("cos_cached", emb.cos().to(dtype), persistent=False) self.register_buffer("sin_cached", emb.sin().to(dtype), persistent=False) # Inverse dim formula to find dim based on number of rotations def yarn_find_correction_dim( num_rotations, dim, base=10000, max_position_embeddings=2048 ): return (dim * math.log(max_position_embeddings / (num_rotations * 2 * math.pi))) / ( 2 * math.log(base) ) # Find dim range bounds based on rotations def yarn_find_correction_range( low_rot, high_rot, dim, base=10000, max_position_embeddings=2048 ): low = math.floor( yarn_find_correction_dim(low_rot, dim, base, max_position_embeddings) ) high = math.ceil( yarn_find_correction_dim(high_rot, dim, base, max_position_embeddings) ) return max(low, 0), min(high, dim - 1) # Clamp values just in case def yarn_get_mscale(scale=1, mscale=1): if scale <= 1: return 1.0 return 0.1 * mscale * math.log(scale) + 1.0 def yarn_linear_ramp_mask(min, max, dim): if min == max: max += 0.001 # Prevent singularity linear_func = (torch.arange(dim, dtype=torch.float32) - min) / (max - min) ramp_func = torch.clamp(linear_func, 0, 1) return ramp_func class DeepseekV3YarnRotaryEmbedding(DeepseekV3RotaryEmbedding): def __init__( self, dim, max_position_embeddings=2048, base=10000, device=None, scaling_factor=1.0, original_max_position_embeddings=4096, beta_fast=32, beta_slow=1, mscale=1, mscale_all_dim=0, ): self.scaling_factor = scaling_factor self.original_max_position_embeddings = original_max_position_embeddings self.beta_fast = beta_fast self.beta_slow = beta_slow self.mscale = mscale self.mscale_all_dim = mscale_all_dim super().__init__(dim, max_position_embeddings, base, device) def _set_cos_sin_cache(self, seq_len, device, dtype): self.max_seq_len_cached = seq_len dim = self.dim freq_extra = 1.0 / ( self.base ** (torch.arange(0, dim, 2, dtype=torch.float32, device=device) / dim) ) freq_inter = 1.0 / ( self.scaling_factor * self.base ** (torch.arange(0, dim, 2, dtype=torch.float32, device=device) / dim) ) low, high = yarn_find_correction_range( self.beta_fast, self.beta_slow, dim, self.base, self.original_max_position_embeddings, ) inv_freq_mask = 1.0 - yarn_linear_ramp_mask(low, high, dim // 2).to( device=device, dtype=torch.float32 ) inv_freq = freq_inter * (1 - inv_freq_mask) + freq_extra * inv_freq_mask self.register_buffer("inv_freq", inv_freq, persistent=False) t = torch.arange(seq_len, device=device, dtype=torch.float32) freqs = torch.outer(t, inv_freq) _mscale = float( yarn_get_mscale(self.scaling_factor, self.mscale) / yarn_get_mscale(self.scaling_factor, self.mscale_all_dim) ) emb = torch.cat((freqs, freqs), dim=-1) self.register_buffer( "cos_cached", (emb.cos() * _mscale).to(dtype), persistent=False ) self.register_buffer( "sin_cached", (emb.sin() * _mscale).to(dtype), persistent=False ) # Copied from transformers.models.llama.modeling_llama.rotate_half def rotate_half(x): """Rotates half the hidden dims of the input.""" x1 = x[..., : x.shape[-1] // 2] x2 = x[..., x.shape[-1] // 2 :] return torch.cat((-x2, x1), dim=-1) # Copied from transformers.models.llama.modeling_llama.apply_rotary_pos_emb def apply_rotary_pos_emb(q, k, cos, sin, position_ids, unsqueeze_dim=1): """Applies Rotary Position Embedding to the query and key tensors. Args: q (`torch.Tensor`): The query tensor. k (`torch.Tensor`): The key tensor. cos (`torch.Tensor`): The cosine part of the rotary embedding. sin (`torch.Tensor`): The sine part of the rotary embedding. position_ids (`torch.Tensor`): The position indices of the tokens corresponding to the query and key tensors. For example, this can be used to pass offsetted position ids when working with a KV-cache. unsqueeze_dim (`int`, *optional*, defaults to 1): The 'unsqueeze_dim' argument specifies the dimension along which to unsqueeze cos[position_ids] and sin[position_ids] so that they can be properly broadcasted to the dimensions of q and k. For example, note that cos[position_ids] and sin[position_ids] have the shape [batch_size, seq_len, head_dim]. Then, if q and k have the shape [batch_size, heads, seq_len, head_dim], then setting unsqueeze_dim=1 makes cos[position_ids] and sin[position_ids] broadcastable to the shapes of q and k. Similarly, if q and k have the shape [batch_size, seq_len, heads, head_dim], then set unsqueeze_dim=2. Returns: `tuple(torch.Tensor)` comprising of the query and key tensors rotated using the Rotary Position Embedding. """ cos = cos[position_ids].unsqueeze(unsqueeze_dim) sin = sin[position_ids].unsqueeze(unsqueeze_dim) b, h, s, d = q.shape q = q.view(b, h, s, d // 2, 2).transpose(4, 3).reshape(b, h, s, d) b, h, s, d = k.shape k = k.view(b, h, s, d // 2, 2).transpose(4, 3).reshape(b, h, s, d) q_embed = (q * cos) + (rotate_half(q) * sin) k_embed = (k * cos) + (rotate_half(k) * sin) return q_embed, k_embed class DeepseekV3MLP(nn.Module): def __init__(self, config, hidden_size=None, intermediate_size=None): super().__init__() self.config = config self.hidden_size = config.hidden_size if hidden_size is None else hidden_size self.intermediate_size = ( config.intermediate_size if intermediate_size is None else intermediate_size ) self.gate_proj = nn.Linear(self.hidden_size, self.intermediate_size, bias=False) self.up_proj = nn.Linear(self.hidden_size, self.intermediate_size, bias=False) self.down_proj = nn.Linear(self.intermediate_size, self.hidden_size, bias=False) self.act_fn = ACT2FN[config.hidden_act] def forward(self, x): down_proj = self.down_proj(self.act_fn(self.gate_proj(x)) * self.up_proj(x)) return down_proj class MoEGate(nn.Module): def __init__(self, config): super().__init__() self.config = config self.top_k = config.num_experts_per_tok self.n_routed_experts = config.n_routed_experts self.routed_scaling_factor = config.routed_scaling_factor self.scoring_func = config.scoring_func self.topk_method = config.topk_method self.n_group = config.n_group self.topk_group = config.topk_group # topk selection algorithm self.norm_topk_prob = config.norm_topk_prob self.gating_dim = config.hidden_size self.weight = nn.Parameter( torch.empty((self.n_routed_experts, self.gating_dim)) ) if self.topk_method == "noaux_tc": self.e_score_correction_bias = nn.Parameter( torch.empty((self.n_routed_experts)) ) self.reset_parameters() def reset_parameters(self) -> None: import torch.nn.init as init init.kaiming_uniform_(self.weight, a=math.sqrt(5)) def forward(self, hidden_states): bsz, seq_len, h = hidden_states.shape ### compute gating score hidden_states = hidden_states.view(-1, h) logits = F.linear( hidden_states.type(torch.float32), self.weight.type(torch.float32), None ) if self.scoring_func == "sigmoid": scores = logits.sigmoid() else: raise NotImplementedError( f"insupportable scoring function for MoE gating: {self.scoring_func}" ) ### select top-k experts if self.topk_method == "noaux_tc": assert not self.training scores_for_choice = scores.view(bsz * seq_len, -1) + self.e_score_correction_bias.unsqueeze(0) group_scores = ( scores_for_choice.view(bsz * seq_len, self.n_group, -1).topk(2, dim=-1)[0].sum(dim = -1) ) # [n, n_group] group_idx = torch.topk( group_scores, k=self.topk_group, dim=-1, sorted=False )[ 1 ] # [n, top_k_group] group_mask = torch.zeros_like(group_scores) # [n, n_group] group_mask.scatter_(1, group_idx, 1) # [n, n_group] score_mask = ( group_mask.unsqueeze(-1) .expand( bsz * seq_len, self.n_group, self.n_routed_experts // self.n_group ) .reshape(bsz * seq_len, -1) ) # [n, e] tmp_scores = scores_for_choice.masked_fill(~score_mask.bool(), float("-inf")) # [n, e] _, topk_idx = torch.topk( tmp_scores, k=self.top_k, dim=-1, sorted=False ) topk_weight = scores.gather(1, topk_idx) else: raise NotImplementedError( f"insupportable TopK function for MoE gating: {self.topk_method}" ) ### norm gate to sum 1 if self.top_k > 1 and self.norm_topk_prob: denominator = topk_weight.sum(dim=-1, keepdim=True) + 1e-20 topk_weight = topk_weight / denominator topk_weight = topk_weight * self.routed_scaling_factor # must multiply the scaling factor return topk_idx, topk_weight class DeepseekV3MoE(nn.Module): """ A mixed expert module containing shared experts. """ def __init__(self, config): super().__init__() self.config = config self.num_experts_per_tok = config.num_experts_per_tok if hasattr(config, "ep_size") and config.ep_size > 1: assert config.ep_size == dist.get_world_size() self.ep_size = config.ep_size self.experts_per_rank = config.n_routed_experts // config.ep_size self.ep_rank = dist.get_rank() self.experts = nn.ModuleList( [ ( DeepseekV3MLP( config, intermediate_size=config.moe_intermediate_size ) if i >= self.ep_rank * self.experts_per_rank and i < (self.ep_rank + 1) * self.experts_per_rank else None ) for i in range(config.n_routed_experts) ] ) else: self.ep_size = 1 self.experts_per_rank = config.n_routed_experts self.ep_rank = 0 self.experts = nn.ModuleList( [ DeepseekV3MLP( config, intermediate_size=config.moe_intermediate_size ) for i in range(config.n_routed_experts) ] ) self.gate = MoEGate(config) if config.n_shared_experts is not None: intermediate_size = config.moe_intermediate_size * config.n_shared_experts self.shared_experts = DeepseekV3MLP( config=config, intermediate_size=intermediate_size ) def forward(self, hidden_states): identity = hidden_states orig_shape = hidden_states.shape topk_idx, topk_weight = self.gate(hidden_states) hidden_states = hidden_states.view(-1, hidden_states.shape[-1]) flat_topk_idx = topk_idx.view(-1) if not self.training: y = self.moe_infer(hidden_states, topk_idx, topk_weight).view(*orig_shape) if self.config.n_shared_experts is not None: y = y + self.shared_experts(identity) return y @torch.no_grad() def moe_infer(self, x, topk_ids, topk_weight): cnts = topk_ids.new_zeros((topk_ids.shape[0], len(self.experts))) cnts.scatter_(1, topk_ids, 1) tokens_per_expert = cnts.sum(dim=0) idxs = topk_ids.view(-1).argsort() sorted_tokens = x[idxs // topk_ids.shape[1]] sorted_tokens_shape = sorted_tokens.shape if self.ep_size > 1: tokens_per_ep_rank = tokens_per_expert.view(self.ep_size, -1).sum(dim=1) tokens_per_expert_group = tokens_per_expert.new_empty( tokens_per_expert.shape[0] ) dist.all_to_all_single(tokens_per_expert_group, tokens_per_expert) output_splits = ( tokens_per_expert_group.view(self.ep_size, -1) .sum(1) .cpu() .numpy() .tolist() ) gathered_tokens = sorted_tokens.new_empty( tokens_per_expert_group.sum(dim=0).cpu().item(), sorted_tokens.shape[1] ) input_split_sizes = tokens_per_ep_rank.cpu().numpy().tolist() dist.all_to_all( list(gathered_tokens.split(output_splits)), list(sorted_tokens.split(input_split_sizes)), ) tokens_per_expert_post_gather = tokens_per_expert_group.view( self.ep_size, self.experts_per_rank ).sum(dim=0) gatherd_idxs = np.zeros(shape=(gathered_tokens.shape[0],), dtype=np.int32) s = 0 for i, k in enumerate(tokens_per_expert_group.cpu().numpy()): gatherd_idxs[s : s + k] = i % self.experts_per_rank s += k gatherd_idxs = gatherd_idxs.argsort() sorted_tokens = gathered_tokens[gatherd_idxs] tokens_per_expert = tokens_per_expert_post_gather tokens_per_expert = tokens_per_expert.cpu().numpy() outputs = [] start_idx = 0 for i, num_tokens in enumerate(tokens_per_expert): end_idx = start_idx + num_tokens if num_tokens == 0: continue expert = self.experts[i + self.ep_rank * self.experts_per_rank] tokens_for_this_expert = sorted_tokens[start_idx:end_idx] expert_out = expert(tokens_for_this_expert) outputs.append(expert_out) start_idx = end_idx outs = torch.cat(outputs, dim=0) if len(outputs) else sorted_tokens.new_empty(0) if self.ep_size > 1: new_x = torch.empty_like(outs) new_x[gatherd_idxs] = outs gathered_tokens = new_x.new_empty(*sorted_tokens_shape) dist.all_to_all( list(gathered_tokens.split(input_split_sizes)), list(new_x.split(output_splits)), ) outs = gathered_tokens new_x = torch.empty_like(outs) new_x[idxs] = outs final_out = ( new_x.view(*topk_ids.shape, -1) .type(topk_weight.dtype) .mul_(topk_weight.unsqueeze(dim=-1)) .sum(dim=1) .type(new_x.dtype) ) return final_out # Copied from transformers.models.llama.modeling_llama.repeat_kv def repeat_kv(hidden_states: torch.Tensor, n_rep: int) -> torch.Tensor: """ This is the equivalent of torch.repeat_interleave(x, dim=1, repeats=n_rep). The hidden states go from (batch, num_key_value_heads, seqlen, head_dim) to (batch, num_attention_heads, seqlen, head_dim) """ batch, num_key_value_heads, slen, head_dim = hidden_states.shape if n_rep == 1: return hidden_states hidden_states = hidden_states[:, :, None, :, :].expand( batch, num_key_value_heads, n_rep, slen, head_dim ) return hidden_states.reshape(batch, num_key_value_heads * n_rep, slen, head_dim) # Copied from transformers.models.llama.modeling_llama.LlamaAttention with Llama->DeepseekV3 class DeepseekV3Attention(nn.Module): """Multi-headed attention from 'Attention Is All You Need' paper""" def __init__(self, config: DeepseekV3Config, layer_idx: Optional[int] = None): super().__init__() self.config = config self.layer_idx = layer_idx if layer_idx is None: logger.warning_once( f"Instantiating {self.__class__.__name__} without passing `layer_idx` is not recommended and will " "to errors during the forward call, if caching is used. Please make sure to provide a `layer_idx` " "when creating this class." ) self.attention_dropout = config.attention_dropout self.hidden_size = config.hidden_size self.num_heads = config.num_attention_heads self.max_position_embeddings = config.max_position_embeddings self.rope_theta = config.rope_theta self.q_lora_rank = config.q_lora_rank self.qk_rope_head_dim = config.qk_rope_head_dim self.kv_lora_rank = config.kv_lora_rank self.v_head_dim = config.v_head_dim self.qk_nope_head_dim = config.qk_nope_head_dim self.q_head_dim = config.qk_nope_head_dim + config.qk_rope_head_dim self.is_causal = True if self.q_lora_rank is None: self.q_proj = nn.Linear( self.hidden_size, self.num_heads * self.q_head_dim, bias=False ) else: self.q_a_proj = nn.Linear( self.hidden_size, config.q_lora_rank, bias=config.attention_bias ) self.q_a_layernorm = DeepseekV3RMSNorm(config.q_lora_rank) self.q_b_proj = nn.Linear( config.q_lora_rank, self.num_heads * self.q_head_dim, bias=False ) self.kv_a_proj_with_mqa = nn.Linear( self.hidden_size, config.kv_lora_rank + config.qk_rope_head_dim, bias=config.attention_bias, ) self.kv_a_layernorm = DeepseekV3RMSNorm(config.kv_lora_rank) self.kv_b_proj = nn.Linear( config.kv_lora_rank, self.num_heads * (self.q_head_dim - self.qk_rope_head_dim + self.v_head_dim), bias=False, ) self.o_proj = nn.Linear( self.num_heads * self.v_head_dim, self.hidden_size, bias=config.attention_bias, ) self._init_rope() self.softmax_scale = self.q_head_dim ** (-0.5) if self.config.rope_scaling is not None: mscale_all_dim = self.config.rope_scaling.get("mscale_all_dim", 0) scaling_factor = self.config.rope_scaling["factor"] if mscale_all_dim: mscale = yarn_get_mscale(scaling_factor, mscale_all_dim) self.softmax_scale = self.softmax_scale * mscale * mscale def _init_rope(self): if self.config.rope_scaling is None: self.rotary_emb = DeepseekV3RotaryEmbedding( self.qk_rope_head_dim, max_position_embeddings=self.max_position_embeddings, base=self.rope_theta, ) else: scaling_type = self.config.rope_scaling["type"] scaling_factor = self.config.rope_scaling["factor"] if scaling_type == "linear": self.rotary_emb = DeepseekV3LinearScalingRotaryEmbedding( self.qk_rope_head_dim, max_position_embeddings=self.max_position_embeddings, scaling_factor=scaling_factor, base=self.rope_theta, ) elif scaling_type == "dynamic": self.rotary_emb = DeepseekV3DynamicNTKScalingRotaryEmbedding( self.qk_rope_head_dim, max_position_embeddings=self.max_position_embeddings, scaling_factor=scaling_factor, base=self.rope_theta, ) elif scaling_type == "yarn": kwargs = { key: self.config.rope_scaling[key] for key in [ "original_max_position_embeddings", "beta_fast", "beta_slow", "mscale", "mscale_all_dim", ] if key in self.config.rope_scaling } self.rotary_emb = DeepseekV3YarnRotaryEmbedding( self.qk_rope_head_dim, max_position_embeddings=self.max_position_embeddings, scaling_factor=scaling_factor, base=self.rope_theta, **kwargs, ) else: raise ValueError(f"Unknown RoPE scaling type {scaling_type}") def _shape(self, tensor: torch.Tensor, seq_len: int, bsz: int): return ( tensor.view(bsz, seq_len, self.num_heads, self.v_head_dim) .transpose(1, 2) .contiguous() ) def forward( self, hidden_states: torch.Tensor, attention_mask: Optional[torch.Tensor] = None, position_ids: Optional[torch.LongTensor] = None, past_key_value: Optional[Cache] = None, output_attentions: bool = False, use_cache: bool = False, **kwargs, ) -> Tuple[torch.Tensor, Optional[torch.Tensor], Optional[Tuple[torch.Tensor]]]: if "padding_mask" in kwargs: warnings.warn( "Passing `padding_mask` is deprecated and will be removed in v4.37. Please make sure use `attention_mask` instead.`" ) bsz, q_len, _ = hidden_states.size() if self.q_lora_rank is None: q = self.q_proj(hidden_states) else: q = self.q_b_proj(self.q_a_layernorm(self.q_a_proj(hidden_states))) q = q.view(bsz, q_len, self.num_heads, self.q_head_dim).transpose(1, 2) q_nope, q_pe = torch.split( q, [self.qk_nope_head_dim, self.qk_rope_head_dim], dim=-1 ) compressed_kv = self.kv_a_proj_with_mqa(hidden_states) compressed_kv, k_pe = torch.split( compressed_kv, [self.kv_lora_rank, self.qk_rope_head_dim], dim=-1 ) k_pe = k_pe.view(bsz, q_len, 1, self.qk_rope_head_dim).transpose(1, 2) kv = ( self.kv_b_proj(self.kv_a_layernorm(compressed_kv)) .view(bsz, q_len, self.num_heads, self.qk_nope_head_dim + self.v_head_dim) .transpose(1, 2) ) k_nope, value_states = torch.split( kv, [self.qk_nope_head_dim, self.v_head_dim], dim=-1 ) kv_seq_len = value_states.shape[-2] if past_key_value is not None: if self.layer_idx is None: raise ValueError( f"The cache structure has changed since version v4.36. If you are using {self.__class__.__name__} " "for auto-regressive decoding with k/v caching, please make sure to initialize the attention class " "with a layer index." ) kv_seq_len += past_key_value.get_usable_length(kv_seq_len, self.layer_idx) cos, sin = self.rotary_emb(value_states, seq_len=kv_seq_len) q_pe, k_pe = apply_rotary_pos_emb(q_pe, k_pe, cos, sin, position_ids) query_states = k_pe.new_empty(bsz, self.num_heads, q_len, self.q_head_dim) query_states[:, :, :, : self.qk_nope_head_dim] = q_nope query_states[:, :, :, self.qk_nope_head_dim :] = q_pe key_states = k_pe.new_empty(bsz, self.num_heads, q_len, self.q_head_dim) key_states[:, :, :, : self.qk_nope_head_dim] = k_nope key_states[:, :, :, self.qk_nope_head_dim :] = k_pe if past_key_value is not None: cache_kwargs = {"sin": sin, "cos": cos} # Specific to RoPE models key_states, value_states = past_key_value.update( key_states, value_states, self.layer_idx, cache_kwargs ) attn_weights = ( torch.matmul(query_states, key_states.transpose(2, 3)) * self.softmax_scale ) if attn_weights.size() != (bsz, self.num_heads, q_len, kv_seq_len): raise ValueError( f"Attention weights should be of size {(bsz, self.num_heads, q_len, kv_seq_len)}, but is" f" {attn_weights.size()}" ) assert attention_mask is not None if attention_mask is not None: if attention_mask.size() != (bsz, 1, q_len, kv_seq_len): raise ValueError( f"Attention mask should be of size {(bsz, 1, q_len, kv_seq_len)}, but is {attention_mask.size()}" ) attn_weights = attn_weights + attention_mask # upcast attention to fp32 attn_weights = nn.functional.softmax( attn_weights, dim=-1, dtype=torch.float32 ).to(query_states.dtype) attn_weights = nn.functional.dropout( attn_weights, p=self.attention_dropout, training=self.training ) attn_output = torch.matmul(attn_weights, value_states) if attn_output.size() != (bsz, self.num_heads, q_len, self.v_head_dim): raise ValueError( f"`attn_output` should be of size {(bsz, self.num_heads, q_len, self.v_head_dim)}, but is" f" {attn_output.size()}" ) attn_output = attn_output.transpose(1, 2).contiguous() attn_output = attn_output.reshape(bsz, q_len, self.num_heads * self.v_head_dim) attn_output = self.o_proj(attn_output) if not output_attentions: attn_weights = None return attn_output, attn_weights, past_key_value # Copied from transformers.models.llama.modeling_llama.LlamaFlashAttention2 with Llama->DeepseekV3 class DeepseekV3FlashAttention2(DeepseekV3Attention): """ DeepseekV3 flash attention module. This module inherits from `DeepseekV3Attention` as the weights of the module stays untouched. The only required change would be on the forward pass where it needs to correctly call the public API of flash attention and deal with padding tokens in case the input contains any of them. """ def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) # TODO: Should be removed once Flash Attention for RoCm is bumped to 2.1. # flash_attn<2.1 generates top-left aligned causal mask, while what is needed here is bottom-right alignement, that was made default for flash_attn>=2.1. This attribute is used to handle this difference. Reference: https://github.com/Dao-AILab/flash-attention/releases/tag/v2.1.0. # Beware that with flash_attn<2.1, using q_seqlen != k_seqlen (except for the case q_seqlen == 1) produces a wrong mask (top-left). self._flash_attn_uses_top_left_mask = not is_flash_attn_greater_or_equal_2_10() def forward( self, hidden_states: torch.Tensor, attention_mask: Optional[torch.LongTensor] = None, position_ids: Optional[torch.LongTensor] = None, past_key_value: Optional[Cache] = None, output_attentions: bool = False, use_cache: bool = False, **kwargs, ) -> Tuple[torch.Tensor, Optional[torch.Tensor], Optional[Tuple[torch.Tensor]]]: # DeepseekV3FlashAttention2 attention does not support output_attentions if "padding_mask" in kwargs: warnings.warn( "Passing `padding_mask` is deprecated and will be removed in v4.37. Please make sure use `attention_mask` instead.`" ) # overwrite attention_mask with padding_mask attention_mask = kwargs.pop("padding_mask") output_attentions = False bsz, q_len, _ = hidden_states.size() if self.q_lora_rank is None: q = self.q_proj(hidden_states) else: q = self.q_b_proj(self.q_a_layernorm(self.q_a_proj(hidden_states))) q = q.view(bsz, q_len, self.num_heads, self.q_head_dim).transpose(1, 2) q_nope, q_pe = torch.split( q, [self.qk_nope_head_dim, self.qk_rope_head_dim], dim=-1 ) # Flash attention requires the input to have the shape # batch_size x seq_length x head_dim x hidden_dim # therefore we just need to keep the original shape compressed_kv = self.kv_a_proj_with_mqa(hidden_states) compressed_kv, k_pe = torch.split( compressed_kv, [self.kv_lora_rank, self.qk_rope_head_dim], dim=-1 ) k_pe = k_pe.view(bsz, q_len, 1, self.qk_rope_head_dim).transpose(1, 2) kv = ( self.kv_b_proj(self.kv_a_layernorm(compressed_kv)) .view(bsz, q_len, self.num_heads, self.qk_nope_head_dim + self.v_head_dim) .transpose(1, 2) ) k_nope, value_states = torch.split( kv, [self.qk_nope_head_dim, self.v_head_dim], dim=-1 ) kv_seq_len = value_states.shape[-2] kv_seq_len = value_states.shape[-2] if past_key_value is not None: kv_seq_len += past_key_value.get_usable_length(kv_seq_len, self.layer_idx) cos, sin = self.rotary_emb(value_states, seq_len=kv_seq_len) q_pe, k_pe = apply_rotary_pos_emb(q_pe, k_pe, cos, sin, position_ids) query_states = k_pe.new_empty(bsz, self.num_heads, q_len, self.q_head_dim) query_states[:, :, :, : self.qk_nope_head_dim] = q_nope query_states[:, :, :, self.qk_nope_head_dim :] = q_pe key_states = k_pe.new_empty(bsz, self.num_heads, q_len, self.q_head_dim) key_states[:, :, :, : self.qk_nope_head_dim] = k_nope key_states[:, :, :, self.qk_nope_head_dim :] = k_pe if self.q_head_dim != self.v_head_dim: value_states = F.pad(value_states, [0, self.q_head_dim - self.v_head_dim]) if past_key_value is not None: cache_kwargs = {"sin": sin, "cos": cos} # Specific to RoPE models key_states, value_states = past_key_value.update( key_states, value_states, self.layer_idx, cache_kwargs ) # TODO: These transpose are quite inefficient but Flash Attention requires the layout [batch_size, sequence_length, num_heads, head_dim]. We would need to refactor the KV cache # to be able to avoid many of these transpose/reshape/view. query_states = query_states.transpose(1, 2) key_states = key_states.transpose(1, 2) value_states = value_states.transpose(1, 2) dropout_rate = self.attention_dropout if self.training else 0.0 # In PEFT, usually we cast the layer norms in float32 for training stability reasons # therefore the input hidden states gets silently casted in float32. Hence, we need # cast them back in the correct dtype just to be sure everything works as expected. # This might slowdown training & inference so it is recommended to not cast the LayerNorms # in fp32. (DeepseekV3RMSNorm handles it correctly) input_dtype = query_states.dtype if input_dtype == torch.float32: # Handle the case where the model is quantized if hasattr(self.config, "_pre_quantization_dtype"): target_dtype = self.config._pre_quantization_dtype elif torch.is_autocast_enabled(): target_dtype = torch.get_autocast_gpu_dtype() else: target_dtype = ( self.q_proj.weight.dtype if self.q_lora_rank is None else self.q_a_proj.weight.dtype ) logger.warning_once( f"The input hidden states seems to be silently casted in float32, this might be related to" f" the fact you have upcasted embedding or layer norm layers in float32. We will cast back the input in" f" {target_dtype}." ) query_states = query_states.to(target_dtype) key_states = key_states.to(target_dtype) value_states = value_states.to(target_dtype) attn_output = self._flash_attention_forward( query_states, key_states, value_states, attention_mask, q_len, dropout=dropout_rate, softmax_scale=self.softmax_scale, ) if self.q_head_dim != self.v_head_dim: attn_output = attn_output[:, :, :, : self.v_head_dim] attn_output = attn_output.reshape( bsz, q_len, self.num_heads * self.v_head_dim ).contiguous() attn_output = self.o_proj(attn_output) if not output_attentions: attn_weights = None return attn_output, attn_weights, past_key_value def _flash_attention_forward( self, query_states, key_states, value_states, attention_mask, query_length, dropout=0.0, softmax_scale=None, ): """ Calls the forward method of Flash Attention - if the input hidden states contain at least one padding token first unpad the input, then computes the attention scores and pad the final attention scores. Args: query_states (`torch.Tensor`): Input query states to be passed to Flash Attention API key_states (`torch.Tensor`): Input key states to be passed to Flash Attention API value_states (`torch.Tensor`): Input value states to be passed to Flash Attention API attention_mask (`torch.Tensor`): The padding mask - corresponds to a tensor of size `(batch_size, seq_len)` where 0 stands for the position of padding tokens and 1 for the position of non-padding tokens. dropout (`int`, *optional*): Attention dropout softmax_scale (`float`, *optional*): The scaling of QK^T before applying softmax. Default to 1 / sqrt(head_dim) """ if not self._flash_attn_uses_top_left_mask: causal = self.is_causal else: # TODO: Remove the `query_length != 1` check once Flash Attention for RoCm is bumped to 2.1. For details, please see the comment in DeepseekV3FlashAttention2 __init__. causal = self.is_causal and query_length != 1 # Contains at least one padding token in the sequence if attention_mask is not None: batch_size = query_states.shape[0] ( query_states, key_states, value_states, indices_q, cu_seq_lens, max_seq_lens, ) = self._upad_input( query_states, key_states, value_states, attention_mask, query_length ) cu_seqlens_q, cu_seqlens_k = cu_seq_lens max_seqlen_in_batch_q, max_seqlen_in_batch_k = max_seq_lens attn_output_unpad = flash_attn_varlen_func( query_states, key_states, value_states, cu_seqlens_q=cu_seqlens_q, cu_seqlens_k=cu_seqlens_k, max_seqlen_q=max_seqlen_in_batch_q, max_seqlen_k=max_seqlen_in_batch_k, dropout_p=dropout, softmax_scale=softmax_scale, causal=causal, ) attn_output = pad_input( attn_output_unpad, indices_q, batch_size, query_length ) else: attn_output = flash_attn_func( query_states, key_states, value_states, dropout, softmax_scale=softmax_scale, causal=causal, ) return attn_output def _upad_input( self, query_layer, key_layer, value_layer, attention_mask, query_length ): indices_k, cu_seqlens_k, max_seqlen_in_batch_k = _get_unpad_data(attention_mask) batch_size, kv_seq_len, num_key_value_heads, head_dim = key_layer.shape key_layer = index_first_axis( key_layer.reshape(batch_size * kv_seq_len, num_key_value_heads, head_dim), indices_k, ) value_layer = index_first_axis( value_layer.reshape(batch_size * kv_seq_len, num_key_value_heads, head_dim), indices_k, ) if query_length == kv_seq_len: query_layer = index_first_axis( query_layer.reshape(batch_size * kv_seq_len, self.num_heads, head_dim), indices_k, ) cu_seqlens_q = cu_seqlens_k max_seqlen_in_batch_q = max_seqlen_in_batch_k indices_q = indices_k elif query_length == 1: max_seqlen_in_batch_q = 1 cu_seqlens_q = torch.arange( batch_size + 1, dtype=torch.int32, device=query_layer.device ) # There is a memcpy here, that is very bad. indices_q = cu_seqlens_q[:-1] query_layer = query_layer.squeeze(1) else: # The -q_len: slice assumes left padding. attention_mask = attention_mask[:, -query_length:] query_layer, indices_q, cu_seqlens_q, max_seqlen_in_batch_q = unpad_input( query_layer, attention_mask ) return ( query_layer, key_layer, value_layer, indices_q, (cu_seqlens_q, cu_seqlens_k), (max_seqlen_in_batch_q, max_seqlen_in_batch_k), ) ATTENTION_CLASSES = { "eager": DeepseekV3Attention, "flash_attention_2": DeepseekV3FlashAttention2, } class DeepseekV3DecoderLayer(nn.Module): def __init__(self, config: DeepseekV3Config, layer_idx: int): super().__init__() self.hidden_size = config.hidden_size self.self_attn = ATTENTION_CLASSES[config._attn_implementation]( config=config, layer_idx=layer_idx ) self.mlp = ( DeepseekV3MoE(config) if ( config.n_routed_experts is not None and layer_idx >= config.first_k_dense_replace and layer_idx % config.moe_layer_freq == 0 ) else DeepseekV3MLP(config) ) self.input_layernorm = DeepseekV3RMSNorm( config.hidden_size, eps=config.rms_norm_eps ) self.post_attention_layernorm = DeepseekV3RMSNorm( config.hidden_size, eps=config.rms_norm_eps ) def forward( self, hidden_states: torch.Tensor, attention_mask: Optional[torch.Tensor] = None, position_ids: Optional[torch.LongTensor] = None, past_key_value: Optional[Tuple[torch.Tensor]] = None, output_attentions: Optional[bool] = False, use_cache: Optional[bool] = False, **kwargs, ) -> Tuple[ torch.FloatTensor, Optional[Tuple[torch.FloatTensor, torch.FloatTensor]] ]: """ Args: hidden_states (`torch.FloatTensor`): input to the layer of shape `(batch, seq_len, embed_dim)` attention_mask (`torch.FloatTensor`, *optional*): attention mask of size `(batch_size, sequence_length)` if flash attention is used or `(batch_size, 1, query_sequence_length, key_sequence_length)` if default attention is used. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. use_cache (`bool`, *optional*): If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding (see `past_key_values`). past_key_value (`Tuple(torch.FloatTensor)`, *optional*): cached past key and value projection states """ if "padding_mask" in kwargs: warnings.warn( "Passing `padding_mask` is deprecated and will be removed in v4.37. Please make sure use `attention_mask` instead.`" ) residual = hidden_states hidden_states = self.input_layernorm(hidden_states) # Self Attention hidden_states, self_attn_weights, present_key_value = self.self_attn( hidden_states=hidden_states, attention_mask=attention_mask, position_ids=position_ids, past_key_value=past_key_value, output_attentions=output_attentions, use_cache=use_cache, **kwargs, ) hidden_states = residual + hidden_states # Fully Connected residual = hidden_states hidden_states = self.post_attention_layernorm(hidden_states) hidden_states = self.mlp(hidden_states) hidden_states = residual + hidden_states outputs = (hidden_states,) if output_attentions: outputs += (self_attn_weights,) if use_cache: outputs += (present_key_value,) return outputs DeepseekV3_START_DOCSTRING = r""" This model inherits from [`PreTrainedModel`]. Check the superclass documentation for the generic methods the library implements for all its model (such as downloading or saving, resizing the input embeddings, pruning heads etc.) This model is also a PyTorch [torch.nn.Module](https://pytorch.org/docs/stable/nn.html#torch.nn.Module) subclass. Use it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage and behavior. Parameters: config ([`DeepseekV3Config`]): Model configuration class with all the parameters of the model. Initializing with a config file does not load the weights associated with the model, only the configuration. Check out the [`~PreTrainedModel.from_pretrained`] method to load the model weights. """ @add_start_docstrings( "The bare DeepseekV3 Model outputting raw hidden-states without any specific head on top.", DeepseekV3_START_DOCSTRING, ) class DeepseekV3PreTrainedModel(PreTrainedModel): config_class = DeepseekV3Config base_model_prefix = "model" supports_gradient_checkpointing = True _no_split_modules = ["DeepseekV3DecoderLayer"] _skip_keys_device_placement = "past_key_values" _supports_flash_attn_2 = True _supports_cache_class = True def _init_weights(self, module): std = self.config.initializer_range if isinstance(module, nn.Linear): module.weight.data.normal_(mean=0.0, std=std) if module.bias is not None: module.bias.data.zero_() elif isinstance(module, nn.Embedding): module.weight.data.normal_(mean=0.0, std=std) if module.padding_idx is not None: module.weight.data[module.padding_idx].zero_() DeepseekV3_INPUTS_DOCSTRING = r""" Args: input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. Padding will be ignored by default should you provide it. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. If `past_key_values` is used, optionally only the last `input_ids` have to be input (see `past_key_values`). If you want to change padding behavior, you should read [`modeling_opt._prepare_decoder_attention_mask`] and modify to your needs. See diagram 1 in [the paper](https://arxiv.org/abs/1910.13461) for more information on the default strategy. - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. position_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Indices of positions of each input sequence tokens in the position embeddings. Selected in the range `[0, config.n_positions - 1]`. [What are position IDs?](../glossary#position-ids) past_key_values (`Cache` or `tuple(tuple(torch.FloatTensor))`, *optional*): Pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention blocks) that can be used to speed up sequential decoding. This typically consists in the `past_key_values` returned by the model at a previous stage of decoding, when `use_cache=True` or `config.use_cache=True`. Two formats are allowed: - a [`~cache_utils.Cache`] instance; - Tuple of `tuple(torch.FloatTensor)` of length `config.n_layers`, with each tuple having 2 tensors of shape `(batch_size, num_heads, sequence_length, embed_size_per_head)`). This is also known as the legacy cache format. The model will output the same cache format that is fed as input. If no `past_key_values` are passed, the legacy cache format will be returned. If `past_key_values` are used, the user can optionally input only the last `input_ids` (those that don't have their past key value states given to this model) of shape `(batch_size, 1)` instead of all `input_ids` of shape `(batch_size, sequence_length)`. inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. use_cache (`bool`, *optional*): If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding (see `past_key_values`). output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. """ @add_start_docstrings( "The bare DeepseekV3 Model outputting raw hidden-states without any specific head on top.", DeepseekV3_START_DOCSTRING, ) class DeepseekV3Model(DeepseekV3PreTrainedModel): """ Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`DeepseekV3DecoderLayer`] Args: config: DeepseekV3Config """ def __init__(self, config: DeepseekV3Config): super().__init__(config) self.padding_idx = config.pad_token_id self.vocab_size = config.vocab_size self.embed_tokens = nn.Embedding( config.vocab_size, config.hidden_size, self.padding_idx ) self.layers = nn.ModuleList( [ DeepseekV3DecoderLayer(config, layer_idx) for layer_idx in range(config.num_hidden_layers) ] ) self._use_flash_attention_2 = config._attn_implementation == "flash_attention_2" self.norm = DeepseekV3RMSNorm(config.hidden_size, eps=config.rms_norm_eps) self.gradient_checkpointing = False # Initialize weights and apply final processing self.post_init() def get_input_embeddings(self): return self.embed_tokens def set_input_embeddings(self, value): self.embed_tokens = value @add_start_docstrings_to_model_forward(DeepseekV3_INPUTS_DOCSTRING) def forward( self, input_ids: torch.LongTensor = None, attention_mask: Optional[torch.Tensor] = None, position_ids: Optional[torch.LongTensor] = None, past_key_values: Optional[List[torch.FloatTensor]] = None, inputs_embeds: Optional[torch.FloatTensor] = None, use_cache: Optional[bool] = None, output_attentions: Optional[bool] = None, output_hidden_states: Optional[bool] = None, return_dict: Optional[bool] = None, ) -> Union[Tuple, BaseModelOutputWithPast]: output_attentions = ( output_attentions if output_attentions is not None else self.config.output_attentions ) output_hidden_states = ( output_hidden_states if output_hidden_states is not None else self.config.output_hidden_states ) use_cache = use_cache if use_cache is not None else self.config.use_cache return_dict = ( return_dict if return_dict is not None else self.config.use_return_dict ) # retrieve input_ids and inputs_embeds if input_ids is not None and inputs_embeds is not None: raise ValueError( "You cannot specify both input_ids and inputs_embeds at the same time" ) elif input_ids is not None: batch_size, seq_length = input_ids.shape[:2] elif inputs_embeds is not None: batch_size, seq_length = inputs_embeds.shape[:2] else: raise ValueError("You have to specify either input_ids or inputs_embeds") past_key_values_length = 0 if use_cache: use_legacy_cache = not isinstance(past_key_values, Cache) if use_legacy_cache: past_key_values = DynamicCache.from_legacy_cache(past_key_values) past_key_values_length = past_key_values.get_usable_length(seq_length) if position_ids is None: device = input_ids.device if input_ids is not None else inputs_embeds.device position_ids = torch.arange( past_key_values_length, seq_length + past_key_values_length, dtype=torch.long, device=device, ) position_ids = position_ids.unsqueeze(0) if inputs_embeds is None: inputs_embeds = self.embed_tokens(input_ids) if self._use_flash_attention_2: # 2d mask is passed through the layers attention_mask = ( attention_mask if (attention_mask is not None and 0 in attention_mask) else None ) else: # 4d mask is passed through the layers attention_mask = _prepare_4d_causal_attention_mask( attention_mask, (batch_size, seq_length), inputs_embeds, past_key_values_length, ) # embed positions hidden_states = inputs_embeds # decoder layers all_hidden_states = () if output_hidden_states else None all_self_attns = () if output_attentions else None next_decoder_cache = None for decoder_layer in self.layers: if output_hidden_states: all_hidden_states += (hidden_states,) layer_outputs = decoder_layer( hidden_states, attention_mask=attention_mask, position_ids=position_ids, past_key_value=past_key_values, output_attentions=output_attentions, use_cache=use_cache, ) hidden_states = layer_outputs[0] if use_cache: next_decoder_cache = layer_outputs[2 if output_attentions else 1] if output_attentions: all_self_attns += (layer_outputs[1],) hidden_states = self.norm(hidden_states) # add hidden states from the last decoder layer if output_hidden_states: all_hidden_states += (hidden_states,) next_cache = None if use_cache: next_cache = ( next_decoder_cache.to_legacy_cache() if use_legacy_cache else next_decoder_cache ) if not return_dict: return tuple( v for v in [hidden_states, next_cache, all_hidden_states, all_self_attns] if v is not None ) return BaseModelOutputWithPast( last_hidden_state=hidden_states, past_key_values=next_cache, hidden_states=all_hidden_states, attentions=all_self_attns, ) class DeepseekV3ForCausalLM(DeepseekV3PreTrainedModel): _tied_weights_keys = ["lm_head.weight"] def __init__(self, config): super().__init__(config) self.model = DeepseekV3Model(config) self.vocab_size = config.vocab_size self.lm_head = nn.Linear(config.hidden_size, config.vocab_size, bias=False) # Initialize weights and apply final processing self.post_init() def get_input_embeddings(self): return self.model.embed_tokens def set_input_embeddings(self, value): self.model.embed_tokens = value def get_output_embeddings(self): return self.lm_head def set_output_embeddings(self, new_embeddings): self.lm_head = new_embeddings def set_decoder(self, decoder): self.model = decoder def get_decoder(self): return self.model @add_start_docstrings_to_model_forward(DeepseekV3_INPUTS_DOCSTRING) @replace_return_docstrings( output_type=CausalLMOutputWithPast, config_class=_CONFIG_FOR_DOC ) def forward( self, input_ids: torch.LongTensor = None, attention_mask: Optional[torch.Tensor] = None, position_ids: Optional[torch.LongTensor] = None, past_key_values: Optional[List[torch.FloatTensor]] = None, inputs_embeds: Optional[torch.FloatTensor] = None, labels: Optional[torch.LongTensor] = None, use_cache: Optional[bool] = None, output_attentions: Optional[bool] = None, output_hidden_states: Optional[bool] = None, return_dict: Optional[bool] = None, ) -> Union[Tuple, CausalLMOutputWithPast]: r""" Args: labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, transformers., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, transformers., config.vocab_size]`. Returns: Example: ```python >>> from transformers import AutoTokenizer, DeepseekV3ForCausalLM >>> model = DeepseekV3ForCausalLM.from_pretrained(PATH_TO_CONVERTED_WEIGHTS) >>> tokenizer = AutoTokenizer.from_pretrained(PATH_TO_CONVERTED_TOKENIZER) >>> prompt = "Hey, are you conscious? Can you talk to me?" >>> inputs = tokenizer(prompt, return_tensors="pt") >>> # Generate >>> generate_ids = model.generate(inputs.input_ids, max_length=30) >>> tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0] "Hey, are you conscious? Can you talk to me?\nI'm not conscious, but I can talk to you." ```""" output_attentions = ( output_attentions if output_attentions is not None else self.config.output_attentions ) output_hidden_states = ( output_hidden_states if output_hidden_states is not None else self.config.output_hidden_states ) return_dict = ( return_dict if return_dict is not None else self.config.use_return_dict ) # decoder outputs consists of (dec_features, layer_state, dec_hidden, dec_attn) outputs = self.model( input_ids=input_ids, attention_mask=attention_mask, position_ids=position_ids, past_key_values=past_key_values, inputs_embeds=inputs_embeds, use_cache=use_cache, output_attentions=output_attentions, output_hidden_states=output_hidden_states, return_dict=return_dict, ) hidden_states = outputs[0] logits = self.lm_head(hidden_states) logits = logits.float() loss = None if labels is not None: # Shift so that tokens < n predict n shift_logits = logits[..., :-1, :].contiguous() shift_labels = labels[..., 1:].contiguous() # Flatten the tokens loss_fct = CrossEntropyLoss() shift_logits = shift_logits.view(-1, self.config.vocab_size) shift_labels = shift_labels.view(-1) # Enable model parallelism shift_labels = shift_labels.to(shift_logits.device) loss = loss_fct(shift_logits, shift_labels) if not return_dict: output = (logits,) + outputs[1:] return (loss,) + output if loss is not None else output return CausalLMOutputWithPast( loss=loss, logits=logits, past_key_values=outputs.past_key_values, hidden_states=outputs.hidden_states, attentions=outputs.attentions, ) def prepare_inputs_for_generation( self, input_ids, past_key_values=None, attention_mask=None, inputs_embeds=None, **kwargs, ): if past_key_values is not None: if isinstance(past_key_values, Cache): cache_length = past_key_values.get_seq_length() past_length = past_key_values.seen_tokens max_cache_length = past_key_values.get_max_length() else: cache_length = past_length = past_key_values[0][0].shape[2] max_cache_length = None # Keep only the unprocessed tokens: # 1 - If the length of the attention_mask exceeds the length of input_ids, then we are in a setting where # some of the inputs are exclusivelly passed as part of the cache (e.g. when passing input_embeds as # input) if ( attention_mask is not None and attention_mask.shape[1] > input_ids.shape[1] ): input_ids = input_ids[:, -(attention_mask.shape[1] - past_length) :] # 2 - If the past_length is smaller than input_ids', then input_ids holds all input tokens. We can discard # input_ids based on the past_length. elif past_length < input_ids.shape[1]: input_ids = input_ids[:, past_length:] # 3 - Otherwise (past_length >= input_ids.shape[1]), let's assume input_ids only has unprocessed tokens. # If we are about to go beyond the maximum cache length, we need to crop the input attention mask. if ( max_cache_length is not None and attention_mask is not None and cache_length + input_ids.shape[1] > max_cache_length ): attention_mask = attention_mask[:, -max_cache_length:] position_ids = kwargs.get("position_ids", None) if attention_mask is not None and position_ids is None: # create position_ids on the fly for batch generation position_ids = attention_mask.long().cumsum(-1) - 1 position_ids.masked_fill_(attention_mask == 0, 1) if past_key_values: position_ids = position_ids[:, -input_ids.shape[1] :] # if `inputs_embeds` are passed, we only want to use them in the 1st generation step if inputs_embeds is not None and past_key_values is None: model_inputs = {"inputs_embeds": inputs_embeds} else: model_inputs = {"input_ids": input_ids} model_inputs.update( { "position_ids": position_ids, "past_key_values": past_key_values, "use_cache": kwargs.get("use_cache"), "attention_mask": attention_mask, } ) return model_inputs @staticmethod def _reorder_cache(past_key_values, beam_idx): reordered_past = () for layer_past in past_key_values: reordered_past += ( tuple( past_state.index_select(0, beam_idx.to(past_state.device)) for past_state in layer_past ), ) return reordered_past @add_start_docstrings( """ The DeepseekV3 Model transformer with a sequence classification head on top (linear layer). [`DeepseekV3ForSequenceClassification`] uses the last token in order to do the classification, as other causal models (e.g. GPT-2) do. Since it does classification on the last token, it requires to know the position of the last token. If a `pad_token_id` is defined in the configuration, it finds the last token that is not a padding token in each row. If no `pad_token_id` is defined, it simply takes the last value in each row of the batch. Since it cannot guess the padding tokens when `inputs_embeds` are passed instead of `input_ids`, it does the same (take the last value in each row of the batch). """, DeepseekV3_START_DOCSTRING, ) class DeepseekV3ForSequenceClassification(DeepseekV3PreTrainedModel): def __init__(self, config): super().__init__(config) self.num_labels = config.num_labels self.model = DeepseekV3Model(config) self.score = nn.Linear(config.hidden_size, self.num_labels, bias=False) # Initialize weights and apply final processing self.post_init() def get_input_embeddings(self): return self.model.embed_tokens def set_input_embeddings(self, value): self.model.embed_tokens = value @add_start_docstrings_to_model_forward(DeepseekV3_INPUTS_DOCSTRING) def forward( self, input_ids: torch.LongTensor = None, attention_mask: Optional[torch.Tensor] = None, position_ids: Optional[torch.LongTensor] = None, past_key_values: Optional[List[torch.FloatTensor]] = None, inputs_embeds: Optional[torch.FloatTensor] = None, labels: Optional[torch.LongTensor] = None, use_cache: Optional[bool] = None, output_attentions: Optional[bool] = None, output_hidden_states: Optional[bool] = None, return_dict: Optional[bool] = None, ) -> Union[Tuple, SequenceClassifierOutputWithPast]: r""" labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, transformers., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). """ return_dict = ( return_dict if return_dict is not None else self.config.use_return_dict ) transformer_outputs = self.model( input_ids, attention_mask=attention_mask, position_ids=position_ids, past_key_values=past_key_values, inputs_embeds=inputs_embeds, use_cache=use_cache, output_attentions=output_attentions, output_hidden_states=output_hidden_states, return_dict=return_dict, ) hidden_states = transformer_outputs[0] logits = self.score(hidden_states) if input_ids is not None: batch_size = input_ids.shape[0] else: batch_size = inputs_embeds.shape[0] if self.config.pad_token_id is None and batch_size != 1: raise ValueError( "Cannot handle batch sizes > 1 if no padding token is defined." ) if self.config.pad_token_id is None: sequence_lengths = -1 else: if input_ids is not None: sequence_lengths = ( torch.eq(input_ids, self.config.pad_token_id).int().argmax(-1) - 1 ).to(logits.device) else: sequence_lengths = -1 pooled_logits = logits[ torch.arange(batch_size, device=logits.device), sequence_lengths ] loss = None if labels is not None: labels = labels.to(logits.device) if self.config.problem_type is None: if self.num_labels == 1: self.config.problem_type = "regression" elif self.num_labels > 1 and ( labels.dtype == torch.long or labels.dtype == torch.int ): self.config.problem_type = "single_label_classification" else: self.config.problem_type = "multi_label_classification" if self.config.problem_type == "regression": loss_fct = MSELoss() if self.num_labels == 1: loss = loss_fct(pooled_logits.squeeze(), labels.squeeze()) else: loss = loss_fct(pooled_logits, labels) elif self.config.problem_type == "single_label_classification": loss_fct = CrossEntropyLoss() loss = loss_fct( pooled_logits.view(-1, self.num_labels), labels.view(-1) ) elif self.config.problem_type == "multi_label_classification": loss_fct = BCEWithLogitsLoss() loss = loss_fct(pooled_logits, labels) if not return_dict: output = (pooled_logits,) + transformer_outputs[1:] return ((loss,) + output) if loss is not None else output return SequenceClassifierOutputWithPast( loss=loss, logits=pooled_logits, past_key_values=transformer_outputs.past_key_values, hidden_states=transformer_outputs.hidden_states, attentions=transformer_outputs.attentions, ) ================================================================================ C.4 generation_config.json (171 bytes) ================================================================================ { "_from_model_config": true, "bos_token_id": 0, "eos_token_id": 1, "do_sample": true, "temperature": 0.6, "top_p": 0.95, "transformers_version": "4.39.3" } ================================================================================ C.5 tokenizer_config.json (3,594 bytes) ================================================================================ { "add_bos_token": true, "add_eos_token": false, "bos_token": { "__type": "AddedToken", "content": "<｜begin▁of▁sentence｜>", "lstrip": false, "normalized": true, "rstrip": false, "single_word": false }, "clean_up_tokenization_spaces": false, "eos_token": { "__type": "AddedToken", "content": "<｜end▁of▁sentence｜>", "lstrip": false, "normalized": true, "rstrip": false, "single_word": false }, "legacy": true, "model_max_length": 16384, "pad_token": { "__type": "AddedToken", "content": "<｜end▁of▁sentence｜>", "lstrip": false, "normalized": true, "rstrip": false, "single_word": false }, "sp_model_kwargs": {}, "unk_token": null, "tokenizer_class": "LlamaTokenizerFast", "chat_template": "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set ns = namespace(is_first=false, is_tool=false, is_output_first=true, system_prompt='', is_first_sp=true) %}{%- for message in messages %}{%- if message['role'] == 'system' %}{%- if ns.is_first_sp %}{% set ns.system_prompt = ns.system_prompt + message['content'] %}{% set ns.is_first_sp = false %}{%- else %}{% set ns.system_prompt = ns.system_prompt + '\\n\\n' + message['content'] %}{%- endif %}{%- endif %}{%- endfor %}{{ bos_token }}{{ ns.system_prompt }}{%- for message in messages %}{%- if message['role'] == 'user' %}{%- set ns.is_tool = false -%}{{'<｜User｜>' + message['content']}}{%- endif %}{%- if message['role'] == 'assistant' and 'tool_calls' in message %}{%- set ns.is_tool = false -%}{%- for tool in message['tool_calls'] %}{%- if not ns.is_first %}{%- if message['content'] is none %}{{'<｜Assistant｜><｜tool▁calls▁begin｜><｜tool▁call▁begin｜>' + tool['type'] + '<｜tool▁sep｜>' + tool['function']['name'] + '\\n' + '```json' + '\\n' + tool['function']['arguments'] + '\\n' + '```' + '<｜tool▁call▁end｜>'}}{%- else %}{{'<｜Assistant｜>' + message['content'] + '<｜tool▁calls▁begin｜><｜tool▁call▁begin｜>' + tool['type'] + '<｜tool▁sep｜>' + tool['function']['name'] + '\\n' + '```json' + '\\n' + tool['function']['arguments'] + '\\n' + '```' + '<｜tool▁call▁end｜>'}}{%- endif %}{%- set ns.is_first = true -%}{%- else %}{{'\\n' + '<｜tool▁call▁begin｜>' + tool['type'] + '<｜tool▁sep｜>' + tool['function']['name'] + '\\n' + '```json' + '\\n' + tool['function']['arguments'] + '\\n' + '```' + '<｜tool▁call▁end｜>'}}{%- endif %}{%- endfor %}{{'<｜tool▁calls▁end｜><｜end▁of▁sentence｜>'}}{%- endif %}{%- if message['role'] == 'assistant' and 'tool_calls' not in message %}{%- if ns.is_tool %}{{'<｜tool▁outputs▁end｜>' + message['content'] + '<｜end▁of▁sentence｜>'}}{%- set ns.is_tool = false -%}{%- else %}{% set content = message['content'] %}{% if '' in content %}{% set content = content.split('')[-1] %}{% endif %}{{'<｜Assistant｜>' + content + '<｜end▁of▁sentence｜>'}}{%- endif %}{%- endif %}{%- if message['role'] == 'tool' %}{%- set ns.is_tool = true -%}{%- if ns.is_output_first %}{{'<｜tool▁outputs▁begin｜><｜tool▁output▁begin｜>' + message['content'] + '<｜tool▁output▁end｜>'}}{%- set ns.is_output_first = false %}{%- else %}{{'<｜tool▁output▁begin｜>' + message['content'] + '<｜tool▁output▁end｜>'}}{%- endif %}{%- endif %}{%- endfor -%}{% if ns.is_tool %}{{'<｜tool▁outputs▁end｜>'}}{% endif %}{% if add_generation_prompt and not ns.is_tool %}{{'<｜Assistant｜>\\n'}}{% endif %}" } ================================================================================ C.6 tokenizer.json (7,847,602 bytes) ================================================================================ DELIVERED AS A SIDECAR FILE, not inline. See the note in B.1. path /home/joseph/DeepSeek-R1/tokenizer.json copy /home/joseph/deepseek-megamind-upload/deepseek/tokenizer.json bytes 7,847,602 sha256 ecb6f9fc369894346f0511f4074ca75cee5cd5f3b06d02f1ba35fcd39f8e121d inside 01-deepseek-source.tar.gz STRUCTURE top-level keys : version, truncation, padding, added_tokens, normalizer, pre_tokenizer, post_processor, decoder, model model.type : BPE model.vocab : 128,000 entries, ids 0..127,999 model.merges : 127,741 merge rules added_tokens : 818 entries (798 placeholders, 20 meaningful) truncation : null padding : null normalizer : {"type":"Sequence","normalizers":[]} -- EMPTY, no NFC/NFKC pre_tokenizer : Sequence of three Split rules then ByteLevel 1. Regex \p{N}{1,3} Isolated 2. Regex [CJK + Hiragana + Katakana]+ Isolated 3. a large punctuation/letter/whitespace regex post_processor : ByteLevel, add_prefix_space true, trim_offsets false decoder : ByteLevel, add_prefix_space true, trim_offsets true The 20 meaningful special tokens and their ids are enumerated in DEEPSEEK_R1_COMPLETE.txt part 11.2 and summarized in B.9 of this document. ================================================================================ C.7 model.safetensors.index.json (8,898,324 bytes) ================================================================================ DELIVERED AS A SIDECAR FILE, not inline. See the note in B.1. path /home/joseph/DeepSeek-R1/model.safetensors.index.json copy /home/joseph/deepseek-megamind-upload/deepseek/model.safetensors.index.json bytes 8,898,324 sha256 a2c5d238ab1e8301a117336e9797e0d3551cb8f6dc9dc787318b23d8f54c9248 inside 01-deepseek-source.tar.gz STRUCTURE top-level keys : ["metadata", "weight_map"] metadata : {"total_size": 1369062772000} weight_map : 91,991 entries, "": "model-000NN-of-000163.safetensors" naming 163 distinct shards example : "model.embed_tokens.weight" -> "model-00001-of-000163.safetensors" WARNING: metadata.total_size (1,369,062,772,000 = 1.245 TiB) overstates the real payload (688,574,839,360 = 641.29 GiB) by 1.99x, because the exporter costed FP8 elements at two bytes each. Never size allocations from it. deepseek_tensor_manifest.json (B.2) is strictly more useful than this file: it carries dtype, full shape, absolute byte offset, byte length and the linked scale tensor for all 91,991 tensors, none of which the index contains. ================================================================================ C.8 README.md — official model card (15,994 bytes) ================================================================================ --- license: mit library_name: transformers --- # DeepSeek-R1
DeepSeek-V3
Homepage Chat Hugging Face
Discord Wechat Twitter Follow
License

Paper Link👁️

## 1. Introduction We introduce our first-generation reasoning models, DeepSeek-R1-Zero and DeepSeek-R1. DeepSeek-R1-Zero, a model trained via large-scale reinforcement learning (RL) without supervised fine-tuning (SFT) as a preliminary step, demonstrated remarkable performance on reasoning. With RL, DeepSeek-R1-Zero naturally emerged with numerous powerful and interesting reasoning behaviors. However, DeepSeek-R1-Zero encounters challenges such as endless repetition, poor readability, and language mixing. To address these issues and further enhance reasoning performance, we introduce DeepSeek-R1, which incorporates cold-start data before RL. DeepSeek-R1 achieves performance comparable to OpenAI-o1 across math, code, and reasoning tasks. To support the research community, we have open-sourced DeepSeek-R1-Zero, DeepSeek-R1, and six dense models distilled from DeepSeek-R1 based on Llama and Qwen. DeepSeek-R1-Distill-Qwen-32B outperforms OpenAI-o1-mini across various benchmarks, achieving new state-of-the-art results for dense models. **NOTE: Before running DeepSeek-R1 series models locally, we kindly recommend reviewing the [Usage Recommendation](#usage-recommendations) section.**

## 2. Model Summary --- **Post-Training: Large-Scale Reinforcement Learning on the Base Model** - We directly apply reinforcement learning (RL) to the base model without relying on supervised fine-tuning (SFT) as a preliminary step. This approach allows the model to explore chain-of-thought (CoT) for solving complex problems, resulting in the development of DeepSeek-R1-Zero. DeepSeek-R1-Zero demonstrates capabilities such as self-verification, reflection, and generating long CoTs, marking a significant milestone for the research community. Notably, it is the first open research to validate that reasoning capabilities of LLMs can be incentivized purely through RL, without the need for SFT. This breakthrough paves the way for future advancements in this area. - We introduce our pipeline to develop DeepSeek-R1. The pipeline incorporates two RL stages aimed at discovering improved reasoning patterns and aligning with human preferences, as well as two SFT stages that serve as the seed for the model's reasoning and non-reasoning capabilities. We believe the pipeline will benefit the industry by creating better models. --- **Distillation: Smaller Models Can Be Powerful Too** - We demonstrate that the reasoning patterns of larger models can be distilled into smaller models, resulting in better performance compared to the reasoning patterns discovered through RL on small models. The open source DeepSeek-R1, as well as its API, will benefit the research community to distill better smaller models in the future. - Using the reasoning data generated by DeepSeek-R1, we fine-tuned several dense models that are widely used in the research community. The evaluation results demonstrate that the distilled smaller dense models perform exceptionally well on benchmarks. We open-source distilled 1.5B, 7B, 8B, 14B, 32B, and 70B checkpoints based on Qwen2.5 and Llama3 series to the community. ## 3. Model Downloads ### DeepSeek-R1 Models
| **Model** | **#Total Params** | **#Activated Params** | **Context Length** | **Download** | | :------------: | :------------: | :------------: | :------------: | :------------: | | DeepSeek-R1-Zero | 671B | 37B | 128K | [🤗 HuggingFace](https://huggingface.co/deepseek-ai/DeepSeek-R1-Zero) | | DeepSeek-R1 | 671B | 37B | 128K | [🤗 HuggingFace](https://huggingface.co/deepseek-ai/DeepSeek-R1) |
DeepSeek-R1-Zero & DeepSeek-R1 are trained based on DeepSeek-V3-Base. For more details regarding the model architecture, please refer to [DeepSeek-V3](https://github.com/deepseek-ai/DeepSeek-V3) repository. ### DeepSeek-R1-Distill Models
| **Model** | **Base Model** | **Download** | | :------------: | :------------: | :------------: | | DeepSeek-R1-Distill-Qwen-1.5B | [Qwen2.5-Math-1.5B](https://huggingface.co/Qwen/Qwen2.5-Math-1.5B) | [🤗 HuggingFace](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B) | | DeepSeek-R1-Distill-Qwen-7B | [Qwen2.5-Math-7B](https://huggingface.co/Qwen/Qwen2.5-Math-7B) | [🤗 HuggingFace](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B) | | DeepSeek-R1-Distill-Llama-8B | [Llama-3.1-8B](https://huggingface.co/meta-llama/Llama-3.1-8B) | [🤗 HuggingFace](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-8B) | | DeepSeek-R1-Distill-Qwen-14B | [Qwen2.5-14B](https://huggingface.co/Qwen/Qwen2.5-14B) | [🤗 HuggingFace](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-14B) | |DeepSeek-R1-Distill-Qwen-32B | [Qwen2.5-32B](https://huggingface.co/Qwen/Qwen2.5-32B) | [🤗 HuggingFace](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B) | | DeepSeek-R1-Distill-Llama-70B | [Llama-3.3-70B-Instruct](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct) | [🤗 HuggingFace](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-70B) |
DeepSeek-R1-Distill models are fine-tuned based on open-source models, using samples generated by DeepSeek-R1. We slightly change their configs and tokenizers. Please use our setting to run these models. ## 4. Evaluation Results ### DeepSeek-R1-Evaluation For all our models, the maximum generation length is set to 32,768 tokens. For benchmarks requiring sampling, we use a temperature of $0.6$, a top-p value of $0.95$, and generate 64 responses per query to estimate pass@1.
| Category | Benchmark (Metric) | Claude-3.5-Sonnet-1022 | GPT-4o 0513 | DeepSeek V3 | OpenAI o1-mini | OpenAI o1-1217 | DeepSeek R1 | |----------|-------------------|----------------------|------------|--------------|----------------|------------|--------------| | | Architecture | - | - | MoE | - | - | MoE | | | # Activated Params | - | - | 37B | - | - | 37B | | | # Total Params | - | - | 671B | - | - | 671B | | English | MMLU (Pass@1) | 88.3 | 87.2 | 88.5 | 85.2 | **91.8** | 90.8 | | | MMLU-Redux (EM) | 88.9 | 88.0 | 89.1 | 86.7 | - | **92.9** | | | MMLU-Pro (EM) | 78.0 | 72.6 | 75.9 | 80.3 | - | **84.0** | | | DROP (3-shot F1) | 88.3 | 83.7 | 91.6 | 83.9 | 90.2 | **92.2** | | | IF-Eval (Prompt Strict) | **86.5** | 84.3 | 86.1 | 84.8 | - | 83.3 | | | GPQA-Diamond (Pass@1) | 65.0 | 49.9 | 59.1 | 60.0 | **75.7** | 71.5 | | | SimpleQA (Correct) | 28.4 | 38.2 | 24.9 | 7.0 | **47.0** | 30.1 | | | FRAMES (Acc.) | 72.5 | 80.5 | 73.3 | 76.9 | - | **82.5** | | | AlpacaEval2.0 (LC-winrate) | 52.0 | 51.1 | 70.0 | 57.8 | - | **87.6** | | | ArenaHard (GPT-4-1106) | 85.2 | 80.4 | 85.5 | 92.0 | - | **92.3** | | Code | LiveCodeBench (Pass@1-COT) | 33.8 | 34.2 | - | 53.8 | 63.4 | **65.9** | | | Codeforces (Percentile) | 20.3 | 23.6 | 58.7 | 93.4 | **96.6** | 96.3 | | | Codeforces (Rating) | 717 | 759 | 1134 | 1820 | **2061** | 2029 | | | SWE Verified (Resolved) | **50.8** | 38.8 | 42.0 | 41.6 | 48.9 | 49.2 | | | Aider-Polyglot (Acc.) | 45.3 | 16.0 | 49.6 | 32.9 | **61.7** | 53.3 | | Math | AIME 2024 (Pass@1) | 16.0 | 9.3 | 39.2 | 63.6 | 79.2 | **79.8** | | | MATH-500 (Pass@1) | 78.3 | 74.6 | 90.2 | 90.0 | 96.4 | **97.3** | | | CNMO 2024 (Pass@1) | 13.1 | 10.8 | 43.2 | 67.6 | - | **78.8** | | Chinese | CLUEWSC (EM) | 85.4 | 87.9 | 90.9 | 89.9 | - | **92.8** | | | C-Eval (EM) | 76.7 | 76.0 | 86.5 | 68.9 | - | **91.8** | | | C-SimpleQA (Correct) | 55.4 | 58.7 | **68.0** | 40.3 | - | 63.7 |
### Distilled Model Evaluation
| Model | AIME 2024 pass@1 | AIME 2024 cons@64 | MATH-500 pass@1 | GPQA Diamond pass@1 | LiveCodeBench pass@1 | CodeForces rating | |------------------------------------------|------------------|-------------------|-----------------|----------------------|----------------------|-------------------| | GPT-4o-0513 | 9.3 | 13.4 | 74.6 | 49.9 | 32.9 | 759 | | Claude-3.5-Sonnet-1022 | 16.0 | 26.7 | 78.3 | 65.0 | 38.9 | 717 | | o1-mini | 63.6 | 80.0 | 90.0 | 60.0 | 53.8 | **1820** | | QwQ-32B-Preview | 44.0 | 60.0 | 90.6 | 54.5 | 41.9 | 1316 | | DeepSeek-R1-Distill-Qwen-1.5B | 28.9 | 52.7 | 83.9 | 33.8 | 16.9 | 954 | | DeepSeek-R1-Distill-Qwen-7B | 55.5 | 83.3 | 92.8 | 49.1 | 37.6 | 1189 | | DeepSeek-R1-Distill-Qwen-14B | 69.7 | 80.0 | 93.9 | 59.1 | 53.1 | 1481 | | DeepSeek-R1-Distill-Qwen-32B | **72.6** | 83.3 | 94.3 | 62.1 | 57.2 | 1691 | | DeepSeek-R1-Distill-Llama-8B | 50.4 | 80.0 | 89.1 | 49.0 | 39.6 | 1205 | | DeepSeek-R1-Distill-Llama-70B | 70.0 | **86.7** | **94.5** | **65.2** | **57.5** | 1633 |
## 5. Chat Website & API Platform You can chat with DeepSeek-R1 on DeepSeek's official website: [chat.deepseek.com](https://chat.deepseek.com), and switch on the button "DeepThink" We also provide OpenAI-Compatible API at DeepSeek Platform: [platform.deepseek.com](https://platform.deepseek.com/) ## 6. How to Run Locally ### DeepSeek-R1 Models Please visit [DeepSeek-V3](https://github.com/deepseek-ai/DeepSeek-V3) repo for more information about running DeepSeek-R1 locally. **NOTE: Hugging Face's Transformers has not been directly supported yet.** ### DeepSeek-R1-Distill Models DeepSeek-R1-Distill models can be utilized in the same manner as Qwen or Llama models. For instance, you can easily start a service using [vLLM](https://github.com/vllm-project/vllm): ```shell vllm serve deepseek-ai/DeepSeek-R1-Distill-Qwen-32B --tensor-parallel-size 2 --max-model-len 32768 --enforce-eager ``` You can also easily start a service using [SGLang](https://github.com/sgl-project/sglang) ```bash python3 -m sglang.launch_server --model deepseek-ai/DeepSeek-R1-Distill-Qwen-32B --trust-remote-code --tp 2 ``` ### Usage Recommendations **We recommend adhering to the following configurations when utilizing the DeepSeek-R1 series models, including benchmarking, to achieve the expected performance:** 1. Set the temperature within the range of 0.5-0.7 (0.6 is recommended) to prevent endless repetitions or incoherent outputs. 2. **Avoid adding a system prompt; all instructions should be contained within the user prompt.** 3. For mathematical problems, it is advisable to include a directive in your prompt such as: "Please reason step by step, and put your final answer within \boxed{}." 4. When evaluating model performance, it is recommended to conduct multiple tests and average the results. Additionally, we have observed that the DeepSeek-R1 series models tend to bypass thinking pattern (i.e., outputting "\\n\n\") when responding to certain queries, which can adversely affect the model's performance. **To ensure that the model engages in thorough reasoning, we recommend enforcing the model to initiate its response with "\\n" at the beginning of every output.** ## 7. License This code repository and the model weights are licensed under the [MIT License](https://github.com/deepseek-ai/DeepSeek-R1/blob/main/LICENSE). DeepSeek-R1 series support commercial use, allow for any modifications and derivative works, including, but not limited to, distillation for training other LLMs. Please note that: - DeepSeek-R1-Distill-Qwen-1.5B, DeepSeek-R1-Distill-Qwen-7B, DeepSeek-R1-Distill-Qwen-14B and DeepSeek-R1-Distill-Qwen-32B are derived from [Qwen-2.5 series](https://github.com/QwenLM/Qwen2.5), which are originally licensed under [Apache 2.0 License](https://huggingface.co/Qwen/Qwen2.5-1.5B/blob/main/LICENSE), and now finetuned with 800k samples curated with DeepSeek-R1. - DeepSeek-R1-Distill-Llama-8B is derived from Llama3.1-8B-Base and is originally licensed under [llama3.1 license](https://huggingface.co/meta-llama/Llama-3.1-8B/blob/main/LICENSE). - DeepSeek-R1-Distill-Llama-70B is derived from Llama3.3-70B-Instruct and is originally licensed under [llama3.3 license](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct/blob/main/LICENSE). ## 8. Citation ``` @misc{deepseekai2025deepseekr1incentivizingreasoningcapability, title={DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning}, author={DeepSeek-AI}, year={2025}, eprint={2501.12948}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2501.12948}, } ``` ## 9. Contact If you have any questions, please raise an issue or contact us at [[author/contact email redacted]]([author/contact email redacted]). ================================================================================ C.9 LICENSE (1,064 bytes) ================================================================================ MIT License Copyright (c) 2023 DeepSeek Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ================================================================================ C.10 .gitattributes (1,519 bytes) ================================================================================ *.7z filter=lfs diff=lfs merge=lfs -text *.arrow filter=lfs diff=lfs merge=lfs -text *.bin filter=lfs diff=lfs merge=lfs -text *.bz2 filter=lfs diff=lfs merge=lfs -text *.ckpt filter=lfs diff=lfs merge=lfs -text *.ftz filter=lfs diff=lfs merge=lfs -text *.gz filter=lfs diff=lfs merge=lfs -text *.h5 filter=lfs diff=lfs merge=lfs -text *.joblib filter=lfs diff=lfs merge=lfs -text *.lfs.* filter=lfs diff=lfs merge=lfs -text *.mlmodel filter=lfs diff=lfs merge=lfs -text *.model filter=lfs diff=lfs merge=lfs -text *.msgpack filter=lfs diff=lfs merge=lfs -text *.npy filter=lfs diff=lfs merge=lfs -text *.npz filter=lfs diff=lfs merge=lfs -text *.onnx filter=lfs diff=lfs merge=lfs -text *.ot filter=lfs diff=lfs merge=lfs -text *.parquet filter=lfs diff=lfs merge=lfs -text *.pb filter=lfs diff=lfs merge=lfs -text *.pickle filter=lfs diff=lfs merge=lfs -text *.pkl filter=lfs diff=lfs merge=lfs -text *.pt filter=lfs diff=lfs merge=lfs -text *.pth filter=lfs diff=lfs merge=lfs -text *.rar filter=lfs diff=lfs merge=lfs -text *.safetensors filter=lfs diff=lfs merge=lfs -text saved_model/**/* filter=lfs diff=lfs merge=lfs -text *.tar.* filter=lfs diff=lfs merge=lfs -text *.tar filter=lfs diff=lfs merge=lfs -text *.tflite filter=lfs diff=lfs merge=lfs -text *.tgz filter=lfs diff=lfs merge=lfs -text *.wasm filter=lfs diff=lfs merge=lfs -text *.xz filter=lfs diff=lfs merge=lfs -text *.zip filter=lfs diff=lfs merge=lfs -text *.zst filter=lfs diff=lfs merge=lfs -text *tfevents* filter=lfs diff=lfs merge=lfs -text ================================================================================ C.11 SCANNER AND BENCHMARK SCRIPTS (all created for this work) ================================================================================ -------------------------------------------------------------------------------- C.11.1 download_r1.py (641 bytes) -------------------------------------------------------------------------------- #!/usr/bin/env python3 import os, sys, traceback print("=== download_r1 START pid",os.getpid(),"===",flush=True) try: from huggingface_hub import snapshot_download print("import OK; calling snapshot_download (this resolves ~160 shards first)...",flush=True) path=snapshot_download( "deepseek-ai/DeepSeek-R1", local_dir="/home/joseph/DeepSeek-R1", ignore_patterns=["*.pth","*.gguf","original/*"], max_workers=8, ) print("=== DOWNLOAD COMPLETE ===",path,flush=True) except SystemExit: raise except BaseException: print("=== ERROR ===",flush=True); traceback.print_exc(); sys.exit(1) -------------------------------------------------------------------------------- C.11.2 launch_r1.sh (590 bytes) -------------------------------------------------------------------------------- #!/bin/bash # Persistent R1 download: systemd unit as joseph, with joseph's hf path pinned so python finds it. Resumes prior 719MB. sudo systemctl reset-failed r1full 2>/dev/null sudo systemd-run --unit=r1full --uid=joseph --setenv=HOME=/home/joseph \ --setenv=PYTHONPATH=/home/joseph/.local/lib/python3.14/site-packages \ --working-directory=/home/joseph python3 /home/joseph/download_r1.py sleep 25 echo "=== active: $(systemctl is-active r1full) ===" sudo journalctl -u r1full --no-pager -n 5 --output=cat 2>/dev/null echo "=== size ==="; du -sh /home/joseph/DeepSeek-R1 2>/dev/null -------------------------------------------------------------------------------- C.11.3 dl_test.py (421 bytes) -------------------------------------------------------------------------------- #!/usr/bin/env python3 import time, sys from huggingface_hub import hf_hub_download t=time.time() try: p=hf_hub_download("deepseek-ai/DeepSeek-R1","config.json",local_dir="/home/joseph/DeepSeek-R1") print("GOT config.json ->",p,"in",round(time.time()-t,1),"s",flush=True) import os; print("size:",os.path.getsize(p),"bytes") except Exception as e: print("ERROR:",type(e).__name__,str(e)[:300],flush=True) -------------------------------------------------------------------------------- C.11.4 scan_headers.py (2349 bytes) -------------------------------------------------------------------------------- import json, os, struct, collections, re, sys D=os.path.expanduser("~/DeepSeek-R1") files=sorted(f for f in os.listdir(D) if f.endswith(".safetensors")) print("shards:",len(files)) tot_tensors=0; tot_bytes=0 dtype_ct=collections.Counter(); dtype_bytes=collections.Counter() fam_ct=collections.Counter(); fam_bytes=collections.Counter(); fam_dtypes=collections.defaultdict(set) shapes=collections.defaultdict(set) layers=set(); allnames=[] per_shard=[] for fn in files: p=os.path.join(D,fn) with open(p,'rb') as fh: n=struct.unpack('13,} B {k[len(pref):]}") ne=sorted(set(int(m.group(1)) for k in got if k.startswith(pref) and (m:=re.search(r'experts\.(\d+)\.',k)))) print(f" --- non-expert subtotal: {tot:,} B = {tot/2**20:.1f} MiB ({len(ks)} tensors)") if ne: print(f" --- plus routed experts {min(ne)}..{max(ne)} ({len(ne)} of them) x 6 tensors = {len(ne)*44050944:,} B") print() show("model.layers.0.","LAYER 0 - DENSE LAYER (also layers 1,2) - COMPLETE MANIFEST") show("model.layers.3.","LAYER 3 - MoE LAYER (also layers 4..60) - COMPLETE MANIFEST") show("model.layers.61.","LAYER 61 - MTP LAYER - COMPLETE MANIFEST") -------------------------------------------------------------------------------- C.11.6 bench_ssd.sh (1473 bytes) -------------------------------------------------------------------------------- #!/bin/bash # bench_ssd.sh -- drive diagnosis for the expert-streaming read path. # Decides whether to fan reads out (concurrency headroom) or cut seeks. D=~/DeepSeek-R1 echo "--- single stream, 8 MiB blocks, 2 GiB ---" [REDACTED: credential-associated original line 3561; original retained outside this bundle.] dd if=$D/model-00080-of-000163.safetensors of=/dev/null bs=8M count=256 2>&1 | tail -1 echo "--- same file again (page-cache warm) ---" dd if=$D/model-00080-of-000163.safetensors of=/dev/null bs=8M count=256 2>&1 | tail -1 echo "--- 4 parallel streams, 4 different shards, 512 MiB each ---" S=$(date +%s.%N) for n in 81 82 83 84; do dd if=$D/model-000$n-of-000163.safetensors of=/dev/null bs=8M count=64 2>/dev/null & done; wait E=$(date +%s.%N) echo "4x512MiB = 2048 MiB in $(echo "$E-$S"|bc)s -> $(echo "scale=1;2048/($E-$S)"|bc) MB/s aggregate" echo "--- 56 scattered 4 MiB preads (per-expert scatter simulation) ---" python3 - <<'PY' import os,time,random p=os.path.expanduser("~/DeepSeek-R1/model-00085-of-000163.safetensors") sz=os.path.getsize(p); fd=os.open(p,os.O_RDONLY); random.seed(7) t=time.time(); tot=0 for _ in range(56): off=random.randrange(0,sz-4*1024*1024) tot+=len(os.pread(fd,4*1024*1024,off)) d=time.time()-t print(f" 56 scattered 4MiB preads: {tot/2**20:.0f} MiB in {d:.2f}s -> {tot/2**20/d:.0f} MB/s") os.close(fd) PY echo "--- device ---"; df -h ~/DeepSeek-R1 | tail -1; lsblk 2>/dev/null | head -6 -------------------------------------------------------------------------------- C.11.7 map_shards_to_layers.py (1015 bytes) -------------------------------------------------------------------------------- import json,os,struct,re,collections D=os.path.expanduser("~/DeepSeek-R1") idx=json.load(open(os.path.join(D,"model.safetensors.index.json"))) wm=idx["weight_map"] print("### INDEX.JSON STRUCTURE") print(" top-level keys:",list(idx.keys())) print(" metadata:",idx["metadata"]) k0=next(iter(wm)); print(" weight_map sample: %r -> %r"%(k0,wm[k0])) print() print("### SHARD -> LAYER MAP") sl=collections.defaultdict(set); glob=collections.defaultdict(list) for name,sh in wm.items(): m=re.match(r'model\.layers\.(\d+)\.',name) if m: sl[sh].add(int(m.group(1))) else: glob[sh].append(name) rows=[] for sh in sorted(set(wm.values())): ls=sorted(sl.get(sh,[])) lay = "-" if not ls else (str(ls[0]) if len(ls)==1 else f"{ls[0]}-{ls[-1]}") rows.append((int(sh.split('-')[1]), lay, len([k for k,v in wm.items() if v==sh]), ",".join(glob.get(sh,[])) )) # compress runs print(" shard layers tensors non-layer tensors") for n,lay,ct,g in rows: print(f" {n:>5} {lay:<10} {ct:>7} {g}") -------------------------------------------------------------------------------- C.11.8 verify_integrity.py (1310 bytes) -------------------------------------------------------------------------------- import json,os,struct,collections,re D=os.path.expanduser("~/DeepSeek-R1") idx=json.load(open(os.path.join(D,"model.safetensors.index.json"))) wm=idx["weight_map"]; meta=idx.get("metadata",{}) print("index metadata:",meta) print("index weight_map entries:",len(wm)) print("index distinct shards:",len(set(wm.values()))) hdrnames={}; bad=[] files=sorted(f for f in os.listdir(D) if f.endswith(".safetensors")) for fn in files: p=os.path.join(D,fn); sz=os.path.getsize(p) with open(p,'rb') as fh: n=struct.unpack('12} .. {o[1]:>12} {o[1]-o[0]:>10} {v['dtype']:8s} {str(v['shape']):16s} {k}") # check: are an expert's 3 weights + 3 scales contiguous? import collections byexp=collections.defaultdict(list) for k,v in items: m=re.match(r'model\.layers\.(\d+)\.mlp\.experts\.(\d+)\.',k) if m: byexp[(int(m.group(1)),int(m.group(2)))].append((v["data_offsets"],k)) print("\nexperts in this shard:",len(byexp)) contig=0; noncontig=0; spans=[] for key,lst in byexp.items(): lst.sort(); lo=lst[0][0][0]; hi=max(o[1] for o,_ in lst) used=sum(o[1]-o[0] for o,_ in lst) spans.append(hi-lo) if hi-lo==used: contig+=1 else: noncontig+=1 print(f" fully contiguous experts: {contig} with gaps: {noncontig}") print(f" expert span bytes: min={min(spans)} max={max(spans)} (payload/expert = {used})") k0=sorted(byexp)[0] print(f"\n layout of one expert {k0}:") for o,k in sorted(byexp[k0]): print(f" {o[0]:>12} .. {o[1]:>12} ({o[1]-o[0]:>9}) {k.split('experts.')[1]}") -------------------------------------------------------------------------------- C.11.10 gen_manifests.py (5475 bytes) -------------------------------------------------------------------------------- #!/usr/bin/env python3 """ gen_manifests.py -- produce the five machine-readable manifests for DeepSeek-R1. Reads ONLY the safetensors JSON headers (never the weight payload), so it is fast and touches ~13 MB of the 643 GB checkpoint. Outputs (into OUT): deepseek_tensor_manifest.json every tensor: shard, dtype, shape, absolute offset, byte length, linked scale tensor deepseek_shard_headers.json per-shard container geometry deepseek_layer_shard_map.json layer index -> shards, and shard -> layers deepseek_expert_offset_map.json (layer,expert) -> one contiguous read deepseek_weight_hashes.txt sizes + partial hashes (see gen_hashes.py for the full SHA-256 pass) """ import json, os, struct, re, collections, hashlib, sys D = os.path.expanduser("~/DeepSeek-R1") OUT = os.path.expanduser("~/deepseek-megamind-upload/deepseek") os.makedirs(OUT, exist_ok=True) files = sorted(f for f in os.listdir(D) if f.endswith(".safetensors")) assert len(files) == 163, f"expected 163 shards, found {len(files)}" tensors = {} # name -> record shard_hdrs = [] for fn in files: path = os.path.join(D, fn) size = os.path.getsize(path) with open(path, "rb") as fh: hdr_len = struct.unpack(" shard maps l2s = collections.defaultdict(set); s2l = collections.defaultdict(set) globals_ = collections.defaultdict(list) for name, rec in tensors.items(): m = re.match(r"model\.layers\.(\d+)\.", name) if m: li = int(m.group(1)); l2s[li].add(rec["shard"]); s2l[rec["shard"]].add(li) else: globals_[rec["shard"]].append(name) lmap = { "layer_to_shards": {str(k): sorted(v) for k, v in sorted(l2s.items())}, "shard_to_layers": {k: sorted(v) for k, v in sorted(s2l.items())}, "global_tensors_by_shard": {k: sorted(v) for k, v in sorted(globals_.items())}, "notes": { "dense_layers": [0, 1, 2], "moe_layers": list(range(3, 61)), "mtp_layer": 61, "base_inference_layers": list(range(0, 61)), }, } json.dump(lmap, open(os.path.join(OUT, "deepseek_layer_shard_map.json"), "w"), indent=1) # expert offset map: one contiguous read per expert where possible experts = collections.defaultdict(list) for name, rec in tensors.items(): m = re.match(r"model\.layers\.(\d+)\.mlp\.experts\.(\d+)\.", name) if m: experts[(int(m.group(1)), int(m.group(2)))].append(rec) emap = {} straddlers = 0 for (li, ei), recs in sorted(experts.items()): by_shard = collections.defaultdict(list) for r in recs: by_shard[r["shard"]].append(r) reads = [] for sh in sorted(by_shard): rs = sorted(by_shard[sh], key=lambda r: r["absolute_offset"]) lo = rs[0]["absolute_offset"] hi = max(r["absolute_offset"] + r["byte_length"] for r in rs) used = sum(r["byte_length"] for r in rs) reads.append({ "shard": sh, "offset": lo, "length": hi - lo, "contiguous": (hi - lo) == used, "tensors": [r["name"].split("experts.%d." % ei)[1] for r in rs], }) if len(reads) > 1: straddlers += 1 emap["%d/%d" % (li, ei)] = { "layer": li, "expert": ei, "total_bytes": sum(r["byte_length"] for r in recs), "read_count": len(reads), "reads": reads, } json.dump({"straddling_experts": straddlers, "expert_count": len(emap), "bytes_per_expert_nominal": 44050944, "experts": emap}, open(os.path.join(OUT, "deepseek_expert_offset_map.json"), "w"), indent=1) print("tensors:", len(manifest)) print("shards:", len(shard_hdrs), "all length checks ok:", all(s["length_check_ok"] for s in shard_hdrs)) print("experts:", len(emap), "straddling shard boundaries:", straddlers) for f in ["deepseek_tensor_manifest.json","deepseek_shard_headers.json", "deepseek_layer_shard_map.json","deepseek_expert_offset_map.json"]: print(" wrote %-38s %12d B" % (f, os.path.getsize(os.path.join(OUT,f)))) -------------------------------------------------------------------------------- C.11.11 gen_hashes.py (2523 bytes) -------------------------------------------------------------------------------- #!/usr/bin/env python3 """ gen_hashes.py -- checkpoint integrity hashes. --partial (default) size + SHA-256 of the first 1 MiB, last 1 MiB, and the 1 MiB at the midpoint of each shard. Reads 489 MiB total, finishes in about a second. Enough to detect truncation, a swapped shard, or a corrupted head/tail. --full SHA-256 over every byte of all 163 shards. Reads the whole 643 GB. At the measured 507 MB/s single-stream rate on this machine that is roughly 22 minutes of I/O plus hash time. Required before any "verified" claim. Output: deepseek_weight_hashes.txt (sorted, one shard per stanza) """ import os, sys, hashlib, time D = os.path.expanduser("~/DeepSeek-R1") OUT = os.path.expanduser("~/deepseek-megamind-upload/deepseek/deepseek_weight_hashes.txt") MODE = "--full" if "--full" in sys.argv else "--partial" CH = 1 << 20 files = sorted(f for f in os.listdir(D) if f.endswith(".safetensors")) t0 = time.time() with open(OUT, "w") as out: out.write("# DeepSeek-R1 checkpoint hashes\n") out.write("# mode: %s\n" % MODE) out.write("# generated: %s\n" % time.strftime("%Y-%m-%dT%H:%M:%S%z")) out.write("# source: %s\n#\n" % D) total = 0 for fn in files: p = os.path.join(D, fn); sz = os.path.getsize(p); total += sz if MODE == "--full": h = hashlib.sha256() with open(p, "rb") as fh: while True: b = fh.read(1 << 22) if not b: break h.update(b) out.write("%s\n size %d\n sha256 %s\n" % (fn, sz, h.hexdigest())) else: def part(off, n): with open(p, "rb") as fh: fh.seek(off); return hashlib.sha256(fh.read(n)).hexdigest() out.write("%s\n size %d\n" % (fn, sz)) out.write(" sha256_head %s (bytes 0..%d)\n" % (part(0, CH), CH)) mid = (sz // 2) - (CH // 2) out.write(" sha256_middle %s (bytes %d..%d)\n" % (part(mid, CH), mid, mid + CH)) out.write(" sha256_tail %s (bytes %d..%d)\n" % (part(max(0, sz - CH), CH), max(0, sz - CH), sz)) out.flush() out.write("#\n# shards: %d\n# total bytes: %d\n# elapsed: %.1fs\n" % (len(files), total, time.time() - t0)) print("wrote", OUT, os.path.getsize(OUT), "B in %.1fs" % (time.time() - t0)) ================================================================================ C.12 WSL CONFIGURATION ================================================================================ --- C.12.1 /etc/wsl.conf (inside the WSL2 guest) --- [boot] systemd=true [network] generateResolvConf=true --- C.12.2 C:\Users\Josep\.wslconfig (Windows host) --- [wsl2] memory=26GB processors=12 vmIdleTimeout=-1 swap=8GB ================================================================================ C.13 SYSTEMD UNIT FILES (all 51, credential redacted in mm-loom.service) ================================================================================ -------------------------------------------------------------------------------- mm-api.service -------------------------------------------------------------------------------- [Unit] Description=missionMATCH API on megamind-agi (per tenant routing, control plane on 5433) After=network-online.target [author/contact email redacted] Wants=network-online.target [Service] Type=simple # Root owned 0600, read by systemd rather than by the service user, so the process never needs # permission to read the credentials it is handed. EnvironmentFile=/etc/missionmatch/api.env ExecStart=/opt/missionmatch/mm-api Restart=on-failure RestartSec=5 DynamicUser=yes NoNewPrivileges=yes PrivateTmp=yes ProtectSystem=strict ProtectHome=yes [Install] WantedBy=multi-user.target -------------------------------------------------------------------------------- mm-boot.service -------------------------------------------------------------------------------- [Unit] Description=MEGAMIND-AGI boot supervisor After=postgresql.service Wants=postgresql.service [Service] Type=oneshot ExecStart=/opt/mm/bin/mmboot RemainAfterExit=yes [Install] WantedBy=multi-user.target -------------------------------------------------------------------------------- mm-cap.service -------------------------------------------------------------------------------- [Unit] Description=MEGAMIND-AGI capability manifest After=network-online.target postgresql.service [Service] Type=oneshot ExecStart=/opt/mm/bin/mmcap -------------------------------------------------------------------------------- mm-cap.timer -------------------------------------------------------------------------------- [Unit] Description=MEGAMIND-AGI manifest every 60s [Timer] OnBootSec=45 OnUnitActiveSec=60 [Install] WantedBy=timers.target -------------------------------------------------------------------------------- mm-cp-backup.service -------------------------------------------------------------------------------- [Unit] Description=missionMATCH control plane backup, dumped here and pushed to Bubbles After=network-online.target Wants=network-online.target [Service] Type=oneshot ExecStart=/usr/local/sbin/mm-cp-backup.sh User=root Group=root Nice=10 -------------------------------------------------------------------------------- mm-cp-backup.timer -------------------------------------------------------------------------------- [Unit] Description=Nightly missionMATCH control plane backup push [Timer] # 02:45, twenty minutes BEFORE the Bubbles job at 03:05, so the dump has arrived to be ingested. OnCalendar=*-*-* 02:45:00 Persistent=true RandomizedDelaySec=120 [Install] WantedBy=timers.target -------------------------------------------------------------------------------- mm-durability.service -------------------------------------------------------------------------------- [Unit] Description=PANTHEON two-device durability (M4 <-> cortex) After=network-online.target Wants=network-online.target [Service] Type=oneshot ExecStart=/opt/mm/backup/durability.sh -------------------------------------------------------------------------------- mm-durability.timer -------------------------------------------------------------------------------- [Unit] Description=Nightly PANTHEON durability backup [Timer] OnCalendar=*-*-* 03:30:00 Persistent=true [Install] WantedBy=timers.target -------------------------------------------------------------------------------- mm-eval.service -------------------------------------------------------------------------------- [Unit] Description=MEGAMIND-AGI substrate eval harness After=postgresql.service [Service] Type=oneshot ExecStart=/opt/mm/bin/mmeval -------------------------------------------------------------------------------- mm-eval.timer -------------------------------------------------------------------------------- [Unit] Description=MEGAMIND-AGI hourly substrate quality eval [Timer] OnBootSec=300 OnUnitActiveSec=3600 [Install] WantedBy=timers.target -------------------------------------------------------------------------------- mm-eye.service -------------------------------------------------------------------------------- [Unit] Description=EYE - Eye of the World, PANTHEON binary VSA world-brain After=postgresql.service network.target [Service] Type=simple User=postgres Environment=EYE_PORT=9404 Environment=EYE_HEARTBEAT_SEC=12 Environment="EYE_DB=host=/var/run/postgresql dbname=livingworld" ExecStart=/opt/mm/bin/eye Restart=always RestartSec=3 [Install] WantedBy=multi-user.target -------------------------------------------------------------------------------- mm-game-reflect.service -------------------------------------------------------------------------------- [Unit] Description=MEGAMIND game reflection tick [Service] Type=oneshot ExecStart=/usr/bin/curl -s --max-time 60 http://127.0.0.1:9097/game/tick?min=4 -------------------------------------------------------------------------------- mm-game-reflect.timer -------------------------------------------------------------------------------- [Unit] Description=Reflect active NPCs every 30 min [Timer] OnBootSec=15min OnUnitActiveSec=30min [Install] WantedBy=timers.target -------------------------------------------------------------------------------- mm-game.service -------------------------------------------------------------------------------- [Unit] Description=MEGAMIND binary Game Memory API (mmgame :9097) After=network.target postgresql.service mm-serve.service [Service] Type=simple ExecStart=/opt/mm/bin/mmgame Restart=always RestartSec=5 WorkingDirectory=/opt/mm [Install] WantedBy=multi-user.target -------------------------------------------------------------------------------- mm-gate.service -------------------------------------------------------------------------------- [Unit] Description=MEGAMIND-AGI anti-degeneracy gate After=mm-serve.service [Service] Type=oneshot ExecStart=/opt/mm/bin/mmgate -------------------------------------------------------------------------------- mm-gate.timer -------------------------------------------------------------------------------- [Unit] Description=MEGAMIND-AGI degeneracy check every 5min [Timer] OnBootSec=120 OnUnitActiveSec=300 [Install] WantedBy=timers.target -------------------------------------------------------------------------------- mm-genserve.service -------------------------------------------------------------------------------- [Unit] Description=MEGAMIND-AGI P20 binary generator (mmpredict serve, :9098) After=network.target postgresql.service [Service] Type=simple ExecStart=/opt/mm/bin/mmpredict serve Restart=always RestartSec=5 WorkingDirectory=/opt/mm [Install] WantedBy=multi-user.target -------------------------------------------------------------------------------- mm-heart.service -------------------------------------------------------------------------------- [Unit] Description=MEGAMIND-AGI mesh heartbeat After=network-online.target [Service] Type=oneshot ExecStart=/opt/mm/bin/mmheart -------------------------------------------------------------------------------- mm-heart.timer -------------------------------------------------------------------------------- [Unit] Description=MEGAMIND-AGI heartbeat every 15s [Timer] OnBootSec=30 OnUnitActiveSec=15 [Install] WantedBy=timers.target -------------------------------------------------------------------------------- mm-identity-sync.service -------------------------------------------------------------------------------- [Unit] Description=missionMATCH identity sync, control plane to every active tenant After=network-online.target Wants=network-online.target [Service] Type=oneshot ExecStart=/usr/local/sbin/mm-identity-sync.sh User=root Group=root Nice=10 -------------------------------------------------------------------------------- mm-identity-sync.timer -------------------------------------------------------------------------------- [Unit] Description=Hourly missionMATCH identity sync [Timer] # Hourly, because the drift it corrects appears whenever an invite is accepted or a role changes, and a # daily cadence would leave a new user invisible to tenant foreign keys for most of a working day. OnCalendar=hourly Persistent=true RandomizedDelaySec=300 [Install] WantedBy=timers.target -------------------------------------------------------------------------------- mm-keepalive.service -------------------------------------------------------------------------------- [Unit] Description=MEGAMIND WSL instance keepalive (holds the WSL2 instance against the hardcoded idle timer) After=multi-user.target [Service] Type=exec ExecStart=/usr/bin/sleep infinity Restart=always RestartSec=1 [Install] WantedBy=multi-user.target -------------------------------------------------------------------------------- mm-loom.service -------------------------------------------------------------------------------- [Unit] Description=The Loom - Living World orchestration organ After=network.target mm-eye.service [Service] [REDACTED: credential-associated original line 4196; original retained outside this bundle.] Environment=LOOM_PORT=9410 ExecStart=/opt/mm/bin/loom Restart=always RestartSec=3 [Install] WantedBy=multi-user.target -------------------------------------------------------------------------------- mm-narrate.service -------------------------------------------------------------------------------- [Unit] Description=MEGAMIND narrator After=network.target [Service] ExecStart=/opt/mm/bin/mm-narrate Restart=always User=root [Install] WantedBy=multi-user.target -------------------------------------------------------------------------------- mm-offsite-watch.service -------------------------------------------------------------------------------- [Unit] Description=missionMATCH watch from OFF Bubbles (cross node vantage, not external) After=network-online.target [Service] Type=oneshot ExecStart=/usr/local/sbin/mm-offsite-watch.sh -------------------------------------------------------------------------------- mm-offsite-watch.timer -------------------------------------------------------------------------------- [Unit] Description=run the off Bubbles watch every five minutes [Timer] OnBootSec=3min OnUnitActiveSec=5min AccuracySec=30s [Install] WantedBy=timers.target -------------------------------------------------------------------------------- mm-oracle.service -------------------------------------------------------------------------------- [Unit] Description=ORACLE - MEGAMIND binary VSA reasoning organ (16 powers, WAL-persistent) After=network.target [Service] Type=simple WorkingDirectory=/opt/mm/oracle ExecStart=/opt/mm/bin/oracle Restart=always RestartSec=2 [Install] WantedBy=multi-user.target -------------------------------------------------------------------------------- mm-sem.service -------------------------------------------------------------------------------- [Unit] Description=MEGAMIND shared contextual semantic embedder (mmsem :9096) After=network.target [Service] Type=simple ExecStart=/opt/mm/bin/mmsem Restart=always RestartSec=5 [Install] WantedBy=multi-user.target -------------------------------------------------------------------------------- mm-serve.service -------------------------------------------------------------------------------- [Unit] Description=MEGAMIND-AGI 512-bit recall service After=postgresql.service Wants=postgresql.service PartOf=mm.target [Service] ExecStart=/opt/mm/bin/mmserve Restart=on-failure RestartSec=5 KillSignal=SIGTERM TimeoutStopSec=120 [Install] WantedBy=multi-user.target mm.target -------------------------------------------------------------------------------- mm-shard.service -------------------------------------------------------------------------------- [Unit] Description=MEGAMIND-AGI federated recall shard After=mm-serve.service network-online.target Wants=mm-serve.service PartOf=mm.target [Service] ExecStart=/opt/mm/bin/mmshard Restart=on-failure RestartSec=5 [Install] WantedBy=multi-user.target mm.target -------------------------------------------------------------------------------- mm-wal-compact.service -------------------------------------------------------------------------------- [Unit] Description=ORACLE WAL compaction (keep restarts fast) [Service] Type=oneshot ExecStart=/opt/mm/oracle/compact_wal.sh -------------------------------------------------------------------------------- mm-wal-compact.timer -------------------------------------------------------------------------------- [Unit] Description=daily ORACLE WAL compaction [Timer] OnCalendar=*-*-* 04:17:00 Persistent=true [Install] WantedBy=timers.target -------------------------------------------------------------------------------- mm-world.service -------------------------------------------------------------------------------- [Unit] Description=MEGAMIND shared associative world-model organ (mmworld :9098) After=network.target [Service] Type=simple ExecStart=/opt/mm/bin/mmworld Restart=always RestartSec=5 [Install] WantedBy=multi-user.target -------------------------------------------------------------------------------- mm.target -------------------------------------------------------------------------------- [Unit] Description=MEGAMIND-AGI node stack Wants=postgresql.service mm-serve.service mm-heart.timer mm-cap.timer After=postgresql.service [Install] WantedBy=multi-user.target -------------------------------------------------------------------------------- mmask.service -------------------------------------------------------------------------------- [Unit] Description=mmask (PANTHEON ask brain — bridges mmknow+ORACLE+mmreason -> LLM) After=network.target mm-oracle.service [Service] Type=simple WorkingDirectory=/opt/mm/reason Environment=MMKNOW=http://100.77.114.40:9542/know Environment=MMREASON=http://100.77.114.40:9522/reason Environment=ORACLE=http://100.69.120.61:9402/oracle Environment=LLM=http://100.94.8.94:8082/v1/chat/completions ExecStart=/usr/bin/python3 /opt/mm/reason/mmask_serve.py 9543 Restart=always RestartSec=3 User=joseph [Install] WantedBy=multi-user.target -------------------------------------------------------------------------------- mmbackward.service -------------------------------------------------------------------------------- [Unit] Description=PANTHEON backward circulation loop (nonstop federation dataflow) After=network.target mm-oracle.service [Service] Type=simple ExecStart=/usr/bin/python3 /opt/mm/loops/backward.py Restart=always RestartSec=3 User=joseph WorkingDirectory=/opt/mm/loops [Install] WantedBy=multi-user.target -------------------------------------------------------------------------------- mmcode2.service -------------------------------------------------------------------------------- [Unit] Description=mmcode2 binary recall over CodeSearchNet (code + docstrings) After=network.target [Service] Environment=WX_BSIG=/opt/mm/code/code.bsig Environment=WX_BLOB=/opt/mm/code/code.blob Environment=WX_IDX=/opt/mm/code/code.idx Environment=WX_NAME=code2 ExecStart=/opt/mm/code/mmwikix serve 9534 Restart=always User=joseph WorkingDirectory=/opt/mm/code [Install] WantedBy=multi-user.target -------------------------------------------------------------------------------- mmforward.service -------------------------------------------------------------------------------- [Unit] Description=PANTHEON forward circulation loop (nonstop federation dataflow) After=network.target mm-oracle.service [Service] Type=simple ExecStart=/usr/bin/python3 /opt/mm/loops/forward.py Restart=always RestartSec=3 User=joseph WorkingDirectory=/opt/mm/loops [Install] WantedBy=multi-user.target -------------------------------------------------------------------------------- mmgateway-web.service -------------------------------------------------------------------------------- [Unit] Description=mmgateway-web (read-only public gateway to every PANTHEON organ) After=network.target mm-oracle.service [Service] Type=simple WorkingDirectory=/opt/mm/reason Environment=ORACLE=http://100.69.120.61:9402/oracle ExecStart=/usr/bin/python3 /opt/mm/reason/mmgateway_web.py 9560 Restart=always RestartSec=3 User=joseph [Install] WantedBy=multi-user.target -------------------------------------------------------------------------------- mmhelix.service -------------------------------------------------------------------------------- [Unit] Description=mmhelix G^16 -- 16 strand-loops, 8 antiparallel complement pairs After=network.target [Service] ExecStart=/usr/bin/python3 /opt/mm/loops/mmhelix.py Restart=always RestartSec=3 User=joseph [Install] WantedBy=multi-user.target -------------------------------------------------------------------------------- mmknow.service -------------------------------------------------------------------------------- [Unit] Description=mmknow (PANTHEON mind organ) After=network.target mm-oracle.service [Service] Type=simple WorkingDirectory=/opt/mm/reason ExecStart=/usr/bin/python3 /opt/mm/reason/mmknow_serve.py 9542 Restart=always RestartSec=3 User=joseph [Install] WantedBy=multi-user.target -------------------------------------------------------------------------------- mmlearn.service -------------------------------------------------------------------------------- [Unit] Description=mmlearn (PANTHEON learning organ) After=network.target mm-oracle.service mmreason.service [Service] Type=simple WorkingDirectory=/opt/mm/reason Environment="ORACLE=http://127.0.0.1:9402/oracle" Environment="MMREASON=http://127.0.0.1:9522/reason" ExecStart=/usr/bin/python3 /opt/mm/reason/mmlearn_serve.py 9541 Restart=always RestartSec=3 User=joseph [Install] WantedBy=multi-user.target -------------------------------------------------------------------------------- mmlearned.service -------------------------------------------------------------------------------- [Unit] Description=mmlearned binary recall over the learned/world-knowledge corpus (iCloud AGI_Data learning streams) After=network.target [Service] Environment=WX_BSIG=/opt/mm/learned/learned.bsig Environment=WX_BLOB=/opt/mm/learned/learned.blob Environment=WX_IDX=/opt/mm/learned/learned.idx Environment=WX_NAME=learned ExecStart=/opt/mm/learned/mmwikix serve 9532 Restart=always User=joseph WorkingDirectory=/opt/mm/learned [Install] WantedBy=multi-user.target -------------------------------------------------------------------------------- mmmaddie.service -------------------------------------------------------------------------------- [Unit] Description=mmmaddie binary recall over the MADDIE model-corpus signature store (346k model/tensor architecture facts, 434 recovered LLMs) After=network.target [Service] Environment=WX_BSIG=/opt/mm/maddie/maddie.bsig Environment=WX_BLOB=/opt/mm/maddie/maddie.blob Environment=WX_IDX=/opt/mm/maddie/maddie.idx Environment=WX_NAME=maddie ExecStart=/opt/mm/maddie/mmwikix serve 9531 Restart=always User=joseph WorkingDirectory=/opt/mm/maddie [Install] WantedBy=multi-user.target -------------------------------------------------------------------------------- mmpubmed.service -------------------------------------------------------------------------------- [Unit] Description=mmpubmed binary recall organ After=network.target [Service] Environment=WX_BSIG=/opt/mm/pubmed/pubmed.bsig Environment=WX_BLOB=/opt/mm/pubmed/pubmed.blob Environment=WX_IDX=/opt/mm/pubmed/pubmed.idx Environment=WX_NAME=pubmed ExecStart=/opt/mm/pubmed/mmwikix serve 9533 Restart=always User=joseph WorkingDirectory=/opt/mm/pubmed [Install] WantedBy=multi-user.target -------------------------------------------------------------------------------- mmpulse.service -------------------------------------------------------------------------------- [Unit] Description=mmpulse -- universal circulation loop through the entire brain After=network.target [Service] ExecStart=/usr/bin/python3 /opt/mm/loops/mmpulse.py Restart=always RestartSec=3 User=joseph [Install] WantedBy=multi-user.target -------------------------------------------------------------------------------- mmreason.service -------------------------------------------------------------------------------- [Unit] Description=mmreason (PANTHEON mind organ) After=network.target mm-oracle.service [Service] Type=simple WorkingDirectory=/opt/mm/reason ExecStart=/usr/bin/python3 /opt/mm/reason/mmreason_serve.py 9522 Restart=always RestartSec=3 User=joseph [Install] WantedBy=multi-user.target -------------------------------------------------------------------------------- mmthink.service -------------------------------------------------------------------------------- [Unit] Description=mmthink (PANTHEON thinking agent) After=network.target mm-oracle.service mmreason.service [Service] Type=simple WorkingDirectory=/opt/mm/reason Environment="ORACLE=http://127.0.0.1:9402/oracle" Environment="MMREASON=http://127.0.0.1:9522/reason" Environment="LLM=http://100.94.8.94:8082/v1/chat/completions" ExecStart=/usr/bin/python3 /opt/mm/reason/mmthink_serve.py 9540 Restart=always RestartSec=3 User=joseph [Install] WantedBy=multi-user.target -------------------------------------------------------------------------------- mmthought.service -------------------------------------------------------------------------------- [Unit] Description=mmthought -- visible self-correcting chain-of-thought organ After=network.target [Service] ExecStart=/usr/bin/python3 /opt/mm/loops/mmthought.py Restart=always RestartSec=3 User=joseph [Install] WantedBy=multi-user.target -------------------------------------------------------------------------------- mm-oracle.service.d/mt.conf -------------------------------------------------------------------------------- [Service] LimitNOFILE=65536 Environment=ORACLE_THREADS=24 -------------------------------------------------------------------------------- mm-serve.service.d/bind.conf -------------------------------------------------------------------------------- [Service] Environment=MMSERVE_BIND=0.0.0.0 ================================================================================ C.14 SANITIZED ENVIRONMENT REPORT ================================================================================ ### uname -a Linux MEGAMIND-AGI 6.18.33.2-microsoft-standard-WSL2 #1 SMP PREEMPT_DYNAMIC Thu Jun 18 21:54:43 UTC 2026 x86_64 GNU/Linux ### lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Address sizes: 39 bits physical, 48 bits virtual Byte Order: Little Endian CPU(s): 12 On-line CPU(s) list: 0-11 Vendor ID: GenuineIntel Model name: Intel(R) Core(TM) i7-8700K CPU @ 3.70GHz CPU family: 6 Model: 158 Thread(s) per core: 2 Core(s) per socket: 6 Socket(s): 1 Stepping: 10 BogoMIPS: 7392.02 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm arch_perfmon rep_good nopl xtopology cpuid tsc_known_freq pni pclmulqdq vmx ssse3 fma cx16 pdcm pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch pti ssbd ibrs ibpb stibp tpr_shadow ept vpid ept_ad fsgsbase bmi1 hle avx2 smep bmi2 erms invpcid rtm rdseed adx smap clflushopt xsaveopt xsavec xgetbv1 xsaves vnmi md_clear flush_l1d arch_capabilities Virtualization: VT-x Hypervisor vendor: Microsoft Virtualization type: full L1d cache: 192 KiB (6 instances) L1i cache: 192 KiB (6 instances) L2 cache: 1.5 MiB (6 instances) L3 cache: 12 MiB (1 instance) NUMA node(s): 1 NUMA node0 CPU(s): 0-11 Vulnerability Gather data sampling: Not affected Vulnerability Ghostwrite: Not affected Vulnerability Indirect target selection: Mitigation; Aligned branch/return thunks Vulnerability Itlb multihit: KVM: Mitigation: Split huge pages Vulnerability L1tf: Mitigation; PTE Inversion; VMX conditional cache flushes, SMT vulnerable Vulnerability Mds: Mitigation; Clear CPU buffers; SMT Host state unknown Vulnerability Meltdown: Mitigation; PTI Vulnerability Mmio stale data: Mitigation; Clear CPU buffers; SMT Host state unknown Vulnerability Old microcode: Not affected Vulnerability Reg file data sampling: Not affected Vulnerability Retbleed: Mitigation; IBRS Vulnerability Spec rstack overflow: Not affected Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization Vulnerability Spectre v2: Mitigation; IBRS; IBPB conditional; STIBP conditional; RSB filling; PBRSB-eIBRS Not affected; BHI SW loop, KVM SW loop Vulnerability Srbds: Unknown: Dependent on hypervisor status Vulnerability Tsa: Not affected Vulnerability Tsx async abort: Mitigation; Clear CPU buffers; SMT Host state unknown Vulnerability Vmscape: Not affected ### free -h total used free shared buff/cache available Mem: 25Gi 15Gi 3.9Gi 5.3Gi 11Gi 9.5Gi Swap: 8.0Gi 2.3Gi 5.7Gi ### df -h Filesystem Size Used Avail Use% Mounted on none 13G 0 13G 0% /usr/lib/modules/6.18.33.2-microsoft-standard-WSL2 none 13G 4.0K 13G 1% /mnt/wsl drivers 931G 800G 132G 86% /usr/lib/wsl/drivers /dev/sdd 1007G 729G 228G 77% / none 13G 32K 13G 1% /mnt/wslg none 13G 0 13G 0% /usr/lib/wsl/lib rootfs 13G 2.8M 13G 1% /init none 13G 680K 13G 1% /run none 13G 0 13G 0% /run/lock none 13G 26M 13G 1% /run/shm none 13G 80K 13G 1% /mnt/wslg/versions.txt none 13G 80K 13G 1% /mnt/wslg/doc C:\ 931G 800G 132G 86% /mnt/c tmpfs 13G 252M 13G 2% /tmp none 1.0M 0 1.0M 0% /run/credentials/systemd-journald.service none 1.0M 0 1.0M 0% /run/credentials/systemd-resolved.service none 1.0M 0 1.0M 0% [author/contact email redacted] tmpfs 2.6G 12K 2.6G 1% /run/user/0 tmpfs 2.6G 12K 2.6G 1% /run/user/1000 ### clang Ubuntu clang version 21.1.8 (6ubuntu1) Target: x86_64-pc-linux-gnu Thread model: posix InstalledDir: /usr/lib/llvm-21/bin ### gcc gcc (Ubuntu 15.2.0-16ubuntu1) 15.2.0 Copyright (C) 2025 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. ### python3 Python 3.14.4 ### pip3 list Package Version ------------------------- ------------- anyio 4.14.2 attrs 25.4.0 autocommand 2.2.2 Automat 25.4.16 babel 2.17.0 bcrypt 5.0.0 blinker 1.9.0 certifi 2026.1.4 chardet 5.2.0 click 8.4.2 command-not-found 0.3 configobj 5.0.9 constantly 23.10.4 cryptography 46.0.5 dbus-python 1.4.0 distro 1.9.0 distro-info 1.15 filelock 3.32.2 fsspec 2026.7.0 h11 0.16.0 hf_transfer 0.1.9 hf-xet 1.6.0 httpcore 1.0.9 httplib2 0.22.0 httpx 0.28.1 huggingface_hub 1.26.0 hyperlink 21.0.0 idna 3.11 incremental 24.7.2 inflect 7.5.0 jaraco.context 6.0.1 jaraco.functools 4.1.0 jaraco.text 4.0.0 Jinja2 3.1.6 jsonpatch 1.32 jsonpointer 2.4 jsonschema 4.19.2 jsonschema-specifications 2023.12.1 launchpadlib 2.1.0 lazr.restfulclient 0.14.6 lazr.uri 1.0.6 libpass 1.9.3 linkify-it-py 2.0.3 markdown-it-py 3.0.0 MarkupSafe 3.0.3 mdurl 0.1.2 more-itertools 10.8.0 netifaces 0.11.0 numpy 2.5.1 oauthlib 3.3.1 packaging 26.0 pip 25.1.1 pyasn1 0.6.3 pyasn1_modules 0.4.1 pycurl 7.45.7 Pygments 2.19.2 PyGObject 3.56.2 PyHamcrest 2.1.0 PyJWT 2.10.1 pyOpenSSL 25.3.0 pyparsing 3.3.2 pyserial 3.5 python-apt 3.1.0+ubuntu1 PyYAML 6.0.3 referencing 0.36.2 requests 2.32.5 rich 13.9.4 rpds-py 0.27.1 service-identity 24.2.0 setuptools 78.1.1 systemd-python 235 tqdm 4.70.0 Twisted 25.5.0 typeguard 4.4.4 typing_extensions 4.15.0 ubuntu-pro-client 8001 uc-micro-py 1.0.3 unattended-upgrades 0.1 urllib3 2.6.3 wadllib 2.0.0 wheel 0.46.3 zipp 3.23.0 zope.interface 8.2 ### systemctl list-unit-files --type=service UNIT FILE STATE PRESET apparmor.service enabled enabled apport-autoreport.service static - apport-coredump-hook@.service static - apport-forward@.service static - apport.service enabled enabled apt-daily-upgrade.service static - apt-daily.service static - apt-news.service static - autovt@.service alias - breakpoint-pre-basic.service static - breakpoint-pre-mount.service static - breakpoint-pre-switch-root.service static - breakpoint-pre-udev.service static - caddy-api.service disabled enabled caddy.service enabled enabled capsule@.service static - chrony-dnssrv@.service static - chrony-wait.service disabled enabled chrony.service enabled enabled chronyd.service alias - cloud-config.service enabled enabled cloud-final.service enabled enabled cloud-init-hotplugd.service static - cloud-init-local.service enabled enabled cloud-init-main.service enabled enabled cloud-init-network.service enabled enabled console-getty.service masked-runtime disabled console-setup.service enabled enabled container-getty@.service static - cron.service enabled enabled cryptdisks-early.service masked enabled cryptdisks.service masked enabled dbus-org.freedesktop.hostname1.service alias - dbus-org.freedesktop.locale1.service alias - dbus-org.freedesktop.login1.service alias - dbus-org.freedesktop.resolve1.service alias - dbus-org.freedesktop.timedate1.service alias - dbus.service static - debug-shell.service disabled disabled dmesg.service enabled enabled dpkg-db-backup.service static - e2scrub@.service static - e2scrub_all.service static - e2scrub_fail@.service static - e2scrub_reap.service enabled enabled emergency.service static - esm-cache.service static - fstrim.service static - getty-static.service static - getty@.service enabled enabled hwclock.service masked enabled initrd-cleanup.service static - initrd-parse-etc.service static - initrd-switch-root.service static - initrd-udevadm-cleanup-db.service static - keyboard-setup.service enabled enabled kmod-static-nodes.service static - kmod.service alias - landscape-client.service enabled enabled ldconfig.service static - logrotate.service static - man-db.service static - mm-api.service enabled enabled mm-boot.service enabled enabled mm-cap.service static - mm-cp-backup.service static - mm-durability.service static - mm-eval.service static - mm-eye.service enabled enabled mm-game-reflect.service static - mm-game.service enabled enabled mm-gate.service static - mm-genserve.service disabled enabled mm-heart.service static - mm-identity-sync.service static - mm-keepalive.service enabled enabled mm-loom.service enabled enabled mm-narrate.service enabled enabled mm-offsite-watch.service static - mm-oracle.service enabled enabled mm-sem.service enabled enabled mm-serve.service enabled enabled mm-shard.service enabled enabled mm-wal-compact.service static - mm-world.service enabled enabled mmask.service enabled enabled mmbackward.service disabled enabled mmcode2.service enabled enabled mmforward.service disabled enabled mmgateway-web.service enabled enabled mmhelix.service enabled enabled mmknow.service enabled enabled mmlearn.service enabled enabled mmlearned.service enabled enabled mmmaddie.service enabled enabled mmpubmed.service enabled enabled mmpulse.service enabled enabled mmreason.service enabled enabled mmthink.service enabled enabled mmthought.service enabled enabled modprobe@.service static - motd-news.service static - netplan-configure.service enabled enabled netplan-ovs-cleanup.service generated - networkd-dispatcher.service enabled enabled NetworkManager-wait-online.service masked-runtime enabled nftables.service disabled enabled nginx.service enabled enabled packagekit-offline-update.service static - packagekit.service static - pam_namespace.service static - pg_basebackup@.service static - pg_compresswal@.service static - pg_dump@.service static - pg_receivewal@.service disabled enabled polkit-agent-helper@.service static - polkit.service static - postgresql.service enabled enabled postgresql@.service indirect enabled procps.service alias - quotaon-root.service static - quotaon@.service static - rc-local.service static - rescue.service static - rsync.service disabled enabled rsyslog.service enabled enabled screen-cleanup.service masked enabled serial-getty@.service disabled enabled setvtrgb.service enabled enabled snapd.apparmor.service enabled enabled snapd.autoimport.service enabled enabled snapd.core-fixup.service enabled enabled snapd.failure.service static - snapd.recovery-chooser-trigger.service enabled enabled snapd.seeded.service enabled enabled snapd.service enabled enabled snapd.snap-repair.service static - snapd.system-shutdown.service enabled enabled ssl-cert.service enabled enabled sudo.service masked enabled syslog.service alias - sysstat-collect.service static - sysstat-rotate.service static - sysstat-summary.service static - sysstat.service enabled enabled system-update-cleanup.service static - [REDACTED: credential-associated original line 4957; original retained outside this bundle.] [REDACTED: credential-associated original line 4958; original retained outside this bundle.] [REDACTED: credential-associated original line 4959; original retained outside this bundle.] systemd-backlight@.service static - systemd-battery-check.service static - systemd-binfmt.service static - systemd-boot-check-no-failures.service disabled disabled systemd-bsod.service static - systemd-confext-initrd.service static - systemd-confext.service disabled enabled systemd-creds@.service static - systemd-exit.service static - systemd-factory-reset-complete.service static - systemd-factory-reset-reboot.service static - systemd-factory-reset-request.service static - systemd-factory-reset@.service static - systemd-firstboot.service static - systemd-fsck-root.service static - systemd-fsck@.service static - systemd-growfs-root.service static - systemd-growfs@.service static - systemd-halt.service static - systemd-hibernate-clear.service static - systemd-hibernate-resume.service static - systemd-hibernate.service static - systemd-hostnamed.service static - systemd-hwdb-update.service static - systemd-hybrid-sleep.service static - systemd-journal-catalog-update.service static - systemd-journal-flush.service static - systemd-journald-sync@.service static - systemd-journald.service static - systemd-journald@.service static - systemd-kexec.service static - systemd-localed.service static - systemd-logind.service static - systemd-loop@.service static - systemd-machine-id-commit.service static - systemd-modules-load.service static - systemd-mute-console@.service static - systemd-network-generator.service disabled enabled systemd-networkd-persistent-storage.service static - systemd-networkd-wait-online.service masked-runtime enabled systemd-networkd-wait-online@.service disabled enabled systemd-networkd.service disabled enabled systemd-pcrextend@.service static - systemd-pcrfs-root.service static - systemd-pcrfs@.service static - systemd-pcrlock-file-system.service disabled enabled systemd-pcrlock-firmware-code.service disabled enabled systemd-pcrlock-firmware-config.service disabled enabled systemd-pcrlock-machine-id.service disabled enabled systemd-pcrlock-make-policy.service disabled enabled systemd-pcrlock-secureboot-authority.service disabled enabled systemd-pcrlock-secureboot-policy.service disabled enabled systemd-pcrlock@.service static - systemd-pcrmachine.service static - systemd-pcrnvdone.service static - systemd-pcrphase-factory-reset.service static - systemd-pcrphase-initrd.service static - systemd-pcrphase-storage-target-mode.service static - systemd-pcrphase-sysinit.service static - systemd-pcrphase.service static - systemd-pcrproduct.service static - systemd-poweroff.service static - systemd-pstore.service enabled enabled systemd-quotacheck-root.service static - systemd-quotacheck@.service static - systemd-random-seed.service static - systemd-reboot.service static - systemd-remount-fs.service enabled-runtime enabled systemd-resolved.service enabled enabled systemd-rfkill.service static - systemd-soft-reboot.service static - systemd-storagetm.service static - systemd-suspend-then-hibernate.service static - systemd-suspend.service static - systemd-sysctl.service static - systemd-sysext-initrd.service static - systemd-sysext.service disabled enabled systemd-sysext@.service static - systemd-sysusers.service static - systemd-timedated.service static - systemd-tmpfiles-clean.service static - systemd-tmpfiles-setup-dev-early.service static - systemd-tmpfiles-setup-dev.service static - systemd-tmpfiles-setup.service static - systemd-tpm2-clear.service disabled enabled systemd-tpm2-setup-early.service static - systemd-tpm2-setup.service static - systemd-udev-load-credentials.service disabled enabled systemd-udev-settle.service static - systemd-udev-trigger.service static - systemd-udevd.service static - systemd-update-done.service static - systemd-user-sessions.service static - systemd-validatefs@.service static - systemd-volatile-root.service static - tailscale-wait-online.service disabled enabled tailscaled.service enabled enabled ua-reboot-cmds.service enabled enabled ua-timer.service static - ubuntu-advantage.service enabled enabled unattended-upgrades.service enabled enabled user-runtime-dir@.service static - user@.service static - wsl-pro.service enabled enabled wslg.service generated - x11-common.service masked enabled 255 unit files listed. ### listening ports State Recv-Q Send-Q Local Address:Port Peer Address:PortProcess LISTEN 0 400 127.0.0.1:5433 0.0.0.0:* LISTEN 0 200 127.0.0.1:5432 0.0.0.0:* LISTEN 0 4096 127.0.0.54:53 0.0.0.0:* LISTEN 0 511 0.0.0.0:80 0.0.0.0:* LISTEN 0 4096 127.0.0.1:2019 0.0.0.0:* LISTEN 0 4096 100.77.114.40:62479 0.0.0.0:* LISTEN 0 4096 127.0.0.53%lo:53 0.0.0.0:* LISTEN 0 64 0.0.0.0:9099 0.0.0.0:* LISTEN 0 1000 10.255.255.254:53 0.0.0.0:* LISTEN 0 32 127.0.0.1:9097 0.0.0.0:* LISTEN 0 32 127.0.0.1:9096 0.0.0.0:* LISTEN 0 16 0.0.0.0:9526 0.0.0.0:* users:(("mmparse",pid=2178309,fd=3)) LISTEN 0 16 0.0.0.0:9527 0.0.0.0:* users:(("mmscheme",pid=2371451,fd=3)) LISTEN 0 16 0.0.0.0:9524 0.0.0.0:* users:(("mmwikix",pid=1969174,fd=3)) LISTEN 0 5 0.0.0.0:9522 0.0.0.0:* users:(("python3",pid=888946,fd=4)) LISTEN 0 16 0.0.0.0:9534 0.0.0.0:* users:(("mmwikix",pid=830746,fd=3)) LISTEN 0 16 0.0.0.0:9532 0.0.0.0:* users:(("mmwikix",pid=2378081,fd=3)) LISTEN 0 16 0.0.0.0:9533 0.0.0.0:* users:(("mmwikix",pid=993129,fd=3)) LISTEN 0 16 0.0.0.0:9531 0.0.0.0:* users:(("mmwikix",pid=987056,fd=3)) LISTEN 0 16 0.0.0.0:9528 0.0.0.0:* users:(("mmpy",pid=2511933,fd=3)) LISTEN 0 16 0.0.0.0:9529 0.0.0.0:* users:(("mmc",pid=2580060,fd=3)) LISTEN 0 5 0.0.0.0:9560 0.0.0.0:* users:(("python3",pid=3653953,fd=3)) LISTEN 0 5 0.0.0.0:9542 0.0.0.0:* users:(("python3",pid=889051,fd=3)) LISTEN 0 5 0.0.0.0:9543 0.0.0.0:* users:(("python3",pid=1063015,fd=3)) LISTEN 0 5 0.0.0.0:9540 0.0.0.0:* users:(("python3",pid=890517,fd=3)) LISTEN 0 5 0.0.0.0:9541 0.0.0.0:* users:(("python3",pid=890518,fd=3)) LISTEN 0 32 0.0.0.0:9404 0.0.0.0:* LISTEN 0 256 0.0.0.0:9402 0.0.0.0:* LISTEN 0 16 0.0.0.0:9403 0.0.0.0:* LISTEN 0 32 0.0.0.0:9401 0.0.0.0:* LISTEN 0 32 0.0.0.0:9410 0.0.0.0:* LISTEN 0 5 0.0.0.0:9420 0.0.0.0:* users:(("python3",pid=1604970,fd=3)) LISTEN 0 5 0.0.0.0:9418 0.0.0.0:* users:(("python3",pid=989694,fd=3)) LISTEN 0 5 0.0.0.0:9419 0.0.0.0:* users:(("python3",pid=1500878,fd=3)) LISTEN 0 400 100.77.114.40:5433 0.0.0.0:* LISTEN 0 4096 [fd7a:115c:a1e0::3f34:7229]:38726 [::]:* LISTEN 0 4096 *:8080 *:* LISTEN 0 4096 *:8094 *:* LISTEN 0 511 [::]:80 [::]:* ================================================================================ PART D — PACKAGES AND PACKAGING COMMANDS ================================================================================ -------------------------------------------------------------------------------- D.1 PACKAGES BUILT THIS SESSION -------------------------------------------------------------------------------- On MEGAMIND-AGI, in /home/joseph/deepseek-megamind-upload/ 01-deepseek-source.tar.gz 4,349,260 B deepseek/config.json deepseek/configuration_deepseek.py deepseek/modeling_deepseek.py deepseek/generation_config.json deepseek/tokenizer_config.json deepseek/tokenizer.json deepseek/model.safetensors.index.json deepseek/README.md deepseek/LICENSE deepseek/.gitattributes deepseek/deepseek_tensor_manifest.json 28,923,014 B deepseek/deepseek_shard_headers.json 40,263 B deepseek/deepseek_layer_shard_map.json 19,562 B deepseek/deepseek_expert_offset_map.json 7,042,163 B deepseek/deepseek_weight_hashes.txt 63,906 B deepseek/FILE_MANIFEST.txt deepseek/scanners/ (11 scripts) NO WEIGHT SHARDS. NO .cache/huggingface/download/. 02-megamind-source.tar.gz 5,235,523 B /opt/mm/src/ 120 .cpp, 53 .h, Makefile, ~90 .bak generations /opt/mm/code/src/ the interpreter organs /opt/mm/reason/*.py /opt/mm/loops/ Excluded: *.pyc __pycache__ *.log *.db *.wal* *.jsonl *.tsv *.u8 *.env *.key *.pem, and every compiled binary. 03-pantheon-source.tar.gz 215,585 B oracle.cpp, oracle_orchestrator.h, oracle_frontier.h all 25 seferim_*.h mmshard mmheart mmsub mmcap mmcap2 mmsem mmserve mmembed mmembed2 hdcembed mmgate mmeval mmstatus mmworld mmreason reason/*.py, loops/*.py 06-env-and-units.tar.gz 10,536 B env/wsl.conf, env/wslconfig env/systemd/ 51 unit and timer files, credential redacted env/ENVIRONMENT_REPORT.txt On the M4 mac-mini (100.94.8.94), in /tmp/ 04-kimi-reference.tar.gz 146,502 B all 73 source files from ~/kimi-blanket (*.c *.h *.py *.m *.sh *.jinja) every benchmark log the fixture binaries (h_float.bin, h_int8.bin, h_ex16.bin, h_ex20.bin, h_resident.bin, bridge_ref.bin, embeds.bin, dual_kimi.bsig, dual_lex.bsig) docs/kimi_vector_organ_design.md kimipkg/exact_generators.py from /Volumes/MEGAMIND/kimi_k25/model/ kimipkg/kimi_modeling_deepseek.py Kimi's own modeling_deepseek.py kimipkg/kmod_modeling_deepseek.py the ~/kimi-blanket/kmod/ copy 05-fixtures.tar.gz NOT BUILT — see B.9. Blocked on B.3. -------------------------------------------------------------------------------- D.2 RETRIEVING THE PACKAGES -------------------------------------------------------------------------------- From megamind-agi (Windows host SSH, default shell is cmd.exe): scp -O josep@100.69.120.61:'C:/Users/Josep/...' . # not this way The packages are inside WSL, not on the Windows filesystem. Two options: (a) copy them out to the Windows home first, then scp: ssh josep@100.69.120.61 "wsl.exe -d Ubuntu -- bash -c \ 'cp ~/deepseek-megamind-upload/*.tar.gz /mnt/c/Users/Josep/'" scp -O josep@100.69.120.61:C:/Users/Josep/01-deepseek-source.tar.gz . (b) or pull directly over the WSL guest's own tailnet address once sshd is running there (it is not, at scan time — port 22 on 100.77.114.40 refuses). From the M4: scp joseph@100.94.8.94:/tmp/04-kimi-reference.tar.gz . -------------------------------------------------------------------------------- D.3 COMMANDS TO REBUILD EVERY PACKAGE -------------------------------------------------------------------------------- These are the exact commands used, so the work is reproducible. # ---- 01 : DeepSeek source bundle ------------------------------------------- mkdir -p ~/deepseek-megamind-upload/deepseek/scanners cd ~/DeepSeek-R1 cp config.json configuration_deepseek.py modeling_deepseek.py \ generation_config.json tokenizer_config.json tokenizer.json \ model.safetensors.index.json README.md LICENSE .gitattributes \ ~/deepseek-megamind-upload/deepseek/ python3 ~/deepseek-megamind-upload/deepseek/scanners/gen_manifests.py python3 ~/deepseek-megamind-upload/deepseek/scanners/gen_hashes.py --partial cd ~/deepseek-megamind-upload find deepseek -type f -printf '%p\t%s bytes\n' | sort > deepseek/FILE_MANIFEST.txt tar -czf 01-deepseek-source.tar.gz deepseek/ # ---- 02 : MEGAMIND source -------------------------------------------------- cd /opt/mm tar -czf ~/deepseek-megamind-upload/02-megamind-source.tar.gz \ --exclude='*.pyc' --exclude='__pycache__' --exclude='*.log' --exclude='*.db' \ --exclude='*.wal*' --exclude='*.jsonl' --exclude='*.tsv' --exclude='*.u8' \ --exclude='*.env' --exclude='*.key' --exclude='*.pem' \ src/ code/src/ reason/*.py loops/ # ---- 03 : PANTHEON / ORACLE ------------------------------------------------ cd /opt/mm tar -czf ~/deepseek-megamind-upload/03-pantheon-source.tar.gz \ src/oracle.cpp src/oracle_orchestrator.h src/oracle_frontier.h \ src/seferim_*.h src/mmshard.cpp src/mmheart.cpp src/mmsub.cpp \ src/mmcap.cpp src/mmcap2.cpp src/mmsem.cpp src/mmserve.cpp \ src/mmembed.cpp src/mmembed2.cpp src/hdcembed.cpp src/mmgate.cpp \ src/mmeval.cpp src/mmstatus.cpp src/mmworld.cpp src/mmreason.cpp \ reason/mmask_serve.py reason/mmknow_serve.py reason/mmlearn_serve.py \ reason/mmreason_serve.py reason/mmthink_serve.py reason/mmgateway_web.py \ loops/ # ---- 04 : Kimi reference (run on the M4) ----------------------------------- mkdir -p /tmp/kimipkg && cd ~/kimi-blanket cp /Volumes/MEGAMIND/kimi_k25/model/exact_generators.py /tmp/kimipkg/ cp /Volumes/MEGAMIND/kimi_k25/model/modeling_deepseek.py /tmp/kimipkg/kimi_modeling_deepseek.py cp kmod/modeling_deepseek.py /tmp/kimipkg/kmod_modeling_deepseek.py tar -czf /tmp/04-kimi-reference.tar.gz \ *.c *.h *.py *.m *.sh *.jinja *.log docs/ -C /tmp kimipkg # ---- 06 : environment ------------------------------------------------------ U=~/deepseek-megamind-upload; mkdir -p $U/env/systemd cp /etc/systemd/system/mm*.service /etc/systemd/system/mm*.timer $U/env/systemd/ cp /etc/systemd/system/mm.target $U/env/systemd/ cp -r /etc/systemd/system/mm-oracle.service.d /etc/systemd/system/mm-serve.service.d $U/env/systemd/ cp /etc/wsl.conf $U/env/wsl.conf cp /mnt/c/Users/Josep/.wslconfig $U/env/wslconfig [REDACTED: credential-associated original line 5248; original retained outside this bundle.] tar -czf $U/06-env-and-units.tar.gz -C $U env/ -------------------------------------------------------------------------------- D.4 SECRET SCAN PERFORMED ON EVERYTHING PACKAGED -------------------------------------------------------------------------------- Two passes were run over every file staged for upload. Pass 1, broad keyword: [REDACTED: credential-associated original line 5257; original retained outside this bundle.] BEGIN .*PRIVATE KEY' Flagged ~20 files. Each was inspected. Pass 2, literal credential values only: [REDACTED: credential-associated original line 5262; original retained outside this bundle.] [[:space:]]*[=:][[:space:]]*"[^"]{6,}"' Over src/, code/src/, reason/*.py and loops/: ZERO matches. DISPOSITION OF EVERY FLAG eye.cpp and its .bak files 31 hits on "secret" — Living World game content (secrets as a world concept). Benign. seferim_sec.h 4 hits on "secret" — security primitive naming. No literal values. Benign. mma5_sectest.cpp 1 hit — the security test harness. Benign. [REDACTED: credential-associated original line 5272; original retained outside this bundle.] mm-loom.service 1 real credential — REDACTED in the package. THE ONE REAL FINDING /opt/mm/src/loom.cpp line 250: int main(){ const char* dsn=getenv("LOOM_DB"); string DSN=dsn?dsn:"host=127.0.0.1 dbname=livingworld [REDACTED: credential-associated original line 5279; original retained outside this bundle.] A hardcoded fallback DSN. It is a read-only PostgreSQL role reachable only on 127.0.0.1 inside a NAT'd WSL2 guest, so exposure is low — but it is a [REDACTED: credential-associated original line 5282; original retained outside this bundle.] I redacted the copy in mm-loom.service. I did NOT edit loom.cpp, because the request says explicitly not to clean the source before sending, and silently editing it would hide exactly the kind of dependency you asked to see. Your call — see PART E, item 1. NOT INCLUDED ANYWHERE, by design: /etc/mm/mm.env (mode 600, root) /etc/missionmatch/api.env ~/.ssh/* any .pem, .key or certificate [REDACTED: credential-associated original line 5292; original retained outside this bundle.] values of any secret environment variable ================================================================================ PART E — ACTIONS, DECISIONS AND BLOCKERS ================================================================================ [REDACTED: credential-associated original line 5300; original retained outside this bundle.] Options: (a) ship as-is, accepting a read-only localhost credential in the archive; (b) rotate the livingworld_ro role first, then ship; (c) let me redact the literal in the shipped copy and add a note in the file. I did not choose for you. Default if you say nothing: I will do (c). 2. [ACTION] Full SHA-256 of the checkpoint, before any verification claim: python3 ~/deepseek-megamind-upload/deepseek/scanners/gen_hashes.py --full ~22 minutes of serial reading. Overwrites deepseek_weight_hashes.txt with full-file hashes in place of the partial ones. 3. [BLOCKER] Source an official FP8 reference and pin it to a commit hash. Nothing downstream is verifiable until this exists. This is the single highest-value next step and it is a download, not a build. 4. [BLOCKED] Fixtures (section 9) wait on item 3. The raw-input halves of the FP8 fixtures can be frozen today — say the word and I will extract them for all six requested sites including the layer 31 expert 92 shard-crossing case. 5. [DECISION] Sanitized memory samples (section 7). Tell me which `source` values in the knowledge table are safe to publish and I will extract samples with that filter. The corpus includes an "mm-estate" organ scoped to business and client material, so I did not sample blind. 6. [RECOMMENDED] `git init` in /opt/mm and commit the current state. There is no version control on a 2 MB source tree that runs 61 live services, and history is currently ~90 hand-copied .bak files. 7. [GAP] cell_server.py does not exist anywhere; cell_client.py has no server. If the distributed dataflow matters for R1, that half must be rebuilt. 8. [SCOPE CORRECTION] Any plan written against the Go file list in section 4 needs revising before work starts. There is no Go, no container, no dependency manifest, and no MCP. See findings 1, 2 and 3 in the summary. ================================================================================ END OF DEEPSEEK REQUIREMENTS ================================================================================