Janady13/missionmatch / BUILD_REPORT.md

1# BUILD_REPORT.md — W3 AI Staffing Recommendations 2 3**Date:** 2026-06-29. **Branch:** `w3-ai-staffing-recommendations` (off `ai-intelligence-layer`). 4**Build host:** M2. **Database validated on:** M4 staging (`tenant_w3_staging`, goose v18). 5 6W3 is a backend build: an AEDT compliant recommendation layer on top of the deterministic decimal engine. 7It turns the frozen engine scores into explained, human decided staffing ACTIONS in three views over one 8core. This report records what was built, what was researched and relied on, the test results, what the 9adversarial review found and what was fixed, and what remains. 10 11## Non negotiable constraints, all held 121. **No Python.** `find . -name '*.py'` outside node_modules is empty. The optimizer is a pure Go minimum 13 cost flow solver, no cgo, no native math. Narration runs out of process on the existing MEGAMIND plane. 142. **FIPS intact.** The API binary builds `CGO_ENABLED=0 GOFIPS140=v1.0.0`; the stamp was verified after 15 every build. W3 added no cgo and no dependency (it uses only the standard library plus the packages 16 already present: shopspring/decimal, pgx, chi, uuid). 173. **EEO isolation absolute.** No W3 path names the eeo schema; sqlc models no eeo table. The existing 18 `TestNoAIFileReferencesEEO` tripwire was extended to walk every staffing source file, and a new 19 `TestNoProtectedAttributeInCoreTypes` reflects over the core input and output types (and the engine 20 Candidate and Requirement) and fails on any protected attribute field. `safety.ForbidEEO` guards every 21 string sent to the plane, with a defense in depth re check on the model's output. 224. **Tenant isolation.** Every W3 database operation runs inside `RunInTenantTx`, RLS scoped, on the 23 validated org. The MEGAMIND plane stays stateless. The proactive cron lists orgs through the migrator 24 role but does the per org work under the NOBYPASSRLS app role and RLS. 255. **The deterministic engine is the floor.** Eligibility, the six component scores, the band, and the 26 rank come only from `engine.ScoreCandidate`, reused through the matching package (new exported load 27 wrappers, `matching.LoadEngineRequirement` / `LoadEngineCandidate`) and persisted frozen to 28 `candidate_evaluations`, which each non source recommendation references by composite FK. W3 never 29 recomputes or overrides a score. The optimizer arbitrates allocation only among candidates the engine 30 already qualified; it never qualifies or disqualifies anyone. 316. **Explainable, human in the loop, never auto reject.** Every recommendation traces to frozen numbers 32 and explicit rules. A person approves, rejects, or dismisses each one. Rejecting a suggestion dismisses 33 only that row; it never blacklists a candidate. A source recommendation names no candidate. 347. **No model in the decision path.** The model produces only the rationale prose, behind the existing 35 faithfulness gate, with a deterministic fallback. Who, what action, what rank, what confidence, and the 36 portfolio allocation are computed only from frozen engine outputs, rules, adjacency, availability, 37 commitment, and the deterministic optimizer. 38 39## Research (primary sources, cited in code) 40A 3 agent research workflow verified the optimizer math and the faithfulness pattern before building: 41- **Minimum cost flow by successive shortest paths with node potentials** (Tomizawa 1971; Edmonds and 42 Karp 1972; Ahuja, Magnanti, Orlin, *Network Flows*, ch 9 and 12). Confirmed: seed potentials with one 43 Bellman Ford pass (a max profit objective produces negative costs), then Dijkstra over non negative 44 reduced costs per augmentation; potentials keep all residual reduced costs non negative; augment one 45 unit while the shortest augmenting path has negative real cost and STOP at zero (the cost vs flow curve 46 is convex, so this is the exact max profit optimum); integer exact under unit capacities and total 47 unimodularity, no float in the objective; deterministic via stable node ids and a node id priority 48 queue tiebreak. This is implemented exactly in `optimizer.go`. 49- **Maximum profit unbalanced assignment** as a min cost flow where candidates and slots may be left 50 unmatched (the SOURCE rows), and the value matrix counterfactual as the interpretable, AEDT defensible 51 explanation (the next best alternative held fixed, not an LP re solve marginal). 52- **Faithfulness gating** (CRAG, Self RAG family): constrain the prose to the supplied evidence, withhold 53 and fall back to a deterministic template when the gate cannot confirm, and record why for later tuning. 54 55## What was built 56- **Migration 0018** (`recommendation_runs`, `staffing_recommendations`): RLS enabled and forced, org 57 scoped, composite FKs leading with organization_id, the engine evaluation reference as a composite FK to 58 `candidate_evaluations (organization_id, candidate_id, requirement_id)` that MATCH SIMPLE correctly skips 59 for a source row, candidate scoped for erasure, grants to `missionmatch_app` only. Numbered 0018 on the 60 v17 head. 61- **The pure recommendation core** (`config.go`, `types.go`, `value.go`, `confidence.go`, `classify.go`): 62 global config with per run override and auditable defaults; the int64 scaled value function (engine 63 total times a bounded priority multiplier from the opportunity dollar value, documented rounding); 64 deterministic confidence (fit, clearance provenance, completeness); the action classifier (assign, 65 reassign with a priority delta, reskill gated on the clearance gate plus a KAG adjacency, a single 66 source note). All pure, no DB, no clock, no model. 67- **The optimizer** (`optimizer.go`): the pure Go minimum cost flow solver, a size cap and a context time 68 budget so a solve never hangs, and the value matrix counterfactual extraction. 69- **The three views** (`gap.go`, `portfolio.go`, `proactive.go`): the gap view reads current commitment so 70 a committed candidate is a reassign, not a duplicate assign; the portfolio view runs the optimizer, 71 builds per assignment counterfactuals, and is value fallback safe (an unknown opportunity value is 72 imputed with the median of the known values, never silently zero); the proactive bench watch. 73- **Narration** (`narrate.go`): faithfulness gated prose grounded only in the deterministic evidence, with 74 a deterministic fallback and a recorded fallback reason (kept out of the audit). The recommendation 75 never depends on it; a plane outage degrades the prose, not the correctness. 76- **The service and endpoints** (`service.go`, `handler.go`, `dto.go`, `errors.go`): four endpoints behind 77 the session resolver, CSRF, RBAC, rate limit, and audit, with one typed error envelope (404, 409, 413, 78 422, 503), pagination, a fresh run per POST that supersedes only the prior pending rows of its scope, the 79 decision path with row locking and a 409 on a non pending row, and the two AEDT events 80 (`aedt.staffing.recommended` on a run, `aedt.staffing.decided` on a decision) plus a served score audit 81 for each surfaced candidate (consistent with the W2 four fifths fix). No PII, no names, no demographics, 82 and no rationale prose in any audit payload. 83- **The scheduled job** (`cmd/staffing-proactive-sync`, `infra/systemd/mm-staffing-proactive.{service, 84 timer}`): pure Go, FIPS built, daily at 05:00 after the ingestion timers, idempotent, deterministic 85 rationales. 86- **Erasure**: the right to erasure removes a candidate's recommendations in BOTH the sourcing anonymize 87 and the governance retention paths (the FK cascade does not fire on anonymize in place). 88- **RBAC**: a new write capability `staffing.recommend`, granted to the roles that hold `slate.write`. The 89 decision reuses the slate capability; reads reuse `candidate.read`. 90- **Wiring and contract**: mounted in `cmd/api/main.go` (reusing the one skill matcher, an optional 91 inference client); the full request and response contract added to `docs/openapi.yaml`. 92 93## Test results 94- **Unit (race, no DB):** all pass, including 18 classifier, value, and confidence golden tests and the 9 95 optimizer tests. The decisive optimizer test proves it is a true optimizer, not greedy: an instance 96 where greedy scores 11 and the true max profit is 17, solved to 17, plus determinism over 20 runs, the 97 size cap, the budget, and the counterfactuals. 98- **Safety unit:** the EEO tripwire (now over the staffing package) and the no protected attribute test. 99- **Live integration on M4 staging:** 5 of 5 W3 tests pass (gap assign plus source with a real frozen 100 evaluation reference and the deterministic fallback; portfolio allocation with value and counterfactual; 101 the proactive bench watch; the decision plus a 409 conflict with no auto reject; erasure removal). The 102 full suite on the bare staging database: 37 pass, 6 skip (plane or seed dependent), 1 fail 103 (`TestCommandCenterUpcomingIncumbentAdopt`, which needs the global federal forecast catalog seeded; 104 empty on a fresh staging database, unrelated to W3). Zero W3 caused failures. 105- **Migration:** 0018 applied cleanly through the full v18 schema plus the eeo set on staging. 106- **Build gates:** FIPS stamp present after every build, `make ci-gates` green (backdoor and FIPS), vet 107 and gofmt clean, the whole repo builds, no Python, no cgo. 108 109## What the adversarial review found and what was fixed 110A 5 dimension adversarial review (EEO and AEDT safety, the engine floor and no model in the decision path, 111denial of service and concurrency, data integrity and tenancy, hygiene and contract) with each finding 112independently verified, confirmed two issues, both fixed: 1131. **Medium, concurrency.** The gap and proactive write paths superseded then inserted without the per org 114 advisory lock the portfolio path held. Under READ COMMITTED, two concurrent same scope runs could each 115 leave a live pending set. Fixed: the advisory lock was generalized to a `(view, org, scope)` key and 116 applied to all three write paths, so same scope writes serialize while different scopes still run in 117 parallel. (Decided rows were never at risk; the status guarded supersede and the FOR UPDATE decision 118 path were already correct.) 1192. **Low, hygiene.** `GetRecommendationRun` and `ListRecommendationsByRun` were dead queries. Fixed: 120 removed and regenerated. 121 122## What remains (genuine decisions or sequenced work) 1231. **The frontend** is a separate workstream (out of scope here by direction). The backend interface 124 handoff is `docs/openapi.yaml`: the finalized request and response schemas, the enums, and the one 125 error envelope the future UI consumes. The decision columns mirror `slate_members` so the existing 126 approve and reject component carries over. 1272. **The production cutover** is the operator's manual step in `DEPLOY_PLAN.md` (apply 0018, deploy the 128 binary that mounts the routes, install the proactive timer, verify, roll back). It was validated on M4 129 staging and deliberately NOT run against production. 1303. **Per org configuration** of the floor, the adjacency threshold, and the confidence constants is a 131 future addition; v1 is global defaults plus a per run override, recorded in the run parameters. 1324. **The learned LTR reranker with SHAP** (AI layer Phase 8) and the **learned win probability** (Phase 9) 133 stay OUT of W3 v1 by direction. The **TEAM** action is reserved in the enum and waits on GraphRAG 134 teaming leaving scaffold. 1355. **Proactive change attribution.** The bench watch refreshes the full available bench each run and tags 136 the trigger as a recompute. Attributing a specific change (newly available, newly certified, a new 137 opportunity unlock) via candidates.updated_at diffing is a refinement. 1386. **Feed freshness.** The watch only surfaces an opportunity once the ingestion timers have ingested it; 139 the audit flagged the SAM sync timer as possibly not firing. DEPLOY_PLAN notes confirming the timer 140 before relying on the watch for new postings. W3 does not change the sync timers. 141 142## Commits 143`5485603` migration and queries and RBAC; `10828b2` the recommendation core; `a60ef4c` the optimizer; 144`e183eb0` the three views, narration, service, endpoints, wiring, and safety tests; `e298916` the proactive 145job and timer; `2ac466f` the erasure join and integration tests; `15fc448` the openapi and deploy plan; 146`635a410` the final review fixes. 147