Source identity
| Field | Recorded value |
|---|---|
| Source type | GitHub source |
| Original record | Janady13/missionmatch / docs/evidence/phase2/functional/gate-a-rerun.txt |
| Date | Not specified in source |
| Repository | Janady13/missionmatch |
| Revision | 720d47fae476b0d38477ae4401697d372b1ea2bc |
| Branch | feature/opportunity-to-outcome |
| Path or archive member | docs/evidence/phase2/functional/gate-a-rerun.txt |
| Original SHA-256 | 82142b2cb87ddafd47111e8b5d2ecc92aac3649db4b140cd37a2d12a3ffaf818 |
| Git blob SHA | 7ed72df6d44fede734b0a8df459df346dff71ae9 |
| Reading-copy SHA-256 | 82142b2cb87ddafd47111e8b5d2ecc92aac3649db4b140cd37a2d12a3ffaf818 |
Recovered source transcript
Download complete source text ↓92 lines · 7.3 KB
GATE A RERUN, 29 July 2026 night. VERDICT: PASS.
Unit 55 packages ok, 0 FAIL. Integration 217 PASSED, 1 SKIPPED, 0 FAILED.
Started this session at 195 passed, 23 skipped, 7 FAILED.
SIX DEFECTS FIXED TO GET HERE, and two of them were production defects rather than test defects.
1. THE PRE AUTH REFUSAL COULD NOT BE AUDITED, SO IT WAS NOT SERVED. Code defect.
orchestrator.audit() always wrote through db.RunInTenantTx, which by design refuses without an
organization_id. A refusal raised BEFORE any tenant is resolved (an unauthenticated question, an injection
caught by the safety gate) therefore could not be audited, and because the helper fails closed the REFUSAL
ITSELF was withheld: the caller got a 503 saying the decision could not be recorded, where a clean refusal
was correct. Now a context with no org routes to audit.System, which writes off the pool with a NULL org,
exactly as the auth module does for a failed login, and audit_logs is outside RLS so a NULL org row is a
legitimate platform event.
FAIL CLOSED IS PRESERVED. TestRefusalIsNotServedWhenTheAEDTAuditCannotBeWritten and
TestAnswerIsNotServedWhenTheAEDTAuditCannotBeWritten both still pass. The change is WHICH writer suits an
event that has no tenant, never whether an unrecordable decision may be served.
PROVEN POSITIVELY, not by absence of error: audit_logs now holds 6 aedt.ai.refused rows with a NULL
organization_id alongside 22 org scoped ones. The row is written, not merely un errored.
2. THE STAGING APP ROLE NEVER RECEIVED WHAT THE MIGRATIONS GRANT. Environment defect, migration 0085.
51 of 84 migrations name missionmatch_app or missionmatch_migrator literally. ZERO name the staging roles
the environment actually connects as. Measured: table grants already mirrored 195 and 195; column grants
2055 against 2048, SEVEN missing, every one an organizations UPDATE column from migration 0057; and of 57
RLS policies, 6 name missionmatch_app and 0 name missionmatch_staging_app.
THE POLICY HALF WAS THE WORSE HALF. A policy with a TO clause applies only to the named roles and their
members, and with NO applicable policy RLS denies everything, so the staging app role was refused on
outcome_events while holding INSERT and SELECT. The error read "new row violates row level security policy",
which sounds like a broken policy and was actually a policy never addressed to the caller.
ROLE MEMBERSHIP WAS REFUSED AS THE FIX even though it is one line and the PostgreSQL documentation confirms
membership and inheritance apply to a policy's TO clause. Membership conveys ALL privileges including
database CONNECT, and missionmatch_app has CONNECT on the production database while the staging roles
deliberately have none. Measured: missionmatch app=t staging_app=f. That f is the containment boundary,
enforced at connection time, and a shorter migration is not worth it.
0085 mirrors table privileges, column privileges and policy applicability, touches no database privilege,
is guarded to the staging database because roles are cluster wide, asserts parity afterwards, and RE ASSERTS
that the staging role did not gain production CONNECT. Applied: 0 table grants, 7 column grants, 6 policies.
3. A LIVE TEST NAMED A SCORING CONTRACT THE ENGINE LEFT BEHIND. Test defect.
TestMegamindTeachBoundaryLive hardcoded model_version "v1.0.0"; the engine has been frozen at v1.4.0. It now
reads engine.ModelVersion so the next freeze cannot rot it. It never reported this because it skips itself
when MM_MEGAMIND_URL is unset and nothing set it. A TEST GATED ON AN OPTIONAL ENVIRONMENT VARIABLE DOES NOT
FAIL, IT DISAPPEARS.
4. THE GATEWAY REFUSED EVERY TEACHING SIGNAL EVER SENT. PRODUCTION defect.
scrub.Validate requires a registered engine model version, and that allow list is populated only through
scrub.RegisterModelVersion. The tenant side registers it in internal/modules/ai. THE GATEWAY PROCESS NEVER
DID, so its allow list was EMPTY and /v1/teach answered 422 to every signal whatever version it named. The
federated teaching transport has been closed since the allow list was introduced.
THAT IS WHY teaching_signal_aggregates AND curation_proposals ARE BOTH EMPTY: not unused, unreachable.
Fixed: the gateway registers engine.ModelVersion at boot, accepts extra contracts through
MEGAMIND_TEACH_MODEL_VERSIONS for a redeploy skew window, and LOGS the accepted set, because an allow list
that silently refuses everything looks identical to one nobody is using. Staging boot now logs
"teaching boundary accepts scoring contract(s) v1.4.0".
5. THE WHOLE INTEGRATION SUITE AUTHENTICATED THROUGH A PROXY IT DID NOT KNOW IT NEEDED. Test defect.
Four test files built inference clients with no bearer. Every call returned 401 against a gateway addressed
directly, and the suite passed only because MM_MEGAMIND_URL pointed at port 8090, a fifteen line node
reverse proxy belonging to PANTHEON that injects the credential and forwards to the real gateway. The suite
was proving THE PROXY authenticates. All four now pass MM_MEGAMIND_TOKEN.
6. THE EMBEDDING PROVIDER COULD NOT AUTHENTICATE AT ALL. PRODUCTION defect, and the most consequential one.
embeddings.newMegamind accepted cfg.APIKey and never handed it to the inference client, so the megamind
embedding provider had NO WAY to send a bearer. Every embedding in this platform therefore depended on that
same node proxy staying alive, with no recovery path from inside this codebase.
THE FAILURE SHAPE IS THE LESSON. The unauthenticated call returned 401, the error was swallowed upstream,
and the test reported SIMILARITY 0. It read as "the embedder has no semantics" when the cause was "the
request was refused". A transport failure wearing the costume of a bad model score.
HOW THE SUITE WAS MADE TO STOP DISAPPEARING. 23 tests skipped at the start of the session, 1 at the end. The
difference is that every optional environment variable the suite gates itself on was actually supplied:
MM_MEGAMIND_URL and MM_MEGAMIND_TOKEN, MM_COMPLIANCE_DSN, MM_AIOE_DSN. That un skipped the adverse impact and
fairness snapshot AEDT tests, the WS5 dry run, the aioe isolation tests, the planner regression and EEO refusal
proofs, the retrieval recall gate, and the semantic skill match. FOUR OF THE SIX DEFECTS ABOVE WERE ONLY
REACHABLE ONCE THOSE TESTS RAN.
THE ONE REMAINING SKIP, named rather than hidden: the per tenant database isolation test, gated on
MM_TENANT_A_DSN and MM_TENANT_B_DSN. Those databases do not exist because the database per tenant work is not
merged. It is correctly skipped and it is the only coverage this gate does not have.
POSITIVE CONTROLS, run and passing rather than assumed: TestSuitePositiveControl,
TestBindingSiteDetectorPositiveControl, TestBackstopCatchesAMisdeclaredCandidateRow,
TestBackstopLimitIsKnownAndDocumented, plus TestExportControlledAttachmentIsRefused and
TestTenantDSNUsesTheTenantHostNotTheControlPlaneHost.
REPRODUCE
GOFIPS140=v1.0.0 go test ./... -count=1
MM_MIGRATOR_DSN=<staging migrator> MM_APP_DSN=<staging app> MM_COMPLIANCE_DSN=<staging compliance> \
MM_AIOE_DSN=<staging aioe> MM_MEGAMIND_URL=http://100.94.8.94:8091 MM_MEGAMIND_TOKEN=<gateway token> \
GOFIPS140=v1.0.0 go test -tags=integration ./internal/integration/... -count=1
Supplying every DSN is not optional rigour. Omitting one silently removes tests.