Janady13/missionmatch / docs/archive/INTELLIGENCE_EXPANSION_SPEC.md

1# INTELLIGENCE_EXPANSION_SPEC 2 3missionMATCH work order, generated 27 July 2026 from the July 2026 competitive, regulatory, and data source intelligence report. Hand this to Claude Code as a single spec. It inherits every rule in CLAUDE.md. 4 5-------------------------------------------------------------------------------- 60. NON NEGOTIABLE RULES 7-------------------------------------------------------------------------------- 8 91. No Python anywhere. Go backend, TypeScript frontend, bash scripts. 102. No dashes in written output. Commas or rewrite. Hyphens appear only inside 11 code identifiers, protocol headers, and official designations. 123. Every API key stays on the server. Root owned 0600, never in the repo, 13 never in the browser bundle, never in a tenant database, never in logs. 144. Clearance is clearance intelligence, never verified. 155. Fail closed everywhere. Every new table gets organization_id, forced RLS, 16 and a composite index leading with organization_id. Migrations are the 17 only way schema changes. 18 19-------------------------------------------------------------------------------- 201. EXTERNAL INFERENCE BACKEND (OPENAI) 21-------------------------------------------------------------------------------- 22 231.1 KEY HANDLING, DO THIS FIRST 24 25The key shared in chat on 27 July 2026 is burned. Any secret that has passed 26through a chat client, email, or a browser must be treated as exposed. Tom 27revokes it in the OpenAI dashboard. Provision a fresh project scoped key under 28a Mission Workforce Technologies organization with a monthly usage cap and 29budget alerts. Store it through internal/platform/secrets as the named secret 30openai_api_key, file root owned 0600 on M4, resolved only by megamind-gateway 31at boot. It never appears in the repo, in config committed to git, in 32environment dumps, in error messages, or in the browser bundle. Confirm the 33API organization has training on submitted data disabled and request zero 34data retention where eligible. 35 361.2 PLACEMENT 37 38The FIPS API binary never speaks to OpenAI. megamind-gateway on M4 gains a 39third proxied backend class alongside Ollama and MLX: 40 41 backend: openai 42 endpoint: https://api.openai.com/v1/chat/completions 43 auth: bearer from the secrets resolver 44 model: config value, validated non empty at boot, no default in code 45 enabled: config flag, default false 46 47Calls go through internal/platform/inference with the existing circuit 48breaker and bounded retries. If enabled is true and the secret is missing, 49the gateway fails the boot closed, matching the database role assertion 50pattern. 51 521.3 EGRESS POLICY, THE LOAD BEARING PART 53 54A scrub gate sits in front of every external call and classifies the payload. 55Only these classes may leave the machine: 56 57 PUBLIC_KNOWLEDGE drafting or revising knowledge cards from cited 58 public sources 59 PUBLIC_DATA_SUMMARY summarizing ingested public federal datasets 60 ONTOLOGY_SUGGEST skill ontology and term alias suggestions from 61 public vocabulary 62 REQUIREMENT_NORMALIZE normalizing text from public solicitations only 63 64Never external, enforced by the gate and by tests: 65 66 candidate rows, resume text, names, emails, phone numbers 67 any tenant identifier or organization_id 68 anything from the eeo schema (already unreadable by the app role, 69 assert it anyway) 70 RouteRAG and RouteGraph over tenant data 71 scoring inputs, scoring outputs, and slate contents 72 73The gate fails closed. If it cannot prove the class, the call does not 74happen and the router falls back to the local plane. The verifier still 75gates all returned prose exactly as it does for local models. Every external 76call writes an audit row containing backend name, payload class, byte count, 77and content hash, never content. 78 791.4 GOVERNANCE 80 81Record the backend in the model registry and the AEDT inventory as outside 82the FIPS boundary and restricted to non regulated tasks. Update MODEL_CARD 83and STACK_DECISIONS with the reasoning. Curation time use, meaning Tom 84drafting cards, briefs, and public data summaries with no tenant data 85involved, is unrestricted and needs no gateway path at all. 86 87-------------------------------------------------------------------------------- 882. INTELLIGENCE INGESTION 89-------------------------------------------------------------------------------- 90 912.1 TIER A, BEFORE DESIGN PARTNER GO LIVE 92 93KNOWLEDGE CARDS, added through curation.curate (superuser only), each cited 94to primary sources, entering the KAG set and GET /knowledge/cards: 95 96 1. NYC Local Law 144: annual independent bias audit, ten business day 97 candidate notice, four fifths impact ratios, DCWP enforcement. 98 2. Illinois HB 3773, effective 1 January 2026: AI use notice duty, zip 99 code proxy prohibition, private right of action. Include the 100 separate AI Video Interview Act. 101 3. California FEHA automated decision system rules, effective 102 1 October 2025: agent liability reaching staffing firms, four year 103 recordkeeping, bias testing relevance. 104 4. California CCPA ADMT rules, effective 1 January 2026: pre use 105 notice, opt out, logic disclosure, risk assessments. 106 5. Colorado SB 26-189: duties begin 1 January 2027, enforcement 107 currently stayed, attorney general rulemaking pending. 108 6. Texas TRAIGA, effective 1 January 2026: intent based liability, 109 attorney general enforcement only. 110 7. New Jersey LAD guidance, January 2025: disparate impact without 111 intent, vendor liability. 112 8. Mobley v. Workday through the 22 June 2026 order: vendor as agent, 113 bias testing discoverable, collective scope. 114 9. Kistler v. Eightfold FCRA theory: third party enrichment as 115 consumer reporting, and why missionMATCH is first party only. 116 10. OFCCP after EO 11246: Section 503 and VEVRAA survive, the VEVRAA 117 benchmark is 5.1 percent, the ESDS state job bank listing duty 118 remains. 119 11. EU AI Act: employment is Annex III high risk, Digital Omnibus 120 timing, relevant only if a tenant is EU facing. 121 122SYNC BINARIES, each Go, batch, idempotent, following the sam-sync pattern, 123each writing provenance into the ingest_manifest table defined in 2.3: 124 125 oews-metro-sync BLS OEWS at MSA level via the BLS public data API 126 version 2. Free registration key raises daily 127 limits. Annual release. Powers metro wage 128 benchmarking for LocationAvail and slate pricing. 129 sca-wd-sync SAM.gov wage determinations REST API. Roughly 130 ninety thousand records across DBA, SCA, and CBA 131 types. Current revision is flagged and superseded 132 revisions persist. No bulk download, so page and 133 cache. 134 usajobs-sync USAJOBS search API. Authentication needs three 135 headers: Authorization-Key, User-Agent set to the 136 registered email, and Host data.usajobs.gov. Pull 137 clearance fields, occupational series, pay plans, 138 and the unauthenticated code list API. 139 awards depth Extend awards-sync to task and delivery order 140 granularity using USASpending File D1 through the 141 api.usaspending.gov bulk_download endpoints. This 142 powers the recompete radar. 143 144ALSO IN TIER A: 145 146 SAM system account. Personal keys allow roughly ten calls a day, role 147 backed accounts roughly one thousand, federal system accounts roughly 148 ten thousand. Opportunities pages cap at one thousand records and the 149 description field costs a second authenticated call, so budget double. 150 Registration approval can take weeks, start now. 151 152 CALC+ verification. Confirm whether CALC+ under buy.gsa.gov exposes a 153 usable bulk path before building on it. Legacy CALC data may be stale. 154 If there is no API, ceiling rates load manually and are versioned. 155 156 ontology-sync bump. Move to O*NET 30.3 now and 31.0 when it lands in 157 late August 2026. Keep the CC BY 4.0 attribution string in the 158 manifest. The database license is CC BY 4.0, but Career Exploration 159 Tools carry a no derivatives license, do not ingest those. 160 1612.2 TIER B, DURING THE 90 DAY PROGRAM 162 163 esds-post VEVRAA state job bank posting generated from 164 requisitions. Formats vary by state and most lack 165 APIs, so build state by state starting with the 166 states where design partners operate. 167 skillbridge-sync The DoD SkillBridge authorized organizations 168 directory only, roughly three thousand listings. 169 Organizations, never people. No candidate PII ever. 170 forecast-sync Agency procurement forecasts. Mostly spreadsheets, 171 normalize into the requirement radar. 172 dsbs data SBA socioeconomic and 8(a) data through SAM.gov 173 entity extracts and the public entity APIs, never 174 through third party scrapers. 175 clearance runway A small versioned reference table, hand curated 176 each quarter from GAO reports and performance.gov 177 Trusted Workforce quarterly reports, surfacing 178 realistic adjudication timelines on slates. 179 comp benchmarks Cleared compensation benchmarks only under license. 180 Never scraped. 181 1822.3 PROVENANCE 183 184New table ingest_manifest: source, url, license, fetched_at, record_count, 185content_hash. Every knowledge card and every synced row traces back to a 186manifest row. This is the evidence trail behind the first party data story 187and it feeds the AEDT evidence pack. 188 189-------------------------------------------------------------------------------- 1903. WHERE THE INTELLIGENCE SURFACES 191-------------------------------------------------------------------------------- 192 193New closed TAG intents, read only, parameterized, model selects the NAME 194only, exactly like the existing registry: 195 196 wage_benchmark_by_msa(soc_code, msa) 197 sca_wd_lookup(state, county) 198 recompete_radar(agency, naics, months_ahead) 199 vevraa_benchmark_status() 200 clearance_runway(clearance_level) 201 202The eleven cards enter RouteDirect and the public knowledge feed. The AEDT 203evidence pack export lands in the governance module as a stub: scoring 204inputs, engine version, weights, band caps applied, model registry entry, 205exportable per decision. 206 207-------------------------------------------------------------------------------- 2084. ACCEPTANCE CHECKS 209-------------------------------------------------------------------------------- 210 2111. Gateway with the openai backend enabled and no secret refuses to boot. 2122. A fixture candidate row placed anywhere in an external bound payload 213 fails the scrub test suite. Prove it by racing it, not by reasoning. 2143. Every new sync writes ingest_manifest rows and reruns without 215 duplicates. 2164. All eleven cards render with citations and appear in 217 GET /knowledge/cards. 2185. rg confirms zero new Python and zero dashes in written docs. 2196. No new table is missing organization_id, forced RLS, or the composite 220 index. Down migrations refuse to delete application authored rows. 221 222-------------------------------------------------------------------------------- 2235. EXPLICIT NON GOALS 224-------------------------------------------------------------------------------- 225 226No scraping of candidate data anywhere, for FCRA and CUI reasons. No NL to 227SQL. No request time external calls carrying tenant candidate data. No 228multi master writes. No Python. 229