Janady13/missionmatch / internal/modules/ai/graph_bench.go

1package ai 2 3// G4 the bench graph. This file adds the FIRST person touching structural plane as FIXED, named, 4// parameterized QueryIntents in the existing closed Registry. Every G4 intent reads the RLS protected 5// person tables (candidates, candidate_skills, candidate_evaluations), so each runs inside RunInTenantTx 6// (the app role sees only the caller's bench through app.current_org_id) and pipes its aggregate partition 7// through the deterministic small cell suppression layer before any count leaves the process. The model 8// authors no SQL: every statement is a static, reviewed constant, registered in the closed Registry, 9// asserted by ValidateRegistry at boot, and re guarded by GuardReadOnlyQuery on every execution. 10// 11// The graph NEVER computes a match score, a band, or any employment decision number. The frozen engine in 12// internal/modules/matching/engine stays the only scorer and is untouched. These intents report STRUCTURAL 13// counts over the bench: 14// 15// - bench_skill_coverage partitions the bench into {holds the skill, does not hold the skill} for one 16// skill, a count to size who already carries a capability. 17// - bench_readiness_distribution reports the distribution of bands the frozen engine ALREADY decided and 18// audited at evaluation time, read from candidate_evaluations. It recomputes nothing; it counts the 19// decisions the engine already made. 20// - bench_gap_for_requirement partitions the bench into {meets, does not meet} where meets is a STRUCTURAL 21// coverage test (holds at least the required clearance level by the rank ladder AND holds at least a 22// fixed fraction of the required skills). This is a count to SIZE a gap, explicitly not a match score; 23// the per candidate score and ranking remain the frozen engine. 24// 25// Each intent's cells form a PARTITION of the matched set (every matched candidate lands in exactly one 26// bucket, so the counts sum to the matched total), which is the assumption Suppress's complementary rule 27// relies on. The executors build the COMPLETE partition in Go (always both buckets, always the three bands) 28// so an empty bucket is a true zero in the partition and a suppressed cell is indistinguishable from an 29// empty one. Headcounts are int (a headcount is an integer, suppressed as a unit); no float touches any 30// surfaced number. 31// 32// DEFERRED, by design: a multi skill ontology family rollup for bench_skill_coverage. A candidate can hold 33// many skills, so grouping by a skill family is NOT a partition (the cells overlap, the counts do not sum 34// to the bench), which breaks the assumption Suppress's complementary suppression relies on. Phase 2 ships 35// ONLY the partition safe single skill version; the family rollup waits for a suppression rule that handles 36// overlapping cells. 37 38import ( 39 "context" 40 "errors" 41 42 "github.com/jackc/pgx/v5" 43 44 "github.com/missionworkforce/missionmatch/internal/platform/tenant" 45) 46 47// The registered names of the three G4 bench intents. Each is a FIXED entry in the closed Registry in 48// tag.go, so ValidateRegistry asserts it at boot and ExecuteIntent runs it through the same guard, tenant 49// transaction, and audit posture as every other intent. 50const ( 51 IntentBenchSkillCoverage = "bench_skill_coverage" 52 IntentBenchReadinessDistribution = "bench_readiness_distribution" 53 IntentBenchGapForRequirement = "bench_gap_for_requirement" 54) 55 56// The fixed buckets and bands. Building the COMPLETE partition in Go (rather than only the buckets SQL 57// returns) guarantees the partition spans every bucket, so Suppress always sees a closed set whose counts 58// sum to the matched total and a suppressed cell is indistinguishable from an empty one. 59const ( 60 bucketHolds = "holds" 61 bucketLacks = "lacks" 62 bucketMeets = "meets" 63 bucketDoesNotMeet = "does_not_meet" 64) 65 66// readinessBands is the fixed, ordered set of bands the frozen engine assigns. The readiness distribution 67// is reported over exactly these three, in this order, so the partition is complete and byte stable. 68var readinessBands = []string{"green", "yellow", "red"} 69 70// benchSkillCoverageSQL counts DISTINCT candidates that HOLD a given skill versus those that do not, for the 71// caller's org. RLS scopes it to the caller's bench. The bucket is computed with an EXISTS subquery over 72// candidate_skills (a SCALAR predicate, never a FROM clause set returning function), so the read stays over 73// the allowlisted candidates and candidate_skills tables and the bucket is single valued (a candidate holds 74// the skill or it does not), a true two bucket PARTITION whose counts sum to the bench size. The soft delete 75// and anonymize filters mirror count_candidates_by_clearance so an erased candidate is never counted. 76// Deterministic ORDER BY on the bucket label, a total order over the two buckets. The executor assembles the 77// complete {holds, lacks} partition in Go, filling a true zero for a bucket SQL omits. 78const benchSkillCoverageSQL = ` 79SELECT 80 CASE WHEN EXISTS ( 81 SELECT 1 FROM candidate_skills cs 82 WHERE cs.organization_id = c.organization_id 83 AND cs.candidate_id = c.id 84 AND cs.skill_id = $1 85 ) THEN 'holds' ELSE 'lacks' END AS bucket, 86 count(DISTINCT c.id) AS headcount 87FROM candidates c 88WHERE c.deleted_at IS NULL 89 AND c.anonymized_at IS NULL 90GROUP BY bucket 91ORDER BY bucket ASC` 92 93// benchReadinessDistributionSQL counts DISTINCT evaluated candidates per band for one requirement, reading 94// the ALREADY AUDITED candidate_evaluations table. The band column is the frozen engine's decision at 95// evaluation time; this intent NEVER recomputes a score, it reports the distribution of decisions the engine 96// already made and audited. RLS scopes it to the caller's org, and the requirement filter ($1) scopes it to 97// one requirement. It JOINs candidates and filters c.deleted_at and c.anonymized_at IS NULL so an erased or 98// anonymized in place candidate is never counted: anonymize in place keeps the candidate row and its 99// evaluations (no cascade fires), so without this join an erased person's band would still feed the count. 100// The band is single valued (each evaluation carries exactly one band), so the cells form a PARTITION whose 101// counts sum to the live evaluated count. Deterministic ORDER BY on the band label. The executor assembles 102// the complete {green, yellow, red} partition in Go, filling a true zero for a band with no evaluations. 103const benchReadinessDistributionSQL = ` 104SELECT 105 ce.band AS band, 106 count(DISTINCT ce.candidate_id) AS headcount 107FROM candidate_evaluations ce 108JOIN candidates c 109 ON c.organization_id = ce.organization_id 110 AND c.id = ce.candidate_id 111WHERE ce.requirement_id = $1 112 AND c.deleted_at IS NULL 113 AND c.anonymized_at IS NULL 114GROUP BY ce.band 115ORDER BY ce.band ASC` 116 117// benchGapForRequirementSQL counts DISTINCT candidates that STRUCTURALLY meet a requirement versus those 118// that do not, for the caller's org. A candidate MEETS when it holds at least the required clearance level 119// (the rank ladder through clearance_levels, the same hierarchy the frozen engine encodes: a held rank at or 120// above the required rank) AND holds at least a fixed fraction (four fifths, the engine's full skills 121// threshold) of the requirement's required skills. The skill fraction is compared in integer arithmetic 122// (held matched times five is at least the required total times four, equivalent to held over total at least 123// four fifths) so NO float touches the test; a requirement with no required skills is vacuously met on the 124// skills arm. The required total and the held matched count are SCALAR subqueries over allowlisted tables 125// (required_skills joined to candidate_skills), never a FROM clause set returning function. This is a 126// STRUCTURAL coverage COUNT to SIZE a gap, explicitly NOT a match score: the per candidate score and the 127// ranking remain the frozen engine in internal/modules/matching/engine, which is untouched. RLS scopes the 128// read to the caller's bench and the requirement filter ($1) plus RLS means a foreign org requirement 129// returns no rows. The soft delete and anonymize filters exclude an erased candidate. Deterministic ORDER BY 130// on the bucket label, a total order over the two buckets. The executor assembles the complete {meets, 131// does_not_meet} partition in Go. 132const benchGapForRequirementSQL = ` 133SELECT 134 CASE 135 WHEN cl_c.rank >= cl_r.rank 136 AND ( 137 (SELECT count(*) FROM required_skills rs_total WHERE rs_total.requirement_id = r.id) = 0 138 OR 139 (SELECT count(*) FROM required_skills rs_held 140 JOIN candidate_skills cs 141 ON cs.organization_id = c.organization_id 142 AND cs.candidate_id = c.id 143 AND cs.skill_id = rs_held.skill_id 144 WHERE rs_held.requirement_id = r.id) * 5 145 >= 146 (SELECT count(*) FROM required_skills rs_req WHERE rs_req.requirement_id = r.id) * 4 147 ) 148 THEN 'meets' ELSE 'does_not_meet' 149 END AS bucket, 150 count(DISTINCT c.id) AS headcount 151FROM requirements r 152CROSS JOIN candidates c 153JOIN clearance_levels cl_c ON cl_c.code = c.clearance_level 154JOIN clearance_levels cl_r ON cl_r.code = r.required_level 155WHERE r.id = $1 156 AND r.deleted_at IS NULL 157 AND c.deleted_at IS NULL 158 AND c.anonymized_at IS NULL 159GROUP BY bucket 160ORDER BY bucket ASC` 161 162// BenchSkillRow is one bucket in the skill coverage partition: the bucket label (holds or lacks) and the 163// suppressed Cell carrying the headcount. The caller renders RenderCount(Cell), never the raw Count, so a 164// suppressed sub k headcount renders the sentinel and never leaves the process as a number. It carries no 165// score and no candidate id, only a structural headcount. 166type BenchSkillRow struct { 167 Bucket string `json:"bucket"` 168 Cell Cell `json:"cell"` 169} 170 171// BenchBandRow is one band in the readiness distribution: the band label and the suppressed Cell carrying 172// the headcount. The band is the frozen engine's stored decision, never a recomputed score. 173type BenchBandRow struct { 174 Band string `json:"band"` 175 Cell Cell `json:"cell"` 176} 177 178// BenchGapRow is one bucket in the structural gap partition: the bucket label (meets or does_not_meet) and 179// the suppressed Cell carrying the headcount. The meets test is structural coverage, never a match score. 180type BenchGapRow struct { 181 Bucket string `json:"bucket"` 182 Cell Cell `json:"cell"` 183} 184 185// BenchSkillCoverage runs the registered bench_skill_coverage intent under RLS (the app role sees only the 186// caller's bench), counting DISTINCT candidates that hold a given skill versus those that do not, then pipes 187// the complete {holds, lacks} partition through Suppress with the caller supplied threshold k (the caller 188// passes ConfiguredSuppressionThreshold). The two buckets sum to the bench size, so complementary 189// suppression is well defined. This implements ONLY the partition safe single skill version; a multi skill 190// ontology family rollup is deferred by design because a family grouping is not a partition (a candidate can 191// hold many skills, so the cells overlap and the counts do not sum to the bench), which would break the 192// assumption Suppress's complementary rule relies on. The caller renders RenderCount, never the raw Count, 193// for a suppressed cell. 194func BenchSkillCoverage(ctx context.Context, db *tenant.DB, skillID string, k int) ([]BenchSkillRow, error) { 195 if db == nil { 196 return nil, errors.New("ai: no database") 197 } 198 // Seed the complete two bucket partition with true zeros so an empty bucket is present and a suppressed 199 // cell is indistinguishable from an empty one. The SQL fills whichever buckets carry candidates. 200 counts := map[string]int{bucketHolds: 0, bucketLacks: 0} 201 err := db.RunInTenantTx(ctx, func(ctx context.Context, tx pgx.Tx) error { 202 if err := GuardReadOnlyQuery(benchSkillCoverageSQL, ReadAllowlist); err != nil { 203 return err 204 } 205 rows, err := tx.Query(ctx, benchSkillCoverageSQL, skillID) 206 if err != nil { 207 return err 208 } 209 defer rows.Close() 210 for rows.Next() { 211 var ( 212 bucket string 213 headcount int 214 ) 215 if err := rows.Scan(&bucket, &headcount); err != nil { 216 return err 217 } 218 if _, ok := counts[bucket]; ok { 219 counts[bucket] = headcount 220 } 221 } 222 return rows.Err() 223 }) 224 if err != nil { 225 return nil, err 226 } 227 // Build the partition in the fixed, byte stable bucket order, then suppress over the closed set. 228 order := []string{bucketHolds, bucketLacks} 229 out := make([]BenchSkillRow, len(order)) 230 cells := make([]Cell, len(order)) 231 for i, b := range order { 232 cells[i] = Cell{Key: []string{b}, Count: counts[b]} 233 } 234 cells = Suppress(cells, k) 235 for i, b := range order { 236 out[i] = BenchSkillRow{Bucket: b, Cell: cells[i]} 237 } 238 return out, nil 239} 240 241// BenchReadinessDistribution runs the registered bench_readiness_distribution intent under RLS, reading the 242// ALREADY AUDITED candidate_evaluations band for one requirement and reporting the distribution of bands the 243// frozen engine ALREADY decided at evaluation time. It NEVER recomputes a score; it counts the decisions the 244// engine already made. The complete {green, yellow, red} partition is piped through Suppress with the caller 245// supplied threshold k. Each evaluation carries exactly one band, so the cells sum to the evaluated count and 246// complementary suppression is well defined. The caller renders RenderCount, never the raw Count, for a 247// suppressed cell. 248func BenchReadinessDistribution(ctx context.Context, db *tenant.DB, requirementID string, k int) ([]BenchBandRow, error) { 249 if db == nil { 250 return nil, errors.New("ai: no database") 251 } 252 counts := map[string]int{} 253 for _, b := range readinessBands { 254 counts[b] = 0 255 } 256 err := db.RunInTenantTx(ctx, func(ctx context.Context, tx pgx.Tx) error { 257 if err := GuardReadOnlyQuery(benchReadinessDistributionSQL, ReadAllowlist); err != nil { 258 return err 259 } 260 rows, err := tx.Query(ctx, benchReadinessDistributionSQL, requirementID) 261 if err != nil { 262 return err 263 } 264 defer rows.Close() 265 for rows.Next() { 266 var ( 267 band string 268 headcount int 269 ) 270 if err := rows.Scan(&band, &headcount); err != nil { 271 return err 272 } 273 if _, ok := counts[band]; ok { 274 counts[band] = headcount 275 } 276 } 277 return rows.Err() 278 }) 279 if err != nil { 280 return nil, err 281 } 282 out := make([]BenchBandRow, len(readinessBands)) 283 cells := make([]Cell, len(readinessBands)) 284 for i, b := range readinessBands { 285 cells[i] = Cell{Key: []string{b}, Count: counts[b]} 286 } 287 cells = Suppress(cells, k) 288 for i, b := range readinessBands { 289 out[i] = BenchBandRow{Band: b, Cell: cells[i]} 290 } 291 return out, nil 292} 293 294// BenchGapForRequirement runs the registered bench_gap_for_requirement intent under RLS, partitioning the 295// caller's bench into {meets, does not meet} a requirement by a STRUCTURAL coverage test (holds at least the 296// required clearance level by the rank ladder AND holds at least four fifths of the required skills), then 297// pipes the complete partition through Suppress with the caller supplied threshold k. This is a structural 298// COUNT to SIZE a gap, explicitly NOT a match score: the per candidate score and the ranking remain the 299// frozen engine in internal/modules/matching/engine, which is untouched. Each candidate lands in exactly one 300// bucket, so the cells sum to the bench size and complementary suppression is well defined. The caller 301// renders RenderCount, never the raw Count, for a suppressed cell. 302func BenchGapForRequirement(ctx context.Context, db *tenant.DB, requirementID string, k int) ([]BenchGapRow, error) { 303 if db == nil { 304 return nil, errors.New("ai: no database") 305 } 306 counts := map[string]int{bucketMeets: 0, bucketDoesNotMeet: 0} 307 err := db.RunInTenantTx(ctx, func(ctx context.Context, tx pgx.Tx) error { 308 if err := GuardReadOnlyQuery(benchGapForRequirementSQL, ReadAllowlist); err != nil { 309 return err 310 } 311 rows, err := tx.Query(ctx, benchGapForRequirementSQL, requirementID) 312 if err != nil { 313 return err 314 } 315 defer rows.Close() 316 for rows.Next() { 317 var ( 318 bucket string 319 headcount int 320 ) 321 if err := rows.Scan(&bucket, &headcount); err != nil { 322 return err 323 } 324 if _, ok := counts[bucket]; ok { 325 counts[bucket] = headcount 326 } 327 } 328 return rows.Err() 329 }) 330 if err != nil { 331 return nil, err 332 } 333 order := []string{bucketDoesNotMeet, bucketMeets} 334 out := make([]BenchGapRow, len(order)) 335 cells := make([]Cell, len(order)) 336 for i, b := range order { 337 cells[i] = Cell{Key: []string{b}, Count: counts[b]} 338 } 339 cells = Suppress(cells, k) 340 for i, b := range order { 341 out[i] = BenchGapRow{Bucket: b, Cell: cells[i]} 342 } 343 return out, nil 344} 345