// Package benchimport is the CUI safe bulk loader port for a real cleared workforce bench. It turns a // directory of resumes and a structured CSV or JSONL roster into normalized records mapped onto the // existing sourcing.SourcedPerson, so the same dedup, skills, certs, and agency writes that back a // sourced lead also back an imported bench person. It is a platform package: it may read files, call the // pure Go docextract reader and the deterministic extraction.Extractor, and shape sourcing types, but it // never imports the scoring engine and never touches the database. Ingestion (dedup, embed, tag) is the // caller's job through sourcing and the import command. // // Two invariants are encoded in the types here. First, BenchRecord carries ONLY structured workforce // tokens and NO demographic field, so no protected attribute can ride into the matching plane (the AEDT // rule). Second, the resume mapping keeps a person name, an email, and a phone OUT of every embeddable or // scored field: a full name lands only on the candidate name column, never in a job title, a role, or a // headline that is composed into the embed profile (invariant 5). package benchimport import ( "context" "github.com/missionworkforce/missionmatch/internal/platform/sourcing" ) // BenchRecord is a normalized bench entry: structured workforce tokens only, never a demographic field // and never a raw resume span. Every field here is a scoring or routing signal, none is a protected // attribute or a close proxy. The clearance field holds a self reported DB CODE captured for triage only; // it is recorded against bench_import_log, never written to candidates.clearance_level (invariant 3), so // a resume regex can never score as a held clearance. // // The full name is carried so the candidate row gets a name, but the mapping to SourcedPerson places it // ONLY on the name field that lands in candidates.full_name. A name, an email, or a phone is NEVER placed // into JobTitle, Role, or any field that the embed profile composer reads (invariant 5). type BenchRecord struct { FullName string // person name, lands ONLY on candidates.full_name, never embedded Location string // a work location, "City, ST" JobTitle string // a sanitized title token, never a name or an email Skills []string // structured skill tokens Certs []string // structured certification tokens Agencies []string // structured customer or agency tokens Years int // years of experience ClearanceCode string // self reported clearance DB code, triage only, NEVER scored as held ExternalID string // a stable per record id for dedup, empty for resumes without one Email string // a contact email, used only for dedup, NEVER embedded Phone string // a contact phone, used only for dedup, NEVER embedded } // ToSourcedPerson maps a BenchRecord onto the sourcing.SourcedPerson the ingest path already knows how to // dedup and write. It enforces invariant 5 at the boundary: FullName goes only to SourcedPerson.FullName // (which the upsert routes to candidates.full_name), the email and phone go only to the contact slices // (used for dedup, never embedded), and JobTitle is the only title token, so a name or an email can never // reach a headline or a scored field. The clearance code is intentionally NOT mapped onto SourcedPerson, // which has no clearance field, so it can never reach a clearance column; the caller records it against // bench_import_log for triage only. func (r BenchRecord) ToSourcedPerson() sourcing.SourcedPerson { sp := sourcing.SourcedPerson{ ExternalID: r.ExternalID, FullName: r.FullName, JobTitle: r.JobTitle, Location: r.Location, Skills: append([]string(nil), r.Skills...), Years: r.Years, } if r.Email != "" { sp.Emails = []string{r.Email} } if r.Phone != "" { sp.Phones = []string{r.Phone} } return sp } // BatchResult counts the outcome of one import run, for the single append only audit summary the import // command writes (counts and the model only, never PII). type BatchResult struct { Read int // files or rows read from the source Skipped int // malformed or empty entries skipped rather than aborting the batch Mapped int // entries that produced a usable BenchRecord // UnmappedHeaders names the columns nobody could match. It is the single most useful thing to hand back // after an import: it turns "only 40 of my 200 people came in" into "your clearance column is called // Eligibility Status and we did not read it", which is a sentence a recruiter can act on. UnmappedHeaders []string // Inferred is every column whose heading nobody recognised and whose CONTENTS suggested what it is. // Reported whether or not it was applied, so an import never sorts something silently. Inferred []Inference } // FileSource yields bench records from local files. Resumes walks a directory of resume files (PDF, DOCX, // HTML, or text) and yields one record per readable file. Profiles decodes a structured CSV or JSONL // roster and yields one record per well formed row, skipping a malformed row rather than aborting. An // implementation is pure: it reads files and shapes records, it never writes to the database. type FileSource interface { // Resumes walks dir, extracts each resume to text, and yields a BenchRecord per readable file along // with the file path for the compliance log. It returns the running counts in the BatchResult. Resumes(ctx context.Context, dir string, yield func(rec BenchRecord, file string) error) (BatchResult, error) // Profiles decodes the CSV or JSONL roster at path and yields a BenchRecord per well formed row, // skipping a malformed row. It returns the running counts in the BatchResult. Profiles(ctx context.Context, path string, yield func(rec BenchRecord, batch string) error) (BatchResult, error) }