//go:build linux package main import ( "os/exec" "syscall" ) // THE HARD BOUND IS THE ISOLATION, NOT AN RLIMIT, and that correction cost a silent regression to learn. // // The first version set RLIMIT_AS to 1 GiB on the worker, reasoning that a local cap makes a runaway fail on // the process that caused it rather than inviting the kernel to choose a victim. The reasoning still holds. The // implementation did not: RLIMIT_AS bounds VIRTUAL address space, and the Go runtime reserves large virtual // arenas at startup, so every worker died immediately with "fatal error: runtime: out of memory" before reading // a single byte. // // THE CONSEQUENCE WAS WORSE THAN THE BUG I WAS FIXING. Every worker failure falls back to the hand rolled // reader, which is the correct behaviour for an unreadable document and the WRONG outcome for a readable one. // So a fix aimed at one pathological file silently degraded extraction for ALL files: the original PWS went // from 55 chunks to 3 and reported success while doing it. That is the exact failure shape this build keeps // finding, a green result over a quietly worse outcome, and I introduced it. // // WHAT ACTUALLY DELIVERS THE PROPERTY D12 NEEDED: the library runs in a CHILD. A runaway kills the child, the // parent learns the library could not read the document, names it, and the batch continues. No rlimit is // required for that and none is set. // // GOMEMLIMIT IS SET AS A SOFT BOUND, and it is described as soft rather than sold as a cap. It makes the // collector work harder as the heap approaches the value and it does NOT refuse a large allocation, so it // reduces the chance of a balloon without pretending to prevent one. const workerSoftMemLimit = "GOMEMLIMIT=512MiB" func applyWorkerLimits(cmd *exec.Cmd) { // Pdeathsig ties the child's life to the parent's, so an orphaned worker cannot outlive a killed batch and // keep allocating. This is the one kernel level guarantee worth taking here. cmd.SysProcAttr = &syscall.SysProcAttr{Pdeathsig: syscall.SIGKILL} cmd.Env = append(cmd.Environ(), workerSoftMemLimit) } // selfLimit deliberately does NOTHING. It exists so the call site in main reads the same on every platform and // so the reason is recorded here rather than as an absence somebody has to reconstruct. func selfLimit() {}