Process hygiene and slots

Release 1 of this section (layer 2 of the plan in docs/specs/2026-11-04-procs.md): whatever a worker starts dies with the worker, and a worktree is not removed from under a running process. Heavy-command slots, the Linux cgroup path and the CPU stall monitor come with release 2 and are not built yet; this section grows with them.

Why. pi stops a bash call’s process group (killProcessTree, SIGKILL to the group of the shell it started). timeout, setsid, detached: true spawns and test runners put their children in groups of their own, so a worker that ends, or is killed, leaves them behind, reparented to launchd or init. On 11/03 four such node test processes ran for six hours at 55 to 60% CPU each in a worktree the parent had already removed.

The owner marker

Every pi that loads the src/hygiene.ts extension (a main, a worker, any session) sets STRAW_BOSS_OWNER=<owner id> in its own process.env. pi’s bash passes that environment to its shell, so every descendant carries it. The id is <subagent id>-<pid> for a pi-herdr-agents worker and pi-<pid> for any other pi. Readers look only for that one key (/proc/<pid>/environ on Linux, the environment option of ps on macOS) and match it in memory: no command line or environment of any process is ever logged, returned or shown. The server’s own spawn environment drops the key, and a standing tower run (a pane the server owns) and Windows get no marker.

Registering a worker

A pi with PI_SUBAGENT_ID set (a pi-herdr-agents worker) sends procs-hello {pid, started, subagentId, sessionFile, pane, terminal, cwd} to the sb server over the control socket at session_start, beats every 30 s, and sends procs-bye when it quits (a reload or a switch of session keeps the process, so it is not a bye). The server finds the pid in its own process table and checks the start time; it finds the worktree root from cwd (the nearest .git going up is a file with gitdir:; a worker in the main checkout has no worktree and gets no cwd rule). Records live in <slot dir>/procs/owners.json (mode 600).

Details

If the server turns a hello down, the worker says so once and what it starts is not reaped; with no server running the hello is tried again at the next beat. The server refuses a hello for a pid that is the Herdr server, a pane’s shell, a shell, a server or the broker, or that has Herdr or a pane’s shell below it. The folder, session file and notify function of the session are read when it starts, never after an await, so a reload in flight cannot raise on a stale context.

What the server keeps. Every 10 s (30 s after a first tick that cost more than a second; the cost is logged once as procs-tick-cost) the server snapshots the descendants of each live owner by ppid, as (pid, start time, pgid). A process that reparents later is still known as that owner’s. Identity is (pid, start time) everywhere, and every action checks it again right before acting.

When an owner ends. On procs-bye, when its pid is gone or reused, or when Herdr reports pane.exited or pane.closed for its pane and the terminal id matches (Herdr reuses pane ids, so an event for a pane whose terminal differs is ignored). An owner that ended while the server was down is reaped when the server starts, from owners.json.

What is reaped

flowchart TD
    e["An owner ends"] --> k["Kill set: its live descendants,<br/>processes with its marker,<br/>orphans in its worktree"]
    k --> t["SIGTERM to each group,<br/>then each pid"]
    t --> s["After 10 s, SIGKILL<br/>what is left"]

The kill set of an ended owner is: the snapshot descendants still alive with the same identity, plus the processes that carry its marker, plus the orphans whose working folder is in its worktree or in a removed checkout last seen as that root (an orphan has parent pid 1, or a systemd --user on Linux), plus what those started below them. The protected set is subtracted: the Herdr server, every live pane’s shell and foreground, an sb server named in a server.json, the intercom broker, every other live registered owner with its tree, and processes that carry another live owner’s marker.

Details

A pane that closes in a linked-worktree workspace with no registered owner (an agent that is not pi) gets only the cwd rule, and not into a worktree where another live session works. A main’s quit reaps nothing: mains start long-lived daemons on purpose.

The kill sequence. SIGTERM to each target group, then to each pid; after 10 s, SIGKILL to what is left with the same identity; a last look is logged. The log has one line per reap, procs-reaped {owner, reason, targets, termed, killed, left, protected}, and procs-left when something stayed alive (its record is then kept for reap). Under a test run every signal goes through src/procs/killguard.ts, which refuses a pid the test did not start or whose cwd is outside SB_TEST_ROOT and fails the process.

cli.mjs procs.

Command	Effect
procs	List the owners the server knows: id, pid, start time, live or ended (with the reason), worktree roots, snapshot size, pane.
procs reap --owner <id>	Run the kill sequence again for an owner that has ended and left something alive. Refused for an owner that is still alive.
procs reap --checkout <path>	Kill what works in a linked worktree (or in a removed checkout a worker was known to work in): the orphans and the leftovers of ended owners. Refused while a live owner works there: end that worker or close its pane first. Exit 1 when a process stayed alive; the JSON on stdout says which.

The removal guard

The extension refuses worktree_remove and a bash command with git worktree remove or herdr worktree remove (also after cd, git -C, &&, ;, a pipe, or inside bash -c "…") while any process has its working folder in the target checkout. The target is resolved from a path, a branch name or a Herdr workspace id with git worktree list --porcelain and Herdr’s workspace list. The refusal lists each holder by pid, program name (comm; on Linux the file name of its executable, because there comm is the main thread’s name, which a program sets for itself and node 24 sets to MainThread; never the command line), CPU time, age and owner state (live, ended, unknown, or no marker), and gives cli.mjs procs reap --checkout '<path>'; after the reap the same call passes.

Details

A target also resolves by git’s unique trailing path components, and herdr worktree remove without --workspace takes the workspace in HERDR_WORKSPACE_ID. A check that cannot run (git or Herdr does not answer) is not a pass. A finished worker’s retained idle pane shell (Herdr says it is its pane’s foreground) in the target’s own workspace does not hold the removal; anything else with its folder there does. The /worktree remove slash command does not pass tool_call: that is a documented limit, and the reaper plugin still refuses its own removals.

The reaper plugin’s hint. herdr-reaper still only refuses to remove a checkout a process works in, and never kills. Its message now ends with node <pi-straw-boss>/dist/server/cli.mjs procs reap --checkout '<path>', since the plugin cannot know where the package is installed. Updating the running plugin is a separate step from the package release: herdr plugin install wayne930242/pi-straw-boss/herdr-reaper again, or a link to the checkout.

The stall watch. A pi that froze is told to its workspace’s tower; the server never signals it and sends it no key. On 11/12 a main’s pi pegged a core for 40 hours, took no keys, and had fallen out of pi-intercom’s list, and nobody noticed until a tower was asked to rescue it by hand. Once a minute the server takes the table of that tick and looks at the pi of every pane where Herdr says one runs (the process Herdr lists in the pane; a pid a session reports counts only when pi-intercom finds that session in the pane now, or could not place it at all; at most eight panes are asked about at a sample, in turn, since pane process-info starts Herdr’s CLI, so after a server start the panes are taken in over a few minutes; a pane whose pi was not found is asked again after 2, 4, 8, 16, then every 32 samples, and is logged once as stall-unresolved). Three signals, each counted in samples a minute apart: cpu, the process averaged cpuPercent (75) of a core or more over cpuMinutes (5) samples in a row, none of them calm (under half of cpuPercent), while its session file was not written: a pegged pi on a loaded machine dips under the threshold for a minute now and then, so one such sample does not start the count again, while a calm sample or a write does, and a burst and a session that still writes are not told (healthy mains with a long history were measured at 73% and 83% of a core over a minute while they wrote their files, so the write, not a lower threshold, is what tells work from a freeze); missing, the process is alive and pi-intercom has not listed it for missingMinutes (3) samples in a row, for a session the server saw listed before or whose pane carries a role and a session file (the broker drops a client whose event loop stopped; a list the server cannot read counts for nothing); silent, the pane says working, the session file was not written for silentMinutes (30) samples in a row, and no tool call is open in it, so a long command, a build, or a subagent is never a stall. A sample that comes more than three minutes late (the computer slept, the server was stopped) starts every count again. The sessions known and the open incidents are kept in <slot dir>/procs/stallwatch.json, so after a server restart a session that does not come back is told once the first three samples have passed, and an incident told before the restart is not told again. An incident is one notice event in the tower’s queue (the parent workspace’s tower for a sub workspace, the head tower where the server stands in for no tower; with neither it is tried again each minute and logged once as stall-untold; with towerRuns.enabled: false the towers are resident sessions, the server tells none, and stall-no-tower is logged once): it names the pane, the session’s id and name (both quoted; the id keeps only letters, digits, and ._:-), the pid and its start time, the cpu minutes, and the signals, and carries the runbook: read the pane twice, and leave a screen that moves; send Esc to one that stands still and read again; only with a handoff file or the boss’s approval stop the process and start a fresh session with the handoff. It is told again only when a further signal fires, when the signals have been gone for three samples (a recovery: do not stop it; a silent session that answered and now has a tool call open is one), and as a reminder after remindMinutes (60), then at twice the wait each time up to 8 hours; a reminder waits while the signals are gone. A pi that ends, or whose pane closes, takes its incident with it without a message. The settings are stallWatch in config.json, read at each sample; 0 minutes turns one signal or the reminders off, and enabled: false the watch. The log has stall-told, stall-untold, stall-no-tower, stall-unresolved, stall-ended and stall-error. Not seen: a tool call that hangs with the process calm (an open call is work); a session that froze with a calm process more than ten minutes before the watch first saw its pane, because the role token, which a session renews itself, has lapsed by then and nothing else says of the process that it registered (a session file’s entries travel with a resume or a fork), which holds for the first sample of a server whose stallwatch.json has no row for that pi (the release that brings the watch, a watch turned on, a lost file); and anything on Windows, which has no process table here. Told wrongly: a session in another pi-intercom scope is never in the server’s list, so with a role on its pane it is told as missing.

Limits. Windows has no hygiene (the extension is inert and says so once). A process a worker started on purpose to outlive it (a dev server, a broker) is reaped unless it is protected; long-lived processes go through MonitorCreate, which pi stops at session end anyway. A pi that loaded Straw Boss before release 1 is not registered either, so what its worker leaves is found only by the cwd rule when the pane closes.

Three limits the replay on macOS showed. (1) The marker is readable only on processes whose program is not an Apple platform binary: node and Homebrew tools show it, /bin/bash, /bin/sleep and /usr/bin/perl do not, because the system hides their environment, and the refusal of the removal guard then says “no owner marker” for them. The snapshot and the cwd rule carry those. The pi itself shows no marker either (it sets the variable after it started; only what it starts inherits it).

(2) The cwd rule does not reach into a worktree where another pane also works (the boss’s shell, a main), because that session may own what runs there; a process that escaped every snapshot by a double fork (adopted by launchd within milliseconds) and cannot be read by marker stays in that case, and cli.mjs procs reap --checkout <path> is the explicit way to stop it. (3) The server learns the linked-worktree workspaces at its tick (every 10 s): a pane without an owner that closes within 10 s of its workspace being created is not seen as a worktree’s.

Timing

A reap sends SIGTERM as soon as it has read the process table, the working folders of the orphans, the markers and the protected set (about 3 s at a load of 25 to 35, more under a higher load), then SIGKILL 10 s later. In the replay every escaped process was gone 12 to 14 s after the worker ended, once 22 s while the machine was at a load of 60 or more. A pane that Herdr closes ends a plain process through the pane event at once (Herdr spells it pane_closed and pane_exited on the wire), and a real pi through the procs-bye its own shutdown sends.