server/replay: semaphore-gate drift detection (audit #13)

[dennisonbertram]

Intent

Gate POST /v1/runs/replay?detect_drift=true behind a small constant-size semaphore so concurrent drift requests cannot multiply entire worker pools + temp rollout dirs + SQLite state into OOM/FD exhaustion.

Plan reference

• Plan: docs/plans/2026-06-24-harness-reliability-plan.md — slice T14
• Umbrella epic: epic: harnessd long-session reliability hardening (15 TDD slices, audit 2026-06-24) #644
• Audit finding: High priority: make apply_patch compatible with unified diff payloads #13

Source finding

POST /v1/runs/replay?detect_drift=true spawns a full harness.Runner per request, unbounded (internal/server/http_replay.go:325-388). No semaphore gating. Concurrent drift requests multiply entire worker pools + temp rollout dirs + SQLite state until OOM/FD exhaustion.

Scope

• Allowed files (test): internal/server/http_replay_test.go (extend)
• Allowed files (impl): internal/server/http_replay.go
• Dependencies: none
• Complexity: low — Risk: low

Acceptance criteria

Use slice T14 in the plan as detailed acceptance criteria. Summary:

• Failing test written first, compiles and fails on main.
• Pre-fill the semaphore to its size (e.g. 2) with stub acquisitions that never release, then issue a third POST /v1/runs/replay?detect_drift=true. Assert a 503/429 within a few hundred ms; assert no Runner was constructed.
• Impl: add Server.replayDriftSem chan struct{} (size 2, configurable via ServerConfig.ReplayDriftConcurrency). In the drift branch, select { case s.replayDriftSem <- struct{}{}: case <-ctx.Done(): return drain } then defer func(){ <-s.replayDriftSem }().
• go test ./internal/server/... -race green; ./scripts/test-regression.sh green.

Guardrails

• Strict TDD; failing regression test must compile and fail on main first.
• Worktree flow; own branch, merge via verify-and-merge.sh.
• Update .context/harness-reliability/tracker.md on pick-up and merge.
• Deterministic tests only — fake runner constructor.
• Append engineering-log entry on merge.