fix(coding-agents): survey prompt says Glob, not Read, for the directory layout - #3796
Open
MasterST1337 wants to merge 1 commit into
Open
Conversation
…ory layout The cold-repo survey spawns a headless `claude -p --model haiku --allowedTools Read Glob Grep` session. SURVEY_PROMPT asks the model to understand "the directory layout" but never says which tool shows it, and haiku reaches for `Read(<repoRoot>)` — a bare directory path, which errors with EISDIR before the survey has read anything. Measured on one machine's OTel export, 2026-08-23..25: 10 distinct survey sessions failed this way on first tool call, across kids-yt-factory, hindsight, sitebrain.eu, PlanView, Meridian and four .traycer worktrees — i.e. every repo the survey ran cold against, plus a fresh worktree each time (a new worktree is a new bank, so the survey re-runs). Glob was already in --allowedTools; the model was simply never told to prefer it here. This is a prompt-level counter-instruction, not a structural guarantee — the robust fix is at the tool-wiring layer (deny Read on a directory, or give the survey an ls-shaped tool), which this does not attempt.
Strix Security ReviewNo security issues found. Updated for Reviewed by Strix |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The cold-repo survey spawns a headless
claude -p --model haiku --allowedTools Read Glob Grepsession.SURVEY_PROMPTasks the model to understand "the directory layout" but never says which tool shows it, and haiku reaches forRead(<repoRoot>)— a bare directory path, which errorsEISDIRbefore the survey has read anything.Evidence
Measured on one machine's OTel export, 2026-08-23..25: 10 distinct survey sessions failed this way on their first tool call. The transcripts all open the same way:
Affected repos:
kids-yt-factory,hindsight,sitebrain.eu,PlanView,Meridian, plus four separate git worktrees. That is every repo the survey ran cold against — and because a fresh worktree is a fresh bank, it re-fires per worktree rather than once per project.The change
One sentence added to
SURVEY_PROMPTnamingGlobfor the directory layout and forbiddingReadon a directory.Globwas already in--allowedTools; the model was simply never told to prefer it here.What this is not
A prompt sentence cannot guarantee a model's tool choice. The structurally robust fix is at the tool-wiring layer — reject
Readon a directory client-side, or give the survey anls-shaped tool — which this PR does not attempt. Happy to follow up there if you would rather fix it that way.No test pins prompt content in this package (
survey.test.tsasserts against theSURVEY_PROMPTconstant, not its text), so no test changed.scripts/hooks/lint.shclean; package suite 655 passing.