Summary
Per-milestone verification consumes nearly as many LLM turns as implementation itself, roughly doubling the cost of each milestone.
Evidence (run 0aea2a4f9e95)
Turn counts by node across the whole run:
Implement: 321 turns
VerifyMilestone: 305 turns
Verification ran once per milestone (clean node walk, no fix-loops at graph level) and cost ~95% of what implementation cost. Over 10 milestones that is a large fraction of total spend on re-reading and re-checking work that TestMilestone already gates.
Proposed change
- Consider running the heavy cross-review at phase / dependency boundaries rather than every single milestone, relying on
TestMilestone for per-milestone gating.
- Or scope
VerifyMilestone's reading to the milestone diff rather than re-deriving context each time.
Notes
VerifyMilestone is thorough and catches real scope issues (over-build guard etc.), so this is a tuning/efficiency question, not a request to remove it. Pairs naturally with the test-fidelity issue (spend the verify budget where it adds signal).
Summary
Per-milestone verification consumes nearly as many LLM turns as implementation itself, roughly doubling the cost of each milestone.
Evidence (run
0aea2a4f9e95)Turn counts by node across the whole run:
Implement: 321 turnsVerifyMilestone: 305 turnsVerification ran once per milestone (clean node walk, no fix-loops at graph level) and cost ~95% of what implementation cost. Over 10 milestones that is a large fraction of total spend on re-reading and re-checking work that
TestMilestonealready gates.Proposed change
TestMilestonefor per-milestone gating.VerifyMilestone's reading to the milestone diff rather than re-deriving context each time.Notes
VerifyMilestoneis thorough and catches real scope issues (over-build guard etc.), so this is a tuning/efficiency question, not a request to remove it. Pairs naturally with the test-fidelity issue (spend the verify budget where it adds signal).