Skip to content

Verification cost ~= implementation cost per milestone #490

Description

@clintecker

Summary

Per-milestone verification consumes nearly as many LLM turns as implementation itself, roughly doubling the cost of each milestone.

Evidence (run 0aea2a4f9e95)

Turn counts by node across the whole run:

  • Implement: 321 turns
  • VerifyMilestone: 305 turns

Verification ran once per milestone (clean node walk, no fix-loops at graph level) and cost ~95% of what implementation cost. Over 10 milestones that is a large fraction of total spend on re-reading and re-checking work that TestMilestone already gates.

Proposed change

  • Consider running the heavy cross-review at phase / dependency boundaries rather than every single milestone, relying on TestMilestone for per-milestone gating.
  • Or scope VerifyMilestone's reading to the milestone diff rather than re-deriving context each time.

Notes

VerifyMilestone is thorough and catches real scope issues (over-build guard etc.), so this is a tuning/efficiency question, not a request to remove it. Pairs naturally with the test-fidelity issue (spend the verify budget where it adds signal).

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2: mediumImportant but not blockingarea/enginePipeline execution engineperformanceSpeed or resource usage improvement

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions