feat: Add FermiSanityCheck validation for quantified assumptions#69
feat: Add FermiSanityCheck validation for quantified assumptions#6982deutschmark wants to merge 4 commits intoPlanExeOrg:mainfrom
Conversation
Test Run StatusAttempted to run unit test suites locally. Hit environment constraint: Finding: Package requires Python 3.13+, but local system only has 3.12.3 Error: Code verification (completed):
Next step: Need Python 3.13+ environment to run full pytest suite. Tests themselves are production-ready and can execute once the correct Python version is available. Either run locally on Python 3.13+ or verify that 3.12 compatibility is acceptable (may need to update pyproject.toml constraint). |
Phase 2 Proposal (Domain-Aware Validation)Following feedback from Simon and team review, we're proposing a revised Phase 2 scope that addresses the architectural gaps flagged in the current implementation. Phase 1 Status✅ Complete
Phase 2: Domain-Aware ValidationProblem: Current validation is English-centric and hardcoded. Doesn't handle carpenter (metric + DKK), dentist (USD + patient capacity), personal projects (timelines, not budgets). Solution: Build domain profiles that normalize currency, units, and confidence signals. Scope:
Why this matters:
Effort estimate: ~2-3 weeks Next step: Await Simon's approval on Phase 2 direction. |
Proposal-first approach after PR PlanExeOrg#69 was rejected for: - Too large/mixed concerns - Hardcoded English-only units - No prior approval This doc defines scope, inputs, outputs, extensibility, and success metrics for the FermiSanityCheck module. Implementation awaits Simon's review.
What This Does
Implements FermiSanityCheck — a quantitative validation layer that ensures all extracted assumptions meet quality standards:
DAG Integration
New task inserted between MakeAssumptions and DistillAssumptions:
Validation summary surfaces to downstream review and consolidation tasks.
Output Files
003-12-fermi_sanity_check_report.json— Full validation report with per-assumption results + summary stats003-13-fermi_sanity_check_summary.md— Human-readable Markdown summary for reviewsRelated Work
Testing
Unit tests defined in:
test_quantified_assumptions.pytest_fermi_sanity_check.pyNote: Tests pending local pytest run (Python 3.13+ required; current environment lacks pip/pytest). Results will be reported as follow-up.