Skip to content

Add adaptive fetcher escalation (http → browser on failure) #5

@gregoryfoster

Description

@gregoryfoster

Summary

Implement automatic fetcher escalation: when the HTTP fetcher fails or produces empty/unusable content for a watch, escalate to a browser-based fetcher on the next attempt.

Context

Depends on at least one browser-based fetcher (#3 or #4) being available. The fetcher_used field on snapshots already tracks which fetcher produced each result.

Scope

  • Add fetcher_preference field to Watch fetch_config (default: "auto")
  • Implement escalation logic in check_watch: if HTTP fetch produces empty extraction or repeated failures, set preference to browser
  • Record escalation decisions in audit log
  • Allow manual override via API (user sets fetch_config.fetcher_preference = "browser")
  • De-escalation: if browser fetcher succeeds with content similar to what HTTP could produce, suggest reverting

Dependencies

Design doc

docs/plans/2026-03-20-url-change-monitoring-design.md — Phase 7

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions