Skip to content

feat(dashboard): show artifact processing status using existing evidence #2130

Open
0 of 3 issues completed

Description

@Zxf-xufeng

Feature description

Show a lightweight, read-only processing-status section on each applicable artifact page in the Dashboard, scoped to the selected Scope. Reuse existing persistence and available runtime observations through a small inspection layer. Suggested priority: P2.

The deliverable is processing visibility, not task management: no new task IDs, task tables, scheduler, execution-history store, or replacement lifecycle model.

Problem

A successful Source capture confirms durable receipt, not artifact generation. Users need to see whether their data remains pending, whether a known block prevents progress, and what execution observations are available without reconstructing logs. Other paths use explicit operations or Dream rather than Source windows and must retain their existing semantics.

Reuse existing data

Inventory the implementation first. The following baseline was inspected at 2d11abd6:

Existing data What it can support Limit
Source journal positions/head and per-Scope, per-binding consumption cursors Consumed position and input head Cursor progress does not prove new artifact creation or current execution.
Pending watermarks, flush generations, dirty/clean and requested/handled counters Retained processing targets and outstanding work These are coalesced progress/intent records, not per-Source task outcomes or retry history.
Topic Memory processing status and retained blocks Existing idle/pending/blocked semantics, cursor/head/target, safe block and budget details Preserve its existing interpretation; pending does not establish a live worker.
Existing Dream records and queries Known operation status, outcome and result references Reuse existing operation records; do not introduce a parallel task model.
Local Supervisor observations Available active-worker, retry-wait and block counts, recent success/failure observations Often family-level and process-local; not necessarily scoped to the selected Scope, durable, or observable from an API-only process.

Missing data must remain explicitly unknown/unavailable. Do not create new persistence just to fill every display field. Reuse applicable existing APIs before adding an inspection endpoint.

Minimal Dashboard design

Place a compact Processing status section above the existing artifact list on each applicable artifact page. It follows the selected Scope and does not interfere with viewing existing artifacts. Use shared presentation components with family/path-specific mappings.

  • Show supported facts such as outstanding work, consumption progress, or a known persistent block.
  • Show a current Scope's running/retry state only when fresh observations actually identify that Scope and binding. Never infer it from family-wide worker counts.
  • Show a safe recent failure only when its association and observation range are known. A historical failure is not automatically an unresolved failure.
  • A details drawer shows available cursor/head/target positions, safe block reasons and existing budget facts, observation timestamps, and whether the evidence is durable or process-local.
  • Display "pending; execution state unknown" when pending work is established but current execution is not observable. Missing observations must not become zero, healthy, stopped, or completed.
  • Do not display a generated-artifact completion percentage or describe a journal-position difference as the number of artifacts still to generate.

Example, using only facts available for the selected Scope:

Pending inputs. Consumed through position 120; Source head is 135. Current execution state is unavailable. View details.

Family/path mapping

Page / path Minimal display
Memory, Profile, Experience Source-consumption paths Existing progress/intent facts and genuinely Scope-associated observations, where available.
Topic Memory Reuse current processing status and retained block details.
Skill, Prompt and other explicit/Dream paths Reuse applicable existing operation status or link to existing records. Source consumption is not applicable.
Handoff Show an existing corresponding operation outcome when available; do not invent an asynchronous lifecycle for paths without one.

Keep unavailable and inapplicable distinct. Do not force every family into a single running/failed state machine. Family-wide or service-wide observations must be separately labeled, permission-gated, and must not be presented as facts about the selected Scope. A global overview is optional follow-up work, not required here.

Read-only inspection contract

  • Compose existing repository queries and available observations; do not duplicate state or create work on reads.
  • Identify family/path, Scope where applicable, evidence origin, observation freshness, and unsupported fields. Preserve independent dimensions: outstanding work, executor visibility, persistent block, and historical outcome.
  • Keep reads bounded and enforce existing Scope authorization. Service-wide metrics require the existing observation permissions and must not leak other Scopes.
  • Inspection must not invoke models, trigger readiness probes, activate processing, request retries, or write business/processing state. If an existing helper materializes legacy diagnostic records on read, use a read-only projection for this view rather than copying that side effect.
  • Keep existing explicit recovery operations separate. This issue adds no new retry, cancel, resume, or budget-reset controls.

Acceptance criteria

  • Document a compact family/path coverage matrix and map each displayed fact to an existing authoritative record or an explicitly scoped runtime observation.
  • Provide the minimum read-only query adaptation needed by the Dashboard, reusing current APIs where suitable. Follow the generated OpenAPI workflow for contract changes.
  • Add the status section and details drawer to applicable artifact pages; preserve existing lists and navigation. Unsupported paths clearly explain unavailable/not-applicable fields.
  • Keep selected-Scope state separate from family-wide counts. A Supervisor running does not prove this Scope is being processed; no active worker does not by itself mean failure.
  • Preserve distinctions between input consumption, invocation acknowledgement, artifact publication, and known no-change outcomes. Do not infer missing output or per-Source failure attribution.
  • Clearly label local observation windows, external/unknown execution and stale/unavailable observations. After restart, missing local history remains unavailable while existing durable progress/block records remain readable.
  • Verify observable behavior for pending input, caught-up progress without proven output, retained blocks, historical failure followed by success, external-worker uncertainty, Scope switching/access isolation, and passive bounded reads. Use fixtures/local observations; production model calls are unnecessary.
  • Complete this scope without new task/state tables, full attempt history, remote-worker state replication, or mandatory CLI/global-overview work. Future consumers can reuse the inspection contract.

Out of scope

Full task management; new durable per-Scope failure history; complete Source-to-output lineage exploration; causal attribution to individual Sources in a failed batch; new recovery workflows; a global monitoring page; and diagnostic export. These require separately justified follow-ups if existing evidence is insufficient.

Related work

Coordinate with #1984 for broader business diagnostics and #2105 for Desktop consumers. Reuse #1356 receipts where relevant without replacing them. Consume currently available safe reasons and budget/block facts from #2126, #2127 and #2128; do not make this minimal view wait for every enhancement to finish. #2128 owns recovery behavior and #2129 owns diagnostic export. This issue owns presenting existing processing evidence consistently in artifact pages.

Activity

  1. changed the title [-]feat(artifacts): unify processing status, input lineage, and recovery visibility[/-] [+]feat(dashboard): show artifact processing status using existing evidence[/+] on Oct 21, 2026
  2. guamiao commented on Nov 9, 2026

    @guamiao
    Contributor

    Hi maintainers, I'd like to work on this issue. It builds directly on the processing diagnostics I'm already contributing in #2126 (PR #2539). Could you please assign it to me? Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions