Skip to content

refactor(orchestrator): route metrics through observability/metrics Emitter#216

Open
xytan0056 wants to merge 2 commits into
pr2-controller-metricsfrom
pr3-orchestrator-metrics
Open

refactor(orchestrator): route metrics through observability/metrics Emitter#216
xytan0056 wants to merge 2 commits into
pr2-controller-metricsfrom
pr3-orchestrator-metrics

Conversation

@xytan0056

@xytan0056 xytan0056 commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Ports native_orchestrator's GetTargetGraph off raw tally.Scope to *metrics.Emitter with the Begin/Complete helper.

  • Params keeps Scope tally.Scope; the orchestrator self-subscopes to orchestrator and builds its own emitter (still forwards the scope to the graph runner).
  • start + result-tagged finish (replaces calls/success/failure).
  • Per-step duration histograms under get_target_graph: lease_duration, checkout_duration, apply_requests_duration, cache_read_duration, compute_duration, cache_write_duration.

Stacked on #215.

@xytan0056
xytan0056 requested review from a team as code owners July 17, 2026 21:04
@xytan0056
xytan0056 force-pushed the pr3-orchestrator-metrics branch 3 times, most recently from 3b9c7e9 to 97b3839 Compare July 17, 2026 21:47
…tter

Port the controller off tally.Scope to *metrics.Emitter using the
Begin/Complete lifecycle helper per docs/observability/metrics.md.

- Params.Scope tally.Scope -> Emitter *metrics.Emitter (optional, nil->Nop)
- Each RPC bakes the repo tag into a per-request emitter, then Begin/Complete
  emit start + result-tagged finish (replacing calls/success/failure and the
  total_duration timers); no domain metrics struct or memoization
- Sub-operations (get_graph, compare_target_graphs) get their own Begin/Complete
  lifecycle; compare_target_graphs flattens to its own op
- Custom timers -> DurationHistogram on the repo-tagged emitter with explicit
  package-level buckets (controller/metrics.go)
- emitFailureMetric(e, op, err) keeps the failure_type/reason axis
- Add controller/metrics_test.go asserting the start/finish path shape
…mitter

Port native_orchestrator's GetTargetGraph metrics off raw tally.Scope to the
Begin/Complete lifecycle helper.

- Params keeps Scope tally.Scope; the orchestrator self-subscopes to
  "orchestrator" and builds its own emitter (still forwards the scope to the
  graph runner).
- start + result-tagged finish (replaces calls/success/failure); recordFailure
  keeps the failure_type/failure_reason axis as a "failures" counter.
- Buckets are package-level in orchestrator/metrics.go.
@xytan0056
xytan0056 force-pushed the pr2-controller-metrics branch from d62bb17 to 7a8d276 Compare July 17, 2026 23:10
@xytan0056
xytan0056 force-pushed the pr3-orchestrator-metrics branch from 97b3839 to 57d3c09 Compare July 17, 2026 23:10
@xytan0056 xytan0056 changed the title [WIP] refactor(orchestrator): route metrics through observability/metrics Emitter refactor(orchestrator): route metrics through observability/metrics Emitter Jul 17, 2026
@xytan0056
xytan0056 force-pushed the pr2-controller-metrics branch from 7a8d276 to e77a1cb Compare July 21, 2026 06:30
return nil, fmt.Errorf("no repository configuration found for remote %q", remote)
}
leaseStart := time.Now()
ws, err := b.repoManager.Lease(ctx, build)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: emit metrics before Lease error

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant