CNV-80608: k8s: add alert listing/query and filter primitives by sradco · Pull Request #832 · openshift/monitoring-plugin

sradco · 2026-03-11T09:08:52Z

Alert Management API — Part 2/15: alert listing, query & filter primitives

Summary

PrometheusAlerts implementation: fetches alerts and rule groups from both platform and user-workload Prometheus/Thanos endpoints
Namespace-scoped Thanos tenancy queries for user-workload rules
State-based alert filtering (firing, pending, silenced)
Exact-match label filtering (key=value) with special handling for namespace (used for tenancy selection, not rule filtering)
Prometheus-style label matchers (match[] semantics) supporting full selectors and selector bodies
TLS transport with service CA for in-cluster communication
Unit tests for label matcher compilation and matching semantics
Alerting health types (AlertingHealth, AlertingStackHealth, AlertingRouteHealth, RouteStatus) and health-checking functions (alertingHealth, stackHealth, routeHealth)
Alertmanager integration — fetching alerts from both Prometheus and Alertmanager APIs, parsing Alertmanager v2 responses, state mapping between Alertmanager/Prometheus states

Dependencies

This PR is part of a stacked series. Please review in order.

alert-mgmt-01: k8s foundation + health stub skeleton ✅ merged
→ This PR — alert listing, query & filter primitives
3–15. Pending — relabel config, alerting health, management foundation, CRUD endpoints, classification, bulk update, docs/CI/e2e, single-rule API, orphan GC, effective metric

Made with Cursor

openshift-ci-robot · 2026-03-11T09:08:57Z

@sradco: This pull request references CNV-80608 which is a valid jira issue.

Details

In response to this:

Alert Management API — Part 2/15: alert listing, query & filter primitives

Summary

PrometheusAlerts implementation: fetches alerts and rule groups from both platform and user-workload Prometheus/Thanos endpoints

Namespace-scoped Thanos tenancy queries for user-workload rules

State-based alert filtering (firing, pending, silenced)

Exact-match label filtering (key=value) with special handling for namespace (used for tenancy selection, not rule filtering)

Prometheus-style label matchers (match[] semantics) supporting full selectors and selector bodies

TLS transport with service CA for in-cluster communication

Unit tests for label matcher compilation and matching semantics

Dependencies

This PR is part of a stacked series. Please review in order.

alert-mgmt-01: k8s foundation + health stub skeleton ✅ merged

→ This PR — alert listing, query & filter primitives
3–15. Pending — relabel config, alerting health, management foundation, CRUD endpoints, classification, bulk update, docs/CI/e2e, single-rule API, orphan GC, effective metric

Made with Cursor

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci · 2026-03-11T09:09:32Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: sradco
Once this PR has been reviewed and has the lgtm label, please assign etmurasaki for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

coderabbitai · 2026-03-11T09:10:02Z

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 2e74893d-7fe5-491e-b6a4-efcd57f13522

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

openshift-ci-robot · 2026-03-11T09:13:52Z

@sradco: This pull request references CNV-80608 which is a valid jira issue.

Details

In response to this:

Alert Management API — Part 2/15: alert listing, query & filter primitives

Summary

PrometheusAlerts implementation: fetches alerts and rule groups from both platform and user-workload Prometheus/Thanos endpoints

Namespace-scoped Thanos tenancy queries for user-workload rules

State-based alert filtering (firing, pending, silenced)

Exact-match label filtering (key=value) with special handling for namespace (used for tenancy selection, not rule filtering)

Prometheus-style label matchers (match[] semantics) supporting full selectors and selector bodies

TLS transport with service CA for in-cluster communication

Unit tests for label matcher compilation and matching semantics

Alerting health types (AlertingHealth, AlertingStackHealth, AlertingRouteHealth, RouteStatus) and health-checking functions (alertingHealth, stackHealth, routeHealth)

Alertmanager integration — fetching alerts from both Prometheus and Alertmanager APIs, parsing Alertmanager v2 responses, state mapping between Alertmanager/Prometheus states

Dependencies

This PR is part of a stacked series. Please review in order.

alert-mgmt-01: k8s foundation + health stub skeleton ✅ merged

→ This PR — alert listing, query & filter primitives
3–15. Pending — relabel config, alerting health, management foundation, CRUD endpoints, classification, bulk update, docs/CI/e2e, single-rule API, orphan GC, effective metric

Made with Cursor

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Add PrometheusAlerts implementation with: - Alert fetching from platform and user-workload Prometheus/Thanos - Rule group fetching with namespace-scoped Thanos tenancy queries - State-based filtering (firing, pending, silenced) - Label-based flat filtering with key=value matching - TLS transport with service CA for in-cluster communication Signed-off-by: Shirly Radco <sradco@redhat.com> Signed-off-by: João Vilaça <jvilaca@redhat.com> Signed-off-by: Aviv Litman <alitman@redhat.com> Co-authored-by: AI Assistant <noreply@cursor.com>

sradco · 2026-03-11T09:31:08Z

Hi @jgbernalp , @jan--f , @simonpasquier, Will appreciate your review of this pr. Thank you.

simonpasquier

I've made a few comments but overall the PR is hard to review because the added code is often not used. I'd rather have one API endpoint implemented at a time (even if not witha ll the functionality).

simonpasquier · 2026-03-11T09:46:50Z

pkg/k8s/prometheus_alerts.go

+	return pa.executeRequest(ctx, client, requestURL)
+}
+
+func (pa *prometheusAlerts) resolveThanosTenancyRulesPort(ctx context.Context) int32 {


why do we need to resolve the port? it's a fixed value (and part of the CMO contract somehow).

simonpasquier · 2026-03-11T10:58:59Z

pkg/k8s/rule_label_matchers.go

+		if sel == "" {
+			continue
+		}
+		if !strings.Contains(sel, "{") {


what if we have a "{" in the label value?

simonpasquier · 2026-03-11T10:59:31Z

pkg/k8s/rule_label_matchers.go

+func compileRuleLabelMatchers(req GetRulesRequest) ([]*labels.Matcher, error) {
+	var out []*labels.Matcher
+
+	for k, v := range req.Labels {


I don't understand why we need 2 ways (Labels and Matchers) for matching rules...

simonpasquier · 2026-03-11T11:02:11Z

pkg/k8s/prometheus_alerts.go

+		return groups, nil
+	}
+
+	return filterRuleGroupsByLabelMatchers(groups, matchers), nil


why don't we apply label matchers directly in the requests querying the backends?

simonpasquier · 2026-03-11T11:05:12Z

pkg/k8s/client.go

 		return nil, fmt.Errorf("failed to create PrometheusRule manager: %w", err)
 	}

+	c.prometheusAlerts = newPrometheusAlerts(routeClientset, clientset.CoreV1(), config, c.prometheusRuleManager)


why going through the route?

simonpasquier · 2026-03-11T12:50:05Z

pkg/k8s/prometheus_alerts.go

+	token := BearerTokenFromContext(ctx)
+	if token == "" {
+		var err error
+		token, err = pa.loadBearerToken()


does it mean that we use the service account's token?

simonpasquier · 2026-03-11T12:57:45Z

pkg/k8s/prometheus_alerts.go

+	return filterRuleGroupsByLabelMatchers(groups, matchers), nil
+}
+
+func (pa *prometheusAlerts) alertingHealth(ctx context.Context) AlertingHealth {


I don't understand what the purpose of this method?

simonpasquier · 2026-03-11T13:00:05Z

There are also very few tests compared to the number of lines being added.

sradco · 2026-03-12T14:00:32Z

@simonpasquier let me address your concern of the split first to hopefully make it easier to review

sradco · 2026-03-12T16:34:04Z

Closing this PR in favor of #841

openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Mar 11, 2026

openshift-ci bot requested review from PeterYurkovich and kyoto March 11, 2026 09:09

sradco force-pushed the alert-mgmt-02-k8s-alert-queries branch from 08289c7 to ffe8541 Compare March 11, 2026 09:27

simonpasquier reviewed Mar 11, 2026

View reviewed changes

sradco closed this Mar 12, 2026

sradco deleted the alert-mgmt-02-k8s-alert-queries branch March 12, 2026 18:54

Conversation

sradco commented Mar 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Alert Management API — Part 2/15: alert listing, query & filter primitives

Summary

Dependencies

Uh oh!

openshift-ci-robot commented Mar 11, 2026 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Alert Management API — Part 2/15: alert listing, query & filter primitives

Summary

Dependencies

Uh oh!

openshift-ci bot commented Mar 11, 2026

Uh oh!

coderabbitai bot commented Mar 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

openshift-ci-robot commented Mar 11, 2026 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Alert Management API — Part 2/15: alert listing, query & filter primitives

Summary

Dependencies

Uh oh!

sradco commented Mar 11, 2026

Uh oh!

simonpasquier left a comment

Choose a reason for hiding this comment

Uh oh!

simonpasquier Mar 11, 2026

Choose a reason for hiding this comment

Uh oh!

simonpasquier Mar 11, 2026

Choose a reason for hiding this comment

Uh oh!

simonpasquier Mar 11, 2026

Choose a reason for hiding this comment

Uh oh!

simonpasquier Mar 11, 2026

Choose a reason for hiding this comment

Uh oh!

simonpasquier Mar 11, 2026

Choose a reason for hiding this comment

Uh oh!

simonpasquier Mar 11, 2026

Choose a reason for hiding this comment

Uh oh!

simonpasquier Mar 11, 2026

Choose a reason for hiding this comment

Uh oh!

simonpasquier commented Mar 11, 2026

Uh oh!

sradco commented Mar 12, 2026

Uh oh!

sradco commented Mar 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

sradco commented Mar 11, 2026 •

edited

Loading

openshift-ci-robot commented Mar 11, 2026 •

edited by openshift-ci bot

Loading

coderabbitai bot commented Mar 11, 2026 •

edited

Loading

openshift-ci-robot commented Mar 11, 2026 •

edited by openshift-ci bot

Loading