Add incidents and scheduled maintenance#102
Merged
Merged
Conversation
Operator-authored incidents and scheduled maintenance windows, shown on the public status page and managed from the admin area. Maintenance is modelled as an STI subclass of Incident, sharing the update timeline and affected-service links. Status is probe-derived, so an incident is a narrative overlay rather than a status source; an active maintenance suppresses its services from the live alarm and shows an "under maintenance" state, while a real outage still outranks it for the top-level banner. A recurring job advances maintenance windows through their lifecycle.
Rebuild the incident/maintenance index and form with a status-spine visual language: a colored rail that adopts the public-page status colour as impact and kind are chosen, a segmented type toggle, a colour-swatch impact picker, service chips, and a lifecycle stepper and timeline on edit. Keep native datetime-local inputs, made legible in dark mode via color-scheme. Also give admin pages meaningful titles via a content_for-driven layout title.
Add a public permalink for each incident and maintenance with its full update timeline, linked from the status page cards and the RSS feed. An active incident's impact now raises the top-level banner alongside probe-derived status (worst wins; maintenance stays lowest). Refactor to house style: incidents seed their default status and first update via model callbacks so the controller just builds and saves, params.expect for strong params, conditional checks over guard returns, and a page_title_tag helper driving per-page titles. Document the dummy-app test/DB workflow.
A display name isn't unique across probes — an HTTP and a traceroute check can share one — so keying ProbeRollup on (probe_name, period_start) collided them into a single row, silently dropping one probe's uptime and its probe_service for the day. On the public status page that left the affected service showing no history at all. Add probe_type/probe_target to the rollup and dedupe on the full probe identity. Elevate [name, type, probe_target] as probe_key on Probeable and its Probes::Status::Probe value object, reused by Status#build_probes.
- Rename incident_class, auto_advance_status, set_maintenance_impact - Align assignments; collapse has_many to single lines - Rename maintenance_window/outage_duration helpers to _description; drop stale comment - Register month_day_at/_zone and clock_zone time formats; use to_fs over strftime - Extract Probes::Status::Probe.key_for as the single probe-key definition, reused by build_probes - Convert maintenance tests to fixtures + travel_to
Reactive-incident fixtures load resolved (inert), so they stay out of the reactive/active scopes the whole-world overall_status tests read; each test travels to a fixed time and mutates a fixture into the state it needs within its own transaction. Keeps fixtures :all with no cross-test contamination.
service_maintenance_test: maintain/stub_live_status/raise_incident. incident_test: activate.
upright_service_under_maintenance (1/0 per probe_service) is refreshed each minute by MaintenanceAdvanceJob on the primary site. Backs an Alertmanager inhibition that mutes a service's probe alerts during its maintenance window, without pausing probes (metrics/rollups keep flowing).
Emit the maintenance metric from SUPPRESSION_LEAD (3m) before starts_at, not just once in progress, so the Alertmanager inhibition is live before the operator takes the service down — closing the start-of-window race deterministically rather than relying on cron/scrape timing.
record_update now takes the update attributes directly, validates with non-bang saves (no rescue), and returns the update. UpdatesController#create always records and redirects, passing the validation result via flash for the edit view to phrase. Add resolved/unresolved lifecycle scopes and use them across active/upcoming/past, suppressing, and the advance job.
- Default incident impact to "minor" via the enum, dropping the controller-side default - Render flash at the layout level so notices/alerts actually show; fold the update flash onto standard notice/alert - Collapse the incident row link_to to one line
Only status is required on an IncidentUpdate now; the body can be blank. Guard the timeline/detail views so an empty message renders nothing rather than an empty element. The initial update still defaults its body.
Public pages render stable UTC <time> tags that the local-time JS rewrites into the viewer's timezone, so cached HTML localizes per browser. Update timestamps, maintenance windows, and outage durations (now auto-updating relative time) use the gem's helper; admin and the RSS feed stay UTC. Uptime-bar day labels stay UTC-bucketed (they're daily rollups, not clock times).
Requiring local_time alongside the other engine deps registers its Rails engine early, so LocalTimeHelper is a normal autoload constant and the public helper can just include it — no requiring the gem file by path.
There was a problem hiding this comment.
Pull request overview
This PR extends the Upright Rails engine’s status functionality by adding first-class incident + scheduled maintenance tracking (admin CRUD + public display), including lifecycle updates, maintenance-driven alert suppression, and improved probe rollup identity to prevent uptime history collisions.
Changes:
- Add Incident/Maintenance models (STI), lifecycle/update recording, affected-services association, and admin workflows for creating and updating events.
- Update the public status page + RSS feed to include active incidents and maintenance (with per-incident detail pages), and add client-local time rendering via
local_time. - Key 90-day uptime rollups on full probe identity (name/type/target) and add maintenance-related metrics for suppression/visibility.
Tip
If you aren't ready for review, convert to a draft PR.
Click "Convert to draft" or run gh pr ready --undo.
Click "Ready for review" or run gh pr ready to reengage.
Reviewed changes
Copilot reviewed 57 out of 58 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| upright.gemspec | Add local_time dependency. |
| Gemfile.lock | Lock local_time gem version/checksum. |
| lib/upright.rb | Require local_time in engine boot. |
| lib/upright/metrics.rb | Add service_under_maintenance gauge. |
| config/importmap.rb | Pin local-time ESM asset for importmap. |
| config/initializers/date_formats.rb | Add shared Time format keys for event display. |
| config/routes.rb | Add admin incident routes and public incident show route. |
| app/javascript/upright/application.js | Start LocalTime client-side formatting. |
| app/helpers/upright/application_helper.rb | Add page_title_tag helper. |
| app/helpers/upright/public/services_helper.rb | Add maintenance window helpers; integrate LocalTime helpers; expand overall labels. |
| app/controllers/upright/public/services_controller.rb | Load active incidents/maintenances + upcoming maintenances for public page. |
| app/controllers/upright/public/incidents_controller.rb | Public incident detail controller. |
| app/controllers/upright/incidents_controller.rb | Admin CRUD controller for incidents/maintenance. |
| app/controllers/upright/incidents/updates_controller.rb | Admin “post update” endpoint using lifecycle recording. |
| app/models/upright/status.rb | Add worst helper for combining status sources. |
| app/models/upright/service.rb | Integrate maintenance-aware degraded/overall calculations. |
| app/models/upright/incident.rb | Add incident model, enums, associations, validations, and initial update seeding. |
| app/models/upright/maintenance.rb | Add maintenance subclass, validation, auto-advance, and suppression metrics export. |
| app/models/upright/incident_update.rb | Add incident update model. |
| app/models/upright/incident_affected_service.rb | Add affected-service join model and validation. |
| app/models/concerns/upright/incidents/lifecycle.rb | Add scopes and record_update lifecycle helper. |
| app/models/concerns/upright/services/maintenance_status.rb | Add per-service maintenance/incident helpers. |
| app/models/concerns/upright/probeable.rb | Add probe_key helper to standardize probe identity. |
| app/models/upright/rollups/probe_rollup.rb | Rollups now key on probe name/type/target; extract identity. |
| app/models/upright/probes/status/probe.rb | Add probe identity key helpers. |
| app/models/upright/probes/status.rb | Group probe status series using shared identity key. |
| app/jobs/upright/maintenance_advance_job.rb | Job to auto-advance maintenance and export maintenance metrics. |
| db/persistent_migrate/20260702000001_create_upright_incidents.rb | Create incidents, updates, and affected-services tables. |
| db/persistent_migrate/20260707000001_add_probe_identity_to_upright_rollups.rb | Add probe identity columns + new unique index for rollups. |
| test/dummy/db/persistent_schema.rb | Update persistent schema to include incidents and probe identity changes. |
| app/views/layouts/upright/application.html.erb | Use page_title_tag; render flash notices. |
| app/views/layouts/upright/public.html.erb | Use page_title_tag for public layout. |
| app/views/layouts/upright/_header.html.erb | Add admin nav link to Incidents. |
| app/views/upright/public/services/index.html.erb | Render active events + upcoming maintenance blocks. |
| app/views/upright/public/services/_active_events.html.erb | New public block for active incidents/maintenance. |
| app/views/upright/public/services/_upcoming_maintenance.html.erb | New public block for scheduled maintenance list. |
| app/views/upright/public/services/_service.html.erb | Show “Under maintenance” state per service. |
| app/views/upright/public/services/_degraded_list.html.erb | Use LocalTime “ago” formatting for outages. |
| app/views/upright/public/services/index.rss.builder | Add incidents/maintenance items to RSS feed. |
| app/views/upright/public/incidents/show.html.erb | Public incident/maintenance detail page. |
| app/views/upright/incidents/index.html.erb | Admin index for incidents + maintenance. |
| app/views/upright/incidents/new.html.erb | Admin new page wrapper. |
| app/views/upright/incidents/edit.html.erb | Admin edit page with lifecycle + update composer + timeline + delete. |
| app/views/upright/incidents/_form.html.erb | Admin form for incident/maintenance creation/editing. |
| app/views/upright/incidents/_row.html.erb | Admin incident row partial for lists. |
| app/assets/stylesheets/upright/public_status.css | Add public styling for events + incident detail, and maintenance color. |
| app/assets/stylesheets/upright/incidents.css | Add admin UI styling for incidents/maintenance editor and lists. |
| app/assets/stylesheets/upright/flash.css | Add flash styling for notice/alert. |
| test/models/upright/status_test.rb | Add tests for new Status.worst. |
| test/models/upright/service_maintenance_test.rb | Add tests for maintenance suppression behavior and overall status interactions. |
| test/models/upright/rollups/probe_rollup_test.rb | Update and extend tests for probe identity uniqueness in rollups. |
| test/models/upright/maintenance_test.rb | Add tests for maintenance STI behavior, validation, scopes, and auto-advance. |
| test/models/upright/incident_test.rb | Add tests for incident lifecycle, updates, scopes, and affected services. |
| test/fixtures/upright_rollups_probe_rollups.yml | Add probe_type/target fixture fields. |
| test/fixtures/upright_incidents.yml | Add incident/maintenance fixtures. |
| test/fixtures/upright_incident_updates.yml | Add incident update fixtures. |
| test/fixtures/upright_incident_affected_services.yml | Add affected-services fixtures. |
| AGENTS.md | Add troubleshooting steps for stray sqlite files causing test DB creation failures. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+31
to
+34
| transaction do | ||
| update.save | ||
| save | ||
| end |
Comment on lines
+3
to
+9
| add_column :upright_rollups_probe_rollups, :probe_type, :string | ||
| add_column :upright_rollups_probe_rollups, :probe_target, :string | ||
|
|
||
| remove_index :upright_rollups_probe_rollups, column: [ :probe_name, :period_start ], unique: true | ||
| add_index :upright_rollups_probe_rollups, | ||
| [ :probe_name, :probe_type, :probe_target, :period_start ], | ||
| unique: true, name: "idx_probe_rollups_identity_period" |
| <% @page_title = @incident.title %> | ||
|
|
||
| <article class="incident-detail incident-detail--<%= @incident.maintenance? ? "maintenance" : @incident.impact %>"> | ||
| <%= link_to "← Status", "/", class: "incident-detail__back" %> |
Comment on lines
+20
to
+23
| <p class="event__update"> | ||
| <%= update.body %> | ||
| <%= local_time update.created_at, format: :month_day_at_zone, class: "event__time" %> | ||
| </p> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds incidents and scheduled maintenance to the status page.
What's included