diff --git a/CHANGELOG.md b/CHANGELOG.md index 5ece39d..a193651 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,81 @@ Completed repository changes, newest first. Tracking begins on 2026-10-05; earlier work has not been reconstructed. Planning and archive rules are in [AGENTS.md](AGENTS.md#task-plans-and-documentation). +## 2026-10-11 — Rename the local workspace folder + +- Rename `/Users/hemant/pers/alert_ops` to `/Users/hemant/pers/replytrail`, + preserving files, Git metadata, and all uncommitted work. +- Verify the new working directory, unchanged Git status and origin, and diff + checks. Reopen the workspace at the new path; no application code changed. + +## 2026-10-11 — Align links with the ReplyTrail repository rename + +- Update local origin fetch/push URLs and current documentation links to + `1hemant2/replytrail`; preserve runtime compatibility identifiers and the + existing workspace directory. +- Document repository-derived GHCR naming and the need to verify new image + publication before deployment. No images or cloud resources were migrated. +- Verify GitHub's canonical URL, local remotes, documentation file links, and + whitespace; Maven verify passes with opt-in integration checks skipped. + No old repository URLs remain in the searched source tree. + +## 2026-10-11 — Prepare open-source hosting applications + +- Correct first-email timing and VM-hosting descriptions; add contribution and + security-reporting guidance without changing application behavior. +- Prepare an [unsent hosting request](docs/deployment/hosting-request.md) with + honest adoption and resource limits. GitHub description text is provided for + manual update; sponsorship and private vulnerability reporting are not enabled + by this change. +- Verification: relative documentation file links and diff checks pass; Maven + verify passes outside the sandbox. Opt-in integration/deployed checks remain + unverified; no applications were submitted. + +## 2026-10-11 — Show the feature guide on the public home page + +- Add a responsive, accessible feature guide to the public landing page with + header and hero anchors, light/dark theme styling, and concise explanations of + the current ReplyTrail workflow. +- Keep product boundaries visible by stating that unsupported channels and + deferred test escalations are not included in the current product. +- Verification: UI build and diff checks pass. UI lint passes with two existing + React hook warnings in the flow detail page; no new warnings were introduced. + +## 2026-10-11 — Publish the ReplyTrail feature guide + +- Add a single [feature guide](docs/features.md) with stable feature IDs, + human-readable workflows, canonical lifecycle and security rules, API areas, + implementation status, and deferred boundaries for people and AI agents. +- Link the guide from the README and align scheduling scope text with the local + daily/weekly recurrence implementation. Preserve the distinction between local + implementation and pending PostgreSQL/deployed verification. +- Verification: documentation links and whitespace checked with `git diff --check`; + no backend or frontend behavior changed. + +## 2026-10-11 — Clean up old Oracle deployment and prepare minimal test hosting + +- Remove the approved old OKE cluster, worker pool, two A1 workers, and three + 47 GB boot volumes; retain the compartment and network. Boot-volume data was + permanently deleted. Install OCI CLI and establish local authenticated access. +- Document the Oracle ARM64 test target and dedicated SSH access in the + [deployment guide](docs/deployment/README.md). The 1 OCPU / 4 GB / 50 GB VM + launch failed; the reduced 1 OCPU / 2 GB target also lacks capacity in all + Mumbai fault domains and needs lower Compose limits plus load testing. No new + VM or application deployment was created, and the task remains pending. +- Verify successful OKE cleanup, terminated workers/volumes, capacity reports, + backend packaging with tests skipped, and diff checks. CI/CD remains pending. + +## 2026-10-11 — Separate Kubernetes materials from the first release + +- Move Kubernetes manifests and historical deployment learning guides into the + separate local repository `/Users/hemant/pers/replytrail-kubernetes`; retain + Docker, Compose, and CI here. Preserve the archive alongside the application + so the deployment learning journey can continue later. +- Update the [deployment guide](docs/deployment/README.md) and release checklist + for the agreed EC2/Compose direction. Production setup and deployment remain pending. +- Verification: all 18 archived files match Git history, updated links resolve, + backend packaging with tests skipped and diff checks pass. + ## 2026-10-11 — Implement minimum recurring escalation runs - Implement daily/weekly repeat support on the existing escalation row with diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..27cfe51 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,31 @@ +# Contributing to ReplyTrail + +ReplyTrail welcomes bug reports, documentation improvements, and focused code +contributions. The project is MIT-licensed and is still preparing its first +deployed release; see the [launch checklist](docs/product-launch-readiness.md). + +## Report an issue + +Use [GitHub issues](https://github.com/1hemant2/replytrail/issues) for ordinary bugs +and feature discussions. Include the version or commit, reproduction steps, +expected behavior, and actual behavior. Remove credentials, email action tokens, +personal data, and sensitive logs. Follow [SECURITY.md](SECURITY.md) for security +reports instead of publishing vulnerability details. + +## Make a change + +1. Discuss substantial behavior or architecture changes in an issue first. +2. Fork the repository and create a focused branch from `master`. +3. Follow [AGENTS.md](AGENTS.md), including the current-plan, naming, test, and + changelog rules. Use the [README](README.md#run-locally) to set up locally. +4. Add focused tests for changed behavior and run the commands in + [Development checks](README.md#development-checks). State which integration + or deployed checks were skipped and why. +5. Open a pull request against `master`. Explain the problem, the change, and + verification results. Keep unrelated work out of the pull request and keep + backend, frontend, tests, and documentation in separate commits. + +Preserve PostgreSQL as the durable source of truth, in-memory timers as wake-up +handles, and RabbitMQ as the ready-work queue. Never commit `.env` files, private +keys, provider credentials, or real user data. Contributions remain under the +repository's [MIT license](LICENSE.md). diff --git a/README.md b/README.md index 1a02e03..0e95ea8 100644 --- a/README.md +++ b/README.md @@ -2,6 +2,8 @@ **Keep every response on track.** +The source repository is [1hemant2/replytrail](https://github.com/1hemant2/replytrail). + ReplyTrail is a self-hosted web application for team response coordination. Turn an alert or request into a task, choose an ordered path of people to notify, and track the response from the first email to acknowledgement or resolution. If nobody responds within the configured wait, the run advances to the next person. It helps teams answer three questions: **Who should respond? When should the next person be notified? What happened to this request?** A team member can start a run in the UI, schedule it for later, or let an external system trigger it through a webhook. This repository contains the React UI, Spring Boot API and workers, database migrations, and deployment configuration. @@ -34,14 +36,14 @@ You can also connect another tool through a webhook to send tasks to ReplyTrail. That tool detects the problem and sends the details; ReplyTrail emails the people you choose. They carry out the work in the relevant app or system. -### Example: daily reminders for ten days +### Example: ten reminders spaced one day apart Create a flow with ten steps and put your own team-member email on every step. Set each wait to **1,440 minutes (24 hours)** and leave **resolution timeout** disabled. Start it with the task you want to be reminded about. -The first reminder is scheduled for 24 hours after you start. Each later -reminder is scheduled for 24 hours after the previous email is sent. If you +The first reminder is sent as soon as the run starts. Each later +reminder becomes due 24 hours after the previous email is sent. If you click **Acknowledge** and confirm, the remaining reminders stop. If you do not acknowledge, the ten-step sequence continues and then ends; @@ -61,10 +63,13 @@ emails arrive, so they may not reach you at the same clock time every day. ## Features +The [feature guide](docs/features.md) explains each feature's purpose, actors, +workflow, lifecycle rules, API area, and verification status. + - **Accounts and teams:** Registration requires email verification before login. Users can belong to multiple teams. Owners and admins can invite members; admins can invite users only. Acceptance requires the invited, verified email address. - **Tasks:** Name, description, and source, plus optional priority, category, and HTTP(S) reference URL. Priority is a user-defined label. Manually created tasks default to source `Manual`. - **Response paths:** Ordered steps with a team-member recipient and wait time, plus optional resolution timeouts. Add, edit, delete, and drag steps to reorder them. Started runs retain their saved task context, step details, and timing when a task or path changes. -- **Immediate or scheduled runs:** Start a run from a task and path immediately, or schedule a one-time start using a date, time, and IANA timezone. Reschedule, cancel, or start a scheduled run early before it begins. The first step keeps its configured wait. +- **Immediate, scheduled, and recurring runs:** Start a run from a task and path immediately, schedule a one-time start using a date, time, and IANA timezone, or repeat a schedule daily or weekly. Reschedule, cancel, stop future repeats, or start a scheduled run early before it begins. The first email is sent immediately when the run starts; its configured acknowledgement wait begins after delivery. - **Webhooks:** Owners and admins can create a webhook with a default path, rotate its secret, and enable or disable it. An incoming event creates a task and starts a run. The complete JSON event is stored with links to both records. - **Acknowledgement and resolution:** Recipient-scoped email links open a preview before confirmation. Acknowledgement either completes the run or pauses later steps for a configured resolution window. Unresolved timeouts resume escalation; resolution stops remaining steps. - **Manual escalation:** Escalate now makes the next eligible response step due immediately from the signed-in UI or an eligible email action link. @@ -74,7 +79,7 @@ emails arrive, so they may not reach you at the same clock time every day. ## Current scope - Notifications use **SMTP email**. SMS, phone calls, push notifications, and native Slack/Teams delivery are not implemented. -- Scheduling supports **one-time starts**, not recurring or cron schedules. A finite reminder sequence can use multiple steps assigned to the same person, as in the ten-day example above. +- Scheduling supports one-time starts and the minimum daily/weekly recurrence implementation. Monthly, yearly, custom-interval, and cron schedules are not implemented. A finite reminder sequence can still use multiple steps assigned to the same person, as in the ten-reminder example above. - A response path names individual team-member recipients; rotating on-call calendars are not implemented. - Opening an email link does not accept responsibility automatically; the recipient must confirm the action. - Local checks do not establish production readiness. Database/broker integration checks and deployed response journeys are tracked in the launch checklist. @@ -112,7 +117,7 @@ PostgreSQL stores tasks, paths, events, runs, step states, and due times. The ba | Ready-work delivery | RabbitMQ; database-backed timers determine when work becomes ready | | Short-lived shared state | Redis for workflow intents and webhook rate limiting | | Email | Spring Mail with an SMTP provider | -| Packaging and deployment | Docker Compose for local services; Kubernetes manifests and GitHub Actions for backend build, verification, and image scanning | +| Packaging and deployment | Docker Compose for local services and planned VM hosting; GitHub Actions for backend build, verification, and image scanning | The UI can be deployed as static files separately from the backend. Existing `com.alertops` packages, `ALERTOPS_*` settings, container/resource names, and the `X-AlertOps-Webhook-Secret` header are retained compatibility identifiers from the earlier product name. Use these exact identifiers in configuration and integrations. @@ -250,6 +255,9 @@ Stop Compose services with `docker compose down`. `docker compose down -v` also ## Development checks +Contributions are welcome. Read [CONTRIBUTING.md](CONTRIBUTING.md) for the workflow +and [SECURITY.md](SECURITY.md) before reporting a potential vulnerability. + From the repository root: ```bash @@ -270,7 +278,9 @@ The backend command runs tests and packages the application; the UI build checks - `docker-compose.yml`: local backend and dependency stack. - `.github/workflows/ci.yml`: Maven verification, container build, and Trivy image scan on pull requests and configured branch pushes. - [`docs/product-launch-readiness.md`](docs/product-launch-readiness.md): release checklist and remaining end-to-end checks. -- [`docs/deployment/README.md`](docs/deployment/README.md): staged deployment learning plan. +- [`docs/features.md`](docs/features.md): human- and AI-readable feature guide. +- [`docs/deployment/README.md`](docs/deployment/README.md): first-release hosting and Docker Compose deployment direction. +- [`docs/deployment/hosting-request.md`](docs/deployment/hosting-request.md): an unsent open-source test-hosting request and application preparation guidance. - [`CHANGELOG.md`](CHANGELOG.md): completed repository changes and verification notes. Manual and webhook response flows, one-time scheduling, resolution timeouts, manual actions, and activity history are implemented locally. The release checklist still calls for deployed end-to-end runs, including restart and duplicate-delivery scenarios. Domain ownership, sender verification, and rebrand rollout are tracked in the [product rebranding plan](docs/product-rebranding-plan.md). diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index 0000000..9925261 --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,21 @@ +# Reporting a security issue + +Do not publish vulnerability details, exploit code, credentials, private keys, +or email action tokens in public issues or pull requests. + +Use GitHub's **Report a vulnerability** option in this repository's +[Security tab](https://github.com/1hemant2/replytrail/security) if it is available. +This option depends on the owner enabling private vulnerability reporting. +If it is unavailable, open an issue titled “Request a private security contact” +with no vulnerability details, and wait for the maintainer to provide a private +channel before sharing the report. + +In the private report, include the affected commit, the impact, reproduction +steps using synthetic data, and any suggested mitigation. Redact secrets and +personal data. Do not test against systems or accounts you do not own or have +explicit permission to assess. + +ReplyTrail is preparing its first deployed release. There is no published +security-support window, guaranteed response time, or bug-bounty program. +Report the affected version even if it is older; review and fixes are handled +as maintainer capacity permits. diff --git a/docs/current-plan.md b/docs/current-plan.md index cd18a92..490d9a0 100644 --- a/docs/current-plan.md +++ b/docs/current-plan.md @@ -1,50 +1,33 @@ # Current task plan -## Task: Implement minimum recurring escalation schedules +## Task: Rename the local ReplyTrail workspace folder Started: 2026-10-11 -Status: Complete — local implementation finished; deployment remains pending +Status: Complete ### Goal and scope -Implement the minimum recurring schedule scope from the [feature plan](recurring-escalation-implementation-plan.md) -while preserving one-time escalation behavior. This task includes backend, -frontend, migrations, focused tests, and documentation updates; deployment remains -excluded and is the final release step. +Rename `/Users/hemant/pers/alert_ops` to `/Users/hemant/pers/replytrail` as +explicitly requested. Preserve all files, Git metadata, and uncommitted changes. ### Decision-compliance note -- Preserve the existing one-time schedule and escalation lifecycle. -- Each occurrence has a new escalation ID; overlapping runs are allowed. -- The existing escalation table owns repeatType, nextRepeatAt, and repeatSourceId. - The original row owns repetition; children never own a repeating chain. -- PostgreSQL owns calendar progress and run state; timers only provide wake-ups. -- Reuse each run's scheduledStartAt and existing start/retry/publication path. -- Exclude a new schedule entity/table, duplicate repeat boolean, separate schedule - lifecycle, and custom intervals unless separately agreed. -- Minimum defaults are daily/weekly, scheduled first start, latest missed repeat - only, and stop without editing. Do not introduce repeat-after-completion, - monthly/yearly, custom intervals, or a separate schedule table. - -### Acceptance criteria - -- [x] Add the three recurrence fields and daily/weekly calendar calculation. -- [x] Create independent child runs atomically and recover repeat timers safely. -- [x] Add stop-repeat and the existing-screen scheduling controls. -- [x] Run focused backend/frontend checks and update readiness documentation. - -### Steps and verification - -- [x] Inspect existing scheduling, execution, timer guidance, and launch tracking. -- [x] Confirm the canonical fields, minimum defaults, and excluded improvements. -- [x] Implement backend persistence, occurrence creation, timer recovery, and API. -- [x] Implement the existing UI controls and source-run context. -- [x] Run focused tests, normal builds, and diff checks; record limitations. - -### Verification results - -`EscalationRepeatCalculatorTest` passes. Backend test compilation and packaging, -UI build, UI lint, and `git diff --check` pass; UI lint retains two existing -warnings in `FlowDetailPage.tsx`. The full Mockito-backed service suite could not -start because Byte Buddy cannot self-attach on this local JDK/process environment. -PostgreSQL, broker, SMTP, restart, and deployed checks remain intentionally last. +- Do not overwrite an existing destination or change runtime identifiers. +- Keep the existing Git origin and historical documentation unchanged. +- No commits, rebuilds, cloud changes, or process restarts are required. + +### Acceptance criteria and steps + +- [x] Inspect working tree and confirm the destination does not exist. +- [x] Rename the exact folder without replacing another directory. +- [x] Verify Git status, origin, and file preservation from the new path. +- [x] Record the outcome and any workspace-session limitations. + +### Verification + +The directory rename succeeded. Git status retains the same modified/untracked +files; origin fetch/push still point to `1hemant2/replytrail`. Diff checks pass. +Reopen the workspace at the new path; shells or tools using the old absolute +path need updating. No source behavior changed, so builds were not rerun. +The repository-link task is complete in the changelog. Oracle deployment +remains blocked by capacity; resume via the [deployment guide](deployment/README.md). diff --git a/docs/deployment/README.md b/docs/deployment/README.md index d777208..2581574 100644 --- a/docs/deployment/README.md +++ b/docs/deployment/README.md @@ -1,146 +1,89 @@ -# ReplyTrail cloud deployment learning path - -This guide takes ReplyTrail from a local Spring Boot application to a GitOps-managed -Kubernetes workload. OCI is the implementation target; AWS equivalents are included -so the same design can be explained in interviews. - -The public product name is ReplyTrail. Existing `alertops` image, service, namespace, -cache, and resource names in the commands below remain compatibility identifiers. - -## Target architecture - -```text -Developer -> GitHub -> GitHub Actions -> GHCR - \-> manifests repository -> Argo CD - | -Internet -> load balancer -> Kubernetes Service -> ReplyTrail pods - |-- PostgreSQL - |-- RabbitMQ - `-- Redis - -OCI: VCN + OKE/Compute + OCIR + Object Storage -AWS: VPC + EKS/EC2 + ECR + S3 +# ReplyTrail first-release deployment + +The first release will use Docker Compose on a virtual machine. Initial testing +targets Oracle Cloud Always Free A1 in Mumbai, replacing the earlier EC2-first +direction. Production hosting will be confirmed after the test deployment. The +application image, PostgreSQL, RabbitMQ, and Redis will share a private Docker +network. The React UI and an HTTPS reverse proxy will provide the public entry +point. Email delivery will use an external SMTP provider. + +PostgreSQL remains the source of truth for escalation state and schedules. +RabbitMQ delivers ready notification work. Redis stores temporary workflow +intents and supports shared webhook rate limiting. In-memory timers are rebuilt +from saved database state after application startup. + +## Existing foundations + +The source repository is [1hemant2/replytrail](https://github.com/1hemant2/replytrail). +CI derives its image name from `github.repository`, so new builds target +`ghcr.io/1hemant2/replytrail:`. The repository rename does not migrate +old container images; verify a successful image publication and package access +before configuring deployment to pull the new name. + +- [Dockerfile](../../Dockerfile) builds the backend image. +- [Docker Compose](../../docker-compose.yml) runs the local backend and dependencies. +- [UI Dockerfile](../../ui/Dockerfile) builds and serves the React application. +- [GitHub Actions](../../.github/workflows/ci.yml) verifies, scans, and publishes + backend images on its configured events. +- [Production properties](../../src/main/resources/application-prod.properties) + read runtime configuration from environment variables. + +## Remaining deployment work + +The existing Compose file is a local stack. The production configuration has not +been implemented or deployed yet. Before release: + +- Pull an immutable application image from GHCR instead of building on the server. +- Publish ARM64-compatible images for Oracle A1 and verify dependency image support. +- Serve the UI and API through the public domain with HTTPS. +- Keep database, broker, and Redis ports private. +- Supply production credentials, JWT configuration, email links, and SMTP settings. +- Configure persistent volumes, restart policies, and log rotation for the services. +- Back up PostgreSQL outside the EC2 machine and verify restoration. +- Verify health, email delivery, scheduling, and recovery after a host restart. + +One machine is a single point of failure. Persistent volumes do not replace +external backups. Deployment and end-to-end verification remain the final release +step; readiness is tracked in the [product launch checklist](../product-launch-readiness.md#release-check). + +## Oracle test resource target + +Use the existing `alertops` compartment and public subnet in the Mumbai home +region. The small test target is `replytrail-test`, using `VM.Standard.A1.Flex` +with 1 OCPU, 2 GB RAM, a 50 GB boot volume, and Ubuntu 24.04 ARM64. +This is a test allocation, not a measured throughput guarantee. Three waiting +escalations need little processing, but the OS, Java, PostgreSQL, RabbitMQ, and +Redis all need memory. Reduce the current local Compose limits, which total +2.25 GB before OS overhead, before deploying on this VM. Verify memory use and +the requested 5–10 requests/second with a representative load test. + +The old OKE cluster, worker pool, two workers, and three 47 GB boot volumes were +removed with the owner's approval. Their boot-volume data was permanently deleted. +The compartment and network were retained. + +The initial 1 OCPU / 4 GB launch failed with `Out of host capacity` on 2026-10-11. +Capacity reports for the reduced 1 OCPU / 2 GB target also show no room in any +of the three Mumbai fault domains. No new +VM was created, and ReplyTrail is not deployed. Retry when home-region capacity +becomes available; choosing a paid shape or another region requires a new decision. + +The dedicated SSH key is stored locally at +`/Users/hemant/.ssh/replytrail_oci_test`; never commit or publish its private key. +Use profile `REPLYTRAIL` with `--auth security_token` for local Oracle CLI access. +When the session expires, run: + +```bash +oci session authenticate --region ap-mumbai-1 --profile-name REPLYTRAIL ``` -PostgreSQL, RabbitMQ, and Redis can run inside Kubernetes for learning. For a -production design, prefer managed services where the chosen cloud provides them. -Stateful systems require backups, persistent volumes, disruption planning, and -different scaling rules from the stateless ReplyTrail application. - -## Course roadmap - -| Phase | Build | Main concepts | Evidence of completion | -|---|---|---|---| -| 0 | Deployment readiness | config, secrets, health, tests | clean build and production checklist | -| 1 | Local containers | images, networks, volumes, Compose | healthy local stack after restart | -| 2 | Kubernetes locally | Pod, Deployment, Service, ConfigMap, Secret, PVC | stack runs in kind/minikube | -| 3 | CI with GitHub Actions | workflows, jobs, cache, artifacts, image tags, scanning | commit produces tested image | -| 4 | OCI foundation | compartments, IAM, VCN, subnets, gateways, security lists | private network and cluster exist | -| 5 | Cloud deployment | registry, ingress, DNS, TLS, storage | public HTTPS health endpoint | -| 6 | Argo CD GitOps | desired state, sync, drift, rollback | Git change deploys automatically | -| 7 | Reliability | probes, resources, HPA, PDB, backups, observability | failure and restore drills pass | -| 8 | Advanced delivery | Helm/Kustomize, promotion, canary, policy | controlled multi-environment release | - -Do the phases in order. In particular, do not begin by copying YAML into a cloud -cluster: first prove the same image and configuration locally. - -Completed foundation: [Phase 0 - production readiness](phase-0-readiness.md) - -Completed: [Phase 1 - Docker and Compose foundations](phase-1-docker-foundations.md) - -Completed: [Phase 2 - local Kubernetes](phase-2-kubernetes.md) - -Completed: [Phase 3 - GitHub Actions CI](phase-3-github-actions.md) - -Next lesson: Phase 4 - OCI foundation. Phase 3 uses GitHub Actions with GitHub -Container Registry (GHCR); its GitLab CI terminology mapping is recorded in the -completed Phase 3 notes. - -## Phase 0 audit (current repository) - -The application builds on Java 17 and has a multi-stage, non-root Docker image plus a -Compose stack for PostgreSQL, RabbitMQ, and authenticated Redis. Spring Boot Actuator -is present. The Maven build now runs the first JWT unit tests; broader service and -integration coverage is still required. Phase 1 migrated one-time intent state from -process memory to authenticated Redis with TTL and atomic consumption. - -Before treating the image as production-ready, fix these items: - -- Move the JWT signing secret out of Java source and rotate the exposed value. -- Stop tracking `.env`; assume every committed value has been exposed and rotate it. -- Add actual unit/integration tests. A green build with zero tests is not a quality gate. -- Use a production profile with schema migrations (Flyway/Liquibase), not - `spring.jpa.hibernate.ddl-auto=update`. -- Expose only health endpoints publicly; protect detailed Actuator endpoints. -- Replace development database and RabbitMQ credentials. -- Add readiness, liveness, and startup health groups. -- Decide how PostgreSQL, RabbitMQ, and Redis data are backed up and restored. -- Verify Redis-backed intent sharing and atomic consumption before multiple app pods. (Completed in Phase 1.) -- Pin container versions and add image/dependency scanning in CI. - -## Configuration ownership - -Use this rule throughout the project: - -| Kind | Example | Storage | -|---|---|---| -| Non-secret config | port, log level, queue name | Git, ConfigMap | -| Secret | JWT key, database password | secret manager / Kubernetes Secret source | -| Build output | JAR, test report | GitHub Actions artifact | -| Deployable image | immutable image tagged with commit SHA | OCIR (OCI) / ECR (AWS) | -| Application data | relational records | PostgreSQL with backups | -| Unstructured backup/export | database dump, report | OCI Object Storage / AWS S3 | - -Never commit real credentials, base64-encoded or otherwise. Kubernetes Secret values -are only encoded, not encrypted by default. - -## OCI to AWS interview map - -| Purpose | OCI | AWS | Interview explanation | -|---|---|---|---| -| Isolation/billing boundary | Compartment | Account/OU (approximate) | OCI compartments are IAM and resource organization boundaries | -| Virtual network | VCN | VPC | CIDR, routing, subnets, and network controls | -| Virtual machine | Compute Instance | EC2 | host for self-managed workloads or cluster nodes | -| Managed Kubernetes | OKE | EKS | managed control plane with worker nodes | -| Container registry | OCIR | ECR | stores immutable application images | -| Object storage | Object Storage | S3 | backups and unstructured objects, not a mounted app database | -| Network firewall | NSG/Security List | Security Group/NACL | stateful workload rules vs subnet rules | -| Public entry point | OCI Load Balancer | ALB/NLB | routes traffic to Kubernetes ingress/services | -| Identity | IAM policies/dynamic groups | IAM policies/roles | least-privilege human and workload access | -| Keys/secrets | Vault | KMS/Secrets Manager | encryption keys and runtime secrets | - -The mappings are conceptual, not exact one-to-one product equivalences. - -## Recommended repository layout - -Later phases will grow toward: - -```text -. -|-- Dockerfile -|-- docker-compose.yml -|-- .github/workflows/ -| `-- ci.yml -|-- docs/deployment/ -|-- k8s/ -| |-- base/ -| `-- overlays/dev/ -`-- src/ -``` - -For serious GitOps, use a separate environment repository so CI can update an image -tag without granting the application repository direct cluster access. Argo CD pulls -desired state from GitHub; CI should not run `kubectl apply` against production. - -## Learning journal template - -For every phase, record: +The browser callback requires local port 8181. SeaweedFS is currently stopped at +the owner's request to leave that port available. Keep test data, credentials, +queues, and volumes separate from production. CI/CD automation follows successful +manual deployment and recovery checks. -1. What problem the technology solves. -2. The command/configuration used and why. -3. One failure you caused deliberately. -4. How you diagnosed it using logs, events, metrics, or health endpoints. -5. How the design maps from OCI to AWS. -6. The rollback and data-recovery procedure. +## Kubernetes archive -This turns the deployment into concrete interview stories rather than a list of tools. +The old `k8s/` manifests and deployment learning guides were moved into the separate +local Git repository `/Users/hemant/pers/replytrail-kubernetes` on 2026-10-11, so the +deployment learning journey can continue later. They are also recoverable from +this repository's Git history. Kubernetes is deferred beyond the first release. diff --git a/docs/deployment/hosting-request.md b/docs/deployment/hosting-request.md new file mode 100644 index 0000000..6a084c2 --- /dev/null +++ b/docs/deployment/hosting-request.md @@ -0,0 +1,84 @@ +# Open-source test-hosting request + +Status: Draft only. No application has been submitted and no sponsorship is +confirmed. Review the details below before sending them to a provider. + +## Request draft + +Hello, + +I maintain ReplyTrail, an MIT-licensed, self-hosted application that helps teams +coordinate responses to tasks and alerts. It emails an assigned person, waits +for acknowledgement or resolution, and advances through an ordered response +path when the response window expires. + +Repository: https://github.com/1hemant2/replytrail + +I am requesting one always-running Linux VM for development, integration +testing, and a small demonstration environment. The project owner currently +funds no paid hosting and plans to keep the application open source without +charging users. Production hosting is outside this initial request. + +The VM would run Docker Compose with a Java 17 Spring Boot backend, PostgreSQL, +RabbitMQ, Redis, and the React UI served as static files. PostgreSQL stores +durable workflow state. In-memory timers wake the backend for due work, and +RabbitMQ delivers ready notifications. The backend must remain running even +when no HTTP requests arrive, so scale-to-zero hosting is not suitable. + +The initial resource request is 1 vCPU, 2 GB RAM, and approximately 40–50 GB of +persistent storage. Either x86-64 or ARM64 Linux is acceptable; ARM64 requires +compatible image builds before deployment. Two GB is an experimental target: +we still need to reduce local container limits and measure memory use. +We intend to test three simultaneous escalation runs and roughly 5–10 HTTP +requests per second, but have not benchmarked that workload on these resources. + +We need SSH administration, HTTPS access for the UI/API, and permission to reach +an external SMTP provider using authenticated TLS. Database, broker, and Redis +ports will not be publicly exposed. Testing will use synthetic data and +controlled recipients, not an unrestricted public email service. + +The code is actively maintained and includes tests and CI, but deployed +end-to-end verification is still pending. As checked on 2026-10-11, the public +repository has zero stars and zero forks. We do not claim an established user +community or measured adoption. Please let us know whether an early-stage +project with these needs qualifies and what attribution or usage conditions +would apply. + +Thank you for considering the request. + +## Before submission + +- Add the maintainer's preferred contact details and the requested duration. +- Confirm repository statistics and include genuine usage evidence if available. +- Confirm whether a demonstration environment and external SMTP are allowed. +- Confirm storage, backups, IP addresses, outbound traffic, credit expiry, + and any required payment method or automatic billing. Budget alerts are not + a hard spending cap; do not provision paid resources without owner approval. +- Add sponsor attribution only after approval and agreement on its wording. + +## Provider-specific guidance + +- [Vultr](https://discover.vultr.com/open-source-credits) considers public, + OSI-licensed, recently maintained projects with adoption evidence. Its + advertised Starter tier is $500/year at 100+ stars; smaller projects may + qualify through other evidence. Approval and renewal are not guaranteed. +- [OSU Open Source Lab](https://osuosl.org/services/hosting/policy/) evaluates + community impact and active development. Explain why this workload cannot + use static hosting; do not imply that the project already has a large community. + For [ARM64 testing](https://osuosl.org/services/aarch64/request-hosting/), explain + the actual architecture-validation work rather than claiming a need for ARM64 + solely to obtain hosting. +- [AWS](https://aws.amazon.com/blogs/opensource/aws-cloud-credits-for-open-source-projects-affirming-our-commitment/) + evaluates active maintenance, community engagement, and relevance to its + ecosystem. Credits, expiration, and eligible services require confirmation. + +## Suggested GitHub repository description + +ReplyTrail is an MIT-licensed, self-hosted application for team alerts and email +escalation, with PostgreSQL-backed in-memory scheduling and RabbitMQ delivery. + +The GitHub description is separate from repository files. The owner can paste +this text into the repository's About settings; this draft does not update it. + +Deployment status and remaining setup are maintained in the +[deployment guide](README.md) and [launch checklist](../product-launch-readiness.md). diff --git a/docs/deployment/phase-0-readiness.md b/docs/deployment/phase-0-readiness.md deleted file mode 100644 index 3d414c7..0000000 --- a/docs/deployment/phase-0-readiness.md +++ /dev/null @@ -1,150 +0,0 @@ -# Phase 0: production readiness - -> **ReplyTrail rebrand note:** This readiness record uses older AlertOps wording in -> historical observations. It refers to the current ReplyTrail product; executable -> `alertops` resource and configuration identifiers remain unchanged. - -Phase 0 makes failures visible before deployment automation is introduced. Kubernetes -cannot repair an application that starts with invalid configuration, silently changes -its database schema, or shares a signing key committed to Git. - -## What changed - -### Secrets are injected at runtime - -`JwtUtil` now receives `security.jwt.secret-base64`, which is populated from the -required `JWT_SECRET_BASE64` environment variable. There is no signing-key fallback: -a missing secret stops startup immediately. - -Generate a new 256-bit secret and store it only in your local `.env` or secret manager. - -PowerShell: - -```powershell -$bytes = New-Object byte[] 32 -[Security.Cryptography.RandomNumberGenerator]::Fill($bytes) -[Convert]::ToBase64String($bytes) -``` - -Linux: - -```bash -openssl rand -base64 32 -``` - -Add the result locally as: - -```dotenv -JWT_SECRET_BASE64=generated-value -``` - -Do not reuse the old `JWT_SECRET`. Because `.env` was tracked previously, treat all -values it contained as compromised. Removing the file from the latest commit does not -erase it from Git history; rotate the credentials. History rewriting is a separate, -coordinated operation and is not required for local learning. - -### Profiles have explicit responsibilities - -- `dev`: local PostgreSQL/RabbitMQ defaults and useful development logging. -- `prod`: environment-only connection settings, restrained logging, graceful shutdown, - health probes, Flyway, and Hibernate schema validation. - -No profile is activated from inside the application artifact. The runtime selects it -with `SPRING_PROFILES_ACTIVE`. The container defaults to `prod`; Compose supplies its -dependencies and secret. - -### Flyway owns database evolution - -`V1__create_initial_schema.sql` describes the first schema. On a fresh database, -Flyway applies it before Hibernate starts. Hibernate then validates entity mappings. - -For an existing non-empty development database, `baseline-on-migrate` records a -baseline rather than trying to recreate existing tables. Back up meaningful data before -the first migration-enabled startup. Future schema changes must be new files such as: - -```text -V2__add_escalation_index.sql -V3__add_notification_channel.sql -``` - -Never edit an applied migration: Flyway checksums make that drift visible. - -### Health endpoints are narrow - -Only `/actuator/health/**` is anonymously accessible. The application exposes health -and info, not every Actuator endpoint. Kubernetes will later use: - -```text -/actuator/health/liveness -/actuator/health/readiness -``` - -Liveness answers “should this process be restarted?” Readiness answers “should this -instance receive traffic?” A database outage should usually make the app unready, not -cause an endless restart loop. - -### The build has an actual test - -The JWT test verifies claim round-tripping and rejects an undersized HMAC key. This is -only the first quality gate; service and database integration tests remain future work. - -## Run Phase 0 locally - -1. Copy `.env.example` to `.env` if necessary. -2. Generate and set a new `JWT_SECRET_BASE64`. -3. Use `SPRING_PROFILES_ACTIVE=dev` when running directly from the IDE. -4. Run the quality gate: - -```powershell -.\mvnw.cmd clean test -``` - -5. Complete Phase 0 by running Compose and verifying the database migration, Redis - persistence, and health endpoints against real containers. - -## Why local services still use credentials - -Docker containers share a private network, but they remain independent processes. -Authentication protects against accidental access from another container and keeps -local configuration structurally similar to production. Compose reads the values from -`.env` and passes them at runtime; the image does not contain them. - -Environment variables are acceptable for this local learning stack, but they are not a -complete production secret solution: container metadata and privileged host users can -inspect them. Kubernetes will later receive secrets from a dedicated secret-management -workflow such as OCI Vault rather than from a committed `.env` file. - -## Redis infrastructure boundary - -Phase 0 provisions an authenticated Redis 7 container, an append-only persistence file, -a named data volume, and a health check. This proves the infrastructure contract before -horizontal application scaling is introduced. - -At the Phase 0 boundary, AlertOps did not connect to Redis and the `intent` cache -remained process-local. Phase 1 has since added the Redis client, JSON serialization, -expiry policy, integration tests, and atomic consume operation. This history preserves -the distinction: installing Redis alone does not make application state shared; the -application must explicitly use it. - -## Interview notes - -- Configuration is externalized so one immutable image can move between environments. -- Secrets are injected at runtime and should originate from OCI Vault (AWS Secrets - Manager equivalent), not ConfigMaps or Git. -- Flyway makes database changes ordered, reviewable, repeatable, and checksum-verified. -- Hibernate `validate` detects mapping drift without mutating production. -- Readiness removes unhealthy instances from service; liveness is a restart signal. -- A CI job must report how many tests ran; “build success” with zero tests is misleading. - -## Definition of done - -- [x] JWT signing key removed from source code. -- [x] `.env` removed from Git tracking and covered by `.gitignore`. -- [x] Production configuration fails fast when required values are missing. -- [x] Flyway owns schema creation; production Hibernate mode is `validate`. -- [x] Public Actuator access is limited to health. -- [x] Liveness and readiness endpoints are enabled. -- [x] At least one meaningful unit-test class runs in Maven. -- [x] Authenticated, persistent Redis is declared in the local infrastructure stack. -- [x] Local credentials have been rotated. -- [x] PostgreSQL migration, Redis persistence, and probes have been exercised locally. diff --git a/docs/deployment/phase-1-docker-foundations.md b/docs/deployment/phase-1-docker-foundations.md deleted file mode 100644 index cc8f889..0000000 --- a/docs/deployment/phase-1-docker-foundations.md +++ /dev/null @@ -1,690 +0,0 @@ -# Phase 1: Docker and Compose foundations - -> **ReplyTrail rebrand note:** This Docker and Compose learning record uses older -> AlertOps wording in historical observations. It refers to the current ReplyTrail -> product; executable `alertops` resource and configuration identifiers remain unchanged. - -This note explains what Docker Compose creates for AlertOps, why each resource exists, -and which commands are safe to use. It is intended as a reference to revisit while -learning; memorizing every command is not required. - -## The problem Docker solves - -AlertOps needs a compatible Java runtime, PostgreSQL, RabbitMQ, and eventually Redis. -Installing and configuring each dependency directly on every developer machine leads -to inconsistent environments. Docker packages software and its runtime dependencies -into **images**, then runs those images as isolated **containers**. - -Docker Compose describes several related containers in one `docker-compose.yml` file. -It acts as a small local orchestrator: we declare the desired services and Compose -creates and connects them. - -Compose is excellent for development and learning on one machine. It is not our final -production orchestrator; Kubernetes will fill that role in later phases. - -## The core mental model - -```text -Dockerfile --build--> image --run--> container - |-- environment variables - |-- network connection - |-- published ports - `-- mounted volume - -docker-compose.yml describes all of the above for multiple services. -``` - -An **image** is a read-only application template, such as `postgres:16-alpine`. -A **container** is a running instance of an image. A useful Java comparison is: - -```text -Java class -> Java object -Docker image -> Docker container -``` - -Deleting a container does not delete its image. A new container can be created from -the same image. - -## What `docker compose up -d` does - -From the repository root, run: - -```powershell -docker compose up -d -``` - -Compose then: - -1. Reads `docker-compose.yml` and substitutes values from the local `.env` file. -2. Downloads the PostgreSQL, RabbitMQ, and Redis images if they are missing. -3. Builds the AlertOps image using the `Dockerfile`. -4. Creates a project network and the declared named volumes. -5. Creates the four containers and connects them to the network. -6. Starts PostgreSQL and RabbitMQ and evaluates their health checks. -7. Starts AlertOps after those two dependencies report healthy. -8. Runs everything in the background because `-d` means **detached**. - -The files have different responsibilities: - -| File | Responsibility | -|---|---| -| `Dockerfile` | Instructions for building the AlertOps image | -| `docker-compose.yml` | Services, ports, variables, health checks, networks, and volumes | -| `.env` | Local configuration and real local secrets; never commit it | -| `.env.example` | Safe template documenting the required variable names | - -Compose creates resources from this configuration. It does not invent the application -settings: we declare them in the files above. - -## Services in the AlertOps stack - -| Compose service | Container | Purpose | -|---|---|---| -| `app` | `alertops-app` | Spring Boot AlertOps application | -| `postgres` | `alertops-postgres` | Relational application data | -| `rabbitmq` | `alertops-rabbitmq` | Asynchronous message broker | -| `redis` | `alertops-redis` | Shared one-time workflow intent state | - -Redis was initially infrastructure-only. AlertOps now connects through Spring Data -Redis and Lettuce. This distinction matters: running a dependency does not make state -shared until the application explicitly reads and writes it. - -## Reproducible image pinning (completed) - -Broad image tags such as `postgres:16-alpine` are mutable labels: the same Git commit -could download different patch releases or operating-system packages at different -times. AlertOps now combines a readable exact tag with an immutable content digest: - -```text -repository:exact-version@sha256:exact-image-content -``` - -The selected service versions are PostgreSQL 16.14, RabbitMQ 3.13.7, and Redis 7.4.9. -The Dockerfile also pins both stages of the application build: - -- Maven 3.9.9 with the Temurin 17 JDK compiles and packages the fat JAR. -- Temurin JRE 17.0.19+10 runs the JAR in the final image. - -Tags communicate the version to humans; digests ensure that developers, CI, and -deployment nodes receive identical image content. Pinning does not mean avoiding -upgrades. It makes upgrades deliberate Git changes that can be tested and rolled back. - -The application image does not bake in `SPRING_PROFILES_ACTIVE`. Compose supplies -`prod` when creating this local container, and future Kubernetes manifests will select -the profile at runtime. This lets the same immutable application image move through -development, staging, and production with environment-specific configuration. - -Verification included a clean test run, an image rebuild with `--pull`, all four -containers reporting healthy, Java 17.0.19 inside the app container, the `prod` profile -in Spring startup logs, an `UP` Actuator response, and the durable RabbitMQ queue still -present. - -## Networks, DNS, and `localhost` - -Compose creates a private project network, normally named `alert_ops_default`, and -connects the containers to it: - -```text - Compose private network - +------------------------------------------------+ - | app ---- postgres | - | | \---- rabbitmq | - | `------- redis | - +------------------------------------------------+ - | - | published application port - v - host/laptop -``` - -Docker provides DNS on this network. A container reaches another container using its -**Compose service name**: - -```text -postgres:5432 -rabbitmq:5672 -redis:6379 -``` - -For example, the application's JDBC URL uses `postgres`, not an IP address: - -```text -jdbc:postgresql://postgres:5432/alert_ops -``` - -Container IP addresses may change when containers are recreated, but service names -remain stable. Hard-coding a container IP is therefore incorrect. - -Inside the app container, `localhost` means **the app container itself**. It does not -mean the laptop, PostgreSQL, or another container. This is one of the most common -container networking mistakes. - -## Ports: host side versus container side - -A Compose port mapping has this form: - -```text -HOST_PORT:CONTAINER_PORT -``` - -For AlertOps, `${APP_HOST_PORT:-8096}:8096` means that a request to port 8096 on the -laptop is forwarded to port 8096 inside the app container. `${NAME:-default}` means -"use the value from `.env`, or use the default when it is absent." - -Published ports are needed when the laptop must access a container. Communication -between containers uses the private network, service name, and container port; it does -not need the published host port. - -Examples: - -| Connection | Address | -|---|---| -| Browser on laptop to AlertOps | `http://localhost:8096` | -| App container to PostgreSQL | `postgres:5432` | -| App container to RabbitMQ | `rabbitmq:5672` | -| Browser on laptop to RabbitMQ UI | `http://localhost:15672` | - -## Volumes and data persistence - -A container's writable filesystem is temporary. When a container is deleted and -recreated, data stored only inside it is lost. A **volume** stores data independently -of the container lifecycle. - -This project currently declares: - -```yaml -volumes: - postgres_data: - redis_data: -``` - -They are mounted as follows: - -```text -PostgreSQL container -> postgres_data -> database files survive recreation -RabbitMQ container -> rabbitmq_data -> broker definitions and data survive recreation -Redis container -> redis_data -> append-only data survives recreation -``` - -Docker manages the physical location of named volumes. We normally refer to them by -name instead of editing their files directly. - -RabbitMQ also has a stable `hostname: rabbitmq`. RabbitMQ stores its node database -under an identity derived from its node name/hostname. A named volume alone is not -enough when every recreated container starts with a different generated hostname. - -### RabbitMQ persistence exercise (completed) - -The RabbitMQ service now declares both stable storage and stable node identity: - -```yaml -rabbitmq: - hostname: rabbitmq - volumes: - - rabbitmq_data:/var/lib/rabbitmq - -volumes: - rabbitmq_data: -``` - -The first test used the named volume without a stable hostname. Docker preserved the -volume, but after container recreation RabbitMQ selected a new node data directory and -the durable test queue appeared to be lost. Adding the stable hostname fixed that -node-identity problem. - -The successful test was: - -1. Recreate RabbitMQ with the named volume and stable hostname. -2. Declare the durable queue `phase1.persistence.test`. -3. Stop and remove only the RabbitMQ container. -4. Create RabbitMQ again from the Compose configuration. -5. Wait for its health check to pass. -6. List queues and confirm `phase1.persistence.test` still exists and is durable. - -The container was new, while `alert_ops_rabbitmq_data` remained. This proves that the -queue definition lived in persistent storage rather than only in the removed -container. A durable queue is only part of message durability: publishers must also -mark important messages as persistent, and production RabbitMQ still needs backups, -replication, monitoring, and restore testing. - -### Safe and destructive cleanup - -```powershell -docker compose down -``` - -Stops and removes this project's containers and network. Named volumes remain, so -PostgreSQL and Redis can reuse their data after the next `up`. - -```powershell -docker compose down -v -``` - -Also deletes the project's named volumes. For this stack, that erases the local -PostgreSQL database and Redis data. Use `-v` only when intentionally resetting the -environment. It is correct to skip this command during normal work. - -## Health checks and startup order - -A running process is not necessarily ready to serve requests. PostgreSQL may be -starting, recovering files, or applying initialization before accepting connections. -A health check runs a small command repeatedly to classify the container: - -```text -starting -> healthy - -> unhealthy (after repeated failures) -``` - -The current Compose file checks PostgreSQL, RabbitMQ, and Redis. The app has -`depends_on` conditions for healthy PostgreSQL and RabbitMQ, so Compose waits before -starting it. This improves startup behavior, but it is not a substitute for resilient -application retries. - -An AlertOps container health check will be added during Phase 1. Spring Actuator -already supplies the endpoints that it can test. - -### AlertOps health-check exercise (completed) - -The `app` service now calls the Actuator health endpoint from inside its own container: - -```yaml -healthcheck: - test: ["CMD", "curl", "--fail", "--silent", "http://localhost:8096/actuator/health/liveness"] - interval: 10s - timeout: 5s - retries: 5 - start_period: 30s -``` - -The check uses `localhost` because it is executed inside the AlertOps container. It -uses the container port, not a host port. `curl --fail` returns a non-zero exit code -for an HTTP error, which Docker interprets as a failed check. The original aggregate -health URL was changed to liveness after the dependency-failure drill proved that -remote database and broker outages should not classify the Java process as dead. - -The configuration was applied by recreating the app container and verified with: - -```powershell -docker compose config --quiet -docker compose up -d --no-deps --force-recreate app -docker compose ps app -``` - -The observed status was `Up ... (healthy)`. The JWT filter initially logged every -probe request, so `/actuator/health` and its subpaths are now excluded from JWT token -processing. Spring Security still explicitly controls whether those endpoints are -public; filter exclusion and endpoint authorization are different responsibilities. - -## Graceful shutdown exercise (completed) - -Docker sends its main container process `SIGTERM` when stopping it. Because the -Dockerfile uses the exec-form `ENTRYPOINT`, Java is PID 1 and receives that signal -directly. Spring Boot can then stop accepting new requests, wait for active work, and -close application resources before exiting. - -Production configuration allows Spring up to 30 seconds per shutdown phase. Compose -allows the container slightly longer so it does not send a forced `SIGKILL` while -Spring is still cleaning up: - -```yaml -app: - stop_grace_period: 40s -``` - -The applied container configuration reported a 40-second stop timeout. Its logs -showed graceful Tomcat shutdown beginning and completing, followed by closure of the -JPA entity manager and Hikari database pool. The container then started again and -returned to `healthy`. - -`docker compose start app` and `docker compose up -d app` are not interchangeable: - -- `start` starts an existing stopped container with its existing configuration. It - does not build an image, create a missing container, or apply Compose-file changes. -- `up` reconciles the declared Compose state. It creates missing resources and may - recreate a container when its image or configuration has changed. It also processes - declared dependencies; `-d` leaves the services running in the background. - -Use `start` after a simple manual stop when nothing changed. Use `up -d` after editing -Compose, changing an image/build, or when the resources might not exist. - -## Container resource limits (completed) - -Containers are isolated processes, but without resource controls any one service can -consume most of the Docker host. CPU and memory behave differently: - -```text -CPU demand above limit -> execution is throttled; work becomes slower -Memory usage above limit -> process may be OOM-killed; container can exit with 137 -``` - -The local learning limits were selected after measuring the idle stack: - -| Service | CPU limit | Memory reservation | Memory limit | -|---|---:|---:|---:| -| AlertOps | 1 CPU | 512 MiB | 1 GiB | -| PostgreSQL | 1 CPU | 256 MiB | 512 MiB | -| RabbitMQ | 1 CPU | 256 MiB | 512 MiB | -| Redis | 0.5 CPU | 64 MiB | 256 MiB | - -Compose expresses these settings as: - -```yaml -app: - cpus: 1.0 - mem_reservation: 512m - mem_limit: 1g -``` - -The limit is a hard control. A Compose memory reservation is a soft target used under -host memory pressure; it is not the same scheduling guarantee as a Kubernetes request. -These values are learning defaults, not production capacity recommendations. -Production values require load tests, peak measurements, and safety headroom. - -### CPU cores, millicpu, and parallel work - -One CPU represents the processing capacity of one logical CPU over time. An eight-core -or eight-logical-CPU machine can execute roughly eight CPU instruction streams at one -instant, while the operating-system scheduler lets many more processes and threads -make progress by rapidly sharing those CPUs. Work waiting for a database, disk, or -network does not continuously occupy a CPU. - -Compose uses decimal CPU units while Kubernetes commonly uses millicpu: - -| CPU capacity | Compose | Kubernetes | -|---:|---:|---:| -| Quarter CPU | `0.25` | `250m` | -| Half CPU | `0.5` | `500m` | -| One CPU | `1.0` | `1000m` or `1` | -| Two CPUs | `2.0` | `2000m` or `2` | - -Here `m` means one-thousandth of a CPU, not megabytes, milliseconds, or minutes. -Docker does not normally dedicate a particular physical core; all container threads -share an aggregate processing quota unless CPU affinity is configured separately. - -### CPU quota and scheduling periods - -A simplified Linux CPU limit uses a quota and a period. The verified one-CPU AlertOps -configuration reported: - -```text -cpu.cfs_quota_us = 100000 -cpu.cfs_period_us = 100000 -``` - -This permits up to 100,000 microseconds of aggregate CPU time per 100,000-microsecond -quota period: one CPU. A `0.5` CPU limit is conceptually about 50,000 microseconds per -100,000-microsecond period. - -The quota applies to the whole container, not independently to each Java task. The -scheduler can distribute it across many shorter executions and multiple threads. Two -threads running simultaneously for 25 ms can together consume 50 ms of CPU time. If -the container exhausts its quota while work remains runnable, Linux throttles it until -more quota becomes available. It is not a rule that every task runs exactly 50 ms and -then waits. - -### Reading CPU in Grafana - -A CPU panel may display cores, percentage of one core, percentage of the Kubernetes -request, percentage of the limit, or percentage of the node. Always identify the -denominator. For example, usage of `500m` is 50% of a one-CPU limit but 200% of a -`250m` request. - -High CPU usage is not automatically a defect. Correlate it with: - -- request or message rate; -- response latency and errors; -- CPU throttled periods/time; -- JVM garbage-collection time and heap pressure; -- deployments and configuration changes; -- whether one replica or every replica is affected. - -Rising traffic with rising CPU and stable latency can be healthy. Unchanged traffic -followed by sustained CPU at the limit, throttling, and increased latency can indicate -an expensive code path, retry storm, excessive logging, garbage collection, or a -regression. Investigate before merely increasing the limit. - -### Resource-limit verification - -Before limits, every container saw approximately 3.6 GiB of Docker Desktop memory and -the JVM estimated a 924 MiB maximum heap. After recreation, Docker reported the exact -configured byte and CPU quotas. AlertOps saw a 1 GiB cgroup memory limit, and -container-aware Java 17 recalculated its estimated maximum heap to 247.5 MiB. - -All four health checks passed, Actuator returned `UP`, and RabbitMQ's durable queues -remained after recreation. Current idle usage fits the limits, but that does not prove -production capacity; load testing remains necessary. - -Depending on the host, cgroup files differ. Cgroup v2 commonly exposes `memory.max` -and `cpu.max`; this Docker Desktop environment uses cgroup v1 files such as -`memory.limit_in_bytes`, `cpu.cfs_quota_us`, and `cpu.cfs_period_us`. Docker inspection -and `docker stats` provide a more portable high-level view. - -## Dependency failure and recovery drill (completed) - -`depends_on` controls Compose startup ordering; it is not continuous supervision. -Stopping PostgreSQL or RabbitMQ after startup left the AlertOps Java process running. -This demonstrated why process state, liveness, readiness, and dependency health are -different signals. - -### PostgreSQL outage - -With PostgreSQL stopped, the aggregate `/actuator/health` endpoint became unavailable -or `DOWN`, while `/actuator/health/liveness` remained `UP`. Hikari attempted to obtain -a connection, timed out after 30 seconds, and logged the database failure. Starting -the existing PostgreSQL container restored the connection and AlertOps recovered -without container recreation. - -The test exposed an operational problem: the Docker health check called the aggregate -endpoint every ten seconds. During the outage, overlapping checks repeatedly triggered -slow database health queries. The Compose app probe now calls the lightweight -`/actuator/health/liveness` endpoint instead. - -### RabbitMQ outage - -RabbitMQ behaved differently from JDBC. Its connection closed, the Spring AMQP -listener reported connection refusal, and the listener container repeatedly attempted -consumer recovery. After RabbitMQ restarted and became healthy, AlertOps created a new -AMQP connection automatically. The durable test and application queues remained. - -The aggregate health request timed out during the broker outage while liveness stayed -`UP`. Docker does not automatically restart a container merely because it is marked -unhealthy; `restart: unless-stopped` applies when its main process exits. A dependency -outage should not cause a liveness failure and restart loop. - -### Redis outage - -The Redis drill was completed after intent-cache integration. With Redis stopped, -aggregate health returned HTTP 503 `DOWN`, while liveness remained HTTP 200 `UP` and -the application container stayed healthy. After Redis restarted, aggregate health -returned to `UP` without recreating AlertOps. Intent operations fail closed rather -than silently treating an unavailable store as a missing one-time intent. - -Readiness policy will be made explicit in the Kubernetes phase. Whether an external -dependency should make a pod unready is a service-design decision: it can fail traffic -quickly, but it can also remove every replica during a shared dependency outage. -Liveness should normally answer whether this process is stuck and needs restarting, -not whether a remote dependency is available. - -## Shared Redis intent state (completed) - -The original `ConcurrentMapCacheManager` stored intents inside one JVM. An intent -written by replica A could not be read by replica B, entries had no TTL, and the -declared delete method was never called. That made horizontal scaling unsafe. - -AlertOps now uses Spring Data Redis with Lettuce and a direct `StringRedisTemplate`: - -```text -AlertOps replica A ----\ - -> Redis key: alertops:intent: -AlertOps replica B ----/ value: {"type":"JOIN_TEAM"} - TTL: 10 minutes -``` - -Connection host, port, password, timeouts, and intent TTL are runtime configuration. -Compose waits for authenticated Redis health before starting AlertOps. Development -defaults to localhost, while production requires injected Redis connection values. - -Intent creation writes JSON with a configurable ten-minute default expiry. Successful -login calls Redis `GETDEL`, which reads and deletes the key atomically. A separate -`GET` followed by `DELETE` would allow two replicas to read the same one-time value -between operations; one atomic server operation permits only one consumer. - -Verification covered: - -- four unit tests for key namespacing, JSON, TTL, reads, and atomic consumption; -- a real Redis integration test using two independent Lettuce connection factories; -- replica A writing an intent and replica B reading and consuming it; -- the first replica observing that the consumed intent no longer exists; -- a Redis key receiving a 600-second TTL and disappearing after `GETDEL`; -- aggregate Actuator health including Redis; -- Redis outage returning aggregate `DOWN` while liveness stayed `UP`; -- automatic health recovery after Redis restarted. - -The integration test is enabled with `REDIS_INTEGRATION_TEST=true` and Redis connection -environment variables. It remains infrastructure-independent during ordinary unit -test runs and can be enabled against a GitHub Actions Redis service container in Phase 3. - -## Full-stack persistence drill (completed) - -Individual service restarts do not fully prove Compose lifecycle persistence. The -complete stack was removed with `docker compose down` (without `-v`) and recreated -with `docker compose up -d`. `down` removed all four containers and the project -network, while these named volumes remained: - -```text -alert_ops_postgres_data -alert_ops_rabbitmq_data -alert_ops_redis_data -``` - -Before teardown, the drill recorded independent storage markers: - -- PostgreSQL cluster system identifier `7661185378394243107`; -- durable RabbitMQ queue `phase1.persistence.test`; -- Redis key `phase1:persistence` with value `survived`. - -After recreation, PostgreSQL returned the same system identifier, RabbitMQ listed the -same durable queues, and Redis returned the marker value. Flyway reported that schema -`public` was already up to date instead of initializing a new database. AlertOps and -all infrastructure health checks returned healthy. - -The PostgreSQL system identifier is a read-only fingerprint generated when a database -cluster is initialized. Matching values before and after recreation provide evidence -that PostgreSQL reused the same data directory. On PowerShell, avoiding nested -`sh -c` quoting made the check more reliable: - -```powershell -$pgUser = (docker compose exec -T postgres printenv POSTGRES_USER).Trim() -$pgDb = (docker compose exec -T postgres printenv POSTGRES_DB).Trim() -docker compose exec -T postgres psql -U $pgUser -d $pgDb -tA -c "SELECT system_identifier FROM pg_control_system();" -``` - -Named-volume persistence is not a backup. A volume can still be deleted with -`docker compose down -v`, corrupted, lost with its Docker host, or damaged by an -application error. Production requires independent backups stored outside the host, -retention policy, encryption, and tested restoration procedures. - -## Command reference - -Run these commands from the repository root: - -| Command | Meaning and benefit | -|---|---| -| `docker compose up -d` | Create/start the stack in the background | -| `docker compose up -d --build` | Rebuild the app image, then create/start the stack | -| `docker compose config` | Show the resolved Compose configuration and catch configuration errors | -| `docker compose config --services` | List the service names Compose found | -| `docker compose ps` | Show containers, state, health, and published ports | -| `docker compose logs -f app` | Follow application logs; `Ctrl+C` only stops following | -| `docker compose restart app` | Restart only AlertOps, leaving dependencies and volumes intact | -| `docker compose stop` | Stop containers without removing them | -| `docker compose down` | Remove containers and project network, preserving named volumes | -| `docker compose down -v` | Remove containers, network, and volumes; deletes local persisted data | -| `docker volume ls` | List Docker-managed volumes | -| `docker network ls` | List Docker networks | -| `docker stats --no-stream` | Snapshot container CPU, memory, network, process, and I/O usage | -| `docker network inspect alert_ops_default` | Show attached containers and low-level network details | -| `docker compose exec app ` | Run a command inside the running app container | - -Examples of Docker DNS checks: - -```powershell -docker compose exec app getent hosts postgres -docker compose exec app getent hosts rabbitmq -docker compose exec app getent hosts redis -``` - -These commands prove that service names resolve inside the Compose network. The long -IDs, IP addresses, endpoint IDs, and labels printed by `inspect` are diagnostic detail. -At this stage, understand what resource the output describes; do not memorize every -field. - -## How to read `docker compose ps` - -Focus on these columns: - -- **SERVICE/NAME**: which declared service and container the row represents. -- **STATUS**: running time and health, such as `Up ... (healthy)`. -- **PORTS**: host-to-container forwarding rules. - -Useful meanings: - -| Output | Meaning | -|---|---| -| `Created` | Container exists but has not started | -| `Up` | Main process is running | -| `Up (starting)` | Process is running; health check has not passed yet | -| `Up (healthy)` | Health check is passing | -| `Up (unhealthy)` | Process runs, but the health check repeatedly fails | -| `Exited (code)` | Main process stopped; inspect logs for the reason | - -## Recommended learning exercises - -1. Run `docker compose ps` and identify every service, health state, and host port. -2. Follow app logs, press `Ctrl+C`, and confirm the app is still running. -3. Restart only the app and confirm PostgreSQL and Redis were not restarted. -4. Inspect the project network and find the four attached containers. -5. List volumes and identify the PostgreSQL and Redis volumes. -6. Run `docker compose down` without `-v`, start again, and verify database data remains. -7. Later, deliberately stop one dependency and observe logs, health, and recovery. - -For each exercise, record: what you expected, what happened, which command provided -evidence, and how you would explain the result in an interview. - -## Compose concepts mapped to Kubernetes - -The names differ, but these foundations carry forward: - -| Docker Compose concept | Later Kubernetes concept | -|---|---| -| Service container | Pod managed by a Deployment/StatefulSet | -| Compose service name/DNS | Kubernetes Service and cluster DNS | -| Environment configuration | ConfigMap and Secret | -| Health check | Liveness, readiness, and startup probes | -| Named volume | PersistentVolumeClaim/PersistentVolume | -| Published port | Service, Ingress, and cloud load balancer | -| Compose desired state | Kubernetes manifests reconciled by controllers | - -The key idea is declarative operation: describe the desired state in configuration, -then let an orchestrator create, connect, monitor, and recreate resources. - -## Phase 1 definition of done - -Phase 1 was audited complete on 2026-07-19: - -- [x] Multi-stage, non-root AlertOps image builds from pinned base-image digests. -- [x] PostgreSQL, RabbitMQ, and Redis images use readable versions plus digests. -- [x] Compose networking and service-name DNS are verified. -- [x] PostgreSQL, RabbitMQ, and Redis use named persistent volumes. -- [x] RabbitMQ has stable node identity and survives container recreation. -- [x] AlertOps has a lightweight liveness health check. -- [x] Graceful shutdown completes before Docker's stop timeout. -- [x] CPU, memory reservation, and memory limits are applied and measured. -- [x] PostgreSQL, RabbitMQ, and Redis outage/recovery behaviour is recorded. -- [x] Full-stack teardown preserves all named-volume data without `-v`. -- [x] One-time intent state is shared through Redis with TTL and atomic consumption. -- [x] Unit tests pass and the opt-in two-client Redis integration test passes. -- [x] The full Compose stack returns to healthy after Docker Engine restart. - -Phase 2 reuses these concepts as Kubernetes Deployments, Services, probes, -ConfigMaps, Secrets, resource requests/limits, and PersistentVolumeClaims. diff --git a/docs/deployment/phase-2-kubernetes.md b/docs/deployment/phase-2-kubernetes.md deleted file mode 100644 index 9afd009..0000000 --- a/docs/deployment/phase-2-kubernetes.md +++ /dev/null @@ -1,1192 +0,0 @@ -# Phase 2: Local Kubernetes - -> **ReplyTrail rebrand note:** This Kubernetes learning record uses older AlertOps -> wording in historical observations. It refers to the current ReplyTrail product; -> executable `alertops` resource and configuration identifiers remain unchanged. - -## Objective - -Run the verified AlertOps container stack on a local Kubernetes cluster and understand -how Kubernetes continuously reconciles declared state. The local cluster uses kind -(Kubernetes IN Docker), so Kubernetes nodes themselves run as Docker containers. - -```text -Windows host -`-- Docker Engine - `-- kind node container - `-- Kubernetes - |-- AlertOps pods - |-- PostgreSQL pod + persistent volume - |-- RabbitMQ pod + persistent volume - `-- Redis pod + persistent volume -``` - -This phase runs stateful dependencies inside Kubernetes for learning. A production -cloud design should evaluate managed PostgreSQL, RabbitMQ, and Redis services rather -than assuming every database or broker belongs inside the application cluster. - -Every task in this phase will record the problem being solved, explain each command, -show how to read its output, cause one controlled failure, and map the local concept -to OCI OKE and AWS EKS. - -## What Kubernetes is - -Kubernetes is a container orchestrator. We declare the desired state of containerized -applications through its API, and control-plane components continuously work to make -the cluster's actual state match it. - -For example, desired state can say: - -```text -Run three AlertOps replicas. -Keep only ready replicas behind one stable network address. -Replace failed replicas. -Give every replica defined CPU and memory resources. -Roll gradually from image version A to version B. -``` - -Kubernetes is not a replacement for the Dockerfile or container image. AlertOps still -uses the image built in Phase 1. Kubernetes decides where and how instances of that -image run and continuously reconciles their lifecycle. - -## Direct Docker deployment on EC2 or OCI Compute - -A valid small production architecture can use one virtual machine: - -```text -Internet - | -load balancer / reverse proxy - | -EC2 or OCI Compute instance - |-- AlertOps container - |-- systemd or Docker Compose - `-- logging/monitoring agent - -Managed PostgreSQL, RabbitMQ and Redis are reached over the network. -``` - -The team creates the VM and firewall rules, installs Docker, authenticates to the -registry, pulls the pinned image, injects configuration, starts the container, and -operates TLS, monitoring, backups and deployment automation. This is not wrong. It can -be the best design for one or a few services, a small team, predictable traffic, -modest availability requirements, or strict cost constraints. - -The work grows when there are multiple hosts and replicas: - -- choose a machine with enough CPU and memory; -- keep the desired replica count after process or VM failure; -- discover changing container addresses; -- route traffic only to ready replicas; -- deploy gradually and roll back safely; -- scale services independently; -- drain work before termination; -- apply configuration, access policy and monitoring consistently. - -Scripts can automate these behaviours, but the organization eventually owns a custom -orchestrator made from scripts. Kubernetes supplies a standard API, scheduler and -controllers for these recurring problems. - -## EC2 and Kubernetes are layers, not alternatives - -Kubernetes still needs computers. In AWS, worker nodes are commonly EC2 instances; in -OCI, they are Compute instances in OKE node pools. - -```text -AWS account / OCI tenancy - | -VPC / VCN - | -Managed Kubernetes control plane: EKS / OKE - | -Worker nodes: EC2 / OCI Compute - | -Pods containing AlertOps containers -``` - -The practical comparison is: - -```text -Manage containers directly on VMs - versus -Use Kubernetes to manage containers across a pool of VMs -``` - -EKS and OKE manage the control plane. The team still owns workload manifests, node -capacity, IAM, network policy, observability, upgrades, security, cost and application -reliability. “Managed Kubernetes” does not mean “no operations.” AWS documents the -same separation between the EKS control plane and EC2 worker nodes in its -[Kubernetes concepts](https://docs.aws.amazon.com/eks/latest/userguide/kubernetes-concepts.html). - -## Desired state and reconciliation - -Docker Compose creates a declared stack on one Docker Engine. Kubernetes adds a -continuous control loop: - -```text -Desired state in Kubernetes API - | -controller compares desired and actual state - | - +-- equal: continue watching - `-- different: create, replace, update or remove resources -``` - -Example: - -```text -Desired replicas: 3 -Actual replicas: 2 - | -Deployment controller creates a replacement Pod - | -Actual replicas: 3 -``` - -This is self-healing of declared workload state. Kubernetes can restart containers, -replace Pods and remove failed Pods from Service endpoints. It cannot fix incorrect -business logic, restore data without a backup design, or repair an unavailable remote -database. See the official -[Kubernetes self-healing documentation](https://kubernetes.io/docs/concepts/architecture/self-healing/). - -## Cluster architecture - -A cluster contains a control plane and worker nodes: - -```text -Developer / Argo CD - | - | Kubernetes API through kubectl or GitOps - v -+----------------------- Control plane -----------------------+ -| API server | scheduler | controllers | etcd desired state | -+------------------------------+-------------------------------+ - | - schedules and supervises workloads - | - +--------------------+--------------------+ - | | -+---------v---------+ +---------v---------+ -| Worker node 1 | | Worker node 2 | -| kubelet | | kubelet | -| container runtime | | container runtime | -| AlertOps Pod A | | AlertOps Pod B | -+-------------------+ +-------------------+ -``` - -| Component | Responsibility | -|---|---| -| API server | Front door for resource operations and validation | -| etcd | Durable control-plane store for cluster desired state | -| scheduler | Selects a suitable node for an unscheduled Pod | -| controllers | Reconcile Deployments, replicas, nodes and other resources | -| kubelet | Node agent ensuring assigned Pods and containers run | -| container runtime | Pulls images and runs containers on a node | - -Application requests do not normally pass through the Kubernetes API server. It is the -management interface; Services and ingress/load balancers carry application traffic. - -## Core Kubernetes resources - -```text -Deployment - `-- ReplicaSet - |-- Pod: AlertOps container - |-- Pod: AlertOps container - `-- Pod: AlertOps container - -Service -> stable DNS and traffic distribution to ready Pods -ConfigMap -> non-secret runtime configuration -Secret -> sensitive runtime values -PersistentVolumeClaim -> request for persistent storage -``` - -- **Pod:** smallest scheduling unit. Pods are disposable; their names and IPs can - change, so clients do not depend on a Pod IP. -- **Deployment:** declares stateless replicas and rollout strategy. AlertOps belongs - in a Deployment. -- **Service:** provides stable discovery and selects current Pods using labels. -- **StatefulSet:** provides stable identity and ordered lifecycle for stateful systems. -- **ConfigMap:** stores non-secret configuration. -- **Secret:** stores sensitive values as a dedicated API object, but base64 is not - encryption; production still requires encryption, RBAC and Vault integration. -- **PersistentVolumeClaim:** requests storage whose lifecycle is separate from a Pod. - -Kubernetes also separates application probes: - -| Probe | Question | Failure result | -|---|---|---| -| Startup | Has the process completed initialization? | Delay other probes; eventually restart after repeated failure | -| Liveness | Is this process stuck and should it restart? | Restart the container | -| Readiness | Should this Pod receive traffic? | Remove it from Service endpoints | - -The Phase 1 failure drills showed why a remote database outage should not make -AlertOps fail liveness: restarting AlertOps cannot repair PostgreSQL. - -## AlertOps request flow - -```text -Client - | -Ingress / cloud load balancer - | -Kubernetes Service - | - +-- ready AlertOps Pod A - +-- ready AlertOps Pod B - `-- ready AlertOps Pod C - |-- PostgreSQL - |-- RabbitMQ - `-- shared Redis intent state -``` - -Redis integration from Phase 1 matters here: Pod A can create an intent and Pod B can -atomically consume the same shared value. - -## Why organizations adopt Kubernetes - -Kubernetes becomes useful when consistent operations across many workloads matter more -than its platform complexity. Common reasons include: - -- declarative deployment and repeatable environments; -- replica management and self-healing; -- service discovery and stable networking; -- rolling updates and rollback history; -- workload and node scaling; -- CPU/memory scheduling and isolation; -- standard probes, configuration and secret interfaces; -- a shared platform for multiple teams and services; -- GitOps reconciliation with Argo CD; -- a common API and ecosystem across cloud providers. - -Not every company or application uses or needs Kubernetes. Its costs include cluster -and node spend, a large learning surface, networking/storage/IAM complexity, upgrades, -more layers to debug, and the need for strong observability and security governance. - -| Situation | Often a reasonable starting point | -|---|---| -| One service, small team | Docker on EC2/OCI Compute or a managed app platform | -| Several containers on one host | Docker Compose with automated VM provisioning | -| AWS containers without Kubernetes portability needs | ECS/Fargate | -| Many services/teams and frequent releases | EKS/OKE/Kubernetes | -| Existing company Kubernetes platform | Deploy into that established platform | -| Local Kubernetes learning and manifest tests | kind, minikube or Docker Desktop Kubernetes | - -Kubernetes should be chosen for workload and organizational requirements, not merely -because it is popular. - -## Why kind - -`kind` means **Kubernetes IN Docker**. It creates a real local Kubernetes cluster whose -nodes are Docker containers: - -```text -Windows laptop - `-- Docker Desktop Linux VM - |-- kind control-plane node container - | `-- Kubernetes control-plane components - `-- optional kind worker node containers - `-- application Pods -``` - -The node container provides an environment in which Kubernetes components and a -container runtime operate. This makes a cluster cheap, disposable and reproducible. -The workstation already has Docker Desktop and `kubectl`; kind adds one local-cluster -tool and directly reuses Docker images and networking. - -kind is useful for learning Kubernetes APIs, validating manifests, testing rollouts -and probes, loading a locally built AlertOps image without a cloud registry, and -creating disposable CI clusters. It was primarily designed for Kubernetes testing and -is also useful for local development and CI. - -kind is not the production platform. It does not reproduce cloud availability zones, -load balancers, IAM or managed storage. The same resource concepts will later move to -OKE, with EKS mappings for interviews. See the official [kind overview](https://kind.sigs.k8s.io/) -and [quick start](https://kind.sigs.k8s.io/docs/user/quick-start/). - -We will pin the kind node image by Kubernetes version and digest for the same -reproducibility reasons used in Phase 1. - -## Compose, kind and managed cloud Kubernetes - -| Layer | Docker Compose | kind | OKE / EKS | -|---|---|---|---| -| Purpose | Multi-container stack on one host | Local Kubernetes learning/testing | Managed production Kubernetes | -| Nodes | One Docker Engine | Docker containers acting as nodes | OCI Compute / EC2 node pools | -| Reconciliation | Limited lifecycle operations | Kubernetes controllers | Kubernetes controllers | -| Load balancer | Local port publishing | Local simulation/port-forward | Cloud LB integration | -| Storage | Docker named volumes | Local test storage class | Cloud block/file storage | -| Availability | One host | One laptop | Multi-node and potentially multi-zone | - -## Tool and cluster versions used - -The completed local environment uses: - -| Component | Version or value | -|---|---| -| Docker Desktop | 4.83 | -| WSL | 2.7.11 | -| kubectl client | v1.36.3 | -| kind | v0.32.0 | -| kind cluster | `alertops` | -| Kubernetes node | v1.36.1 | -| Linux control groups | cgroup v2 | - -Helm is intentionally deferred. Plain manifests expose the basic Kubernetes resources -and relationships instead of hiding them behind templates. - -## Commands will be introduced gradually - -Do not memorize this table. Each command will be explained when its resource exists: - -| Command | What it asks the system to do | -|---|---| -| `kind create cluster` | Create Docker node containers and initialize Kubernetes | -| `kubectl cluster-info` | Show control-plane and cluster service addresses | -| `kubectl get nodes` | List cluster nodes and their readiness | -| `kubectl apply -f ` | Submit desired resource state from a manifest | -| `kubectl get ` | Read current resource summaries | -| `kubectl describe ` | Read detailed state, conditions and events | -| `kubectl logs ` | Read application output from a Pod container | -| `kubectl get events` | Inspect chronological operational events | -| `kubectl rollout status deployment/` | Follow Deployment rollout progress | -| `kind delete cluster` | Remove the disposable local cluster node containers | - -For troubleshooting, remember the purpose of each command family rather than every -flag: - -```text -get -> What is the current state? -describe -> Why is it in that state? -logs -> What did the application report? -events -> What did Kubernetes report? -rollout -> What is happening with the Deployment version? -``` - -`describe` and `events` are especially useful when a container never started, such as -`ImagePullBackOff` or `CreateContainerConfigError`; application logs may not exist yet. - -`kubectl` is only a client for the Kubernetes API. It does not run containers. It -reads kubeconfig to find a cluster and credentials, sends API requests, and displays -the resource state returned by the API server. - -## Task sequence - -1. Install compatible kind and kubectl versions. -2. Create a pinned, single-control-plane learning cluster. -3. Learn contexts, clusters, nodes, namespaces, and basic `kubectl` inspection. -4. Build AlertOps and load its local image into kind. -5. Create a dedicated `alertops` namespace. -6. Separate non-secret ConfigMap values from Secret values. -7. Deploy PostgreSQL with a Service and PersistentVolumeClaim. -8. Deploy RabbitMQ with stable identity, a Service, and persistent storage. -9. Deploy Redis with authentication, a Service, and persistent storage. -10. Deploy AlertOps with startup, liveness, and readiness probes plus resources. -11. Expose AlertOps locally through a Kubernetes Service/port-forward. -12. Scale AlertOps to multiple replicas and verify shared Redis intent state. -13. Perform pod deletion, dependency outage, rollout, rollback, and storage drills. -14. Document evidence and complete the Phase 2 checklist. - -## Compose-to-Kubernetes map - -| Compose | Kubernetes | Purpose | -|---|---|---| -| service container | Pod controlled by Deployment/StatefulSet | Run workload processes | -| service name | Service + cluster DNS | Stable network discovery | -| `environment` | ConfigMap and Secret references | Runtime configuration | -| `healthcheck` | startup/liveness/readiness probes | Lifecycle and traffic decisions | -| named volume | PersistentVolumeClaim | Data independent of a pod | -| `cpus`/`mem_limit` | resource requests and limits | Scheduling and enforcement | -| `depends_on` | probes, retries, and init logic | Dependency readiness | -| `restart` | controller reconciliation | Replace failed workload instances | - -Kubernetes has no direct equivalent of Compose `depends_on`. Applications must retry -remote dependencies, and probes/controllers express whether a workload should receive -traffic or be restarted. - -## Phase 2 implementation journal - -### 1. Local cluster and terminology - -The cluster was created as a pinned kind cluster named `alertops`: - -```powershell -kind create cluster --name alertops --image -``` - -- `kind create cluster` creates and initializes the local Kubernetes cluster. -- `--name alertops` gives the cluster a stable local name. -- `--image` selects a kind node image containing Kubernetes components and the node - runtime. It is not the AlertOps application image. - -The commonly confused terms have different scopes: - -```text -kind cluster: alertops -`-- Docker node container: alertops-control-plane - `-- Kubernetes components and containerd - `-- application and system Pods - `-- one or more workload containers -``` - -- A **container** is an isolated process created from an image. -- `alertops-control-plane` is a Docker container acting as the kind Kubernetes node. -- The **cluster** is the complete Kubernetes system: API, desired state, nodes, - networking, storage integration, and workloads. - -The cluster was inspected with: - -```powershell -kubectl config get-contexts -kubectl get nodes -o wide -kubectl get namespaces -kubectl get pods --all-namespaces -``` - -A **context** connects a cluster, credentials, and a default namespace. The `*` in -`kubectl config get-contexts` marks the current context. `-o wide` requests extra -columns such as IP, OS, kernel, and container runtime. `--all-namespaces` searches -across every namespace. - -The node output showed: - -```text -alertops-control-plane Ready control-plane v1.36.1 -``` - -`Ready` means the node can accept workloads. This single-node learning cluster runs -the control plane and application Pods together. Production clusters normally use a -replicated managed control plane and separate worker nodes. - -The initial namespaces were: - -| Namespace | Purpose | -|---|---| -| `default` | Used when no namespace is specified | -| `kube-system` | Kubernetes system components | -| `kube-public` | Publicly readable cluster information | -| `kube-node-lease` | Lightweight node heartbeats | -| `local-path-storage` | kind's local storage provisioner | - -`kubectl get pods` initially returned `No resources found in default namespace` -because the cluster was healthy but no user workload existed in `default`. - -### 2. Local platform troubleshooting - -Docker Desktop initially failed while starting the kind control-plane container. -Docker Desktop and WSL were upgraded. An obsolete `disableHardwareAcceleration` -setting was removed after creating this backup: - -```text -C:\Users\ASUS\AppData\Roaming\Docker\settings.json.pre-4.83-backup-20260725-133516 -``` - -An older `kubectl` executable appeared earlier in `PATH` than the newly installed -client. A PowerShell profile alias was added so new sessions resolve `kubectl` to -v1.36.3: - -```text -C:\Users\ASUS\OneDrive\Documents\WindowsPowerShell\profile.ps1 -``` - -Existing shells do not automatically reread a changed `PATH`. Close and reopen -PowerShell. If only the PowerShell profile changed, reload it with: - -```powershell -. $PROFILE -``` - -`kubectl version --client` verifies the client executable. It does not prove that a -cluster exists or report its server version. - -#### Cgroup and CPU fundamentals - -Linux **control groups**, or cgroups, account for and constrain CPU, memory, and other -resources used by process groups. Kubernetes asks the runtime to place each container -into cgroups according to its resource configuration. This machine uses cgroup v2, -the newer unified hierarchy. - -CPU values express CPU capacity over time: - -- `1000m`, or `1`, means one CPU core of capacity. -- `500m` means half a core. -- `250m` means one quarter of a core. - -A CPU limit is enforced by throttling CPU time over small accounting periods. It does -not mean that one Java task always runs for exactly 50 ms and then stops. An eight-core -machine can execute roughly eight CPU-bound threads simultaneously, while many more -threads can exist and wait for CPU or I/O. A scheduling window is a short interval -during which the kernel accounts for CPU usage. - -High CPU in Grafana is a signal to investigate, not automatically a fault. Correlate -it with traffic, throughput, latency, errors, garbage collection, retries, queue -depth, and CPU-throttling metrics. - -### 3. Application image workflow - -Docker Compose was not needed to build the Kubernetes application image: - -```powershell -docker build --tag alertops:3514a9d28585 . -kind load docker-image alertops:3514a9d28585 --name alertops -``` - -- `docker build` reads the local Dockerfile and creates the AlertOps image. -- `--tag`, or `-t`, assigns the readable image name and tag. -- `.` is the Docker build context. -- `kind load docker-image` copies the built image into the kind node's containerd - image store. -- `--name alertops` selects the kind cluster. - -Compose previously appeared to build the application because its `app` service has a -`build` instruction. Compose only orchestrated the same Docker build; the Dockerfile -remained the build definition. - -Docker and Git tags solve different problems: - -| Item | Points to | Purpose | -|---|---|---| -| Git commit | Exact source snapshot | Source history | -| Git tag | Named Git commit | Release marker or CI trigger | -| Docker image | Packaged runnable artifact | Deployment | -| Docker image tag | Readable image reference | Artifact selection | -| Image digest | Immutable image content | Exact reproducibility | - -`docker build -t alertops:3514a9d28585 .` creates an image and attaches that tag. -`docker tag old-name new-name` does not rebuild or copy its layers; it adds another -reference to the same local image. In a company workflow, a pushed Git tag often -triggers CI, which builds and pushes a Docker image. Kubernetes deploys that image, -not the Git tag. - -The image was loaded into kind because Docker Desktop's image store and the kind -node's containerd store are separate. Cloud worker nodes instead pull images from -OCIR, ECR, or another registry. - -Images are cached at node scope, not namespace scope. Deleting the `alertops` -namespace removes its namespaced API resources but does not remove -`alertops:3514a9d28585` from the kind node's containerd store. Deleting the entire kind -cluster removes that node and its cache. In `kind load docker-image ... --name -alertops`, `--name alertops` selects the cluster; it does not select a namespace. - -### 4. Namespace, configuration, and secrets - -The dedicated namespace was created and inspected with: - -```powershell -kubectl apply -f k8s/base/namespace.yaml -kubectl get namespace alertops --show-labels -``` - -- `apply` creates or updates declared state. -- `-f` means read the following file. -- `--show-labels` adds labels to the table output. - -Every namespaced manifest explicitly contains: - -```yaml -metadata: - namespace: alertops -``` - -`kind: ConfigMap` identifies an API resource type; it does not create a namespace. -`metadata.namespace` places the resource into an already existing namespace. - -Non-secret values are committed in `alertops-config`. Sensitive local values were -created without committing their source file: - -```powershell -kubectl create secret generic alertops-secrets ` - --namespace alertops ` - --from-env-file=.env.k8s.local -``` - -- `generic` creates an ordinary `Opaque` Secret. -- `--namespace`, or `-n`, selects the namespace. -- `--from-env-file` reads key/value pairs from the ignored local file. - -The Deployment injects both resources: - -```yaml -envFrom: - - configMapRef: - name: alertops-config - - secretRef: - name: alertops-secrets -``` - -Spring resolves expressions such as `${SPRING_RABBITMQ_HOST}` from the container's -environment when the application starts. - -ConfigMaps and Secrets are API objects, not Pods. The kubelet resolves required -references and injects their keys before starting the container. If a required Secret -does not exist, the application process cannot start and the Pod commonly reports -`CreateContainerConfigError`; Spring cannot handle an error that prevents Java from -starting. - -Secret values can be retrieved and Base64-decoded, but Base64 is encoding rather than -encryption. Normal inspection should show key names, not values: - -```powershell -kubectl describe secret alertops-secrets -n alertops -``` - -Production needs least-privilege RBAC, encryption at rest, and usually an external -source such as OCI Vault, AWS Secrets Manager, or HashiCorp Vault. - -### 5. Labels and selectors - -Labels are searchable metadata: - -```yaml -labels: - app.kubernetes.io/name: postgres - app.kubernetes.io/part-of: alertops -``` - -- `name: postgres` identifies the component. -- `part-of: alertops` groups it with the complete system. -- Labels under `spec.template.metadata` are copied onto created Pods. - -A selector is an active matching rule: - -```yaml -selector: - matchLabels: - app.kubernetes.io/name: postgres -``` - -A StatefulSet selector identifies the Pods it owns. A Service selector identifies -the Pods that receive its traffic. Selectors must match the Pod-template labels. -Labels do not route traffic by themselves; selectors use them to form relationships. - -The `-l` command option uses the same mechanism for filtering: - -```powershell -kubectl get pods -n alertops -l app.kubernetes.io/name=redis -``` - -A Service is the stable frontend, while EndpointSlices are its changing backend list: - -```text -DNS name postgres - -> Service ClusterIP and port - -> EndpointSlice with ready Pod IPs - -> postgres-0 Pod IP and target port -``` - -Each Service gets its own ClusterIP. That IP remains stable for the Service's lifetime, -but deleting and recreating the Service can allocate a new one. EndpointSlice names, -such as `postgres-xrd5c`, use a generated suffix and are internal implementation -objects, not client DNS names. A Service can exist while its EndpointSlice has no -addresses: this means a stable frontend exists but no matching ready backend exists. - -### 6. StorageClass, provisioner, PV, and PVC - -The local StorageClass reported: - -```text -standard (default) rancher.io/local-path Delete WaitForFirstConsumer -``` - -| Field | Meaning | -|---|---| -| `standard` | Storage policy selected by the PVC | -| `rancher.io/local-path` | Driver that creates backing local storage | -| Reclaim policy `Delete` | Delete the PV when its claim is deleted | -| `WaitForFirstConsumer` | Wait for a Pod before choosing storage placement | -| Expansion `false` | Existing claims cannot be enlarged through this class | - -A provisioner translates a PVC request into real storage. Creating a StorageClass does -not install a new storage driver; its `provisioner` must name an installed driver. -OKE and EKS use CSI drivers to provision OCI or AWS block/file volumes. - -```text -Pod mounts PVC - -> PVC requests storage - -> StorageClass selects policy and provisioner - -> provisioner creates PV and backing storage -``` - -The PVC was initially `Pending` because `WaitForFirstConsumer` waits until a Pod uses -it. It became `Bound` when the StatefulSet Pod was scheduled. `RWO`, or -`ReadWriteOnce`, permits a read/write mount from a single node. - -### 7. Why StatefulSet, Service, and PVC are separate - -Each stateful dependency uses three objects because each solves a different problem: - -```text -StatefulSet -> creates and replaces the process Pod with stable identity -Service -> provides stable DNS and selects the current Pod -PVC -> preserves data independently of Pod lifetime -``` - -For PostgreSQL: - -- StatefulSet creates `postgres-0` and recreates it when necessary. -- Service provides `postgres:5432`; AlertOps never relies on a temporary Pod IP. -- PVC mounts at `/var/lib/postgresql/data`. - -RabbitMQ and Redis use the same model. A StatefulSet alone would not provide the -application-facing Service or persistent disk. A Service alone would not create a -Pod. A PVC alone would only request storage. - -The following command displays all four layers together: - -```powershell -kubectl get statefulset,pod,pvc,service -n alertops -``` - -This single-replica design teaches identity and persistence, but it is not highly -available. Production PostgreSQL, RabbitMQ, and Redis require product-specific -replication, quorum, backups, restore tests, and disruption planning. Simply setting -their replica count to three does not safely create a database or broker cluster. - -### 8. RabbitMQ startup-probe failure - -RabbitMQ initially restarted because an executable startup probe could race with -Erlang cookie creation. A probe running as root could create a root-owned cookie -before RabbitMQ, running as UID 999, could read it. - -The startup probe was changed to: - -```yaml -startupProbe: - tcpSocket: - port: amqp -``` - -This checks whether AMQP is listening without executing a cookie-dependent command. -Readiness and liveness retain RabbitMQ diagnostics after startup. The replacement Pod -became ready with the same PVC, and the Service selected its new IP automatically. - -Probes run inside the container and can have side effects. Probe commands should be -safe, lightweight, and permission-aware. - -### 9. Redis authentication and AOF persistence - -Redis returned an authenticated `PONG`. An unauthenticated request returned: - -```text -NOAUTH Authentication required. -``` - -The configuration enables: - -```text -appendonly yes -appendfsync everysec -``` - -Redis normally serves reads from memory. Its write path is approximately: - -```text -SET/DEL/INCR - -> update memory - -> append the command to AOF under /data - -> flush it to PVC-backed storage -``` - -`appendfsync everysec` balances performance and durability; a sudden host loss can -lose approximately the latest second of writes. On restart, Redis replays the AOF to -reconstruct memory. AOF improves recovery but is not a substitute for backups. - -### 10. AlertOps Deployment and Service - -AlertOps runs in a Deployment because shared intent state was moved to Redis and the -application replicas are intended to be stateless. The final desired count is: - -```yaml -replicas: 3 -``` - -The Deployment uses: - -- the commit-SHA image tag; -- ConfigMap and Secret environment injection; -- resource requests and limits; -- startup, readiness, and liveness probes; -- container security restrictions; -- `maxUnavailable: 0` and `maxSurge: 1` rolling updates. - -The resource-to-runtime chain is: - -```text -Deployment manifest stores replicas, Pod template, and rollout strategy - -> Deployment controller manages ReplicaSet revisions - -> ReplicaSet controller maintains the Pod count - -> scheduler assigns each unscheduled Pod to one node - -> node kubelet asks the runtime to start its containers -``` - -The scheduler schedules Pods, not Services or PVCs. A Pod stays `Pending` with a -`FailedScheduling` event when no node satisfies its requested resources. The kubelet -operates Pods assigned to its node; it does not manage cluster-wide Services. - -Requests are considered by the scheduler. Limits constrain actual use. A `250m` CPU -request reserves scheduling capacity equal to a quarter of a CPU. A CPU limit of `1` -allows up to one CPU of time. A memory limit is a hard ceiling and exceeding it may -cause an OOM kill. - -The ClusterIP Service provides stable internal discovery: - -```text -service/alertops -> ready AlertOps Pod endpoints on port 8096 -``` - -Local access was tested with: - -```powershell -kubectl port-forward service/alertops 8096:8096 -n alertops -Invoke-RestMethod http://localhost:8096/actuator/health -``` - -- `port-forward` creates a temporary local tunnel to the Service. -- It runs in the foreground until `Ctrl+C`. -- It is a debugging mechanism, not production ingress. - -The response was: - -```json -{"status":"UP","groups":["liveness","readiness"]} -``` - -### 11. Scaling and shared state - -After changing `replicas` from one to three, Kubernetes created three AlertOps Pods. -The Service EndpointSlice contained all three Pod IPs, proving Service discovery. - -Every replica receives the same Redis host and credentials. This is the correct -infrastructure for shared intent state. The full invitation/login flow was not -repeated because the application workflow was no longer fresh in memory. The precise -result is: - -> Multi-replica scheduling, readiness, and Service discovery were verified. Shared -> Redis intent state is structurally configured, but cross-replica intent creation -> and atomic consumption still require an application-level integration test. - -Infrastructure configuration is useful evidence but does not replace a functional -test. A future automated test should create an intent through one replica and consume -it through another. - -### 12. Controlled operational drills - -#### Pod self-healing - -One AlertOps Pod was deleted and the replicas were watched: - -```powershell -kubectl delete pod -n alertops -kubectl get pods -n alertops -l app.kubernetes.io/name=alertops --watch -``` - -- `delete pod` removed only the disposable Pod, not its Deployment. -- `--watch` kept the query open and printed state changes. - -The Deployment continued to desire three replicas. Kubernetes created a replacement, -which progressed from `0/1 Running` to `1/1 Running` when readiness succeeded. This -proved controller reconciliation and readiness gating. - -#### Rolling restart - -```powershell -kubectl rollout restart deployment/alertops -n alertops -kubectl rollout status deployment/alertops -n alertops -``` - -`rollout restart` modifies a Pod-template annotation and creates a new ReplicaSet -without changing the application image. `rollout status` follows progress. AlertOps -took around 50 seconds for each new Pod to become ready, and the rollout succeeded. - -With `maxUnavailable: 0`, Kubernetes retained the desired number of ready replicas. -With `maxSurge: 1`, it could temporarily create one extra Pod. - -#### Invalid image and rollback - -An intentionally nonexistent image was set only in the live cluster: - -```powershell -kubectl set image deployment/alertops ` - alertops=alertops:rollback-drill-missing ` - -n alertops - -kubectl rollout status deployment/alertops -n alertops --timeout=90s -kubectl get pods -n alertops -l app.kubernetes.io/name=alertops -``` - -- `set image` updates a named container in the Deployment. -- `--timeout=90s` stops waiting after 90 seconds; it does not repair the rollout. -- This imperative command did not modify the committed YAML. - -The result contained three healthy old Pods and one new `ImagePullBackOff` Pod. -Kubernetes could not pull the missing image, but `maxUnavailable: 0` protected the -healthy replicas. - -Rollback used: - -```powershell -kubectl rollout undo deployment/alertops -n alertops -kubectl rollout status deployment/alertops -n alertops -kubectl get deployment alertops -n alertops ` - -o jsonpath='{.spec.template.spec.containers[0].image}' -``` - -`rollout undo` restored the previous Pod template. JSONPath extracted only the image -field. The failed Pod disappeared and all three healthy replicas remained on: - -```text -alertops:3514a9d28585 -``` - -#### Redis Pod recreation and disk recovery - -A harmless key was stored before deleting Redis: - -```powershell -kubectl exec redis-0 -n alertops -- ` - redis-cli SET phase2:persistence "survives-pod-recreation" - -kubectl delete pod redis-0 -n alertops -kubectl wait --for=condition=Ready pod/redis-0 -n alertops --timeout=180s - -kubectl exec redis-0 -n alertops -- ` - redis-cli GET phase2:persistence -``` - -- `exec` runs a command inside a container. -- `--` ends kubectl options; everything after it is the container command. -- `wait --for=condition=Ready` waits for the recreated Pod to become ready. - -The value survived. The StatefulSet recreated `redis-0`, mounted the same PVC, and -Redis replayed its AOF into memory. Cleanup returned `1`, meaning one key was deleted: - -```powershell -kubectl exec redis-0 -n alertops -- redis-cli DEL phase2:persistence -``` - -#### Focused rebuild and Service-selector drift - -The committed manifests were used to rebuild the namespace without creating duplicate -practice YAML. Deleting the namespace removed its Deployments, StatefulSets, Pods, -Services, ConfigMap, Secret, and PVCs. The cluster and node-level image cache remained. - -Services were deliberately created before their workloads. Each received a ClusterIP, -but its EndpointSlice initially showed `` because no matching Pod existed. When -`postgres-0` became ready, its temporary Pod IP and port `5432` appeared automatically, -and `postgres-data` changed from `Pending` to `Bound`. - -The live AlertOps Service selector was then changed to a value no Pod had: - -```powershell -kubectl set selector service/alertops ` - app.kubernetes.io/name=missing-alertops ` - -n alertops -``` - -All three Pods stayed `1/1 Running`, but the EndpointSlice had no addresses. Comparing -`kubectl describe service alertops -n alertops` with -`kubectl get pods --show-labels -n alertops` exposed the mismatch. Reapplying -`k8s/base/alertops-service.yaml` restored the committed selector and all three -endpoints. This is a small GitOps example: live state drifted, and declared Git state -corrected it. - -#### Redis dependency outage: readiness versus liveness - -Redis was temporarily scaled to zero without deleting its PVC: - -```powershell -kubectl scale statefulset/redis --replicas=0 -n alertops -``` - -All AlertOps Pods changed to `0/1 Running`, and the Service had no usable endpoints. -Direct checks from one Pod returned: - -```text -liveness: 200 -readiness: 503 -``` - -The containers stayed running and their restart counts did not increase. The failure -was in a remote dependency, so restarting AlertOps would not repair it. Reapplying the -Redis StatefulSet restored `replicas: 1`; after Redis became ready, AlertOps readiness -succeeded and its Service endpoints returned automatically. - -If liveness instead fails beyond its threshold, kubelet restarts the container inside -the same Pod and increments `RESTARTS`. Repeated startup failures may result in -`CrashLoopBackOff`; Kubernetes does not immediately delete the Pod object. - -### 13. Environment design: clusters versus namespaces - -Companies use both patterns: - -- namespaces commonly separate teams, applications, or lower-risk environments; -- separate clusters commonly isolate production from non-production; -- large organizations may use multiple clusters per region or security boundary. - -A reasonable example is: - -```text -non-production cluster -|-- alertops-dev namespace -`-- alertops-test namespace - -production cluster -`-- alertops-prod namespace -``` - -Namespaces provide logical isolation but share a control plane and often nodes. They -are not as strong a failure, upgrade, or security boundary as separate clusters. -Production separation depends on risk, compliance, scale, cost, and platform policy. - -### 14. Questions asked during Phase 2 - -**Why not deploy the Docker image directly to EC2 or OCI Compute?** - -That remains valid for smaller systems. Kubernetes becomes valuable when consistent -scheduling, self-healing, discovery, rollout, scaling, policy, and resource management -are needed across many replicas and machines. - -**Does an eight-core machine mean only eight tasks can exist?** - -No. It can execute roughly eight CPU-bound threads simultaneously, but many more -threads can exist or wait for CPU and I/O. The operating system schedules runnable -work across the available CPUs. - -**Does Kubernetes create a replacement immediately after a Pod is deleted?** - -The controller observes that actual replicas are below desired replicas and creates a -replacement. The scheduler selects a suitable node. Creation can be delayed when no -node satisfies the resource or policy requirements. - -**Is a Pod the same as a container?** - -Not exactly. A Pod is the smallest Kubernetes scheduling unit and can contain one or -more containers sharing networking and volumes. AlertOps uses the common -one-container-per-Pod pattern. - -**Why are readiness and liveness separate?** - -Readiness decides whether the Pod receives traffic. Liveness decides whether a -container restart may repair it. A remote database outage should generally fail -readiness, not liveness, because restarting AlertOps cannot fix PostgreSQL. - -**Does a ConfigMap create its namespace?** - -No. Namespace is a separate resource. `metadata.namespace` places the ConfigMap into -an existing namespace. - -**Why put a hostname in ConfigMap but a JWT key in Secret?** - -A hostname is ordinary configuration. Passwords and signing keys are sensitive. -Secrets allow separate access controls and handling, although production also needs -encryption and external secret management. - -**Why does a Service need selectors?** - -Pod IPs change. The Service matches current Pods by label and maintains stable DNS -plus the ready endpoint set. - -**Why was a PVC `Pending` before its Pod existed?** - -The `standard` StorageClass uses `WaitForFirstConsumer`, so it waits for workload -scheduling information before provisioning and binding the volume. - -**Why use StatefulSet when Service and PVC already exist?** - -The PVC stores data and the Service provides networking; neither creates or -supervises the process. The StatefulSet creates and replaces the Pod with stable -identity. - -**Why did a Service find the replacement Pod automatically?** - -The replacement had the same labels. The Service selector matched it, and Kubernetes -updated the EndpointSlice with its new Pod IP. - -**Why did Redis retain a value after Pod deletion?** - -Redis reconstructed the value by replaying AOF data stored on the PVC. It then served -normal requests from memory again. - -**What does `-f` mean?** - -Use the following file as command input, such as `kubectl apply -f manifest.yaml`. - -**What does `-n` mean?** - -It is the short form of `--namespace` and scopes the command to that namespace. - -**What does `-l` mean?** - -It filters resources using a label selector. - -**Does `kubectl` create containers?** - -No. It sends requests to the Kubernetes API. Controllers, the scheduler, kubelet, and -the container runtime cooperate to create the workload. - -**What is the difference between a Git tag and a Docker tag?** - -A Git tag names source history. A Docker tag names a runnable image artifact. CI often -uses the Git tag as a trigger and then produces a Docker image tag for deployment. - -**Why does kind need `kind load docker-image`?** - -The image built by Docker is not automatically present in the kind node's containerd -store. Loading copies it into the node. Production nodes pull from a registry. - -## Final validation evidence - -The live cluster reported: - -```text -Node -alertops-control-plane Ready control-plane v1.36.1 - -Deployment -alertops 3/3 Ready - -StatefulSets -postgres 1/1 Ready -rabbitmq 1/1 Ready -redis 1/1 Ready - -PersistentVolumeClaims -postgres-data Bound 1Gi RWO -rabbitmq-data Bound 1Gi RWO -redis-data Bound 1Gi RWO - -AlertOps Service endpoints -10.244.0.16:8096 -10.244.0.17:8096 -10.244.0.18:8096 -``` - -Pod names and IPs are temporary and will change after recreation. - -## Phase 2 completion checklist - -- [x] Compatible Docker Desktop, WSL, kind, and kubectl installed. -- [x] Pinned kind cluster created and inspected. -- [x] AlertOps image built with a commit-SHA tag and loaded into kind. -- [x] Dedicated Namespace, ConfigMap, and Secret created. -- [x] PostgreSQL deployed with Service and bound PVC. -- [x] RabbitMQ deployed with Service and bound PVC. -- [x] Redis deployed with authentication, AOF, Service, and bound PVC. -- [x] AlertOps deployed with probes, resources, security context, and Service. -- [x] Health endpoint reached using a temporary port-forward. -- [x] AlertOps scaled to three ready replicas. -- [x] Service endpoint discovery across three replicas verified. -- [x] Namespace rebuilt from committed manifests without duplicate practice YAML. -- [x] Broken Service selector diagnosed through labels and empty endpoints. -- [x] Redis outage produced readiness 503 and liveness 200 without container restarts. -- [x] Pod self-healing and readiness transition observed. -- [x] Rolling restart completed. -- [x] Invalid-image rollout failed without removing healthy replicas. -- [x] Previous working rollout restored. -- [x] Redis data survived Pod recreation and the test key was cleaned up. -- [ ] Cross-replica intent create/consume behavior automated and functionally tested. -- [ ] Production-grade backup and restore procedures designed and tested. - -Phase 2 is complete for the local Kubernetes learning objective. The unchecked items -are deliberately carried forward: application integration coverage and production -backup/restore work belong in later application and reliability phases. diff --git a/docs/deployment/phase-3-github-actions.md b/docs/deployment/phase-3-github-actions.md deleted file mode 100644 index 9e6e8bb..0000000 --- a/docs/deployment/phase-3-github-actions.md +++ /dev/null @@ -1,301 +0,0 @@ -# Phase 3: Continuous Integration with GitHub Actions - -> **ReplyTrail rebrand note:** This CI learning record uses older AlertOps wording in -> historical observations. It refers to the current ReplyTrail product; executable -> `alertops` artifact and resource identifiers remain unchanged. - -## Objective - -Turn the verified local build into a repeatable CI pipeline. Every relevant push and -pull request should prove that AlertOps compiles and passes its tests. Trusted branch -builds will then create an immutable Docker image and publish it to GitHub Container -Registry (GHCR) using the Git commit SHA. - -Phase 3 automates artifact creation. It does not deploy directly to Kubernetes: - -```text -Developer pushes commit to GitHub - | - v -GitHub Actions workflow - |-- check out exact commit - |-- install pinned Java toolchain - |-- restore Maven dependency cache - |-- run tests and package JAR - |-- upload test/build evidence - |-- build and scan container image - `-- publish immutable image to GHCR on trusted branches - -Later phase: -Argo CD reads Git desired state and deploys the selected image -``` - -CI answers, "Is this commit a valid release candidate?" GitOps deployment answers, -"Does the cluster match the version declared in Git?" Keeping those responsibilities -separate prevents a CI job from holding broad production-cluster credentials. - -## Implementation decision - -AlertOps already uses GitHub as its remote repository, so this phase uses: - -- GitHub Actions for CI automation; -- GitHub-hosted runners initially; -- GitHub Actions artifacts for JARs and reports; -- GitHub Container Registry (`ghcr.io`) for Phase 3 images; -- the built-in `GITHUB_TOKEN` with explicitly limited permissions where possible. - -There will be no parallel GitLab repository or `.gitlab-ci.yml` during implementation. -GitLab CI mapping and a small YAML translation exercise are deferred until the GitHub -workflow is complete. This keeps one source repository and one CI mental model active. - -## CI is not the developer machine - -A GitHub-hosted runner is a temporary clean machine allocated for a job. It does not -reuse the developer's local JDK, Maven cache, `.env`, Docker images, or kind cluster. -The workflow must declare everything required to reproduce the build. - -```text -Local machine GitHub-hosted runner ------------- -------------------- -long-lived files temporary clean VM -developer credentials scoped workflow token/secrets -warm Maven cache cache restored only when configured -local Docker image image rebuilt from checked-out source -local .env unavailable and must remain uncommitted -``` - -A successful local build is necessary evidence, but CI proves that the repository can -build from a clean checkout without hidden machine-specific state. - -## Core terminology - -| Term | Meaning in this phase | -|---|---| -| Workflow | YAML automation definition under `.github/workflows/` | -| Event | Trigger such as a push, pull request, tag, or manual run | -| Job | Group of steps executed on one runner | -| Runner | Machine that executes a job | -| Step | One command or reusable action within a job | -| Action | Reusable workflow building block such as checkout or Java setup | -| Artifact | Output retained from a run, such as a JAR or test report | -| Cache | Reusable dependencies that improve speed but are not release outputs | -| Service container | Temporary dependency, such as Redis, available to a test job | -| Workflow secret | Sensitive value supplied to a job without committing it | -| Container registry | Remote store for versioned Docker/OCI images | - -Artifacts and caches are not interchangeable: - -```text -Maven cache -> speeds up later builds; safe to regenerate -JAR artifact -> evidence/output produced by this exact workflow run -Docker image -> deployable release artifact stored in a registry -``` - -## Image identity - -The primary image tag will use the Git commit SHA: - -```text -ghcr.io//alert_ops: -``` - -The commit SHA connects source, workflow evidence, image, and later Kubernetes -deployment. Human-friendly release tags can be added, but they should not replace an -immutable commit or digest reference for traceability. - -## Security rules - -- Never copy `.env` or `.env.k8s.local` into the workflow or image. -- Give `GITHUB_TOKEN` only the permissions required by each job. -- Do not publish images from untrusted pull-request code with write credentials. -- Pin third-party actions to reviewed versions and later to commit SHAs. -- Keep runtime credentials out of image-build arguments and image layers. -- Treat test reports and logs as potentially sensitive output. -- Build once and promote the same image rather than rebuilding per environment. - -## Task sequence - -1. Understand workflow, runner, job, step, event, cache, artifact, and registry. -2. Add a minimal workflow triggered by pushes and pull requests. -3. Run Maven tests and package the JAR on Java 17. -4. Cache Maven dependencies and upload useful test/build artifacts. -5. Run the Redis integration test with a temporary service container. -6. Build the Docker image in CI without publishing from pull requests. -7. Authenticate safely and publish commit-SHA images to GHCR from trusted branches. -8. Add image scanning and preserve useful security evidence. -9. Configure required checks/branch protection after the workflow is stable. -10. Cause and diagnose a controlled pipeline failure, then document recovery. -11. Document the completed workflow and only then map it to GitLab CI terminology. - -Each task will be introduced separately. The first version will remain small; caching, -publishing, scanning, and policy will be added only after the Maven test job is proven. - -## Completion evidence - -Phase 3 is complete when: - -- a pull request automatically runs the Maven quality gate; -- a failed test blocks the workflow; -- a successful run retains useful reports or a JAR artifact; -- the Docker image is built from the same tested commit; -- a trusted branch publishes an image tagged with its commit SHA to GHCR; -- secrets are not stored in Git or printed in logs; -- one controlled failure can be diagnosed from job logs; -- the process is documented well enough to explain in an interview. - -## Completed workflow - -The implemented workflow is [`.github/workflows/ci.yml`](../../.github/workflows/ci.yml). -It runs for pushes to `deployment` and `master`, pull requests targeting `master`, -and manual dispatches. - -```text -pull request to master push to deployment or master ------------------------ ---------------------------- -Maven verification Maven verification - | | - v v -container build + Trivy scan container build + Trivy scan -GHCR login and push: skipped GHCR login and push: allowed -``` - -The two jobs run on separate temporary runners. `actions/checkout` is therefore -required in each job: it puts the triggering commit's source files on that job's -otherwise empty runner. `needs: test` makes `container-build` wait for a successful -Maven verification job; a Maven failure skips the container job. - -### Pipeline outputs - -| Output | Location | Retention | Purpose | -|---|---|---:|---| -| Fat JAR | GitHub Actions artifact | 7 days | Build evidence for the exact commit | -| Surefire reports | GitHub Actions artifact | 7 days | Test-failure diagnosis | -| `trivy-results.txt` | GitHub Actions artifact | 30 days | Container vulnerability evidence | -| Container image | GHCR | Package retention policy | Deployable release artifact | - -An artifact is downloaded as a ZIP because GitHub Actions packages uploaded files -for transport. GHCR is separate from Actions artifacts: it stores OCI/Docker images, -not build reports. - -### Image name, tag, and digest - -The workflow builds and pushes an image named: - -```text -ghcr.io/1hemant2/alert_ops: -``` - -`` is the readable source-version tag. `docker push` uploads the -image and GHCR returns a content digest such as `sha256:...`; that digest identifies -the exact immutable image content. The tag is used for source traceability, while a -digest is the strongest deployment pin: - -```text -ghcr.io/1hemant2/alert_ops@sha256: -``` - -The local image disappears when the GitHub-hosted runner is destroyed. The pushed -GHCR image remains available for a later Kubernetes cluster to pull. - -## Container security gate - -After Docker builds the final image, Trivy scans the image before registry login and -push. Scanning after the build is necessary because the final image contains the JRE, -operating-system packages, application JAR, and all runtime Java dependencies. - -```text -build image -> scan HIGH and CRITICAL findings -> upload report -> push only if clean -``` - -The scan uses a reviewed SHA-pinned Trivy Action and pins the Trivy binary to -`v0.74.0`. It writes a readable `trivy-results.txt` file and -`actions/upload-artifact` uploads it with `if: always()`. -That condition means the report is retained even when the scan fails; it does not run -another scan. The later GHCR steps use their normal implicit `success()` condition, -so they are skipped after a scan failure. - -`exit-code: "1"` turns HIGH or CRITICAL findings into a release gate. The backend -now uses Spring Boot `4.0.8`, which brings Spring Framework `7.0.9` and removes the -critical Spring MVC finding. Existing Jackson 2 application code remains supported -through Spring Boot's `spring-boot-jackson2` compatibility module. The POM also -pins the current fixed versions for Jackson 2 (`2.21.7`), RabbitMQ Java client -(`5.34.0`), and Tomcat (`11.0.25`) until the Boot dependency line manages those -versions without overrides. - -Netty is transitive rather than an AlertOps direct dependency: - -```text -spring-boot-starter-data-redis -> lettuce-core -> Netty -spring-boot-starter-amqp -> spring-rabbit -> amqp-client -spring-boot-starter-webmvc -> spring-boot-starter-tomcat -> Tomcat -``` - -Do not add an individual Netty module directly just to patch a CVE. The PostgreSQL -driver is a direct runtime dependency supplied by Spring Boot dependency management. -Review the three security properties above when upgrading Spring Boot and remove an -override only after the managed version is equal to or newer than the fixed version. - -## Protected master branch - -The GitHub repository has an active ruleset targeting `master`: - -- pull requests are required before merging; -- branches must be up to date before merging; -- `Maven verification` and `Container image build` are required checks from GitHub - Actions; -- force pushes and deletion are blocked. - -The branch rule requires passing checks; it does not decide which workflow steps run. -The `if: github.event_name != 'pull_request'` condition on GHCR login and image push -is what prevents a pull-request build from publishing a container image. - -## Controlled failure and recovery drill - -An unmerged `ci-failure-drill` pull request added a temporary JUnit assertion of -`assertTrue(false)`. The Maven job reported the exact failing class, method, and -assertion message. Because `container-build` has `needs: test`, its image build was -skipped. The required checks prevented the PR from merging into `master`. - -Recovery consisted of removing only the temporary test with `git rm`, committing, -and pushing the same branch. The PR then became green. It was closed without merging -and its remote and local branches were deleted. - -Use this diagnostic order for a real CI incident: - -```text -failed workflow -> failed job -> failed step -> first concrete error -> source/config fix -``` - -## GitHub Actions to GitLab CI mapping - -The implementation remains GitHub Actions; this mapping is for interviews and for a -future deliberate GitLab migration. - -| GitHub Actions | GitLab CI | Meaning | -|---|---|---| -| `.github/workflows/ci.yml` | `.gitlab-ci.yml` | Pipeline definition | -| `on: push` / `pull_request` | `rules:` / `only` / `except` | Pipeline trigger conditions | -| workflow job | GitLab job | Unit of work on a runner | -| `runs-on: ubuntu-24.04` | runner tags / executor selection | Runner environment | -| `needs: test` | `needs: [test]` | Dependency and ordering between jobs | -| `run:` | `script:` | Shell commands executed by a job | -| `uses:` | `image:`, `before_script`, included templates | Reusable implementation; no exact one-to-one equivalent | -| `services: redis` | `services:` | Temporary test dependency | -| `actions/upload-artifact` | `artifacts:` | Retained job output | -| `actions/setup-java` Maven cache | `cache:` | Reusable dependency cache | -| `GITHUB_TOKEN` | `CI_JOB_TOKEN` / protected CI variables | Scoped job authentication | -| GHCR | GitLab Container Registry | Remote OCI image registry | -| GitHub ruleset | protected branches + merge-request approval/pipeline rules | Merge policy | - -In an interview: “The concepts are portable. I used GitHub Actions to test, build, -scan, and publish an immutable image; GitLab CI expresses the same lifecycle through -stages/jobs, `rules`, `services`, artifacts, caches, and the GitLab registry. CI still -creates evidence and release artifacts; GitOps later deploys the version declared in -Git.” - -## Official references - -- [Building and testing Java with Maven](https://docs.github.com/en/actions/tutorials/build-and-test-code/java-with-maven) -- [Publishing Docker images](https://docs.github.com/en/actions/tutorials/publish-packages/publish-docker-images) -- [GitHub repository rulesets](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-rulesets/available-rules-for-rulesets) -- [Trivy GitHub Action](https://github.com/aquasecurity/trivy-action) diff --git a/docs/features.md b/docs/features.md new file mode 100644 index 0000000..1ec6d51 --- /dev/null +++ b/docs/features.md @@ -0,0 +1,505 @@ +# ReplyTrail feature guide + +This is the maintained, human-readable and AI-readable overview of ReplyTrail. +It explains what each feature does, who can use it, the important rules, and +where the exact implementation or release decision is recorded. + +**Last reviewed:** 2026-10-11 +**Product:** ReplyTrail +**Audience:** users, operators, contributors, and AI agents working in this repository + +## How to read this guide + +Each feature follows the same order: + +1. **Purpose** — the user problem the feature solves. +2. **Who can use it** — the actor and team boundary. +3. **How it works** — the normal workflow. +4. **Rules and limits** — behavior that must not be inferred differently. +5. **Status and sources** — what is implemented locally and what still needs release verification. + +“Implemented locally” means the repository contains the backend and UI behavior +and focused local checks have passed. It does not mean that PostgreSQL, +RabbitMQ, Redis, SMTP, restart recovery, or a deployed end-to-end journey has +been verified. The [product launch checklist](product-launch-readiness.md) is +the source of truth for release readiness. + +## Product in one sentence + +ReplyTrail turns an alert or request into a task, sends it through an ordered +response path, and records who responded, when the next person was notified, +and what happened afterward. + +## Core vocabulary + +| Term | Meaning | Do not confuse it with | +| --- | --- | --- | +| **Team** | A workspace containing members, tasks, response paths, webhooks, and escalation runs. | A user account. | +| **Task** | Reusable context describing the work that needs attention. | An escalation run. | +| **Response path** | A reusable ordered list of team-member recipients and timing. The API and Java code call this a **flow**. | The task being acted on. | +| **Response step** | One recipient and its timing inside a response path or saved run snapshot. | A whole response path. | +| **Escalation run** | One execution of a task through a response path. | The reusable task or path. | +| **Acknowledgement** | A recipient confirms that they accept responsibility. | Resolution of the underlying issue. | +| **Resolution** | The current recipient confirms that the issue is finished while a resolution window is open. | Acknowledgement. | +| **Activity event** | A saved user or system milestone in the run history. | Timer housekeeping or a raw stack trace. | + +The public product name is **ReplyTrail**. Existing `com.alertops` packages, +`ALERTOPS_*` settings, `alertops.*` storage keys, and the +`X-AlertOps-Webhook-Secret` header remain compatibility identifiers. Do not +replace those technical identifiers as part of an ordinary feature change. + +## End-to-end journey + +```text +Register and verify email + | + v +Create or select a team and add members + | + v +Create a task --------------+ + | | + v v +Create a response path Configure a webhook + | | + +--------+--------+ + v + Create an escalation run + | | + Start immediately Schedule for later + | | + +--------+--------+ + v + Send response steps by email + | + Acknowledge, resolve, or escalate now + | + View saved activity history +``` + +## Feature catalog + +| ID | Feature | Primary outcome | Current status | +| --- | --- | --- | --- | +| F-01 | Accounts and teams | A verified user works inside the correct team. | Implemented locally; membership-policy and deployed verification remain. | +| F-02 | Tasks | A request has reusable, searchable context. | Implemented locally; deployed verification remains. | +| F-03 | Response paths | A team defines ordered recipients and waits once, then reuses that path. | Implemented locally; deployed verification remains. | +| F-04 | Escalation runs | One task follows one saved path snapshot with an explicit lifecycle. | Implemented locally; PostgreSQL/deployed verification remains. | +| F-05 | Start and scheduling | A run starts now, starts later, or repeats daily/weekly. | Implemented locally; capacity and deployed verification remain. | +| F-06 | Email acknowledgement and resolution | A recipient can accept responsibility and, when enabled, resolve the issue. | Implemented locally; PostgreSQL/deployed verification remains. | +| F-07 | Escalate now | An eligible user can make the next response step due immediately. | Implemented locally; PostgreSQL/deployed verification remains. | +| F-08 | Webhook intake | An external system can create an idempotent task and run. | Implemented locally; PostgreSQL/Redis/deployed verification remains. | +| F-09 | Activity history | People can investigate the complete saved run timeline. | Implemented locally; PostgreSQL/deployed verification remains. | +| F-10 | Recovery and duplicate safety | Restart, retry, and duplicate delivery do not silently lose workflow state. | Implemented locally; infrastructure and deployed verification remain. | + +## F-01 — Accounts and teams + +### Purpose + +Keep users, recipients, tasks, paths, webhooks, and runs inside an explicit team +boundary. + +### Who can use it + +- A new user registers and verifies an email address before normal sign-in. +- A verified user can create or select a team and can belong to more than one team. +- Team owners and administrators can invite members. The invited email address + must be the verified address that accepts the invitation. +- The server, not the browser, is authoritative for membership and permissions. + +### How it works + +1. Register and follow the email verification link. +2. Sign in and select the team to work in. +3. Invite the people who may receive response steps. +4. Create tasks and response paths using members of that team. + +### Rules and limits + +Team-scoped reads and writes must not expose another team's data. A token that +contains a team identifier does not replace a live membership check. Email +action tokens are scoped to their specific run and recipient; they do not grant +access to the team's full history. + +### Status and sources + +The core account, team selection, invitation preview, and invitation acceptance +journeys are present locally. The remaining role-management policy is tracked +in the [team management task list](team-management-ui-task-list.md). See the +[authentication controller](../src/main/java/com/alertops/auth/controller/UserController.java) +and [team controller](../src/main/java/com/alertops/team/controller/TeamController.java). + +## F-02 — Tasks + +### Purpose + +Give every response run a clear description of what needs attention and enough +context for the recipient to act. + +### Who can use it + +Authenticated members of the selected team can create and view team tasks. + +### How it works + +1. Create a task with a name and description. +2. Set a source. Manually created tasks default to `Manual` when no source is supplied. +3. Optionally add priority, category, and an HTTP(S) reference URL. +4. Edit reusable task details later without changing the context already saved in a running run. + +### Rules and limits + +When a run starts, ReplyTrail snapshots the task context into its response +steps. Later edits to the reusable task do not rewrite history or change an +active run's saved context. A webhook must provide `eventId`, `taskName`, +`description`, and `source`; priority, category, and `referenceUrl` are optional. + +### Status and sources + +Task creation, editing, listing, and detail UI are implemented locally. See the +[task service](../src/main/java/com/alertops/task/service/TaskService.java), +[task controller](../src/main/java/com/alertops/task/controller/TaskController.java), +and the [webhook feature plan](webhook-escalation-implementation-plan.md). + +## F-03 — Response paths + +### Purpose + +Define the ordered people and timing used whenever a task needs a response. + +### Who can use it + +Authenticated members of the selected team can create, view, and configure +paths. Every recipient in a path must be a member of that team. + +### How it works + +1. Create a named response path. +2. Add one or more ordered response steps. +3. For each step, choose a team-member email recipient and a wait duration. +4. Optionally enable resolution timeout for the path and set a positive resolution timeout on every step. +5. Reorder, edit, or remove steps before starting a run. + +### Timing rules + +- The first response email is sent immediately when the run starts. +- The configured step duration is the acknowledgement wait after delivery is accepted. +- If nobody acknowledges, the next eligible step becomes due after that same wait. +- Resolution timeout is an optional path-level mode, not a separate run-wide timer. +- With resolution timeout enabled, acknowledgement pauses the next step and + starts the current recipient's resolution window. +- A started run keeps a runtime snapshot of path recipients and timing. Editing + the reusable path does not rewrite the active run. + +### Status and sources + +Path and step editing, ordering, timing, and runtime snapshots are implemented +locally. The exact timing decisions are binding in the +[resolution timeout plan](resolution-timeout-implementation-plan.md) and the +[flow controller](../src/main/java/com/alertops/flow/controller/FlowController.java). + +## F-04 — Escalation runs and lifecycle + +### Purpose + +Execute one task through one saved response path while making every lifecycle +transition explicit, auditable, and safe under concurrent requests. + +### Lifecycle states + +| State | Meaning | Terminal? | +| --- | --- | --- | +| `IDLE` | Created but not started or scheduled. | No | +| `SCHEDULED` | Waiting for its start time or an eligible start retry. | No | +| `OPEN` | Actively processing response steps. | No | +| `ACKNOWLEDGED` | A recipient owns an active resolution window. | No | +| `RESOLVED` | The current recipient resolved the issue within the resolution window. | Yes | +| `COMPLETED` | The run ended after acknowledgement or after all steps were exhausted. | Yes | +| `CANCELLED` | A scheduled run was cancelled before it started. | Yes | +| `START_FAILED` | All attempts to start a scheduled run failed. | Yes | + +### Allowed transitions + +```text +IDLE -> OPEN Start escalation +IDLE -> SCHEDULED Schedule later +SCHEDULED -> OPEN Due start or authenticated Start now +SCHEDULED -> SCHEDULED Reschedule or an allowed retry +SCHEDULED -> CANCELLED Cancel before start +SCHEDULED -> START_FAILED Start retries exhausted +OPEN -> ACKNOWLEDGED Acknowledgement with resolution enabled +OPEN -> COMPLETED Acknowledgement without resolution, or exhaustion +ACKNOWLEDGED -> RESOLVED Timely resolution +ACKNOWLEDGED -> OPEN Resolution timeout with a next step +ACKNOWLEDGED -> COMPLETED Resolution timeout on the final step +``` + +The server owns the state machine. There is no supported generic status-update +operation. The first valid database transition that commits wins; a stale +request returns a conflict and a stale timer callback becomes a no-op. Terminal +runs remain visible for history and cannot be reopened in v1. + +### Step outcomes + +Each saved response step has one of these statuses: + +`PENDING` → not yet due; `SCHEDULED` → waiting for its due time; `PAUSED` → +held during an acknowledgement resolution window; `SENDING` → delivery in +progress; `SENT` → SMTP accepted the message; `FAILED` → delivery failed after +the applicable retry behavior; `SKIPPED` → no longer eligible after a terminal +decision. + +`SENT` means that the SMTP server accepted the email. It does not prove that +the message reached the recipient's inbox. + +### Status and sources + +Lifecycle behavior, concurrency rules, API error mapping, and audit boundaries +are defined in the [launch checklist](product-launch-readiness.md#2-incident-lifecycle), +the [resolution timeout plan](resolution-timeout-implementation-plan.md), and +the [escalation service](../src/main/java/com/alertops/flow_execution_engine/service/EscalationService.java). + +## F-05 — Start, one-time scheduling, and recurring runs + +### Start choices + +- **Start immediately** creates the run and starts the same workflow used by a + scheduled run when it becomes due. +- **Schedule for later** requires a future local date/time and an IANA timezone, + such as `Asia/Kolkata`. +- **Start now** starts a scheduled run early and sends the first email immediately. + The first step's configured acknowledgement wait begins after delivery. + +ReplyTrail stores the resolved UTC instant and the submitted timezone. A +scheduled run can be rescheduled or cancelled only while it remains +`SCHEDULED`. Those operations race safely with a due start because the database +decides which transition commits first. + +### Recurring runs + +Daily and weekly recurrence is supported in the current local implementation. +The original escalation stores the recurrence settings. Each calendar +occurrence creates an independent child run with a different ID; child runs do +not repeat themselves, and unresolved earlier runs do not block later ones. +Stopping recurrence prevents future children while preserving existing runs. +After downtime, recovery creates only the latest missed occurrence. Monthly, +yearly, custom-interval, preview, and recurrence-editing features are outside +the current minimum scope. + +### Start failure behavior + +If every scheduled-start attempt fails, the run becomes `START_FAILED`. ReplyTrail +persists a pending notification obligation for the scheduling user and the team +owner or administrators. Startup recovery can deliver an obligation that was +still pending when the application stopped. + +### Status and sources + +The scheduling and recurrence decisions are recorded in the +[launch checklist](product-launch-readiness.md#1-scheduled-escalation-start), +the [recurring escalation plan](recurring-escalation-implementation-plan.md), +and the [scheduler](../src/main/java/com/alertops/flow_execution_engine/service/EscalationStartScheduler.java). +Local implementation is complete; Oracle capacity, PostgreSQL integration, and +deployed recovery remain open. + +## F-06 — Email acknowledgement and resolution + +### Purpose + +Let the current recipient respond from an email without requiring a full login +and, when the path enables it, keep ownership open until the issue is resolved. + +### How it works + +1. ReplyTrail sends an email containing a recipient-scoped action link. +2. Opening the link shows a read-only preview. Opening a link does not change state. +3. The recipient confirms acknowledgement explicitly. +4. With resolution timeout disabled, acknowledgement is terminal for the run and later unsent steps are skipped. +5. With resolution timeout enabled, acknowledgement changes the run to + `ACKNOWLEDGED` and displays the saved resolution deadline. +6. The current recipient or an authorized team member resolves before the + deadline. Resolution changes the run to `RESOLVED` and skips later steps. +7. If the deadline expires first, the next step becomes due immediately. If no + next step exists, the run completes as exhausted. + +### Security and timing rules + +Only the intended recipient's valid, unexpired token can confirm an email +action. Only its hash is stored. The backend rechecks recipient ownership, +expected step, current state, and deadline under the run lock. Repeating an +already accepted action returns the saved result without restarting a timer. + +### Status and sources + +See the [acknowledgement controller](../src/main/java/com/alertops/flow_execution_engine/controller/EscalationAcknowledgementController.java), +[resolution controller](../src/main/java/com/alertops/flow_execution_engine/controller/EscalationResolutionController.java), +and [launch requirement 3](product-launch-readiness.md#3-resolution-timeout-after-acknowledgement). + +## F-07 — Escalate now + +### Purpose + +Allow an eligible person to make the next response step due immediately when +waiting longer is not useful. + +### Who can use it + +- An authenticated member of the selected team can use it while the run is `OPEN` or `ACKNOWLEDGED`. +- The current recipient can use the recipient-scoped email action while their + acknowledgement or resolution window is still open. + +### How it works + +The user first sees a read-only preview of the next recipient, then explicitly +confirms. ReplyTrail advances the next eligible unsent step immediately. It +does not resend the previous email, skip unfinished steps, or let an old link +advance a later step after the run has already moved on. + +There is no action when the run is terminal or no eligible next step exists. +The action differs from **Start now**: Start now begins a scheduled run, while +Escalate now advances an already open or acknowledged run. + +### Status and sources + +The behavior is defined in [launch requirement 4](product-launch-readiness.md#4-escalate-now) +and implemented by the [manual action controller](../src/main/java/com/alertops/flow_execution_engine/controller/EscalationManualActionController.java). + +## F-08 — Webhook intake + +### Purpose + +Let an external monitoring or business system create a task and start its +configured response path without a signed-in browser session. + +### Configuration workflow + +1. An owner or administrator creates a webhook and chooses its default response path. +2. ReplyTrail shows the secret only when the webhook is created or its secret is rotated. +3. Disable the webhook or rotate its secret if it should no longer accept traffic. + +### Event workflow + +Send a JSON object to `POST /api/v1/webhooks/{webhookId}/events` with the +`X-AlertOps-Webhook-Secret` header. A new event creates one task and one run. +The event can include `flowId` to select another path in the same team; +otherwise the configured default path is used. + +Required fields are `eventId`, `taskName`, `description`, and `source`. +Optional fields are `priority`, `category`, and `referenceUrl`. Other JSON +fields are retained as event context. + +### Idempotency and limits + +- A new event returns HTTP `202` and includes the created task and run IDs. +- Repeating the same `eventId` with the same payload returns HTTP `200` and the original records. +- Reusing an `eventId` with different data returns HTTP `409` and creates nothing new. +- The server stores only a hash of the webhook secret. +- The default serialized payload limit is 64 KiB and the default rate limit is 120 requests per webhook per minute. +- Redis shares the rate limit across application instances when available; the fallback counter is per instance. + +### Status and sources + +See the [webhook plan](webhook-escalation-implementation-plan.md), +[webhook configuration controller](../src/main/java/com/alertops/webhook/controller/WebhookConfigurationController.java), +[event controller](../src/main/java/com/alertops/webhook/controller/WebhookEventController.java), +and the [README request example](../README.md#trigger-a-task-through-a-webhook). + +## F-09 — Activity history + +### Purpose + +Show what happened to a run in plain language after it starts, finishes, or +encounters a failure. + +### What is recorded + +The saved timeline includes creation, scheduling, rescheduling, start, early +start, cancellation, notification acceptance, notification failure and retry, +acknowledgement, resolution deadline, resolution timeout, Escalate now, +resolution, exhaustion, completion, and scheduled-start failure. + +Each visible event includes its time, actor or system source, and relevant step +or recipient context. Repeated delivery failures are grouped in the default +view with expandable attempt details. Timer housekeeping, secrets, raw tokens, +and stack traces are not exposed as user history. + +History is append-only and remains available for terminal runs. Viewing history +does not change the run lifecycle. + +### Status and sources + +The [history controller](../src/main/java/com/alertops/flow_execution_engine/controller/EscalationHistoryController.java) +and [generic audit service](../src/main/java/com/alertops/audit/service/AuditService.java) +implement the local behavior. The [timeline requirement](product-launch-readiness.md#5-incident-activity-timeline) +defines the event and safety rules. + +## F-10 — Recovery and duplicate safety + +### Durable and temporary responsibilities + +- **PostgreSQL** is the source of truth for tasks, paths, runs, steps, statuses, + timestamps, audit events, schedules, and pending notification obligations. +- **In-memory timers** are wake-up handles only. A callback reloads current state + before acting, so cancellation, replacement, restart recovery, and duplicate + callbacks are safe. +- **RabbitMQ** carries ready work and consumer retries. It is not the source of + truth for delayed workflow timing. +- **Redis** stores short-lived workflow intents and shares webhook rate limiting. + +### Reliability behavior + +Lifecycle transitions, audit events, and durable publication obligations are +committed before timers or message publication are changed. Duplicate messages, +stale callbacks, repeated actions, and restart recovery recheck saved state and +must not create a second winning transition. Delivery may be at least once: an +SMTP-accepted message can be sent again if the application stops before it +saves the `SENT` result. + +### Status and sources + +See the [in-memory timer plan](in-memory-timer-implementation-plan.md), the +[launch reliability checks](product-launch-readiness.md#make-escalation-execution-safe-under-duplicate-messages-and-restarts), +and the [scheduler implementation](../src/main/java/com/alertops/flow_execution_engine/service/EscalationStartScheduler.java). +Infrastructure integration and deployed restart checks remain open. + +## API route map + +The UI uses these API areas. Routes below are stable technical entry points; +the user-facing feature names above are the preferred product vocabulary. + +| API area | Base route | Main responsibility | +| --- | --- | --- | +| Authentication | `/api/v1/auth` | Registration, verification, and login. | +| Teams | `/api/v1/team` | Create, select, list, invite, and inspect team members. | +| Tasks | `/api/v1/task` | Create, list, read, edit, and delete task context. | +| Response paths | `/api/v1/flow` | Create paths, manage steps, timing, and order. | +| Escalation runs | `/api/v1/escalation` | Create, list, start, schedule, reschedule, cancel, repeat, and inspect runs. | +| Acknowledgement | `/api/v1/escalation/acknowledgement` | Preview and confirm recipient acknowledgement. | +| Resolution | `/api/v1/escalation/{id}/resolve` and `/api/v1/escalation/resolution/confirm` | Resolve as a team member or recipient. | +| Escalate now | `/api/v1/escalation/*/escalate-now` | Preview and confirm the next-step action. | +| Activity history | `/api/v1/escalation/{id}/history` | Read same-team paginated history. | +| Webhook configuration | `/api/v1/team/webhooks` | Create, list, rotate, disable, and inspect webhook events. | +| Webhook intake | `/api/v1/webhooks/{id}/events` | Accept authenticated external events. | + +For API questions, read the relevant controller and service together with the +linked feature plan. Do not infer permissions, lifecycle transitions, or +idempotency behavior from route names alone. + +## Current boundaries and deferred work + +The current product sends SMTP email. SMS, phone calls, push notifications, and +native Slack or Teams delivery are not implemented. Response paths use named +team members; rotating on-call calendars are not implemented. **Send test +escalation** is deferred until initial users provide feedback; its behavior is +not yet agreed. Production hosting, verified sender/domain setup, and deployed +end-to-end validation are still open in the [launch checklist](product-launch-readiness.md#release-check). + +## Canonical references + +- [Product launch checklist](product-launch-readiness.md) — agreed behavior, release status, and verification gaps. +- [Webhook escalation plan](webhook-escalation-implementation-plan.md) — webhook contract, task/run creation, and idempotency. +- [Resolution timeout plan](resolution-timeout-implementation-plan.md) — lifecycle, acknowledgement, resolution, and manual action rules. +- [Recurring escalation plan](recurring-escalation-implementation-plan.md) — daily/weekly recurrence and restart behavior. +- [UI implementation plan](ui-implementation-plan.md) — browser routes and frontend/backend deployment boundary. +- [Deployment guide](deployment/README.md) — hosting, infrastructure, and operational readiness. +- [Feature backlog](feature-backlog.md) — deferred feature decisions. diff --git a/docs/product-launch-readiness.md b/docs/product-launch-readiness.md index c3fef5e..d40d38c 100644 --- a/docs/product-launch-readiness.md +++ b/docs/product-launch-readiness.md @@ -19,7 +19,7 @@ Review these requirements in order. A checked **Requirements agreed** box means - Reject a scheduled time that is not in the future. Default the UI timezone to the browser timezone and allow the user to change it. - Allow reschedule and cancel only while the run is still `SCHEDULED`, enforced atomically so a start cannot race with either action. - Treat **Start immediately** as create-and-start, using the same start use case invoked when a scheduled time becomes due. - - Allow **Start now** to start a scheduled run early through the existing manual start API. Commit `SCHEDULED → OPEN` and step creation before cancelling the in-memory start timer. The first step retains its configured delay. This extension is implemented locally; PostgreSQL/deployed verification remains pending. See the [shared lifecycle plan](resolution-timeout-implementation-plan.md#how-start-now-will-work). + - Allow **Start now** to start a scheduled run early through the existing manual start API. Commit `SCHEDULED → OPEN` and step creation before cancelling the in-memory start timer. Send the first email immediately; its configured acknowledgement wait begins after delivery. This extension is implemented locally; PostgreSQL/deployed verification remains pending. See the [shared lifecycle plan](resolution-timeout-implementation-plan.md#how-start-now-will-work). - Add a terminal `CANCELLED` state for a cancelled scheduled run, consistent with the incident lifecycle in Requirement 2. - When the retry limit is exhausted, change the escalation to `START_FAILED` and atomically persist a pending failure-notification obligation in PostgreSQL. Do not rely on the scheduler callback or an in-memory queue to remember that notification. - Notify the user who scheduled the escalation and the team owner or administrators. If the application stops after recording `START_FAILED` but before notification delivery, startup recovery must find the pending notification and send it when the application is available again. @@ -117,7 +117,7 @@ Review these requirements in order. A checked **Requirements agreed** box means - [x] **Requirements agreed** - [x] **Implementation complete locally** - [ ] **Implemented and verified in PostgreSQL/deployed flow** -- **Requested behavior:** An authenticated team member, or an eligible notification recipient using a valid email token, can end the wait for the next notification and make that step due immediately, following saved step order. This differs from **Start now**, which begins a not-yet-started run and keeps normal node delays. +- **Requested behavior:** An authenticated team member, or an eligible notification recipient using a valid email token, can end the wait for the next notification and make that step due immediately, following saved step order. This differs from **Start now**, which begins a not-yet-started run with an immediate first email and its normal acknowledgement window. - **Agreed decisions:** Allow the action when nobody has acknowledged (`OPEN`) and while somebody is working on resolution (`ACKNOWLEDGED`). In `OPEN`, the run stays open and the next eligible unsent step becomes due now. In `ACKNOWLEDGED`, confirm the user's intent, end the active resolution wait, return to `OPEN`, and make the next eligible paused step due now. Do not resend the previous notification or skip unfinished steps. No next eligible step or a terminal run means the action is unavailable. Sends already in progress finish normally. - **Email action:** Add an **Escalate now** button opening a read-only preview/confirmation page that shows the current next recipient. Allow explicit POST confirmation without login using a hashed recipient token bound to the source step and validated against the expected target/current state. Before acknowledgement, only the current recipient within their acknowledgement window may act; after acknowledgement, only the current acknowledging recipient within their resolution window may act. At expiry or advancement, new actions are unavailable but the scoped link stays viewable. Opening/scanning never changes state. Reject invalid/foreign tokens, terminal runs, and no-next-step actions. Recheck ownership, deadline, and target under the run lock; repeated Step 1 confirmation cannot advance Step 3 after advancing Step 2. Audit the recipient email and affected steps; reuse the UI lifecycle operation. See [email source rules](resolution-timeout-implementation-plan.md#escalate-now-from-email). - **State and reliability rules:** Reuse the step-status enum from Requirement 3. Bind manual requests to the expected step so duplicate/stale clicks cannot advance another step. Save the state change, due time, publication obligation, and actor audit together; change timers/publish only after commit. Manual escalation, timeout, and resolution must have one winner for the same acknowledgement/step. Reuse durable immediate-delivery scheduling with resolution timeout expiry. @@ -206,6 +206,13 @@ The scheduling integration tests cover publication after commit, no publication ## Release check +- **Deployment decision (2026-10-11):** Version one uses Docker Compose for the + backend, PostgreSQL, RabbitMQ, and Redis, plus UI hosting and HTTPS. Initial + testing targets a minimal Oracle A1 VM in Mumbai; production hosting is pending + confirmation. Kubernetes is deferred and its learning materials are preserved + separately. See the [deployment guide](deployment/README.md). +- [ ] Prepare and verify the production VM/Compose setup, private dependency + ports, HTTPS, external database backups, and recovery after a host restart. - [x] Select and implement the public product identity using the [product rebranding plan](product-rebranding-plan.md). ReplyTrail is implemented locally; domain ownership, sender verification, and deployed rollout remain pending. Retain existing integration and storage identifiers during the initial rebrand. - [x] All four critical fixes above are complete locally. - [ ] A scheduled escalation that exhausts its start retries notifies the responsible user and team administrators. The pending notification survives an application crash and is recovered after restart. diff --git a/docs/product-rebranding-plan.md b/docs/product-rebranding-plan.md index 20634d1..75bdf82 100644 --- a/docs/product-rebranding-plan.md +++ b/docs/product-rebranding-plan.md @@ -65,7 +65,7 @@ trademark clearance remain open before any public launch. | Outgoing email | [Notification](../src/main/java/com/alertops/messaging/Notification.java), [verification mailer](../src/main/java/com/alertops/messaging/EmailVerificationMailer.java), [invitation mailer](../src/main/java/com/alertops/messaging/TeamInvitationMailer.java) | Update subjects, HTML/plain text, headers/footers, and sender display name; preserve action behavior | | External webhook contract | [Event controller](../src/main/java/com/alertops/webhook/controller/WebhookEventController.java), [CORS](../src/main/java/com/alertops/security/SecurityConfig.java), [API examples](../README.md) | Keep `X-AlertOps-Webhook-Secret`, routes, payloads, and secret handling compatible | | Configuration and durable runtime identity | [Properties](../src/main/resources/application.properties), [environment example](../.env.example), [RabbitMQ configuration](../src/main/java/com/alertops/messaging/RabbitMqConfig.java), [intent cache](../src/main/java/com/alertops/caching/IntentCache.java), [webhook service](../src/main/java/com/alertops/webhook/service/WebhookService.java) | Retain `ALERTOPS_*`, `alertops.*`, queue/exchange/routing names, and Redis prefixes | -| Build and deployment identity | [Maven](../pom.xml), [Dockerfile](../Dockerfile), [Compose](../docker-compose.yml), [Kubernetes base](../k8s/base/), [CI](../.github/workflows/ci.yml) | Update human-readable descriptions; retain coordinates, `com.alertops` packages, resource names/selectors, volumes, and image paths initially | +| Build and deployment identity | [Maven](../pom.xml), [Dockerfile](../Dockerfile), [Compose](../docker-compose.yml), [deployment guide](deployment/README.md), [CI](../.github/workflows/ci.yml) | Update human-readable descriptions; retain coordinates, `com.alertops` packages, container names, volumes, and image paths initially. Kubernetes materials are preserved separately for later use. | | Current documentation | [README](../README.md), [UI README](../ui/README.md), [launch checklist](product-launch-readiness.md), [deployment guide](deployment/README.md), [agent instructions](../AGENTS.md) | Update current product references and explain retained compatibility identifiers | Several pages repeat the split wordmark markup. The initial implementation updates diff --git a/k8s/base/alertops-deployment.yaml b/k8s/base/alertops-deployment.yaml deleted file mode 100644 index 362ee78..0000000 --- a/k8s/base/alertops-deployment.yaml +++ /dev/null @@ -1,80 +0,0 @@ -apiVersion: apps/v1 -kind: Deployment -metadata: - name: alertops - namespace: alertops - labels: - app.kubernetes.io/name: alertops - app.kubernetes.io/component: backend - app.kubernetes.io/part-of: alertops -spec: - replicas: 3 - revisionHistoryLimit: 5 - minReadySeconds: 5 - progressDeadlineSeconds: 300 - strategy: - type: RollingUpdate - rollingUpdate: - maxUnavailable: 0 - maxSurge: 1 - selector: - matchLabels: - app.kubernetes.io/name: alertops - template: - metadata: - labels: - app.kubernetes.io/name: alertops - app.kubernetes.io/component: backend - app.kubernetes.io/part-of: alertops - spec: - automountServiceAccountToken: false - terminationGracePeriodSeconds: 40 - securityContext: - seccompProfile: - type: RuntimeDefault - containers: - - name: alertops - image: alertops:3514a9d28585 - imagePullPolicy: IfNotPresent - ports: - - name: http - containerPort: 8096 - protocol: TCP - envFrom: - - configMapRef: - name: alertops-config - - secretRef: - name: alertops-secrets - resources: - requests: - cpu: 250m - memory: 512Mi - limits: - cpu: "1" - memory: 1Gi - securityContext: - allowPrivilegeEscalation: false - capabilities: - drop: - - ALL - startupProbe: - httpGet: - path: /actuator/health/liveness - port: http - periodSeconds: 5 - timeoutSeconds: 3 - failureThreshold: 30 - readinessProbe: - httpGet: - path: /actuator/health - port: http - periodSeconds: 5 - timeoutSeconds: 3 - failureThreshold: 3 - livenessProbe: - httpGet: - path: /actuator/health/liveness - port: http - periodSeconds: 10 - timeoutSeconds: 3 - failureThreshold: 6 \ No newline at end of file diff --git a/k8s/base/alertops-service.yaml b/k8s/base/alertops-service.yaml deleted file mode 100644 index e272e95..0000000 --- a/k8s/base/alertops-service.yaml +++ /dev/null @@ -1,18 +0,0 @@ -apiVersion: v1 -kind: Service -metadata: - name: alertops - namespace: alertops - labels: - app.kubernetes.io/name: alertops - app.kubernetes.io/component: backend - app.kubernetes.io/part-of: alertops -spec: - type: ClusterIP - selector: - app.kubernetes.io/name: alertops - ports: - - name: http - protocol: TCP - port: 8096 - targetPort: http \ No newline at end of file diff --git a/k8s/base/configmap.yaml b/k8s/base/configmap.yaml deleted file mode 100644 index 4e3ca92..0000000 --- a/k8s/base/configmap.yaml +++ /dev/null @@ -1,25 +0,0 @@ -apiVersion: v1 -kind: ConfigMap -metadata: - name: alertops-config - namespace: alertops - labels: - app.kubernetes.io/part-of: alertops -data: - SPRING_PROFILES_ACTIVE: "prod" - SERVER_PORT: "8096" - - POSTGRES_DB: "alert_ops" - SPRING_DATASOURCE_URL: "jdbc:postgresql://postgres:5432/alert_ops" - SPRING_DATASOURCE_USERNAME: "alertops" - - SPRING_RABBITMQ_HOST: "rabbitmq" - SPRING_RABBITMQ_PORT: "5672" - SPRING_RABBITMQ_USERNAME: "alertops" - - SPRING_DATA_REDIS_HOST: "redis" - SPRING_DATA_REDIS_PORT: "6379" - SPRING_DATA_REDIS_CONNECT_TIMEOUT: "2s" - SPRING_DATA_REDIS_TIMEOUT: "2s" - - INTENT_CACHE_TTL: "10m" \ No newline at end of file diff --git a/k8s/base/namespace.yaml b/k8s/base/namespace.yaml deleted file mode 100644 index db146bc..0000000 --- a/k8s/base/namespace.yaml +++ /dev/null @@ -1,6 +0,0 @@ -apiVersion: v1 -kind: Namespace -metadata: - name: alertops - labels: - app.kubernetes.io/part-of: alertops \ No newline at end of file diff --git a/k8s/base/postgres-pvc.yaml b/k8s/base/postgres-pvc.yaml deleted file mode 100644 index c6a54b7..0000000 --- a/k8s/base/postgres-pvc.yaml +++ /dev/null @@ -1,15 +0,0 @@ -apiVersion: v1 -kind: PersistentVolumeClaim -metadata: - name: postgres-data - namespace: alertops - labels: - app.kubernetes.io/name: postgres - app.kubernetes.io/part-of: alertops -spec: - accessModes: - - ReadWriteOnce - storageClassName: standard - resources: - requests: - storage: 1Gi \ No newline at end of file diff --git a/k8s/base/postgres-service.yaml b/k8s/base/postgres-service.yaml deleted file mode 100644 index 10e7f8c..0000000 --- a/k8s/base/postgres-service.yaml +++ /dev/null @@ -1,17 +0,0 @@ -apiVersion: v1 -kind: Service -metadata: - name: postgres - namespace: alertops - labels: - app.kubernetes.io/name: postgres - app.kubernetes.io/part-of: alertops -spec: - type: ClusterIP - selector: - app.kubernetes.io/name: postgres - ports: - - name: postgres - protocol: TCP - port: 5432 - targetPort: 5432 \ No newline at end of file diff --git a/k8s/base/postgres-statefulset.yaml b/k8s/base/postgres-statefulset.yaml deleted file mode 100644 index 4b4b7bf..0000000 --- a/k8s/base/postgres-statefulset.yaml +++ /dev/null @@ -1,86 +0,0 @@ -apiVersion: apps/v1 -kind: StatefulSet -metadata: - name: postgres - namespace: alertops - labels: - app.kubernetes.io/name: postgres - app.kubernetes.io/part-of: alertops -spec: - serviceName: postgres - replicas: 1 - selector: - matchLabels: - app.kubernetes.io/name: postgres - template: - metadata: - labels: - app.kubernetes.io/name: postgres - app.kubernetes.io/part-of: alertops - spec: - terminationGracePeriodSeconds: 30 - containers: - - name: postgres - image: postgres:16.14-alpine@sha256:57c72fd2a128e416c7fcc499958864df5301e940bca0a56f58fddf30ffc07777 - imagePullPolicy: IfNotPresent - ports: - - name: postgres - containerPort: 5432 - protocol: TCP - env: - - name: POSTGRES_DB - valueFrom: - configMapKeyRef: - name: alertops-config - key: POSTGRES_DB - - name: POSTGRES_USER - valueFrom: - configMapKeyRef: - name: alertops-config - key: SPRING_DATASOURCE_USERNAME - - name: POSTGRES_PASSWORD - valueFrom: - secretKeyRef: - name: alertops-secrets - key: SPRING_DATASOURCE_PASSWORD - resources: - requests: - cpu: 250m - memory: 256Mi - limits: - cpu: "1" - memory: 512Mi - startupProbe: - exec: - command: - - sh - - -c - - pg_isready -U "$POSTGRES_USER" -d "$POSTGRES_DB" - periodSeconds: 2 - timeoutSeconds: 2 - failureThreshold: 30 - readinessProbe: - exec: - command: - - sh - - -c - - pg_isready -U "$POSTGRES_USER" -d "$POSTGRES_DB" - periodSeconds: 5 - timeoutSeconds: 2 - failureThreshold: 3 - livenessProbe: - exec: - command: - - sh - - -c - - pg_isready -U "$POSTGRES_USER" -d "$POSTGRES_DB" - periodSeconds: 10 - timeoutSeconds: 3 - failureThreshold: 6 - volumeMounts: - - name: postgres-data - mountPath: /var/lib/postgresql/data - volumes: - - name: postgres-data - persistentVolumeClaim: - claimName: postgres-data \ No newline at end of file diff --git a/k8s/base/rabbitmq-pvc.yaml b/k8s/base/rabbitmq-pvc.yaml deleted file mode 100644 index 4cb10ce..0000000 --- a/k8s/base/rabbitmq-pvc.yaml +++ /dev/null @@ -1,15 +0,0 @@ -apiVersion: v1 -kind: PersistentVolumeClaim -metadata: - name: rabbitmq-data - namespace: alertops - labels: - app.kubernetes.io/name: rabbitmq - app.kubernetes.io/part-of: alertops -spec: - accessModes: - - ReadWriteOnce - storageClassName: standard - resources: - requests: - storage: 1Gi \ No newline at end of file diff --git a/k8s/base/rabbitmq-service.yaml b/k8s/base/rabbitmq-service.yaml deleted file mode 100644 index 8fa9b67..0000000 --- a/k8s/base/rabbitmq-service.yaml +++ /dev/null @@ -1,21 +0,0 @@ -apiVersion: v1 -kind: Service -metadata: - name: rabbitmq - namespace: alertops - labels: - app.kubernetes.io/name: rabbitmq - app.kubernetes.io/part-of: alertops -spec: - type: ClusterIP - selector: - app.kubernetes.io/name: rabbitmq - ports: - - name: amqp - protocol: TCP - port: 5672 - targetPort: 5672 - - name: management - protocol: TCP - port: 15672 - targetPort: 15672 \ No newline at end of file diff --git a/k8s/base/rabbitmq-statefulset.yaml b/k8s/base/rabbitmq-statefulset.yaml deleted file mode 100644 index 736618a..0000000 --- a/k8s/base/rabbitmq-statefulset.yaml +++ /dev/null @@ -1,81 +0,0 @@ -apiVersion: apps/v1 -kind: StatefulSet -metadata: - name: rabbitmq - namespace: alertops - labels: - app.kubernetes.io/name: rabbitmq - app.kubernetes.io/part-of: alertops -spec: - serviceName: rabbitmq - replicas: 1 - selector: - matchLabels: - app.kubernetes.io/name: rabbitmq - template: - metadata: - labels: - app.kubernetes.io/name: rabbitmq - app.kubernetes.io/part-of: alertops - spec: - terminationGracePeriodSeconds: 60 - containers: - - name: rabbitmq - image: rabbitmq:3.13.7-management@sha256:e582c0bc7766f3342496d8485efb5a1df782b5ce3886ad017e2eaae442311f69 - imagePullPolicy: IfNotPresent - ports: - - name: amqp - containerPort: 5672 - protocol: TCP - - name: management - containerPort: 15672 - protocol: TCP - env: - - name: RABBITMQ_DEFAULT_USER - valueFrom: - configMapKeyRef: - name: alertops-config - key: SPRING_RABBITMQ_USERNAME - - name: RABBITMQ_DEFAULT_PASS - valueFrom: - secretKeyRef: - name: alertops-secrets - key: SPRING_RABBITMQ_PASSWORD - resources: - requests: - cpu: 250m - memory: 256Mi - limits: - cpu: "1" - memory: 512Mi - startupProbe: - tcpSocket: - port: amqp - periodSeconds: 5 - timeoutSeconds: 5 - failureThreshold: 30 - readinessProbe: - exec: - command: - - rabbitmq-diagnostics - - -q - - check_running - periodSeconds: 5 - timeoutSeconds: 5 - failureThreshold: 3 - livenessProbe: - exec: - command: - - rabbitmq-diagnostics - - -q - - ping - periodSeconds: 10 - timeoutSeconds: 5 - failureThreshold: 6 - volumeMounts: - - name: rabbitmq-data - mountPath: /var/lib/rabbitmq - volumes: - - name: rabbitmq-data - persistentVolumeClaim: - claimName: rabbitmq-data diff --git a/k8s/base/redis-pvc.yaml b/k8s/base/redis-pvc.yaml deleted file mode 100644 index 4584b4f..0000000 --- a/k8s/base/redis-pvc.yaml +++ /dev/null @@ -1,15 +0,0 @@ -apiVersion: v1 -kind: PersistentVolumeClaim -metadata: - name: redis-data - namespace: alertops - labels: - app.kubernetes.io/name: redis - app.kubernetes.io/part-of: alertops -spec: - accessModes: - - ReadWriteOnce - storageClassName: standard - resources: - requests: - storage: 1Gi \ No newline at end of file diff --git a/k8s/base/redis-service.yaml b/k8s/base/redis-service.yaml deleted file mode 100644 index 5744e2d..0000000 --- a/k8s/base/redis-service.yaml +++ /dev/null @@ -1,17 +0,0 @@ -apiVersion: v1 -kind: Service -metadata: - name: redis - namespace: alertops - labels: - app.kubernetes.io/name: redis - app.kubernetes.io/part-of: alertops -spec: - type: ClusterIP - selector: - app.kubernetes.io/name: redis - ports: - - name: redis - protocol: TCP - port: 6379 - targetPort: 6379 \ No newline at end of file diff --git a/k8s/base/redis-statefulset.yaml b/k8s/base/redis-statefulset.yaml deleted file mode 100644 index d40381e..0000000 --- a/k8s/base/redis-statefulset.yaml +++ /dev/null @@ -1,88 +0,0 @@ -apiVersion: apps/v1 -kind: StatefulSet -metadata: - name: redis - namespace: alertops - labels: - app.kubernetes.io/name: redis - app.kubernetes.io/part-of: alertops -spec: - serviceName: redis - replicas: 1 - selector: - matchLabels: - app.kubernetes.io/name: redis - template: - metadata: - labels: - app.kubernetes.io/name: redis - app.kubernetes.io/part-of: alertops - spec: - terminationGracePeriodSeconds: 30 - containers: - - name: redis - image: redis:7.4.9-alpine@sha256:6ab0b6e7381779332f97b8ca76193e45b0756f38d4c0dcda72dbb3c32061ab99 - imagePullPolicy: IfNotPresent - command: - - sh - - -c - - | - umask 077 - printf 'appendonly yes\nappendfsync everysec\nrequirepass %s\n' "$REDIS_PASSWORD" > /tmp/redis.conf - exec redis-server /tmp/redis.conf - ports: - - name: redis - containerPort: 6379 - protocol: TCP - env: - - name: REDIS_PASSWORD - valueFrom: - secretKeyRef: - name: alertops-secrets - key: SPRING_DATA_REDIS_PASSWORD - - name: REDISCLI_AUTH - valueFrom: - secretKeyRef: - name: alertops-secrets - key: SPRING_DATA_REDIS_PASSWORD - resources: - requests: - cpu: 100m - memory: 64Mi - limits: - cpu: 500m - memory: 256Mi - startupProbe: - exec: - command: - - sh - - -c - - redis-cli ping | grep -q '^PONG$' - periodSeconds: 2 - timeoutSeconds: 2 - failureThreshold: 30 - readinessProbe: - exec: - command: - - sh - - -c - - redis-cli ping | grep -q '^PONG$' - periodSeconds: 5 - timeoutSeconds: 2 - failureThreshold: 3 - livenessProbe: - exec: - command: - - sh - - -c - - redis-cli ping | grep -q '^PONG$' - periodSeconds: 10 - timeoutSeconds: 2 - failureThreshold: 6 - volumeMounts: - - name: redis-data - mountPath: /data - volumes: - - name: redis-data - persistentVolumeClaim: - claimName: redis-data \ No newline at end of file diff --git a/ui/src/index.css b/ui/src/index.css index 9abc70d..1f33606 100644 --- a/ui/src/index.css +++ b/ui/src/index.css @@ -2220,3 +2220,67 @@ html[data-theme="dark"] .flow-detail-page .path-settings-status { border-color: @media (max-width: 600px) { .flow-detail-page .path-route-heading-tools { width: 100%; justify-content: space-between; margin-left: 0; } } + +/* Gives visitors a readable feature guide before they enter the workspace. */ +.public-nav-link { color: #d0d5dd; font-size: 13px; font-weight: 750; text-decoration: none; } +.public-nav-link:hover { color: #8fe9ba; } +.public-features { position: relative; overflow: hidden; padding: 96px max(calc((100vw - 1190px)/2), 6vw) 104px; color: #eef4fb; background: radial-gradient(circle at 15% 18%, rgba(143,233,186,.08), transparent 25%), #0d1524; } +.feature-guide-heading { display: flex; align-items: end; justify-content: space-between; gap: 54px; max-width: 1080px; } +.feature-guide-eyebrow { color: #8fe9ba; } +.feature-guide-heading h2 { margin: 12px 0 0; color: #eef4fb; font: 700 clamp(34px, 4vw, 49px)/1.08 var(--font-sans); letter-spacing: -.05em; } +.feature-guide-heading h2 em { color: #8fe9ba; font-style: normal; font-weight: 400; } +.feature-guide-heading > p { max-width: 420px; margin: 0 0 3px; color: #a5b2c2; font-size: 16px; line-height: 1.7; } +.feature-guide-grid { display: grid; grid-template-columns: repeat(4, minmax(0, 1fr)); gap: 14px; margin-top: 48px; } +.feature-guide-card { display: flex; min-height: 252px; flex-direction: column; gap: 22px; padding: 21px 20px 19px; border: 1px solid #2a3a50; border-radius: 15px; background: linear-gradient(145deg, #111c2e, #152238); box-shadow: 0 12px 28px rgba(0,0,0,.12); transition: transform .18s ease, border-color .18s ease, box-shadow .18s ease; } +.feature-guide-card:hover { transform: translateY(-3px); border-color: #526b9a; box-shadow: 0 18px 36px rgba(0,0,0,.2); } +.feature-guide-card-top { display: flex; align-items: center; justify-content: space-between; gap: 12px; } +.feature-guide-index { color: #8fe9ba; font-size: 10px; font-weight: 850; letter-spacing: .1em; } +.feature-guide-icon { display: grid; width: 38px; height: 38px; place-items: center; border: 1px solid #315743; border-radius: 12px; color: #a8e8bd; background: #173429; font-size: 17px; font-weight: 800; } +.feature-guide-card:nth-child(3n + 2) .feature-guide-icon { border-color: #405583; color: #b8c6ff; background: #202d4c; } +.feature-guide-card:nth-child(3n + 3) .feature-guide-icon { border-color: #5a4a2a; color: #f0cf8b; background: #2b2415; } +.feature-guide-card-copy { display: flex; flex: 1; flex-direction: column; gap: 9px; } +.feature-guide-card-copy h3 { margin: 0; color: #eef4fb; font: 650 16px/1.3 var(--font-sans); letter-spacing: -.015em; } +.feature-guide-card-copy p { margin: 0; color: #a5b2c2; font-size: 13px; line-height: 1.65; } +.feature-guide-card-detail { display: flex; align-items: end; justify-content: space-between; gap: 12px; padding-top: 13px; border-top: 1px solid #2a3a50; } +.feature-guide-card-detail span { color: #7f8da2; font-size: 9px; font-weight: 850; letter-spacing: .08em; } +.feature-guide-card-detail strong { color: #dce6f2; font-size: 11px; text-align: right; } +.feature-guide-boundary { display: flex; align-items: baseline; gap: 13px; max-width: 900px; margin-top: 37px; color: #a5b2c2; font-size: 13px; line-height: 1.6; } +.feature-guide-boundary span { flex: 0 0 auto; color: #dce6f2; font-size: 10px; font-weight: 850; letter-spacing: .1em; } +.feature-guide-boundary p { margin: 0; } + +html:not([data-theme="dark"]) .public-nav-link { color: #d0d5dd; } +html:not([data-theme="dark"]) .public-nav-link:hover { color: #a6f4c5; } +html:not([data-theme="dark"]) .public-features { color: #172033; background: radial-gradient(circle at 15% 18%, rgba(85,115,243,.09), transparent 25%), #eef3fb; } +html:not([data-theme="dark"]) .feature-guide-eyebrow { color: #345cf5; } +html:not([data-theme="dark"]) .feature-guide-heading h2 { color: #172033; } +html:not([data-theme="dark"]) .feature-guide-heading h2 em { color: #345cf5; } +html:not([data-theme="dark"]) .feature-guide-heading > p { color: #526074; } +html:not([data-theme="dark"]) .feature-guide-card { border-color: #cbd5e1; background: linear-gradient(145deg, #fff, #f7faff); box-shadow: 0 9px 22px rgba(24,39,64,.07); } +html:not([data-theme="dark"]) .feature-guide-card:hover { border-color: #9eb1f2; box-shadow: 0 16px 32px rgba(49,86,212,.13); } +html:not([data-theme="dark"]) .feature-guide-index { color: #345cf5; } +html:not([data-theme="dark"]) .feature-guide-icon { border-color: #c7d7f8; color: #087443; background: #ecfdf3; } +html:not([data-theme="dark"]) .feature-guide-card:nth-child(3n + 2) .feature-guide-icon { border-color: #dbe3ff; color: #345cf5; background: #eff2ff; } +html:not([data-theme="dark"]) .feature-guide-card:nth-child(3n + 3) .feature-guide-icon { border-color: #f6e2b4; color: #9a6700; background: #fff8e7; } +html:not([data-theme="dark"]) .feature-guide-card-copy h3 { color: #172033; } +html:not([data-theme="dark"]) .feature-guide-card-copy p { color: #526074; } +html:not([data-theme="dark"]) .feature-guide-card-detail { border-color: #e4e7ec; } +html:not([data-theme="dark"]) .feature-guide-card-detail span { color: #667085; } +html:not([data-theme="dark"]) .feature-guide-card-detail strong { color: #344054; } +html:not([data-theme="dark"]) .feature-guide-boundary { color: #526074; } +html:not([data-theme="dark"]) .feature-guide-boundary span { color: #344054; } + +@media (max-width: 880px) { + .feature-guide-heading { align-items: flex-start; flex-direction: column; gap: 22px; } + .feature-guide-heading > p { max-width: 680px; } + .feature-guide-grid { grid-template-columns: repeat(2, minmax(0, 1fr)); } +} + +@media (max-width: 600px) { + .public-nav-link { display: none; } + .public-features { padding: 62px 20px 68px; } + .feature-guide-heading h2 { font-size: 37px; } + .feature-guide-heading > p { font-size: 15px; } + .feature-guide-grid { grid-template-columns: 1fr; gap: 12px; margin-top: 32px; } + .feature-guide-card { min-height: 0; } + .feature-guide-boundary { align-items: flex-start; flex-direction: column; gap: 7px; margin-top: 29px; } +} diff --git a/ui/src/pages/PublicPage.tsx b/ui/src/pages/PublicPage.tsx index 67f48d9..dbc3e60 100644 --- a/ui/src/pages/PublicPage.tsx +++ b/ui/src/pages/PublicPage.tsx @@ -31,6 +31,27 @@ const useCases: UseCase[] = [ { icon: '✓', category: 'PERSONAL', title: 'Personal reminders', description: 'Remind yourself about the same task once a day for as long as you need.', scenario: 'You need a daily nudge for a finite sequence of reminders over ten days.', primary: 'You', fallback: 'You again', timing: '24 hours', result: 'Repeated reminders stop when you acknowledge or the configured sequence ends.' }, ] +type FeatureGuideItem = { + icon: string + title: string + description: string + detailLabel: string + detail: string +} + +const featureGuideItems: FeatureGuideItem[] = [ + { icon: '◎', title: 'Teams keep work separated', description: 'Work inside an explicit team with verified members, shared tasks, response paths, webhooks, and runs.', detailLabel: 'Team boundary', detail: 'The server checks access' }, + { icon: '▤', title: 'Tasks keep the context', description: 'Give every request a name, description, source, and optional priority, category, or reference link.', detailLabel: 'Reusable context', detail: 'Saved with every run' }, + { icon: '↯', title: 'Paths define the handoff', description: 'Choose team members in order and decide how long each response step should wait.', detailLabel: 'Ordered ownership', detail: 'One path, many runs' }, + { icon: '◇', title: 'Runs follow an explicit lifecycle', description: 'Every run moves through named states so your team can see whether it is open, acknowledged, resolved, or complete.', detailLabel: 'Clear state', detail: 'No hidden transitions' }, + { icon: '◷', title: 'Runs start on your terms', description: 'Start immediately, schedule a future time with its timezone, or repeat a schedule daily or weekly.', detailLabel: 'Flexible starts', detail: 'Now, later, or recurring' }, + { icon: '✓', title: 'Email actions make ownership explicit', description: 'Recipients preview and confirm acknowledgement, then resolve the issue when a resolution window is enabled.', detailLabel: 'Clear responsibility', detail: 'Acknowledge or resolve' }, + { icon: '→', title: 'Escalate now moves the next step', description: 'An eligible team member or recipient can make the next response step due without resending the previous email.', detailLabel: 'Manual control', detail: 'Advance the next step' }, + { icon: '⌁', title: 'Webhooks connect external systems', description: 'An authenticated event creates one task and one run, with safe retries when the same event is sent again.', detailLabel: 'Idempotent intake', detail: 'One event, one run' }, + { icon: '≡', title: 'History shows every milestone', description: 'Review saved user and system activity, including delivery attempts, acknowledgement, resolution, and completion.', detailLabel: 'Investigation ready', detail: 'A durable timeline' }, + { icon: '↻', title: 'Recovery keeps work moving', description: 'Saved workflow state lets the application recover scheduled work and ignore stale or duplicate callbacks.', detailLabel: 'Restart safe', detail: 'PostgreSQL stays authoritative' }, +] + const signedInSteps = [ { number: '01', title: 'Primary on-call', detail: 'ops@replytrail.dev', time: 'ACKNOWLEDGED', state: 'done' }, { number: '02', title: 'Platform lead', detail: 'maya@replytrail.dev', time: '+ 5 MIN', state: 'active' }, @@ -59,14 +80,14 @@ export function PublicPage() {
REPLYTRAIL -
Sign in{token ? 'Open workspace' : 'Create account'} ↗
+
FeaturesSign in{token ? 'Open workspace' : 'Create account'} ↗
INCIDENT RESPONSE, WITHOUT THE CHAOS

Get the right people
on every alert.

Turn alerts, requests, and reminders into clear follow-up paths with an owner, a waiting time, and a next step.

-
{token ? 'Open workspace' : 'Create account'} →Explore use cases ↓
+
{token ? 'Open workspace' : 'Create account'} →See the features ↓Explore use cases ↓
ALERTS REQUESTS REMINDERS
✓ Clear owner↗ Timed handoff↻ Restart-safe
@@ -128,6 +149,25 @@ export function PublicPage() {
+
+
+
+ THE REPLYTRAIL FEATURE GUIDE +

Everything that keeps
the handoff moving.

+
+

ReplyTrail gives your team one place to define the work, route the response, and understand what happened next.

+
+
+ {featureGuideItems.map((feature, index) => ( +
+
0{index + 1}
+

{feature.title}

{feature.description}

+
{feature.detailLabel}{feature.detail}
+
+ ))} +
+
PRODUCT BOUNDARY

ReplyTrail currently sends email. SMS, phone calls, push notifications, native Slack or Teams delivery, rotating on-call calendars, and test escalations are not part of the current product.

+