feat: harden cron windows, KYC dual-key, FX quorum, payout runbook

[Calebstack]

Overview

This PR closes remaining Wave acceptance gaps for four related backend hardening issues: deferred distribution cron windows, payout reconciliation runbook/alerting, KYC webhook dual-key rotation, and FX quorum variance guard.

Related Issue

Closes #661
Closes #666
Closes #676
Closes #704

Changes

🗓️ Deferred distribution cron windows (#661)

• [ADD] src/db/migrations/025_create_distribution_schedules.sql
  • Persists per-offering cron + timezone window definitions.
• [ADD] src/services/cronScheduleService.ts
  • Validates via CronWindowValidator before write; rejects Stellar maintenance overlaps and logs diffs.
• [MODIFY] src/services/distributionScheduler.ts
  • Cron gate in scheduler, 1st-Monday maintenance constraint, year-skip lookahead, offering-scoped window de-dupe, scheduler.window.rejected metrics/logs.
• [MODIFY] Offering / revenue report repositories to read & persist cron schedules.

📘 Payout reconciliation runbook (#666)

• [MODIFY] docs/runbooks/payout-reconciliation.md
  • PayoutDriftClass playbook matrix, SEV-1 in 15 minutes rotations, Grafana screenshots-of-truth PromQL.
• [MODIFY] src/services/payoutDriftDetector.ts
  • Embeds runbook_url + PagerDuty description with per-class counts when alarm fires.
• [MODIFY] PayoutDriftClass type exported from payout drift repository.

🔐 KYC webhook dual-key rotation (#676)

• [ADD] src/routes/kycWebhooks.ts — mounted at /api/v1/webhooks/kyc when secret configured.
• [MODIFY] kycWebhookAuth / verifyWebhookPayloadDualKey
  • Requires hard expiry when next key is set; rejects after deadline; emits kyc.webhook.verified_by_key.

💱 FX quorum variance guard (#704)

• [ADD] src/services/fxQuorumBootstrap.ts
  • Bootstraps FxProviderRouter in quorum mode with paging + audit.
• [MODIFY] src/index.ts — opt-in via FX_QUORUM_ENABLED=true.
• [MODIFY] Alert mapping for fx_quorum_failed_total.

Verification Results

npx jest --runInBand --forceExit \
  src/services/distributionScheduler.test.ts \
  src/services/__tests__/distributionScheduler.test.ts \
  src/middleware/webhookAuth.test.ts \
  src/lib/webhookSignature.test.ts \
  src/services/cronScheduleService.test.ts \
  src/services/fxQuorumBootstrap.test.ts \
  src/routes/kycWebhooks.test.ts \
  src/services/payoutDriftDetector.test.ts \
  src/services/fxQuorumEvaluator.test.ts \
  src/services/fxProviderRouter.quorum.test.ts \
  --coverage=false

✅ 386/386 passed

npm run validate:alert-mappings
✅ OK: All 23 known alerts have mapping entries.

Acceptance Criteria	Status
Cron windows validated vs Stellar maintenance + overlaps before persist	✅ CronScheduleService + CronWindowValidator
DST / leap-day / year-skip coverage	✅ Tests included; year-skip uses extended lookahead
PayoutDriftClass → owner / ETA / rollback + SEV-1 table	✅ Runbook matrix + named rotations
Detector alert descriptions link runbook	✅ runbook_url + pagerduty_description
Dual-key KYC window with hard deadline	✅ Route mounted; missing expiry fail-closed; deadline rejects next key
FX K-of-N quorum blocks + pages on divergence	✅ Bootstrap + fx_quorum_failed_total alert mapping
Single-provider outage still meets quorum if N−1 agree	✅ Covered in bootstrap / quorum tests

Timeline

• Calebstack committed

[drips-wave]

@Calebstack Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

[Calebstack]

Splitting into four issue-scoped PRs (#661, #666, #676, #704) for Wave tracking.