Fix enum discriminant collision, fuzz multisig, trace outbox boundary, reproducible builds, SLOs - #1148
Merged
joelpeace48-cell merged 5 commits intoAug 28, 2026
Conversation
Closes FinesseStudioLab#851 Fixes a build-blocking bug on main first: two independently-merged PRs each added a new Error variant at discriminant 46 (TimelockNotFound, from FinesseStudioLab#1134, and BelowMinClaim, from a separately-merged issues-850-860- 861-848 batch) — colliding once merged together, so `cargo check -p trivela-rewards-contract` currently fails with E0081 "discriminant value 46 assigned more than once" on main. Renumbers BelowMinClaim to 48 (the next free slot); safe since nothing could have successfully deployed relying on that specific discriminant value while the crate didn't compile. Adds contracts/rewards/fuzz/fuzz_targets/fuzz_multisig.rs: adversarial fuzzing of verify_multisig (used by set_paused) covering duplicate signers, unknown (unregistered) signers, corrupted signatures, and nonce replay — extending the existing fuzz_balance.rs coverage to the co-admin ed25519 multisig path. Asserts try_set_paused never panics for any signature-set shape, and that success implies a sufficiently large set of distinct, valid, registered signatures plus an unconsumed nonce; replaying the exact same call after a success must then be rejected. Wires both fuzz_balance and the new fuzz_multisig into contract-fuzzing.yml as libFuzzer (cargo-fuzz) runs, time-boxed to the workflow's existing FUZZ_DURATION, alongside the pre-existing proptest- based fuzz_* unit tests that workflow already ran (which weren't actually running the cargo-fuzz targets in contracts/rewards/fuzz/ at all before this).
Closes FinesseStudioLab#778 The outbox pattern (writeOutbox now, OutboxRelay._deliver later, on a completely separate poll-loop tick) breaks OTel's automatic in-process context propagation — a webhook delivery span had no way to link back to the HTTP request that enqueued it, so root-causing latency across that async hop meant manually correlating timestamps instead of following one trace. Adds two helpers to tracing.js: captureTraceparent() snapshots the active span's W3C traceparent string (reusing the same format already built for traceparentMiddleware's response header), and linkedSpan() re-establishes that context as the parent of a new span via OTel's propagation.extract, falling back to a plain unlinked withSpan() when no traceparent is available (e.g. rows enqueued before this change). writeOutbox now wraps the stored payload in { payload, _traceparent } rather than adding a migration column — the reserved _traceparent key round-trips through the existing payload TEXT column. _deliver detects the wrapped shape (vs. legacy raw-payload rows already queued) and runs the handler inside linkedSpan(..., 'outbox.deliver', ...). tracing.test.js adds the trace assertion test the issue's Verification section asks for: using an in-memory OTel exporter, confirms a span captured before the async boundary is recorded as the parent of the outbox.deliver span created after it, sharing the same trace id — plus a fallback case confirming delivery is still traced (unlinked) when no traceparent was stored. Note: DB query and Stellar RPC call spans (the issue's other scope items) aren't added here — withSpan() already exists for exactly that purpose and is documented for it in tracing.js's module doc comment; wiring it into every individual query/RPC call site across the codebase is a much larger, mechanical follow-up beyond what fits in this pass.
… doc Closes FinesseStudioLab#775 Adds rust-toolchain.toml pinning an exact rustc version (1.87.0) plus the wasm32v1-none target — contracts-ci.yml currently installs the floating `stable` channel, which can silently pick up a different compiler release on different days, incompatible with "anyone can rebuild from a tag and get the published hash." Adds scripts/reproducible-build.sh: builds a given contract package twice, from a clean CARGO_TARGET_DIR each time, with SOURCE_DATE_EPOCH fixed and --locked --release --target wasm32v1-none, then asserts both passes produce byte-identical WASM via sha256. Adds docs/PROVENANCE.md documenting the rebuild-and-verify process and what to publish per release. Explicitly scopes out, rather than half-implementing, the two heavier remaining pieces: (1) updating contracts-ci.yml / contract-fuzzing.yml to respect rust-toolchain.toml instead of the floating stable channel they currently install, and (2) SLSA/in-toto signed provenance attestation plus tying the attested hash into the contract's own upgrade allowlist — both real infrastructure projects needing CI signing-identity setup and design decisions beyond this pass. Note: scripts/reproducible-build.sh's logic was reviewed but not executed end-to-end in this environment (a double wasm32v1-none build takes several minutes and wasn't run here) — the CI wiring called out as follow-up above is also where "CI rebuilds twice -> identical hash" from the issue's Verification section would actually get exercised.
Closes FinesseStudioLab#779 Adds docs/SLO.md defining SLIs/SLOs (availability + p95 latency) for the four critical journeys the issue names — campaign read, register, redeem, webhook delivery — grounded in the metrics /metrics (backend/src/routes/health.js) actually exposes today (trivela_requests_total, trivela_request_errors_total, trivela_route_hits_total{route}, trivela_http_request_duration_ms_bucket). Includes example Prometheus alert rules using the Google SRE workbook's multiwindow multi-burn-rate pattern (fast: 14.4x/1h+5m, slow: 6x/6h+30m) for HTTP error budget burn, plus a webhook-delivery-specific pair. Flags two real, concrete gaps rather than writing alert rules against metrics that don't exist: (1) trivela_requests_total/ trivela_request_errors_total are global, not broken down per-route — trivela_route_hits_total{route} exists but there's no per-route error or latency-bucket metric yet, so a redeem-specific outage would only show up diluted in the aggregate rate today; and (2) the webhook-delivery alert rules reference trivela_outbox_delivered_total / trivela_outbox_failed_total metrics that don't exist yet either — they'd need to be added to outboxService.js's _deliver/_markFailed paths. Dashboard provisioning (Grafana, as dashboards-as-code) and on-call paging integration are infrastructure setup outside this repo and are also not done here — see the Follow-up section for the full list.
joelpeace48-cell
merged commit Aug 28, 2026
2ce7cf4
into
FinesseStudioLab:main
4 of 20 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #851, closes #778, closes #775, closes #779.
Found and fixed first:
maincurrently fails to compile (cargo check -p trivela-rewards-contract→ E0081) — two independently-merged PRs (#1134 for #838, and a separatelymerged issues-850-860-861-848 batch) each added a new
Errorvariant at discriminant46(
TimelockNotFoundandBelowMinClaim), colliding once both landed onmain. RenumberedBelowMinClaimto48; safe since nothing could have deployed relying on that value while thecrate didn't build.
#851 — fuzz the multisig. Adds
fuzz_targets/fuzz_multisig.rs, extending the existingfuzz_balance.rscoverage toverify_multisig(used byset_paused): duplicate signers, unknown/unregistered signers, corrupted signatures, nonce replay. Asserts
try_set_pausednever panics forany input shape, success implies a sufficiently large distinct-valid-registered signer set plus an
unconsumed nonce, and replaying a successful call is rejected. Wires both
fuzz_balanceand the newfuzz_multisigintocontract-fuzzing.ymlas actualcargo fuzz run(libFuzzer) jobs — thatworkflow was only running the separate proptest-based
fuzz_*unit tests before, never thecargo-fuzztargets incontracts/rewards/fuzz/at all.#778 — trace context across the outbox async boundary. The outbox pattern (write now, deliver
later on an unrelated poll tick) breaks OTel's automatic context propagation.
writeOutboxnowcaptures the enqueuing request's
traceparentand stores it alongside the payload;OutboxRelay._deliverre-establishes it as the parent of a newoutbox.deliverspan viacaptureTraceparent()/linkedSpan()(new helpers intracing.js).tracing.test.jsadds thetrace assertion test the issue's Verification section asks for, using an in-memory OTel exporter to
confirm parent/child linkage across the boundary. DB-query and RPC-call span instrumentation aren't
added here —
withSpan()already exists and is documented for exactly that; wiring it into everycall site is a larger mechanical follow-up.
#775 — reproducible builds.
rust-toolchain.tomlpins an exactrustcversion (CI currentlyinstalls the floating
stablechannel).scripts/reproducible-build.shbuilds a package twice froma clean target dir with
SOURCE_DATE_EPOCHfixed and asserts identicalsha256.docs/PROVENANCE.mddocuments the rebuild/verify process and explicitly scopes out (rather thanhalf-building) updating CI to respect the pinned toolchain and full SLSA/in-toto signed attestation.
#779 — SLOs.
docs/SLO.mddefines SLIs/SLOs for the four named journeys, grounded in themetrics
/metricsactually exposes today, with example multi-window burn-rate Prometheus alertrules (Google SRE workbook pattern). Flags two concrete gaps instead of writing rules against
nonexistent metrics: HTTP errors/latency aren't broken down per-route yet, and webhook-delivery
metrics referenced in the alert rules don't exist yet either.
Test plan
cargo check -p trivela-rewards-contractpasses (was failing onmain)cargo checkincontracts/rewards/fuzzpasses (both fuzz targets compile)node --checkon all three changed/new backend JS files passesnode --test src/tracing.test.js— not run; needs@opentelemetry/sdk-trace-baseinstalled,and
npm installin this environment hits unrelated platform-lock issues on other optionaldeps (seen in prior PRs against this repo)
scripts/reproducible-build.sh— logic reviewed, not executed end-to-end (a doublewasm32v1-nonebuild takes several minutes)