feat(0119): reject every invalid input with a 400 envelope - #217
Conversation
Axum's own Query/Json/Path rejections answered in text/plain with 415/422 for body failures, bypassing the ErrorEnvelope contract. New wrapper extractors (ValidatedQuery/ValidatedJson/ValidatedPath) map every rejection to 400 + envelope; new invalid_body code; bad_request now carries Cache-Control: no-store; /prices/batch gets an explicit 16 KB body limit so an oversized body is refused before parsing. CH-less negative tests prove each rejection fires before any ClickHouse call.
… fns The six param enums (SortCol, Order, TypeFilter, Timeframe, Granularity, BaseCurrency) now derive Deserialize + ToSchema with explicit per-variant renames, so params deserialize straight into typed forms: an unknown token fails serde with a message enumerating the valid values (400 envelope via ValidatedQuery), and the same enums publish into OpenAPI. Deletes the six stringly parse() fns. Case policy: exact documented tokens; base_currency keeps lowercase usd/xlm aliases (historically case-insensitive). New: search capped at 1-12 ASCII alphanumeric (SEP-11 alphanum12 prefix).
Cursor: deny_unknown_fields (foreign lookalikes 400), 256-char token cap, and valid_for(sort) — a numeric sort requires a finite-parseable v, which previously reached ClickHouse toFloat64() and threw a 500 from client input; code sorts require an asset-code shape. OHLCV: parse_time (chrono; already in the build graph transitively) replaces the shape-only valid_iso8601, rejecting impossible dates that CH would have interpreted freely; the validated epoch (not the raw string) is bound via toDateTime(?) so exactly one interpretation of the window exists. New window rule before any DB call: start < end, and ceil(span/granularity) <= OHLCV_MAX_POINTS with a message naming the count — replacing the silent newest-5000 truncation. The timeframe window anchors to end when only end is given (?end=..&timeframe=7d is the 7d window ending there); timeframe=all starts at Stellar genesis.
…s-api Registers the six query-param enums in ApiDoc components — utoipa-axum collects body schemas from routes but a params-tuple $ref is NOT auto-registered, so the served document carried dangling refs (caught by the new enum-publication contract test). BatchRequest.assets publishes minItems/maxItems mirroring MAX_BATCH; search publishes its 1-12 length bounds. Task file gains the AC 1 param -> rule -> error-code table and the recorded policies (unknown params ignored, exact-token case with the base_currency lowercase exception, cursor sort-binding limitation, CH-in-CI deliberately deferred to 0120/0122). CI clippy line now includes prices-api.
…eframe cliff Local-CH smoke caught the one positive-path change 0119 makes: timeframe=all&granularity=1h is now an up-front 400 (genesis to now is ~95k hourly buckets against the 5000 cap), where the merge test leaned on the old silent newest-5000 truncation. Narrowed with explicit start/end around the seeded candles — merge + backfill_note coverage unchanged. All 21 integration tests pass against the prod-pinned CH 26.3.10.60, confirming the toDateTime epoch binds. Task notes record the consumer-visible change and the ~2029-06 cliff where bare timeframe=all at 1d itself crosses 5000 buckets.
…t-validation-hardening
develop gained AppConfig.portal_enabled (task 0183, PR #207) while this branch was in flight; the new tests/common harness constructs AppConfig literally and did not compile against the merge. Off in tests — the portal gate is irrelevant to the validation suite.
…e_time /code-review (high) on the branch surfaced four correctness bugs and a set of contract gaps; all fixed: - sort=code cursors: the charset/non-empty rule rejected tokens the API itself issues (Soroban rows carry empty codes; lossy-decoded on-chain garbage exists), dead-ending pagination. String sorts now cap length only — no 500 risk exists on that path (plain string bind). - end-anchored windows could derive a negative eff_start and bind it into toDateTime(?) unvalidated (CH throw -> 500). Clamped to epoch 0. - fencepost: inclusive SQL bounds hold span/gran + 1 bucket starts, so an aligned 5000-bucket span was passed through and silently trimmed by LIMIT. Point count now +1. - parse_time regressions vs the old pipeline: minute-precision offsets (11:30Z, 11:30+02:00) and colonless +0200 now parse; millis decided by digit count (13+) as documented, so 12-digit values reject. - one window interpretation for real: the validated eff_start always binds into SQL; the parallel now() - INTERVAL path (calendar-aware 1 YEAR vs 365d seconds() — divergent on leap years) is deleted along with Timeframe::interval() and OhlcvArgs.since_interval. - spec honesty: base_currency documents the lowercase legacy aliases, search documents empty-as-absent, start/end document epoch forms; maxItems asserted against MAX_BATCH (const, not literal); search cap derives from identity::MAX_CODE_LEN. - task notes record the full blast radius of the 5000-bucket 400s (not just timeframe=all), the UTC pinning of naive datetimes, and the end-anchoring flip for far-future ?end.
Review notes — input-validation hardeningReviewed The plumbing looks correct: the Everything below is the same shape instead: a validation rule stricter than the data model it guards, so requests that worked before now return 400. 1.
|
…RL realities All seven review points addressed: 1. search: length-only cap (64B, shared MAX_STRING_PAYLOAD_LEN with the cursor) — the charset rule made lossy-decoded but listed asset codes unsearchable, contradicting this branch's own cursor reasoning. 2. start == end allowed (inclusive SQL bounds = one bucket); guard is now strictly greater-than. 3. granularity omitted on an explicit start/end window derives from the window span (finest fitting 5000 points), not from a timeframe the caller never sent — ?start=2020-01-01 alone is answerable again. 4. parse_time undoes query-string percent-decoding of '+': the space separator is fixed positionally (byte 10) and a trailing ' HH:MM' / ' HHMM' after a time is recovered as a +offset; raw-in-URL offsets work over HTTP, not only in unit tests. 5. cursors are minted URL_SAFE_NO_PAD (STANDARD still decoded for in-flight tokens) — next_cursor echoed verbatim survives a query string. 6. future-only start now says 'start is in the future' instead of blaming an end the client never sent. 7. the 2029 timeframe=all cliff is defused as a side effect of (3): all's default granularity is span-derived, so it self-coarsens (1d today, 1w post-2029) instead of hard-400ing. Spec + validation table updated to match; 21 live-CH integration tests green (cursor walk exercises the URL-safe tokens end-to-end).
…low-ups All 8 ACs checked off. Completion notes record the two hardening rounds (8-angle /code-review + okarcz's PR #217 review), the emerged decisions (length-only asset-code rules, URL-safe cursors, '+'-decoding recovery, span-derived auto-granularity defusing the 2029 all-timeframe cliff) and the one intentionally modified integration test. 0206 owns the deferred remainders: cursor {sort,order} binding, ValidatedPath<AssetIdentifier>, negative-test assert helper, parse_time dedup.
Summary
ValidatedQuery/ValidatedJson/ValidatedPathextractors route every axum rejection through the standardErrorEnvelope— the old text/plain 400/415/422 bodies are gone; 400s now carryCache-Control: no-store, andPOST /prices/batchgets an explicit 16 KB body limitparse_*fns deleted): unknown tokens 400 with the valid values enumerated,searchis capped at 1–12 ASCII alphanumerics, and the same enums publish into OpenAPI (contract-tested — the test caught dangling $refs utoipa-axum does not auto-register)deny_unknown_fields, 256-char token cap, and payload type-checked against the active sort — closes a client-reachable 500 (corruptvreaching ClickHousetoFloat64)toDateTime(?), plus an explicit window rule (start < end, ≤5000 points at the chosen granularity) replacing silent truncation; the timeframe window anchors toendwhen onlyendis givenAppState::without_ch()(panics on any CH access, proving validation precedes every query) and run in the existing CIcargo teststep;prices-apiadded to the clippy-D warningsgate; all 21 live-CH integration tests pass against the prod-pinned 26.3.10.60Task:
lore/1-tasks/active/0119_FEATURE_api-input-validation-hardening.md(param → rule → error-code table + recorded policies)