Repository navigation
chore(deps): merge upstream Pingora 0.9 and patch zstd advisories - #15
Merged
Merged
Conversation
Adapt the change to the released crates.io boring 5.x packages while preserving the open source feature defaults.
Set the GitHub Actions MSRV lane to Rust 1.85.
Once a queued downstream proxy-task write reports an error during cache fill, the proxy loop already marks the downstream state errored and propagates that first error to the downstream session. Continuing to enter the queued-task branch after that point only polls the same failed downstream path unnecessarily while cache fill continues.
openssl 0.10.77 deprecated `Asn1StringRef::as_utf8`, which truncates at the first interior NUL byte. Decode subject-name ASN.1 strings with `as_slice()` + `String::from_utf8_lossy` instead. This API is not deprecated, is available on both the openssl and boringssl backends, and no longer truncates at interior NUL bytes.
Configure the H2 client with a response header-list size limit so oversized H2 upstream response headers fail while reading from the upstream, before they can be forwarded to downstream sessions. The limit is a dedicated constant sized for H1-equivalence: h2 bounds the HPACK-decoded header list rather than raw bytes, so it is derived to accept exactly the responses the HTTP/1 client path would. This prevents oversized response headers from being carried over custom downstream transports as successful responses.
Add a per-shard atomic shadow of total weight (shard_weights: [AtomicUsize; N]) maintained alongside the existing global weight counter at every weight-mutating site (admit, increment_weight, evict_shard_at, remove, insert_tail). Rewrite Lru::shard_weight() to read from the shadow with a Relaxed load instead of acquiring the shard's RwLock to read the LruUnit::used_weight field. This mirrors the pattern already established for shard_lens/shard_len(). Like shard_len(), the same 'best-effort, no cross-thread ordering' caveat applies: the sum of shard_weight(i) is not guaranteed to equal weight() at any single instant, since the global and per-shard atomics are updated independently with Relaxed ordering. Motivation: callers (eviction-balance heuristics, observability) that want to scan all N shards every few seconds should not pay N brief read-lock acquisitions when an atomic shadow is cheap enough to maintain on the write path. Adds a single-threaded regression test verifying the shadow stays consistent with the truth across all five weight-mutating paths.
… invariant violation which triggers panics or silent truncation
allows callers to update fresh_until, stale_while_revalidate_sec, and stale_if_error_sec on an existing CacheMeta without replacing the response header or other metadata.
This reverts commit 5a99587f415897cd5ca0ff53eabc896b20f9f876.
The infallible HeaderMap::append calls .expect("size overflows
MAX_SIZE") internally, which aborts the process if the case-preserving
name map ever exceeds http's MAX_SIZE (1 << 15). Switch the name_map
append to the non-panicking try_append and surface MaxSizeReached as an
InvalidHTTPHeader error, matching the existing error-handling style in
the function. The function fingerprint is unchanged.
Keep downstream custom message reader ownership in the proxy caller instead of inside the downstream forwarding future. This ensures the reader is restored when a retryable upstream error cancels that future. Also restore the writer if reader extraction fails, and restore both handles when custom forwarding setup fails.
When a response bypasses cache (no cache writer) and the downstream half completes the response by its own framing (e.g. a range or Content-Length shorter than the upstream stream) before the upstream signals end-of-stream, it drops the task pipe. The upstream half's send then fails; previously that surfaced a spurious InternalError ("channel closed") and tore the upstream connection down, even though the client already received a correct response.
Add a shared PipeState (Active / DownstreamComplete) that the downstream half sets before it returns and the upstream half checks at every task-pipe closure point: the body, trailer, and end-of-stream sends, and the HTTP/2 closed-pipe and trailer-race early exits. A closure is treated as benign only when the downstream signaled completion; an unexpected closure still surfaces an error. Applied consistently across HTTP/1, HTTP/2, and custom transports.
On the benign HTTP/1 path, reuse is left to the loop's normal exit, which marks the upstream connection reusable only when both the response was fully read and the request was fully sent. A premature response can leave the request half-sent, so the connection must not be pooled on response completion alone.
Previously, long request body reads in the proxy state machine could interrupt an upstream response header read after it had already consumed partial bytes. Dropping that future lost those bytes, so the next response read could resume mid-header and report a spurious read error.
A subrequest can publish a proxy error while its response channel closes. If response completion exits the pipe loop first, the queued error was not observed and the caller treated the subrequest as successful. Reconcile the non-blocking error receiver before returning success. Add a deterministic regression that holds the completing response task until the error is queued.
Watch the downstream during subrequest pipe waits after its request body is complete, so client disconnects cancel the pipe. Respect HTTP/1 pipelining and half-close behavior, retain pending storage reads across client events, and stop watching once the response is delivered. Make cancellation interrupt upstream pacing delays after abandonment while preserving accepted-task processing and the bounded cancellation drain, so a fully received response can still commit. Cover long body pauses, pending cancellation, normal pacing, and complete-response handling.
Add LRU operations that check an entry and then update, admit, remove or evict it under one shard lock, plus an atomic two-key mutation, so callers can validate identity and move weight without racing. Add a bounded selection of the least-recently-used eligible entry, and count an eviction only when the selected entry is finally removed.
Co-authored-by: Song Hiếu Trần <songhieu2516@gmail.com>
…ht updates Storage that corrects cache weights out of band, for example from a background size scan, needs a finished write's admission or weight update to land before it measures that write. Both finish paths now prepare a MissCompletion, which carries the entry, its freshness, the eviction manager, the storage and trace handles, and the runtime that purges evicted entries, before they consume the miss handler, and pass it to HandleMiss::finish_with_eviction. The default calls finish(), then completes the accounting with no await in between, so existing storage is unaffected. Storage that overrides it can move the completion into the worker that finishes the write and complete it there, so the accounting has one owner even after the caller's future is dropped. complete() returns a MissCompleted receipt that nothing else can create, and a successful finish_with_eviction must return it, so an override cannot report success without applying its completion. Failed writes drop the completion and return their error. Wrapping miss handlers forward the method. pingora-lru gains set_weight_if_unchanged, which replaces an entry's weight only while it still equals an expected value, without admitting, promoting or evicting. Every update to an existing entry's weight now applies one net delta to the global weight, so weight() never reports a total that no entry state ever had.
Atomic counter increments wrap on overflow, but the return-value calculation used ordinary addition and panicked with overflow checks enabled. Use wrapping_add to preserve the documented overflow behavior, and test both overflow directions. --- style(cache): align block comment for nightly rustfmt Includes-commit: 0ec7c48 Includes-commit: b50ea59 Replicated-from: cloudflare#1041
When an upstream backend returns response headers containing non-UTF-8 bytes (e.g. ISO-8859-1 encoding or binary residue), 'str::from_utf8().unwrap()' in 'read_response_task' causes a panic. Replace 'str::from_utf8(...).unwrap()' with 'String::from_utf8_lossy(...)' so raw response header bytes are logged safely without risking process panics. Closes cloudflare#1023 --- style: fix block comment closing for nightly rustfmt Includes-commit: 13ae58a Includes-commit: 4047c4d Replicated-from: cloudflare#1039
Unit tests only, no library change: conditional revalidation header, Vary value in the variance key, inclusive serve-stale window end, uncacheable responses releasing waiting readers, case-insensitive request-target scheme, remove_header dropping the case-preserved name, and repeated headers surviving header-serde round trips. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Includes-commit: 924b472 Replicated-from: cloudflare#1036
RUSTSEC-2026-0346 and RUSTSEC-2026-0349 flag rustls 0.21.12, which comes in through hyper-rustls 0.24 on the same legacy aws-sdk-s3 chain as the rustls-webpki advisories already ignored here. There is no 0.21.x backport (patched >= 0.23.46), and rustls 0.23 already resolves to 0.23.46.
Signed-off-by: Shane Utt <shaneutt@linux.com>
The action's master branch no longer exists, so the scheduled audit fails before it runs. v2.0.0 keeps the same token input.
pingora-runtime's dial9 feature depends on the metrique crates only to pin versions that Dial9 leaves open, so cargo machete reports them as unused. Add them to its ignore list in the CI checkout before it runs.
When the h2 client cancels a stream, the server may already have sent the response. Once h2 forgot the reset stream, it answered each late frame with RST_STREAM(STREAM_CLOSED), and those resets counted against a per-connection lifetime limit of 1024. A long-lived upstream connection that cancels many in-flight requests therefore closed itself with GOAWAY(ENHANCE_YOUR_CALM, "too_many_internal_resets") and failed every request in flight on it. An h2 client that discards those frames, as RFC 9113 section 5.1 allows, keeps the connection (hyperium/h2#983). The new test keeps one request in flight while the client cancels 21 rounds of 100 requests that a raw server had already answered; with released h2 the in-flight request fails with too_many_internal_resets.
Replace plain HTTP origin fixtures with owned Rust origins while preserving wire-level reset, streaming, and informational-response behavior. Share listener lifecycle and HTTP head reading across test origins, migrate rebased cache tests to the shared reader, and remove unused OpenResty locations.
Stop forwarding failed uploads while continuing to relay complete upstream responses, matching HTTP/1 error handling. Preserve existing downstream drain and close policies, and drop failed writers without cleanup. Cancel blocked uploads on response-header read failures. Preserve Custom's existing upgraded half-close behavior when the upload succeeds. Add regression coverage for error precedence, retries, blocked writers, upgrades, cache handling, and downstream finalization.
Let services name a runtime group so that services with similar workloads share one worker pool while each keeps its own name, dependencies and readiness. - runtime_group() on the service traits, with setters on listening services and ProxyServiceBuilder. - Server::add_runtime_group() and RuntimeGroupConf. - A pre-pass resolves each service's runtime. Conflicting overrides, zero threads and duplicate addresses within a group are startup errors. - Group runtimes are built lazily and shut down with the same timeout. Services without a group run exactly as before.
tokio 1.54.0 panics when a BufReader or BufStream is created with zero capacity (tokio-rs/tokio#8494). Stream wraps every new TCP, Unix and virtual socket in BufStream::with_capacity(0, 0, ..) and sizes the buffers later in set_buffer, and L4BufferSettings documents Some(0) as turning buffering off for that direction. On tokio 1.54.0 every accept and connect panics with "capacity must be greater than zero". An empty BufReader never buffered anything here. Stream only reads through poll_read, and BufReader hands a read straight to the stream under it when its own buffer is empty. Hold the L4 buffers in a small enum instead of BufStream: a BufReader over a BufWriter as before, or the BufWriter alone when the read capacity is zero. Reads and writes go to the socket exactly as they did on tokio 1.53, and an unbuffered reader still allocates nothing. A listener test serves a request with L4BufferSettings::unbuffered(). On tokio 1.54.0 it fails with the panic above without this change.
…n streams The test needs the h2 client to drop frames that arrive on streams it no longer tracks (hyperium/h2#983). Released h2 answers each with RST_STREAM(STREAM_CLOSED), so the test fails against crates.io h2.
theduke
changed the title
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.
theduke
marked this pull request as ready for review
October 28, 2026 10:59
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Merge cloudflare/pingora
mainat84f21ee9a2c5074bd3b8d98b92a5cb152940ae99into the Wasmer fork, including all existingedgebranch commits. This updates Pingora 0.8.1 to 0.9.0 while retaining the integrations Edge needs. The upstream merge includes 131 commits and introduces API changes which the Edge integration PR adapts to.Upstream still requires vulnerable zstd versions. Update
pingora-header-serdeto zstd 0.14.3 and zstd-safe 8.2.0, which selects zstd-sys 2.1.2. This addresses RUSTSEC-2026-0391, RUSTSEC-2026-0398, RUSTSEC-2026-0403, and RUSTSEC-2026-0410.Conflict resolution and fork behavior:
is_response_completeAPIs; remove obsolete tracing dependency pins now incompatible with upstream foundations.Validation at the merged revision:
cargo test --quiet --locked -p pingora-header-serde -p pingora-core -p pingora-proxy --lib -- --skip connectors::http::v2::tests::test_h2_late_frames_on_cancelled_streams_keep_connection: passes (612 core, 14 header-serde, 223 proxy tests; 2 ignored).cargo fmt --allandgit diff --check: pass.cargo clippy --quiet --locked -p pingora-header-serde -p pingora-core -p pingora-proxy --all-targets: passes.Local builds use the installed CMake binary on PATH and
RUSTFLAGS='-C linker-features=-lld'to work around missing local tools/toolchain paths. No environment workarounds are repository changes.Merge strategy: merge this PR with a merge commit. Do not squash or rebase it: the upstream merge ancestry and published commit pinned by Edge must remain reachable.