Repository navigation
Promote edge to stable: Polis 2.45.2 - #3450
Merged
Merged
Conversation
…t the image can run, with one admission entry (#3446) * delphi: graph steps and models come from one typed registry, checked against the image before admission Which embedding, cluster, keyword and label models a job graph may name is now defined once, in queue-rs/polis-contracts/job-models.json, read by a typed Rust registry (Step, Model) and by the Delphi worker from a byte-equal copy (a repository check compares the two files; a Rust test compares the registry Python renders with the compiled one). Admission goes through one Rust entry, jobs::admission, used by the shadow scheduler and, as `polis-jobs graph-admit`, by the Python graph client and the importer. Before the database call it asks the image which (step, model, adapter digest) pairs it implements and refuses an unknown model, a model the image does not carry, a wrong input role or producer, and out-of-range resources. The worker child checks each input's producer schema again. The placeholder models used by the graph proofs compile only into test builds (the graph-test-models feature and the Delphi test image); the final image and production binaries refuse them. DELPHI_EMBED_MODEL_BY_ENV, optional, names the embedding model a result environment uses. Unset, every path keeps its present model (granite at 384 dimensions); shipping weights for another model selects nothing. No schema change: pd_graph_admit is unchanged and still checked last. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KvftUAdasWzxmMYdioZJWs * polis-contracts: job_graph.rs formats the same under both of its workspaces The file is compiled in polis-contracts (the CLI workspace, edition 2021) and, by path, in the queue-rs crate (edition 2024). The two rustfmt styles order `use serde_json::{Value, json}` differently, so `cargo fmt --check` in the CLI workspace refused what queue-rs's accepted. Import only `Value` and call `serde_json::json!` by its path; no behaviour change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KvftUAdasWzxmMYdioZJWs * ci: the one-admitter check reads the whole tree, not four directories check_graph_registry.py looked for `pd_graph_admit` only under queue-rs/src, delphi/scripts, delphi/polismath and server/src, so a new caller in delphi/umap_narrative, the Delphi entry scripts, polis-api, the CLI or a shell script passed. It now reads every tracked or unignored file (every file when the root is not a Git checkout), skipping migrations and documentation, and names the files that may mention the function: the Rust admission entry, the database contract tests that call it directly, one documentation comment, and the check itself. New tests cover callers in those places and the allowed files. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KvftUAdasWzxmMYdioZJWs --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…r conversations' full-pipeline runs; optional same-host graph-stage daemon, off by default (#3447) * delphi shadow: a job graph stuck past the job deadline is deferred and holds only its own conversation When a graph's attempt cannot be proven finished (its scope stays held), the scheduler no longer waits on it forever. After the job deadline the graph moves to a deferred list that is polled one per cycle, and the full-pipeline shadow keeps admitting every other conversation; only the conversation whose graph scope is held is skipped. The environments the graph path has used are recorded in the state file before any admission, so a graph admitted under a list that was later emptied is still found and recovered after a restart. The job daemon can bind its retained journal to one host identity file (POLIS_JOBS_JOURNAL_HOST_FILE): a journal written under another host is refused at start-up rather than read as exit evidence. Adds Rust tests for a lost attempt after a restart with the list emptied and for a committed admission whose reply was lost, and two Docker process checks for recovery and for the scheduler under its deployed memory and CPU limits. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KvftUAdasWzxmMYdioZJWs * delphi shadow: an optional graph-stage daemon with its own profile, journal volume and deploy check Adds the delphi-shadow-graph Compose service (profile delphi-shadow-graph, off unless DELPHI_SHADOW_GRAPH_ENABLED=1): the job daemon restricted to the four graph stages at concurrency one, with its own journal volume, a read-only host identity file bound in for the journal's host binding, and its own memory and CPU ceilings (6 GiB and 2 CPUs by default). The daemon refuses to start unless it runs as PID 1 of its own container with exactly those stages, concurrency one and a host identity file set. The deploy hook starts it only when the full-pipeline shadow is selected and the setting is on, and first checks the rendered Compose settings with scripts/check-shadow-graph.py (names only); if they are wrong, neither shadow is started. With the setting off, nothing in the deploy changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KvftUAdasWzxmMYdioZJWs * delphi shadow: graph publication is recognised by its served root and retried a bounded number of times; refusals are reconsidered by build A finished graph's publication is now confirmed by reading the served bundle and checking that its root artifact is this graph's leaf at the next generation, not by the publish call's reply alone. A publish whose commit is durable but whose reply was lost is recognised as published; a transient database error before commit (connection, serialization, deadlock, lock or too-many-connections classes) is retried at most three times, each on a fresh connection, and then counted failed. A definite refusal is not retried. If the served bundle cannot be read, the graph stays in flight and is resolved on a later poll. A refused conversation's recorded observation now carries the admission rule and the build's code version, so a later build looks at it again once. A graph admission that errored is followed by a recovery read, so an admission that committed before its reply was lost is picked up before any full-pipeline run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KvftUAdasWzxmMYdioZJWs * delphi shadow: graph wait and journal recovery log lines say why, and the recovery runbook is documented A graph waiting past its deadline now logs the reason it is still held (nodes still active, a process exit not yet confirmed, or reconciliation pending) and the count of attempts with no confirmed exit, instead of one fixed reason. A job daemon with a host-bound journal logs one line per retained journal entry saying whether the attempt's exit was proven and why. docs/configuration.md gains a section on the same-host graph executor: its profile, host identity file, journal volume, deadlines, memory ceilings and what to drain before a rollback. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KvftUAdasWzxmMYdioZJWs * delphi tests: the graph executor's Compose tests run in the Delphi test container The graph executor's validator test located scripts/check-shadow-graph.py by its own path, which inside the Delphi test container resolves to /scripts and does not exist, so every shard of the Delphi Python tests stopped with a collection error. It now reads the validator from the checkout the other Compose tests read ($POLIS_CHECKOUT_DIR), and the deploy-hook runner copies the validator there with its other inputs. The test that the shadow yields CPU and memory to every other service failed once the graph executor was added, because the executor carries the shadow's own lowest weight. Both shadow daemons must now carry exactly that weight and OOM adjustment, and every other service must still outrank them; no assertion about any other service changed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KvftUAdasWzxmMYdioZJWs * delphi shadow: the graph executor receives the per-environment embedding model setting as the shadow daemon does DELPHI_EMBED_MODEL_BY_ENV is now passed through to delphi-shadow-graph the way the registry passes it to delphi-shadow and the scheduler: listed by name, with no default of its own, so with the setting unset (today) it is absent from the container and nothing changes. Each graph's model is still decided and sealed by the scheduler at admission, and graph children's environment stays the existing allowlist. The deploy check refuses a graph executor whose setting differs from the scheduler's; tests cover the check and that both daemons list the setting the same way. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KvftUAdasWzxmMYdioZJWs * graph registry check: the daemon recovery proof may admit its test node directly delphi/tests/shadow/graph_recovery.py admits one test-model node through the SQL admission function so that a test-build daemon has a real attempt to lose and recover, as the graph proofs already do. The check that every product admission goes through the shared entry now lists it beside those proofs. The proof's description says it needs a test-build image, since a production daemon refuses to run a test-only model. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KvftUAdasWzxmMYdioZJWs --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…y the comments participants are shown (#3448) * shadow: tests that a run embeds every comment and lays out only what participants are shown The Delphi shadow places, clusters and names every comment of a conversation, rejected ones included. These tests say what it should do instead, on comments in every moderation state (accepted, unmoderated, rejected; inactive and muted ones among them; strict and open conversations): - the rule is the comment list participants are served (participantVisibleClause in server/src/comment.ts): a Rust contract test writes a golden of rows and what the rule keeps of them, open and strict, and the Python selection must keep the same comments under the same digest; - a run that names the selection embeds every comment and gives UMAP, EVoC, the stored families and the naming only the selected ones; a run that names none (the served path) is what it was, and its pipeline arguments are byte for byte today's; - the scheduler's census digests both sets, its admission names the selection and is keyed by it: a moderation that changes what participants are shown is a new run, one that does not (an unmoderated comment accepted outside strict moderation, a rejected comment added) is not; the floor counts the selection; a conversation an earlier build ran is run once more and then not again; - the rehearsal sets moderation states on its public fixtures (one conversation under strict moderation; inactive, muted and unmoderated comments), counts its first wave by what participants are shown, adds two moderations that must run nothing, and checks the stored runs: every comment embedded, exactly the shown ones placed, clustered and named. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KvftUAdasWzxmMYdioZJWs * shadow: embed every comment once; lay out, cluster and name what participants are shown Each Delphi shadow run now names its selection, participant_visible/1: what participants are shown, by the rule of the comment list the server gives them (server/src/comment.ts, participantVisibleClause: active, velocity above 0, accepted under strict moderation or not rejected otherwise), stated once in Rust (the input bundle contract's participant_visible) and once in Python (polismath/selection.py). - The pipeline, given --selection, embeds every comment with text and stores every embedding, then gives UMAP, EVoC, the keyword labels and every family derived from them (graph, assignments, keywords, features, names, the conversation's layout config) only the selected comments. The selection is the whole rule: the moderation and report filters must be off. Without --selection nothing changes: the served path, which the poller runs with no selection, embeds and lays out what it always did. - The admission names the selection; the scheduler's census digests every comment (the embedded set, as the run's census records it) and the selection; the idempotency key is the selection's digest; the run record carries both digests and both counts (config_effective, delphi-shadow/2). The floor counts the selection, the cap and the storage estimate every comment with text. A state entry written before runs named a selection is read and admitted once more. - The input bundle carries the conversation's strict_moderation and each comment's velocity only when the run names a selection; every other bundle is byte for byte what it was. No schema change. No change to the vote, comment or next-comment path: the census and the bundle read comments and conversations in read-only snapshots that take ACCESS SHARE only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KvftUAdasWzxmMYdioZJWs * shadow rehearsal: embedded is every comment the newest run could read A comment written after a conversation's newest run, and not shown to participants (an unmoderated comment under strict moderation), starts no run, so it is embedded by the conversation's next run, not before. The rehearsal's selection check now holds the embeddings to every comment with text the newest run could read, and counts the ones added since. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KvftUAdasWzxmMYdioZJWs --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Contributor
Author
Delphi Coverage Report
|
colinmegill
marked this pull request as ready for review
November 19, 2026 04:54
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release plan
Version: 2.45.2
Reason: no new selected migrations
Previous stable:
c4f5d5b0bae985b0756fb73d829bfbfebff3f216Candidate:
6cf1edfe7ee478bc0cbbcc42256b6eefe7dd5240release.txt additions:
Merge the reviewed version preparation into edge, then refresh this plan for the resulting edge SHA before merging the promotion with a merge commit.