Skip to content

Promote edge to stable: Polis 2.45.3 - #3454

Draft
github-actions[bot] wants to merge 2 commits into
stablefrom
edge
Draft

github-actions[bot] wants to merge 2 commits into
stablefrom
edge

Conversation

@github-actions

@github-actions github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Release plan

Version: 2.45.3

Reason: no new selected migrations

Previous stable: 8289aae6781cfdcaa9d78f1b2fe5aa11af37dcc3

Candidate: 99eb05482e9386ba9f1a39dfe05785dfadc8af73

release.txt additions:

  • none

Merge the reviewed version preparation into edge, then refresh this plan for the resulting edge SHA before merging the promotion with a merge commit.

colinmegill and others added 2 commits November 19, 2026 05:13
…go and admitted again, publication is retried across polls, and a copied host identity no longer confirms a renewing attempt (#3452)

* polis-jobs: a host-bound journal entry from another boot is confirmed only after its attempt's lease has expired

The graph executor binds its retained journal to a host identity file. The
binding is that file's contents, so a restored snapshot of the volume, or a
copied or template-baked identity file, passes it on a second machine; the
boot id then differs and the entry read as provably gone, so the copy
recorded a live attempt's exit as confirmed while the original's child ran.

Now, for a host-bound journal only, an entry written under another boot is
confirmed once the database shows its attempt's lease over: while the
attempt still holds the job, when its locked_until has passed by database
time; once the job is no longer held by it, one whole lease after this
daemon first read it so. A live original keeps renewing its lease, so it is
never confirmed; an honest reboot waits at most one lease. Waiting entries
are read again at the reaper cadence, before the reaper. The read is the
existing pq_job_status together with the database clock, in one read-only
statement; no SQL function or table changes.

Daemons without a host binding (the full-pipeline shadow, the worker boxes)
recover exactly as before. New graph_recovery reasons:
another_boot_lease_live, another_boot_lease_expired.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KvftUAdasWzxmMYdioZJWs

* deploy: invalid graph executor settings refuse the graph executor only; the full-pipeline shadow still starts

With DELPHI_SHADOW_GRAPH_ENABLED=1, the Delphi deploy checks the graph
executor's composed settings (scripts/check-shadow-graph.py). A refusal used
to stop both shadows, so a typo in an optional executor's setting also
stopped the full-pipeline shadow. Now it refuses the executor alone: the
full-pipeline shadow, whose own settings were read back first, starts as it
would with the executor switched off. The message names what is not started.

Nothing changes unless DELPHI_SHADOW_GRAPH_ENABLED=1 and the shadow are both
selected. Two deploy-hook tests: valid settings start the executor before
the shadow; invalid settings start the shadow without the executor.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KvftUAdasWzxmMYdioZJWs

* delphi shadow: an unclaimed graph is let go after ten minutes, publication is retried on later polls, and a held graph admission says why

Three changes to the scheduler's graph path, which runs only for the
conversations DELPHI_SHADOW_GRAPH_CONVERSATIONS lists (empty by default):

- An admitted graph none of whose nodes is ever claimed (no graph executor
  running, unstartable, or refused at deploy) waited the whole job deadline
  (default four hours) per listed conversation, holding the full-pipeline
  shadow. After ten minutes with every node still queued and no attempt at
  all, its nodes are now cancelled with the existing fenced call, its scope
  is released by the existing reconciliation, it is counted failed, and the
  full pipeline goes on (graph_unclaimed). A graph any of whose nodes was
  claimed is waited for as before.
- Publication of a finished graph was tried three times back to back, then
  counted failed. Now it is tried once per poll, up to three times, the
  count carried in the flight (in the state file, so a restart keeps it);
  a transient failure keeps the flight (graph_publish_retry). A given-up
  publication is counted failed once: recovery no longer takes the graph up
  again (which reset the count and counted the failure again every poll),
  and its snapshot is not remembered as seen, so a later build or change
  reconsiders the conversation and the same snapshot's admission answers
  with that graph, whose publication alone runs again.
- While a deferred graph holds graph admission, the scheduler says so once,
  with the deferred graphs' ids (graph_admission_held, reason
  graph_deferred).

With the list empty none of this runs and the state file is unchanged (the
new fields are omitted when empty). The existing test of a transient
publication failure now expects the retry on the next poll, as designed;
its assertions on one success, two calls and one generation are kept.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KvftUAdasWzxmMYdioZJWs

* docs: the graph executor's recovery from another boot, its deploy refusal, an unclaimed graph and publication retries

Describes, in the same-host graph executor section of the configuration
docs, what the three preceding changes do and the lines they write, and
that an older scheduler ignores the publication count and the given-up
publications on a rollback.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KvftUAdasWzxmMYdioZJWs

* deploy: a refused graph executor starts the scheduler with its graph list blank, so no graph is admitted that nothing could claim

With the graph executor switched on and its composed settings refused by
the deploy check, the hook started the full-pipeline shadow with the
scheduler's graph list as the environment document gave it. The scheduler
then admitted each listed conversation's graph, which nothing could claim.

Now the shadow's start is given one more Compose file, written by the hook
(docker-compose.shadow-graph-off.yml), that composes the scheduler's
DELPHI_SHADOW_GRAPH_CONVERSATIONS blank: the scheduler's existing off path,
so it admits no graph for that deploy. The hook reads the composed list back
first; should it not be blank, the shadow is not started at all, as when its
own settings are refused. One line beginning "Error:" says which, naming the
refused settings (the validator prints names only) and the list by key,
never a value. The full-pipeline shadow starts as before.

Nothing changes unless DELPHI_SHADOW_GRAPH_ENABLED=1 and the shadow are both
selected and the executor is refused: the shadow's start otherwise runs the
same command with the same files.

Tests: the refused case starts the shadow with the file, reads the list
back, writes one line and prints no value; a list that cannot be composed
blank starts no shadow; the valid case writes no file; the read-back's jq
program with the real jq.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KvftUAdasWzxmMYdioZJWs

* delphi shadow: a graph let go as unclaimed holds graph admission until the next build or an hour, doubling to a day, then its conversation is admitted again as a new graph

A graph let go because nothing claimed it kept its snapshot remembered as
seen, and the same snapshot's admission key answered with the cancelled
graph, so an unchanged listed conversation was never admitted again, with
no line saying so.

Now, when a graph is let go as unclaimed (decided before its cancel, or
found finished with every node cancelled and none ever claimed):
- its conversation's snapshot is forgotten as seen and observed, and the
  graph is recorded per conversation in the scheduler state
  (graph_unclaimed), with the count of graphs let go in a row in that env;
- no graph is admitted in that env until the next code version or the end
  of a hold of one hour, doubling with each let-go in a row up to a day,
  said once per hold (graph_admission_held, reason graph_unclaimed);
- the next admission of that snapshot names the let-go graph in its key,
  so it is a new graph (and the same one again on a lost reply or a retried
  publication); every other admission keeps the key it always had;
- a claim seen in that env resets the count (graph_unclaimed_cleared);
- recovery no longer remembers as seen a snapshot whose graph had every
  node cancelled.
The let-go line (graph_unclaimed) carries the reason, the count and the
hold. The full-pipeline shadow goes on throughout; while held, the graph
path makes no query. No SQL function, migration or Compose change.

Tests, each seen failing on the previous code first: the let-go
conversation is admitted again as a new graph after the hold, across a
restart, with the full pipeline going on meanwhile; at the next build at
once, with the hold doubling after a second let-go, and its given-up
publication later found and only published; found cancelled or past the job
deadline with no claim; a claim racing the cancel; the lines and their
counts from the scheduler binary. And for the ten-minute rule: nothing is
cancelled two seconds short of it; a graph whose claimed node was put back
in the queue, and one with a node out of the queue and no attempt, are not
let go; a re-admitted given-up publication leaves the given-up list and its
nodes ran once. The held-admission test reads the scheduler's lines as they
come instead of sleeping a fixed time.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KvftUAdasWzxmMYdioZJWs

* docs: a refused graph executor's blank list, the unclaimed hold and re-admission, the ten-minute constant, and what the copied-identity and publication-retry fixes do not cover

The ten minutes are a constant, with what a slow box does. A refused
executor starts the scheduler with its list blank. A graph let go as
unclaimed holds admission for an hour, doubling to a day, or until the next
code version, and its conversation is then admitted again as a new graph.

Corrected: a copied identity still confirms an attempt whose original is
alive but has stopped renewing for longer than a lease (writes stay fenced;
the cost is a false exit record and the step computed twice); an honest
reboot waits about one lease, up to a lease plus a reaper interval; a
given-up publication is found and only published again only when the
graph's specification is unchanged.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KvftUAdasWzxmMYdioZJWs

* delphi shadow: the admission-key comment no longer names the admission function

The one-admitter check refuses any file outside the shared entry that names
the SQL admission function; a comment added with the let-go key named it.
Comment only.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KvftUAdasWzxmMYdioZJWs

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor Author

Delphi Coverage Report

File Stmts Miss Cover
polismath/init.py 2 0 100%
polismath/benchmarks/bench_pca.py 128 107 16%
polismath/benchmarks/bench_repness.py 81 65 20%
polismath/benchmarks/bench_update_votes.py 38 28 26%
polismath/benchmarks/benchmark_utils.py 34 0 100%
polismath/components/init.py 1 0 100%
polismath/components/config.py 165 70 58%
polismath/conversation/init.py 2 0 100%
polismath/conversation/conversation.py 1293 264 80%
polismath/conversation/group_vote_counts.py 38 2 95%
polismath/conversation/manager.py 131 42 68%
polismath/database/init.py 1 0 100%
polismath/database/dynamodb.py 392 176 55%
polismath/database/postgres.py 414 95 77%
polismath/delphi_storage/init.py 1 0 100%
polismath/delphi_storage/codec.py 295 22 93%
polismath/delphi_storage/embedding_spool.py 58 6 90%
polismath/delphi_storage/golden_corpus.py 60 9 85%
polismath/delphi_storage/import_report.py 51 3 94%
polismath/delphi_storage/legacy_import.py 171 16 91%
polismath/delphi_storage/postgres.py 49 22 55%
polismath/delphi_storage/production_import.py 614 131 79%
polismath/delphi_storage/resource.py 179 32 82%
polismath/delphi_storage/scan_pacing.py 77 0 100%
polismath/delphi_storage/writer.py 196 13 93%
polismath/embedding_model.py 197 49 75%
polismath/empty_output.py 29 2 93%
polismath/engine_adapter.py 317 265 16%
polismath/job_child/init.py 364 51 86%
polismath/job_child/census.py 74 9 88%
polismath/job_child/input_bundle.py 106 7 93%
polismath/pca_kmeans_rep/init.py 5 0 100%
polismath/pca_kmeans_rep/clusters.py 257 38 85%
polismath/pca_kmeans_rep/corr.py 98 17 83%
polismath/pca_kmeans_rep/group_k_smoother.py 28 0 100%
polismath/pca_kmeans_rep/legacy_kmeans.py 197 7 96%
polismath/pca_kmeans_rep/pca.py 162 19 88%
polismath/pca_kmeans_rep/repness.py 226 5 98%
polismath/poller/init.py 4 0 100%
polismath/poller/admission.py 318 15 95%
polismath/poller/backfill.py 936 32 97%
polismath/poller/capacity.py 485 27 94%
polismath/poller/capacity_queue.py 248 13 95%
polismath/poller/coordinator_bridge.py 317 317 0%
polismath/poller/math_writer.py 120 3 98%
polismath/poller/promotion.py 112 4 96%
polismath/poller/readiness.py 368 13 96%
polismath/poller/rebuild_child.py 133 5 96%
polismath/poller/recompute_line.py 141 9 94%
polismath/poller/service.py 760 50 93%
polismath/poller/worker_pool.py 143 0 100%
polismath/queue/executor.py 221 111 50%
polismath/regression/init.py 4 0 100%
polismath/regression/clojure_comparer.py 188 20 89%
polismath/regression/comparer.py 891 640 28%
polismath/regression/datasets.py 126 27 79%
polismath/regression/recorder.py 36 27 25%
polismath/regression/utils.py 130 85 35%
polismath/replay/init.py 2 0 100%
polismath/replay/attribution_capture.py 49 0 100%
polismath/replay/axis_continuity.py 347 24 93%
polismath/replay/certify.py 999 59 94%
polismath/replay/crosslang.py 128 2 98%
polismath/replay/decision_ties.py 51 8 84%
polismath/replay/diagnostics.py 50 7 86%
polismath/replay/driver.py 73 1 99%
polismath/replay/dsn_admission.py 36 1 97%
polismath/replay/equiv_query.py 10 0 100%
polismath/replay/event_ingress.py 95 1 99%
polismath/replay/fixture_bundle.py 815 62 92%
polismath/replay/fixture_config.py 266 11 96%
polismath/replay/fixture_extract.py 285 12 96%
polismath/replay/fixture_generate.py 103 1 99%
polismath/replay/fixture_samples.py 157 14 91%
polismath/replay/fixture_selection.py 119 0 100%
polismath/replay/fixture_survey.py 147 11 93%
polismath/replay/legacy_pca.py 29 17 41%
polismath/replay/polarity.py 319 25 92%
polismath/replay/poller_equiv.py 776 99 87%
polismath/replay/prodclone.py 193 2 99%
polismath/replay/real_data.py 146 10 93%
polismath/replay/schedule.py 205 10 95%
polismath/replay/served_math.py 159 4 97%
polismath/replay/shard_bench.py 189 65 66%
polismath/replay/stagecompare.py 890 161 82%
polismath/replay/stages.py 256 40 84%
polismath/replay/stepcompare.py 86 1 99%
polismath/replay/store.py 120 11 91%
polismath/replay/tie_capture.py 107 19 82%
polismath/replay/types.py 58 6 90%
polismath/run_math_pipeline.py 284 93 67%
polismath/selection.py 48 0 100%
polismath/step_log.py 95 8 92%
polismath/tools/engine_rebuild.py 230 11 95%
polismath/types.py 6 0 100%
polismath/utils/cli_flags.py 15 0 100%
polismath/utils/clj_hash.py 44 0 100%
polismath/utils/env_flags.py 13 0 100%
polismath/utils/general.py 63 39 38%
polismath/utils/output_profile.py 66 2 97%
polismath/utils/serialization.py 10 5 50%
polismath/utils/vote_convention.py 159 5 97%
umap_narrative/700_datamapplot_for_layer.py 514 470 9%
umap_narrative/701_static_datamapplot_for_layer.py 310 310 0%
umap_narrative/702_consensus_divisive_datamapplot.py 435 275 37%
umap_narrative/801_narrative_report_batch.py 657 209 68%
umap_narrative/803_check_batch_status.py 268 148 45%
umap_narrative/llm_factory_constructor/init.py 2 0 100%
umap_narrative/llm_factory_constructor/model_provider.py 232 106 54%
umap_narrative/narrative_data.py 178 55 69%
umap_narrative/numerical_stages.py 97 17 82%
umap_narrative/polismath_commentgraph/init.py 1 0 100%
umap_narrative/polismath_commentgraph/cli.py 268 268 0%
umap_narrative/polismath_commentgraph/core/init.py 3 3 0%
umap_narrative/polismath_commentgraph/core/clustering.py 108 108 0%
umap_narrative/polismath_commentgraph/core/embedding.py 104 104 0%
umap_narrative/polismath_commentgraph/lambda_handler.py 219 219 0%
umap_narrative/polismath_commentgraph/schemas/init.py 2 0 100%
umap_narrative/polismath_commentgraph/schemas/dynamo_models.py 80 6 92%
umap_narrative/polismath_commentgraph/tests/conftest.py 17 17 0%
umap_narrative/polismath_commentgraph/tests/test_clustering.py 74 74 0%
umap_narrative/polismath_commentgraph/tests/test_embedding.py 55 55 0%
umap_narrative/polismath_commentgraph/tests/test_storage.py 87 87 0%
umap_narrative/polismath_commentgraph/utils/init.py 9 7 22%
umap_narrative/polismath_commentgraph/utils/converter.py 283 48 83%
umap_narrative/polismath_commentgraph/utils/group_data.py 332 279 16%
umap_narrative/polismath_commentgraph/utils/storage.py 616 486 21%
umap_narrative/reset_conversation.py 162 53 67%
umap_narrative/run_pipeline.py 459 158 66%
umap_narrative/topic_naming.py 148 11 93%
Total 26230 7390 72%

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant