Repository navigation
[fence 5/9] libsql-server: migration target lifecycle and import capability - #45
Draft
tszymczyszyn-shopify wants to merge 6 commits into
Draft
tszymczyszyn-shopify wants to merge 6 commits into
tszymczyszyn-shopify wants to merge 6 commits into
Conversation
Add `NamespaceStore::create_target_quarantined` (and route the admin `CreateTargetQuarantined` command through it), which creates a namespace as a quarantined migration target atomically with namespace creation: - A name the server already knows (config in memory, or a namespace cache entry) is refused with FENCE_PRECONDITION_FAILED/namespace_exists without touching its gate. - Otherwise the controller publishes an in-memory target-creation gate before the metastore transaction writes the marker, config row, record and receipt. The gate refuses every class but maintenance and observability, and `check_available` refuses the name, so create, fork and `with()` neither store nor set anything up for it. - The committed record's quarantine gate replaces it; only then is the config published into the in-memory map (from the durable row, with the record's own block values) and the namespace loaded, so its first connection maker is created behind the quarantine gate. - The command runs on its own task under the transition lock; a replay returns the stored result and completes the publication and load of a commit that was not acknowledged, or of a creation interrupted between its marker and its commit. `CreateTargetRequest` is the typed entry point for the admin route and bulk import. The target's log id is not written back into the record, to keep the marker rule for same-revision records intact (documented). Co-authored-by: Tomasz Szymczyszyn <tomasz.szymczyszyn@shopify.com>
Add the operation-owned import path into a quarantined migration target and
the seal that ends it (docs/NAMESPACE_FENCE.md sections 7, 10.2 and 11).
- MigrationCapability (server-issued, fields private) and CapabilityPurpose.
The fence controller keeps the live capability set and a count of running
import calls; every published transition drops capabilities whose state,
owner or revision no longer match.
- FenceConnState::with_capability: the WAL admits a capability connection's
write transaction only while its capability matches the fence and is live;
a validation connection never writes.
- NamespaceStore::open_import_session and ImportSession::{with_raw,
load_dump}: the only way to write into TARGET_QUARANTINED. The dump loader
is split into load_dump_sql so that the loader used for namespaces created
from a dump also runs under an import capability.
- SealTargetImport closes import admission in memory, persists
TARGET_IMPORT_DRAINING (invalidating every import capability), waits on
release notifications for running import calls and for any import
transaction holding the write slot (force_rollback rolls it back at the
deadline), then persists TARGET_VALIDATING. A deadline leaves
TARGET_IMPORT_DRAINING durable and closed until the owner's seal resumes it.
Co-authored-by: Tomasz Szymczyszyn <tomasz.szymczyszyn@shopify.com>
Co-authored-by: Tomasz Szymczyszyn <tomasz.szymczyszyn@shopify.com>
Keep stale DRAINING receipts historical once their exact drain state and revision have been superseded, and preserve resumed attribution when a live drain completes. This prevents old commands from cancelling or completing work in newer states while keeping replay responses and audit classification accurate. Co-authored-by: Tomasz Szymczyszyn <tomasz.szymczyszyn@shopify.com>
Walk a source and two targets through every stored fence state and check the protection an older binary gets (docs/NAMESPACE_FENCE.md section 13.2): the config row's block_* fields hold the fence's mirror and nothing else in the row changes, a config write through the metastore is refused and changes nothing, and an older binary's delete of the config row fails on the fence row's foreign key. Release and write enable put the namespace's own values back. The test found that a restart overwrote the in-memory config of a released source or a writable target with the block_* values saved when the fence was acquired, although config writes after the operation had stored the namespace's own values in the row since: a namespace blocked after its release could come back unblocked. Loading the metastore, the target config publication and adoption now take the namespace's own config through fence_store::own_config, which uses the row as it is once the record no longer mirrors the fence. Crash the server at each persistence boundary of SetSourceReadFence, SealTargetImport and EnableTargetWrites, and while a read or seal drain waits for a reader or an import call, then restart it on the same directory: the prior or the committed state is recovered with no in-memory gate, reader or import call left, reads and import stay closed once their draining state committed, target writes open only if TARGET_WRITABLE committed, the marker is repaired, and a replay of the same command completes an interrupted drain at once. Co-authored-by: Tomasz Szymczyszyn <tomasz.szymczyszyn@shopify.com>
Add a test that the admin shell can neither read nor write a quarantined migration target, through its own entry point and with a raw write that skips its read admission, while the operation's import session keeps working. Map every acceptance requirement in docs/NAMESPACE_FENCE.md section 17 to the tests that cover it, correct the names of the corrupt-state tests, list the transition and CAS tests that cover ownership, revision and replay outcomes, and note the replica connection path in the code-path coverage table. Co-authored-by: Tomasz Szymczyszyn <tomasz.szymczyszyn@shopify.com>
This was referenced Oct 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Creates migration targets in quarantine, issues operation-owned import and validation capabilities, and implements the seal drain, validation, publication (readable, writes fenced) and the irreversible
EnableTargetWrites.open_import_sessioninto a quarantined target is the internal API that bulk import builds on.Also includes, next to the code they cover:
block_*mirror fix and its tests, plus crash-boundary tests for the read fence, seal and write enable;Review focus
Capability checks at the WAL (only the owning operation's import capability can write a quarantined target) and the seal/validate/publish/enable transitions.
Commits
Stack
Part 5 of 9, based on
namespace-fence/3-read-fence. Retargeted from #35 with no feature change: applied in order, the 9 PRs carry #35's fence diff (stable patch ID70d97d6a) onv0.9.30-shopify-patches. Review and land bottom-up, restacking after each squash or rebase merge.