Repository navigation
feat(sync)!: run sync on the sync-v2 engine and plain tables - #323
Merged
Merged
Conversation
This was referenced Oct 14, 2026
mohamede1945
added this pull request to stack #326
October 14, 2026 14:05
- SyncResult.Failed carries its cause (the server's answer or the network error), so the app can report the HTTP status and error code. - sync(account, push = false) fetches and merges only, for pulling while pushes back off after a 403 that may be a scope error. - SyncType.forbidsItems: only collections have items the server refuses with 403. For other types a 403 without a mutationIndex is the request's scope, so the run stops instead of reverting the edit or holding a create. - Collections split read-only (no rename or delete) from system (hidden from app lists): an older app's protected color collection is read-only but still a user's collection. - syncTypes(database) is the one type list, and SyncEngineV2.wipe(database, types) wipes in one transaction for a reset outside the engine. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…refresh errors - AuthRepository.forceRefreshTokens refreshes even when the token has not expired locally, for a 401 from the sync server (S25). AuthService exposes it as forceRefreshAccessToken: a refused refresh token ends the session, which authState reports. - A refresh that fails on the network, with a timeout or a 5xx no longer clears the session; the error is thrown and the next attempt retries (S3). Only a refused refresh token, or an unclassified error as before, ends it. - AuthService.currentAccountId returns the signed-in user's sub, which sync stores next to its token to detect an account switch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…w notes Two merge rules the integration suite needs, with the queries the app's repositories read: - Edit beats delete for a collection renamed locally and deleted remotely now recreates it as this device has it, verses included: the server's cascade of membership deletes is beaten too (COL-14). A collection recreated only because a verse was added keeps just the new verses, as before. - A new local note equal to a new remote note (same clientCreatedAt, ranges and body) is linked to it instead of pushed as a duplicate: two devices importing the same legacy notes, or an import over notes another device already synced (LIFE-05, IMP-03). - App-facing queries: a note by id, notes not archived (newest first), a body edit, a session by id, and every collection a verse is in. - kotlinx-serialization and the Ktor client are api dependencies: State and SyncApi expose them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SyncScheduler decides when the new sync client runs, keeping today's timings: 30 s after app start, 5 s after a local change (a burst runs once), at once for an immediate trigger, and 30 min after every run. Runs never overlap; a local change during a run runs again after it, other triggers during a run or while a retry waits are dropped, as before. A failed attempt is retried with backoff (200 ms x 2.5^n, up to 5 retries), unless it isn't retryable or asks for a longer backoff of its own. After the retries run out the periodic run is still scheduled, so sync keeps trying without user action (S21). The run reports a final failure itself, through willRetry, so the report lands before the next run or a logout's reset. Not wired yet; the next commits run the sync-v2 engine through it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Persistence and the wiring change together, so every commit builds: the old sync engine's bridge can't outlive the pending columns it reads. persistence: - MobileSyncDatabase: QuranDatabase (key-value store, import history) and sync-v2's SyncV2Database (plain domain tables, sync_item, sync_meta) on one driver. MobileSyncSchema is version 2 with foreign keys on; a version-1 database (the old pending columns) is dropped and created again, since mobile-sync is pre-release with no backward compatibility. - Repositories write plain rows through one store per type (NoteStore, CollectionStore, ReadingSessionStore, ReadingBookmarkStore), which the importer uses too: trimmed text, the note clientCreatedAt nudge, one highlight color per verse, the newest 20 reading sessions. App-facing queries hide archived notes; collection lists still hide system ones. - Reset wipes domain rows, sync state (token, account, outbox) and import history in one transaction and bumps the engine's wipe epoch. - API (deliberate): updateNote takes only the body (ranges are immutable); renaming or deleting a read-only collection throws ReadOnlyCollectionException, renaming onto a taken name throws DuplicateCollectionNameException; Collection.isReadOnly; PageReadingBookmark.mushafId. Ayah bookmark ids are "sura:ayah". - Removed: the old tables, *SynchronizationRepository, Remote*/LocalSync* inputs, PersistenceWriteBoundaryGuard, CollectionName(Key). The *Impl classes and PersistenceModule are hidden from Objective-C. - Tests ported to the new tables; the ones for removed sync columns and plumbing are dropped. Two bugs the port found are fixed: replacing a bookmark's only saved collection no longer moves its date, and a visit older than the newest 20 no longer throws. sync-pipelines: - SyncClient runs SyncEngineV2 through SyncScheduler for the signed-in account (the token's sub). A 401 forces one token refresh and runs again; a refused refresh ends the session (S25). A 403 that may be the token's scope stops pushes for its backoff while automatic runs keep pulling; "sync now" pushes. Each request is admitted through the session lifecycle gate. syncStatus keeps #317's semantics. - SyncEnginePipeline only holds the repositories; the result receivers and SyncLocalModificationDateStore are gone (the token lives in sync_meta). - Test harness: device databases are MobileSyncDatabase, FakeAuthRepository implements forced refresh, and updateNote call sites drop their ranges. The integration suite's knownIssues that this fixes are flipped next. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
NOTE-02, NOTE-03, NOTE-06, NOTE-09, NOTE-10, NOTE-14, NOTE-17 and NOTE-18, plus the smoke test's note edit, pass now: their knownIssues (S1, S2, S7, S12, S13, S15, S17) and NOTE-17/18's known duplicates are gone, assertions kept. - NOTE-13 is rewritten: ranges are immutable, so updateNote can't move a note and device and server agree on its range (S4's sync part). - NOTE-12 stays pinned on S4 alone: every range is stored, but the public Note model shows the first one (the iOS model change is step 5). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
COL-03, COL-04, COL-06, COL-07, COL-09, COL-10, COL-11, COL-12, COL-13, COL-14, COL-15, COL-16 and COL-17 pass now (S1, S2, S9, S11, S15, S16, S17, S26). COL-12 and COL-17 now also check the typed errors: DuplicateCollectionNameException, and ReadOnlyCollectionException for an older app's protected color collection. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
BMK-02, BMK-05, BMK-07, BMK-11, BMK-16, BMK-17, BMK-19 (S6, S15, S23) and HL-07 (S24) pass now; their knownIssues are gone and the assertions kept. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- RS-08, RS-10, RS-11, RS-12 and the RS-13 variant pass (S1, S5, S8, S19). RS-09's tablet visit is now newer than the phone's sessions, so the newest-20 prune every device runs keeps it and the test checks S1 alone. - RB-07 (S18) passes, and RB-06 (S13) checks the page's mushaf (7) through the new PageReadingBookmark.mushafId. - TXT-11 (S13) passes. TXT-09's phone collections are dated before the web's: linking a local create to a remote collection of the same name keeps the newer spelling, so the test was timing-dependent. - Fake fix: a reading-bookmark UPDATE that set name to null kept the old name. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- ERR-05, ERR-07, ERR-08, ERR-10 (S11), ERR-12 (S21) and ERR-19 (S27) pass. - ERR-06 (both) is rewritten: the engine merges a remote delete before it pushes, so S10's race can't happen; they check the row is gone before the POST and no 404 follows. - New: ERR-18 variant, pulls go on while pushes back off after a 403. - Adapted to the new engine: ERR-03 (one POST, one lost response), ERR-13 (the GET is merged before a broken POST), OBS-02 (a permanent 503, since a rejected mutation is now held), and the harness paths tests (a 422 with a code is held; a write during a blocked POST is pushed in the same run). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- LIFE-09 (S3) and the LIFE-14 variant (S25: one forced refresh) pass. - CONV-02, CONV-04 and CONV-08 pass (S2, S6, S7, S13). - IMP-07 (S11) and IMP-09 (S20) pass: the importer goes through the repositories, so it prunes to the newest 20 sessions. IMP-08 expects the second note's creation time nudged by 1 ms. - IMP-10 stays pinned on S14 (step 3). - The S6/S7 pins are no longer used and are removed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
README and AGENTS.md now describe the app database (MobileSyncDatabase), the stores that apply the write rules to every local write, the synced types, the sync client and scheduler timings, and the schema policy (no migrations; an older database is dropped). SYNCENGINE_ARCHITECTURE.md is marked as the previous engine's. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
After a forced refresh, a 401 again (a server that rejects every token) used to come back retryable, so each of the run's retries forced another refresh: six in a row. Now that second 401 fails the run without retries; the next run (periodic, or a user's) may refresh once more. Test: LIFE-14 variant, a server that keeps rejecting tokens gets one refresh per run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…the run FakeAuthRepository can refuse the next forced refresh, as a revoked refresh token does. The run then ends without a status of its own (skipped), the session is cleared, and authState is no longer Success (S25). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…h failures The OIDC library wraps every non-HTTP failure in TechnicalFailure, so keeping the session for any TechnicalFailure also kept it for errors that will never succeed (an unreadable response). Now only a TechnicalFailure whose cause chain holds an IOException or a timeout keeps the session; anything else ends it, as before. Tests: a refresh that times out keeps the session; one that fails for another technical reason ends it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The plain-tables port listed collections in row order. The app-facing order is back to the old one, created_at DESC (ties by the newer row), and a synced collection now keeps its own creation time: COLLECTION reads clientCreatedAt into an ignored createdAt field (written on insert, sent as clientCreatedAt), where it used to take clientUpdatedAt. Test: collections are listed newest first by creation time, on the device that made them and on one that synced them, including a collection from the web. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The repository test still expected row order, which the previous commit replaced with the app-facing newest-first order. It now checks that order, with the seeded Favorites (no creation time of its own) last. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… pulls A pull-only run during a push backoff (a 403 that may be the token's scope) reported Succeeded, so the 403 vanished from syncStatus while nothing could be pushed. Such a run now ends as skipped, which keeps the previous status: Failed stays until a run that pushes succeeds. Test: the ERR-18 variant asserts the periodic pull keeps the Failed status, and that the pushing "sync now" turns it into Succeeded. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ng retry An immediate trigger (a user's "sync now") during a run, or while a retry waited, was dropped, so a push the user asked for could wait for the local change delay, the retry backoff or the periodic run. The scheduler now remembers one during a run and runs (and pushes) right after it, ahead of a local change's delay or a retry; one while a retry waits replaces the retry with an immediate run. triggerSync's KDoc says so. Tests: an immediate trigger during a run runs right after it; one while a retry waits runs at once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…unt again The port dropped the lifecycle tests' token assertions with the old token store. The fixture now runs the real PersistenceResetRepositoryImpl over its in-memory database (the fake only adds failures and counts), seeds sync_meta with a token and an account, and checks that every logout and the persisted reset recovery wipe them, while a logout aborted by a local-auth-clear failure keeps them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…vice is cleared QuranDataService.clear() only cancelled the scheduled work: the scheduler's command loop and scope, and the sync client's HTTP client, outlived the service. SyncClient is now AutoCloseable; clear() closes it, which stops the command loop, the timers and any running attempt for good, and closes the HTTP client. cancelAndJoin after close returns at once. Tests: close stops the command loop for good (later triggers run nothing); clearing the service closes its sync client. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The ERR-06 rewrite only showed the remote delete is merged before the POST. The fake server can now hold a GET (ServerFaults.blockGet), and both ERR-06 tests add the catalog's race: the device edits R while the GET carrying R's delete is held, before the merge commits. Edit beats delete, so R is recreated with the edit (the note keeps its range; the collection its new name), with no 404. The earlier assertions stay. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ERR-03 armed a second lost-response fault that the new engine's single POST never reaches, and only asserted that some response was lost. It now arms one fault for the POST that carries the verse and the session, and asserts exactly one 504. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…collection TXT-09 dates the phone's collections before the web's, so the server keeps "ÉTUDE". The new variant covers the other side of that rule: the phone's "étude" is newer, so the link keeps it and renames the server's collection, on every device. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…r tie-break Coverage the persistence port lost, now at the app level: - Bookmark dates of verses from the web: a membership's time is its clientUpdatedAt, or its clientCreatedAt, or the server's timestamp; the bookmark was last updated when its earliest surviving saved membership was made. The fake server can now leave client times out of COLLECTION_BOOKMARK entries (ServerFaults.omitMembershipTimes), and the web can add a verse with explicit client times. - Reading sessions visited at the same time are listed by verse, on every device, the way the prune ranks them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The sync-v2 repositories are stand-ins for the app's, but two of their companion functions are production code: persistence seeds Favorites with CollectionsRepository.ensureFavorites, and its reading-session store prunes with ReadingSessionsRepository.prune, the same ranking the sync type's prune uses. Their KDoc now says so, and the classes say they are test stand-ins. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drop the S-issue tags the wiring added to auth, the sync client and scheduler, and the flipped integration scenarios, and rewrite ERR-06's KDoc without the old-versus-new engine history and the catalog's name for the race. Fix comments the wiring made stale: - ServerFaults.pageSizeCap no longer warns about a client that derives its page count from its own limit; sync-v2 pages by hasMore. - sync-v2's NotesRepository and notes.sq no longer claim every local note write goes through NotesRepository; the app and the importer write through persistence's NoteStore. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Now that QuranDataService runs on sync-v2 and the scenarios pass end to end, remove 34 sync-v2 acceptance tests that replay a scenario's steps and assertions and pin nothing more precise: - notes: NOTE-01 with CONV-02, TXT-01/03, NOTE-10, NOTE-11, NOTE-15 - collections: CONV-02, LIFE-01, BMK-10, COL-04, BMK-02, BMK-05, BMK-03, COL-09, COL-10, COL-14, BMK-16, COL-15, COL-13, COL-17, HL-09 - reading sessions: RS-01, RS-11, RS-13, RS-04, RS-07, IMP-09 - reading bookmarks: RB-02, RB-05, RB-07, ERR-19, RB-04, RB-06 - highlights: HL-01, BMK-06; a Favorites removal from the web: BMK-11 and BMK-17; a create refused for the scope: ERR-18 Remove the harness's "the engine reads the bare backend error envelope": SyncErrorParsingTest covers parsing the backend's own shape, and ERR-07 the 422 hold. Merge LIFE-14 with its server-rejection variant. Since a 401 forces a refresh, both are the same case; the kept test asserts the first request got 401, one refresh, and the note landed. In persistence, remove "a new database has Favorites" (the collections repository test checks the same row) and the one-line overload test for addReadingSession, and fold four "already deleted" cases into their "missing" twins, since hard deletes take the same path. getNotesFlow's archived check joins getAllNotes'. Move NOTE-18's KDoc back onto its test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
persistence no longer generates its own Note table class, so the app-facing Note reaches Swift as Note instead of Note_. The demo still used Note_ and did not compile; CI does not build it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add a Sync section to the README's architecture: the sync-v2 engine, persistence on plain tables, the sync-pipelines wiring, when the scheduler runs and backs off, how auth feeds sync (the account from the token's sub, a forced refresh after a 401), and what syncStatus shows. Drop the stale claim that imports track sync mutations. In AGENTS.md, point "adding a synced type" at syncTypes and fold the development note into the sync-v2 steps, adding the integration fake. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mohamede1945
force-pushed
the
feat/sync-v2-wiring
branch
from
October 14, 2026 15:36
20d10cc to
0bef849
Compare
This was referenced Oct 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Moves persistence onto
sync-v2's plain tables and wires the new engine in behindQuranDataService. The public API is unchanged apart from the deliberate changes listed below. The oldsyncenginestays compiled for its shared types, but it never runs; it gets deleted in step 4.note_range, collections, verse-in-collection, reading bookmarks, reading sessions), plussync_item/sync_meta.SyncClientrunsSyncEngineV2through a newSyncScheduler.retryableandretryAfterMillis.syncStatussemantics are the same as feat(sync): add immediate sync trigger and sync status #317.sub, and switching accounts wipes local sync data (S3).lastUpdated;addReadingSessionthrew for a visit older than the newest 20.Review fixes
Each fix is its own commit, with a regression test:
TechnicalFailurecaused by a network error or a timeout keeps the session. Anything else ends it, as before.clientCreatedAt.syncStatusduring a push backoff: a pull-only run keeps the 403Faileduntil a run that pushes succeeds.QuranDataService.clear()closes the sync client, which stops the scheduler's command loop and closes the HTTP client.ensureFavoritesandpruneare documented as production API.Integration suite
The integration suite goes from 57
knownIssues to 2, with no scenario going from pass to fail. The flip commits remove the pins area by area. The two that remain:Notemodel exposes only the first range. The iOS model change is step 5.About 15 scenarios had their premise adapted to the new engine; each change is explained in its commit message. For example, ERR-06's race can no longer happen, because the remote delete is merged before the push.
Public API changes (iOS/Android follow-ups)
updateNote(id, body[, timestamp]): the range parameters are removed, because ranges are immutable.updateCollection/deleteCollectionthrowReadOnlyCollectionException.updateCollectionthrowsDuplicateCollectionNameException.Collection.isReadOnlyis added.PageReadingBookmark.mushafIdis added: 5, 6 or 7, the representative mushaf of the server's group.Shared.h:SharedNote_is renamed toSharedNoteandSharedCollection_toSharedCollection, so iOS renamesNote_toNote;noteshides archived notes;"sura:ayah";system:are rejected;Tests
./gradlew allTests --continuepasses on every module and target::umbrella:linkDebugFrameworkIosSimulatorArm64and:demo:android:assembleDebugsucceed.The integration suite passed 3 runs in a row with
--rerun.Persistence tests went from 247 to 171. 155 were dropped:
The rest were rewritten for the new tables, and new tests cover the new write rules.
Open
syncStatusstill reportsSucceeded. No public API lists held items yet; that is planned together with sync history.🤖 Generated with Claude Code
Cleanup (rebased 2026-10-14)
Note_→Note), which this PR had broken.syncStatus.