Repository navigation
test(sync): add a multi-device sync integration test harness - #318
Merged
Merged
Conversation
This was referenced Oct 14, 2026
mohamede1945
added this pull request to stack #326
October 14, 2026 14:05
SynchronizationClientIntegrationTest only held four @ignore'd tests that called the testing API with a hard-coded, long-expired access token. They never ran, and the integration harness that follows replaces them with tests against a local fake server. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The sync-pipelines integration harness drives the production graph, but it needs each simulated device to sync only when the test says so and to give up on a failing run in milliseconds, not the 30 s, 5 s and ~13 s of production timings. - SynchronizationClientImpl.withSchedulerTimings returns a copy of a client with other scheduler timings. It is internal; no public API changes. - The sync-pipelines JVM test compilation becomes a friend of the syncengine JVM jar (-Xfriend-paths), so the harness can reach that seam. Kotlin does not support associating compilations across projects. Main code still sees only syncengine's public API. - sync-pipelines jvmTest gets kotlinx-serialization-json for the fake server. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FakeSyncServer serves GET and POST /v1/sync on loopback HTTP for the integration harness, so devices reach it through the production Ktor client. It fakes the network boundary only, and its rules come from quran.com-users-backend pre-live (modules/v1/sync and the services it calls). - Per-user ResourceLog and head: CREATE resets a row, timestamps are monotonic per user, and GET collapses one entry per resource with the stored row. The virtual Favorites entry takes a slot on page 1 and shifts later pages. - POST checks run in the backend's order: query, body validation (the route middleware), scopes, then the head (409 OutOfSyncError). The batch is atomic, collapses reading-bookmark creates per slot, and echoes the validated request data. - Joi rules per resource, including htmlStrip as sanitize-html with no allowed tags (tags dropped, text kept, escaped and trimmed), unknown keys rejected, and note UPDATE accepting only body. - Side effects: a new bookmark goes into Favorites, a membership create bumps its collection, removing from Favorites deletes an orphaned bookmark, BOOKMARK and COLLECTION deletes cascade, sessions upsert by verse and a move deletes the session at the target, reading bookmark slots upsert, clear as an UPDATE and project slot 1 onto a bookmark, duplicate collection names are 409 UniqueViolationError with details.name DUPLICATE, and system and protected collections are 403. - Errors use the API gateway envelope. BE1's details.mutationIndex is off by default and switchable. - Faults: fail or drop requests, outages, network down, lose a response after commit (504 or a dropped connection), block a POST, reject a chosen mutation, run web writes before a POST or between GET pages, malformed responses, vanished resources and a page-size cap. - A web actor writes with REST semantics, and the request log and ServerState support assertions, including a duplicate check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
syncScenario { } runs end-to-end scenarios against FakeSyncServer. Each
SyncDevice is the production AppGraph built with Metro's dynamic graph
factory, so devices are independent of the process-wide graph and of
each other. Tests use only QuranDataService and SyncAuthService.
- A device has its own in-memory SQLDelight database, the real engine
and Ktor client pointed at the fake server, and FakeAuthRepository at
the OIDC boundary (sign in to an account, expire the session, rotate
the token, fail remote logout).
- Short scheduler timings through the internal seam: nothing syncs on
its own, and a failing run gives up in well under a second.
- syncNow() calls triggerSync(immediately = true) and waits on the
public syncStatus flow for the run's end; lastRun keeps its requests.
login(), restart() and awaitSync() wait for the runs they start.
syncAll and syncUntilQuiescent sync several devices.
- assertConverged compares every device's public flows with the
server's live data; each scenario ends with a server duplicate check.
- knownIssue("Sx") is a strict expected failure: it fails once the
issue is fixed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Seven scenarios that pass against today's engine and exercise the
harness end to end: the first sync of an empty account, a note and a
collection with a verse round-tripping between two devices, a web change
reaching a device, repeat syncs being no-ops, a transient 500 retried in
the same sync, and knownIssue("S7") for editing a synced note, which
still fails with 422 on every attempt.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Backend PR #1932 (BE1) adds details.mutationIndex to an unexpected 500 ServerError too, not only to typed errors. The fake now does the same when BE1 mode is on, and a test covers a note range that names a missing verse (S11). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replace the friend-path seam with a parameter, as the harness review asked. - SynchronizationClientBuilder.build and SyncEnginePipeline.setup take schedulerTimings: SchedulerTimings? = null. Both are hidden from Objective-C, and null keeps the internal production defaults. - The harness passes its timings through the internal QuranDataServiceSynchronizationClientFactory, so it no longer casts to SynchronizationClientImpl or builds a client with production timings first. - Remove SynchronizationClientImpl.withSchedulerTimings and the -Xfriend-paths compiler argument. The iOS framework header is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From the harness review.
- Real backend 500s (a note range naming a missing verse, leading zeros,
a reversed range) throw a dedicated BackendCrash and answer 500
ServerError. Any other exception in the fake, including a failing
beforePost or betweenPages callback, is recorded in internalErrors so
the scenario can fail instead of passing it off as a backend error.
- Typed errors always carry details ({} by default), as the backend's
BaseError does. Only the untyped 500 has none.
- errorEnvelope switches between the API gateway envelope (default) and
the bare backend body.
- GET honours metadataOnly.
- BOOKMARK and COLLECTION_BOOKMARK read data.mushaf like the backend, so
a body with only mushafId gets the default group. A legacy mushaf is
copied onto mushafId in the echo.
- Document that dropConnection with a finite count is not deterministic,
because OkHttp may retry by itself. The default lost response stays
the deterministic gateway 504.
- List htmlStrip's unverified assumptions, and drop a needless !!.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…icates From the harness review. - syncNow(), login(), awaitSync() and restart() return the SyncRun: its status, whether it was skipped, and its requests. assertSucceeded(run) fails on a skipped run, so an expired session can no longer pass on the previous Succeeded. - knownIssue(id, description, expect) accepts only an AssertionError that expect allows. Other exceptions, harness timeouts and blocks that run past a short timeout fail the test as errors. syncRejectedWith(422) pins a run that failed for good. Ids are free form. - A run must start within 5 s and end within 15 s, or the harness throws HarnessTimeoutException. A run the lifecycle gate skips, which asks for no headers, is reported that way instead of hanging. The KDoc states the header-based skip detection as a harness assumption. - syncScenario fails when the fake server hit an internal error. The final duplicate check can run inside a known issue (knownDuplicates), and it throws AssertionError. - Convergence compares sorted lists, so duplicates on a device show. - syncUntilQuiescent rotates the device order each round, and needs a round with no skipped run, no POST and no head change. - restart() builds the graph and subscribes to syncStatus on Main, so the stored session's sync cannot finish before the recorder sees it. - The fake auth layer keeps tokens in the graph's token store, and the device database replaces the app's DriverFactory database in a binding that takes it. This clears Metro's unused-graph-input warnings. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Seven scenarios that pass against today's engine: - restart() resumes the stored session and pushes a pending edit once; - an expired session gives a skipped run that assertSucceeded rejects; - logout() clears local data, keeps the server's, and reports a failed revocation; - a lost response (gateway 504) is retried without a duplicate; - a write during a blocked POST is pushed by the follow-up sync; - a head change between GET pages is retried; - pagination reports total and hasMore, and the device reads every page. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The device database binding took the app's DriverFactory only to clear Metro's unused-graph-input warning. A fake factory can't replace it: - The JVM DriverFactory is a final class with no constructor parameters whose makeDriver() always opens jdbc:sqlite:quran.db in the working directory. - The production binding goes through makeDatabase, which caches one QuranDatabase for the whole process and ignores later factories. So the binding no longer takes the factory, its KDoc names both blockers, and Metro reports DriverFactory as an unused input for device graphs until persistence offers a factory seam. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- knownIssue requires expect, and the final duplicate check pins its own "has duplicates" failure. Its default timeout is 20 s. - Harness waits use withTimeoutOrNull, so they turn only their own timeout into HarnessTimeoutException and let knownIssue's timeout through. A run must start within 3 s. - The fake server ignores I/O errors from clients that went away. On close it releases held requests and waits for its handlers before scenarios read internalErrors. - The lost-response test asserts the single POST got the 504, which proves the commit happened before it. - New tests: a run the lifecycle gate skips fails fast with a clear message (FakeAuthRepository.holdNextLogout holds a logout after the reset began), and the engine reads the bare backend error envelope. - FakeAuthRepository keeps the token only in the graph's token store. - A skipped SyncRun has a null status. - dropConnection is gone; networkDown is the only connection-drop fault. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…gn-out
The old engine publishes a run's end (syncSucceeded or syncFailed) inside
the task function, before the Scheduler leaves WaitingForReply. An
IMMEDIATE trigger that lands in that gap is dropped ("Ignoring redundant
trigger: IMMEDIATE"), so a syncNow() right after login() or another
syncNow() could start nothing and time out after 3 s. That is the CI
flake in COL-06, LIFE-07 and CONV-09. syncNow() now repeats the trigger
every 500 ms until a run starts. SyncScheduler remembers an IMMEDIATE
that arrives during a run, so there the first trigger starts the run and
none is repeated.
FakeAuthRepository counts a header denial before AuthService sees the
empty headers and clears the session. So a skipped run could return while
authState still showed Success, which failed "an expired session gives a
skipped run that assertSucceeded rejects". A skipped run now returns once
authState no longer shows a session.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drop private plan references from the harness's comments: the BE1 name for the backend's mutationIndex (now cited as quran.com-users-backend#1932), S-issue tags, and "today's engine". The syncNow KDoc no longer names one scheduler, since the retrigger guards against any scheduler that drops an immediate trigger. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Remove FakeSyncServerTest's "metadataOnly returns only the head" and the fake's metadataOnly branch it tested. No engine or scenario sends metadataOnly, so the branch was dead code. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add an "Integration Tests" section to AGENTS.md: where the suite lives, why test bodies use only the public API and the fake server, how to write a scenario, how knownIssue works, the fault hooks, keeping FakeSyncServer in step with the backend when a synced type is added, and how to run it. The README's command list points to it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mohamede1945
force-pushed
the
test/sync-harness
branch
from
October 14, 2026 15:36
1d093a0 to
6fb7d5e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an end-to-end integration-test harness for sync. Each test drives the public
QuranDataService/AuthServiceof several simulated devices against one fake/v1/syncserver. The tests use only the public API and server state, so the same tests will run unchanged against a rewritten sync engine.Per device: a real in-memory database, the real Ktor/OkHttp client, and a production
AppGraphbuilt per device. Only the database and the OIDC auth layer are replaced. The network is faked only at the boundary, with a loopback HTTPFakeSyncServer.This PR adds the harness plus smoke tests; the scenario suite comes in the next stacked PRs.
What's included
FakeSyncServermirrors the backend's sync rules and side effects:lastMutationAthead check (409), and atomic POSTs;__default__collection;UniqueViolationError;htmlStrip, and note UPDATE accepting onlybody.Errors use the API-gateway envelope by default; the bare backend envelope can be switched on.
details.mutationIndex(from the backend PR) is off by default and can be switched on.Fault injection:
A request log is kept for assertions.
The harness (
syncScenario { }) gives you devices, a web actor,syncNow()returning aSyncRun(status, skipped, requests),syncUntilQuiescent(rotating order, unchanged head, no skipped runs), convergence checks and a duplicate check after every scenario.knownIssue(id, description, expect)is a strict expected failure:AssertionErrormatchingexpectcounts as the issue reproducing;Smoke tests:
knownIssue(S7: editing a synced note gets 422 forever);Deleted
SynchronizationClientIntegrationTest: 4@Ignored tests against the live testing API, using a JWT that had expired.Seams
schedulerTimings: SchedulerTimings? = nullonSynchronizationClientBuilder.buildandSyncEnginePipeline.setup. Both are@HiddenFromObjC, and the generatedShared.his unchanged. In tests, a run that keeps failing gives up in about 0.3 s instead of about 13 s.DriverFactoryis final, andmakeDatabasecaches one database per process, so each device binds its own in-memory database directly. This goes away when persistence is reworked.Known differences from the backend
Tests
./gradlew :sync-pipelines:jvmTest --rerun: 119 tests, 0 failures. The new tests take about 2 s, and were stable across 8 consecutive reruns../gradlew allTests --continue: passes on all targets.Reviewed in two rounds by a separate reviewer; all findings are addressed.
🤖 Generated with Claude Code
Cleanup (rebased 2026-10-14)
syncNow()repeats a "sync now" that a scheduler dropped at the end of a run. After a skipped run, the harness waits for the sign-out. It was verified with 50 loaded runs and no failures.metadataOnlypath.knownIssue/allowDuplicates, and how to model a new synced type inFakeSyncServer.