Skip to content

fix(sync): keep the import history across logout and import in linear time - #324

Merged
mohamede1945 merged 6 commits into
feat/sync-v2-wiringfrom
feat/sync-v2-importer
Oct 14, 2026
Merged

mohamede1945 merged 6 commits into
feat/sync-v2-wiringfrom
feat/sync-v2-importer

Conversation

@mohamede1945

@mohamede1945 mohamede1945 commented Oct 14, 2026 •

Copy link
Copy Markdown
Collaborator

Stacked on #323 (wiring), then #322 (engine) and the integration tests. Port step 3 of 5.

Summary

Moves the importer and reset onto the new engine, and fixes S14: logout no longer clears the import history.

S14: keep the import history across logout

  • The bug: deleteAllData cleared import_history. A tracked re-import after logout could then bring back legacy items the user had deleted while signed in, and push them as new creates on the next login (IMP-10).
  • iOS flow (checked in quran-ios):
    • LegacyDataImportCoordinator imports with trackHistory = true on the 3.1.0 upgrade, on every launch and on every CloudKit change.
    • logoutFromQuranCom() calls disable() before logging out, which turns legacy import off permanently.
    • Android doesn't track history.
  • Decision: keep the history across logout. It records what this device's legacy data has already brought in, and logout doesn't change that.
  • Why not tie it to the account: that wouldn't fix IMP-10. An import that runs after logout has no account.
  • Side benefits:
    • A late import that was queued before the reset only adds legacy items that were never handled.
    • A re-import can't leak legacy data into another account (TEST-PLAN C4).
  • Trade-offs:
    • An import still pending at logout is discarded with the other unsynced changes.
    • If the first login after the upgrade is a throwaway account, the legacy data stays in that account.
    • A reinstall or a second device upgrading still re-imports from CloudKit. That isn't a regression. A possible follow-up is an account-scoped "legacy imported" marker.

Import performance

  • The note creation-time nudge is now batched (NoteStore.batch() / CreationTimes). 20,000 notes sharing one time take about 0.3 s, down from about 15 s, and pick the same times as single inserts.
  • New index collection_ayah(sura, ayah): 20,000 memberships or highlights import in about 0.5 s, down from about 3 s.
  • Atomicity: the whole import is one transaction across both SQLDelight databases (one shared driver).

Reset and sync state (pinned by tests)

  • The wipe is one transaction: user data, sync_item/sync_meta (token, account, outbox, failures) and an epoch bump. A sync that started before the reset can't commit afterwards.
  • An import writes user data rows only, never sync state. So the engine's "token but no account ⇒ wipe" rule can't trip because of an import.
  • New scenarios:
    • LIFE-08 variant: logout while the GET is held.
    • IMP-11: an import made while a POST is held is pushed by the next sync.
    • IMP-04 now pins exactly what a restore (deleteExisting) pushes.

Tests

  • ./gradlew allTests --continue passes on every module and target:
    • persistence: 179 JVM, 187 iOS;
    • sync-pipelines: 299 JVM.
  • The integration suite passed 3 runs in a row. Only S4 (NOTE-12, the iOS Note ranges model, step 5) remains as a knownIssue.
  • Each commit builds on its own.
  • Reviewed by a separate reviewer, who verified the iOS import flow in quran-ios. Its two test nits are addressed in the last commit.

API

  • deleteAllData() no longer clears the import history; the README now says so.
  • PersistenceResetRepositoryImpl takes only SyncV2Database. It's internal to DI and hidden from ObjC.

🤖 Generated with Claude Code

Cleanup (rebased 2026-10-14)

  • Removed 4 comments that carried private issue tags.

mohamede1945 and others added 6 commits October 14, 2026 11:28
Logout cleared the import history with the user's data. A tracked legacy
import run after logout then brought back items the user had deleted while
signed in, and the next login pushed them as new creates (S14, IMP-10).

The history records what this device's legacy data already brought in,
which a logout doesn't change, so the reset now keeps it, like the
key-value store. Tying it to the account would not help: an import after
logout runs with no account and merges into whichever account signs in
next. Keeping it also stops a re-import from adding legacy data to another
account. The iOS app disables legacy import at logout, so it never relies
on the cleared history.

- PersistenceResetRepositoryImpl is the engine's wipe alone and no longer
  takes the app database; clearImportHistory is gone.
- IMP-10 passes without its knownIssue; a variant checks that a re-import
  after logout brings nothing into another account.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two lookups made an import quadratic in its size:

- The note creation-time nudge asked the database once per note, and its
  query reads every note at or after the asked time. Notes imported newest
  first, or sharing one time, rescanned the whole run each time: 20,000
  notes with one creation time took about 15 s. The importer now nudges
  through NoteStore.batch(), which reads the taken times once and skips
  each run of taken milliseconds after walking it (CreationTimes), with
  the same result as a single insert. The app's single inserts keep the
  query.
- The collections a verse is in (collectionAyahsAt) had no index, so each
  imported membership and highlight scanned every saved verse.
  collection_ayah now has an index on (sura, ayah). The schema stays at
  version 2, which is unreleased.

20,000 memberships import in about 0.5 s instead of 3 s, and 20,000 notes
in 0.3 s. Tests: a batch matches single inserts on random times, the
verse lookup's query plan uses the index, and a JVM test bounds the time
of a 20,000-note import (both fail without the change).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The reset clears the outbox and the item failures with the bases, the
  token and the account, and a sync that started before it can no longer
  write: its generation is superseded, so neither a downloaded note nor
  the token lands.
- An import writes domain rows only. Logged in, it leaves the token, the
  account, the epoch and the bases as they were, so a restore's deletes
  still reach the server; logged out, it writes no sync state, so it can
  never leave the token without an account that makes the engine wipe.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d restores

- LIFE-08 variant: a logout while the GET is held. Nothing the GET carries
  lands, nothing is pushed, and a re-login converges.
- IMP-11: a tracked import made while a POST is held is pushed by the next
  sync, next to what the held POST carried.
- IMP-04 pins what a restore with deleteExisting pushes: a DELETE for every
  synced item and a CREATE for every backup item, the note in both sets
  included (S14's restore note).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The logged-out import test now asserts that no sync type has a sync row
  and that the epoch is unchanged, not just the NOTE rows.
- The LIFE-08 variant's comment says what it proves: the logout cancels
  the held GET, so nothing it carries lands and nothing is pushed. The
  epoch guard against an answered GET is covered by the persistence
  reset test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drop the S-issue tags from the reset repository and the import
scenarios, and the bare [NoteStore.insert] KDoc on Batch.insert, which
batch()'s KDoc already covers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mohamede1945
mohamede1945 force-pushed the feat/sync-v2-importer branch from a4dcdd2 to 48724c6 Compare October 14, 2026 15:36
@mohamede1945
mohamede1945 merged commit 63a6e93 into main Oct 14, 2026
1 check passed
@mohamede1945
mohamede1945 deleted the feat/sync-v2-importer branch October 14, 2026 18:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant