Web: In-memory connection pool

[simolus3]

As a follow-up to #1029 adding support for in-memory databases (see in particular the discussion in #1029 (comment)), this adds a multi-threaded variant of the in-memory file system with support for concurrent reads / writes.

This is a fairly specialized VFS, relevant where persistence is not required but fast and parallel queries are. As most users probably need this this, this is exported separately (as @powersync/web/extra/shared-memory-pool), which ensures this doesn't impact default bundle sizes.

To use this, import { InMemoryWriteAheadLogPool } from '@powersync/web/in-memory-wal-experiment'; and open PowerSync databases with the opened: new InMemoryWriteAheadLogPool({ numWorkers: 2 }) option.

This implementation avoids navigator locks and and doesn't give databases a name, each instance is a fresh and completely independent database file. It does require shared memory and cross-origin isolation though. For the most part, this VFS works like the OPFS write-ahead VFS, except that:

1. The write-ahead overlay is not persisted or serialized, we literally just append blobs to a WAL and track offsets in a JavaScript Map we can send between workers. This avoids checksums and other complex state tracking.
2. Checkpointing doesn't rely on navigator locks, since all shared array buffers are bound to a single tab we can manage the checkpointing process with a local Semaphore instead.
3. This doesn't use a WAL2 approach, a checkpointer blocks everything else instead. I believe this should be fine since checkpointing is essentially a memcpy, so fairly fast overall. We could parallelize these copies by delegating them to workers, but that feels like a premature optimization at this point.

This shares the local connection state management with the OPFS write-ahead pool. I have also tested this VFS in the react supabase todolist demo.

AI use: Mostly manual, I used Claude for reviews and minor correctness / concurrency fixes.

[changeset-bot]

🦋 Changeset detected

Latest commit: 3253a06

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages

Name	Type
@powersync/web	Minor
@powersync/diagnostics-app	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR