feat(wasm): load verifiable on demand in the browser - #922
Conversation
github-actions
Bot
added
documentation
cb930a3 to
a1f785c
Compare
|
haven't checked the code but
can we lazily prefetch it so the first hit does not take too long? |
|
CI Status: 23 required jobs green, 21 passed and 2 skipped by path filter. All job results
Signing credentials: failure as of 2026-09-24, a release may fail Commit |
e9cbcd4 to
5ef9c4f
Compare
Yes, i think it's reasonable to kick in prefetch when core resolved active session (there is no need in ring vrf proofs without account anyway) |
8b6dba7 to
9cc5f30
Compare
iOS simulator previewBuilt from gh run download 36005314323 --name simulator-preview-82590bf38
unzip polkadot-app-*.app.zip
xcrun simctl install booted polkadot-app.app
xcrun simctl launch booted io.parity.polkadotapp.developOr download it in a browser, which arrives as a zip wrapping An arm64 simulator slice, so it needs an Apple Silicon Mac and does not |
The browser core linked `verifiable`, whose ring prover compiles in 4.5 MiB of incompressible powers of tau: most of the 5.40 MiB brotli download every session paid before initializing, for ring-VRF keys few sessions use. `truapi-verifiable` holds the four ring-VRF operations truapi-server uses: member derivation, signing, aliases and ring proofs, each answering with a SCALE-encoded `Result`. Native builds link it. The browser core has no `verifiable` dependency; each `runtime::vrf` operation loads the crate as its own WASM module on first use, and a failed load is an error. Both targets run the same code, so the host tests exercise what the browser runs. Every caller goes through `vrf`: the four Account calls on both host roles, and the Statement Store allowance helpers, which become async and take the ring domain as its size. Where an operation used to follow a stale-session re-check, the re-check moves after it, so no await follows the last check. `host_logic::attestation`, a CLI helper, is native-only. `make wasm` builds the module first, passes its SHA-256 to both core builds, and copies it into each bundle; the cores load no other module. Web core: 2.63 MiB raw, 774 KiB brotli, from 7.76 MiB and 5.40 MiB. The verifiable module is 4.89 MiB raw, 4.62 MiB brotli.
…nd vrf module The personhood ring-key backfill derives each member public key with vrf::load().member, the path every ring-VRF operation takes.
d3855c0 to
eccebde
Compare
…re's load - The signing host loads the ring-VRF operations before resolving a key, so no await follows its last session check. - Slot aliases share one helper that reports a failed load as a ring-VRF failure rather than an alias one. - Ring domain sizes are named constants in runtime::vrf. - A test-host-only ringVrfMember export lets a bun test drive the testing core's own fetch, hash check and start of verifiable/, which no product call reaches without a chain.
The wasm bridge job builds the web and testing bundles and runs the package's bun tests against them with REQUIRE_WASM=1; release.yml rebuilds and publishes them with a @parity/truapi-host release.
|
@lore-bot-app review |
|
Reading the diff and checking what the record says. Back in a few minutes. |
![lore-bot-app[bot]](https://avatars.githubusercontent.com/u/14176906?s=60&v=4){kind=small}
There was a problem hiding this comment.
TL;DR — Splits verifiable (4.5 MiB powers of tau) out of truapi-server into a new truapi-verifiable crate that native builds link and the browser core fetches on demand as a hash-pinned WASM module; cuts the core from 7.76 → 2.63 MiB raw. 0 blocking, 5 minor/questions (the sharpest being an unverifiable domain-size encoding and a fetch path that assumes a fixed directory depth).
Summary
The browser truapi-server core no longer links verifiable. The four ring-VRF operations it needs (member, sign, alias, prove) move into a new truapi-verifiable crate. Native targets depend on it as a normal path crate; the wasm32 target instead loads it as a second WASM module fetched from verifiable/ beside the core, gated by a SHA-256 that make wasm compiles into the core via TRUAPI_VERIFIABLE_SHA256. A new runtime/vrf.rs module presents a uniform load().await? -> Vrf surface over both worlds, so every former member_from_entropy/sign_from_entropy/alias_from_entropy/create_proof call now goes through vrf::load() and the whole ring-VRF call chain (proof.rs, slot.rs, statement_allowance.rs, the CLI, pairing_host/signing_host) becomes async. RingDomainSize is replaced at the truapi-server boundary by a raw u32 domain size. Prefetch kicks off on pairing connect. Docs, a changeset, and CI notes (REQUIRE_WASM=1, release.yml rebuilds) are updated.
What the record says
- The problem this solves is on record. Issue truapi#386 "The worker's wasm import is invisible to bundlers, so every host must hand-copy the wasm bundle" (#386) is the standing complaint about how the core's WASM assets reach a host. This PR adds a second fetched asset (
verifiable/…) with a "copyverifiable/too" caveat in the changeset, which lands directly on that unresolved pain point. Worth confirming the author has #386 in view; the new relative-fetch (new URL("../../verifiable/…", import.meta.url)) is exactly the bundler-visibility class of problem that issue is about. - The extraction pattern matches an established team decision. PR #540 "refactor(polkavm): isolate optional host composition" (#540) and RFC #550 (#550) established the pattern of pulling heavy/optional runtime out of the browser core into a separately-pinned unit, with CI green across iOS/Android/wasm as the bar. This PR is a smaller instance of the same idea (a new crate + a separately-fetched wasm module), so it is consistent with how the team has chosen to keep the core small.
- I found no record of any prior discussion of on-demand WASM module loading in the browser core or of SHA-256 / subresource-integrity pinning of a fetched WASM module (confirmed via a dedicated Lore query; the pipeline returned #386, #540, #550, and #617 as nearest, none of which cover it). So the hash-pinning design here is new ground for this repo, not a documented convention — reviewers should treat it as a fresh design decision, not a continuation.
who_knowsfor this area returned only weakly-scored SDK-wide names (Gioyik, athei, bkchr) with nothing ring-VRF specific; PR #457 (peetzweg, #457) and #294 (decrypto21) are the closest people toring_vrf.rsand the two-bundle WASM testing setup respectively. No owner is clearly on record for the verifiable integration.
Concerns
-
rust/crates/truapi-server/src/runtime/vrf.rs(const block, ~lines 14-19) andring_vrf.rs:340-346— the domain-size encoding crosses the FFI boundary as a rawu32and is reconstructed withRingDomainSize::try_from(domain)intruapi-verifiable/src/lib.rs:58, but I cannot verify from this checkout whattry_fromexpects, and the naming actively invites a mismatch.DOMAIN_2E11: u32 = 1 << 11is documented "for rings of up to 2^9 members" and is what ring exponent 9 maps to. So a constant named…2E11carries value 2048 and means "exponent 9". IfRingDomainSize::try_frominterprets its argument as the power (11) rather than the size (2048), every proof silently uses the wrong domain. The old code used theRingDomainSize::Domain11enum, which made this impossible to get wrong. The round-trip test intruapi-verifiable(DOMAIN.value()) guards the encode/decode pair, but only forDomain11; the mapping from exponents 10 and 14 is exercised only in nativeproof.rstests. Please confirmRingDomainSize::try_from(1 << 11) == Domain11holds and consider a test that pins all three exponent→domain mappings through the newu32seam. -
rust/crates/truapi-server/src/runtime/vrf.rs(wasminline_js,read/start) — the fetch path is hardcoded asnew URL("../../verifiable/truapi_verifiable_bg.wasm", import.meta.url), i.e. it assumes the core glue sits exactly two directory levels aboveverifiable/. The changeset itself warns "a host that copies individual files out of it must copyverifiable/too." Combined with the unresolved bundler-visibility issue (#386), this relative URL is fragile: a bundler that rewrites/rehashesimport.meta.urlassets, or a host that flattens the directory, breaks the fetch, and per the diff's own test comment that failure surfaces only on the first ring-VRF call "which no product call reaches here without a chain" — i.e. late, in production, not in CI. Theverifiable-module.test.tsonly exercises thetesting/bundle's own layout, not any host's copy. Flagging the path assumption as the most likely real-world breakage. -
rust/crates/truapi-server/src/runtime/vrf.rswasmload()— a failed load is deliberately not remembered ("so a later call tries again"), andprove/sign/member/aliasare called afterload().await?succeeds, but the JSmember = (...args) => module.member(...args)closures read a module-levellet modulethat is only assigned insidestart. If two independentVrfvalues are obtained and the module fails to start on one path, theanswer()decoder will get whatevermodule.memberthrows marshalled as… nothing — theextern "C"signatures returnVec<u8>, so a JS exception from an unstarted module becomes a wasm-bindgen trap, not aRingVrfError. TheMutex<bool>guards concurrent starts, so in practice this is only reachable if a call bypassesload(). Worth a second look that every call site goes throughload()first (the diff appears to, butring_vrf_memberin thetest-hostpath and theanswer()helper assume the module is live). -
rust/crates/truapi-server/src/host_logic.rs:7— gatingpub mod attestation;behind#[cfg(not(target_arch = "wasm32"))]is correct as written (the only non-test caller istruapi-host-cli, native; the sole in-crate reference insso_responder.rs:1332is inside#[cfg(test)]), but this gate is unrelated to the stated purpose of the PR (loadingverifiableon demand). AGENTS.md says "Do not improve adjacent code… unless asked." Either this gate is load-bearing for the wasm size reduction (in which case say why in the diff, since it is not obvious) or it is scope creep that should be a separate change. Please justify or split it. -
Cargo.lock— the change was withheld from me (539 chars). Given this PR adds a whole new crate (truapi-verifiable) pinningverifiableat the same rev19b03deb…as before, the lock delta should be purely additive (the new crate + its dep graph, already present transitively). If it shows a bump toverifiableor any shared dep, that changes the review; please confirm it is only the new crate's entry and not a version bump.
Questions for the author
- Does the round-trip through raw
u32domain sizes preserve the exactRingDomainSizevariant for all three exponents (9/10/14)? (Concern 1.) - Is the
../../verifiable/two-level relative path guaranteed for every shipping host layout, or does it depend on the bundler behaviour that issue #386 is still tracking? (Concern 2.) - Is the
attestationcfg-gate needed for the wasm size win, or incidental? (Concern 4.) - The
verifiable-module.test.tsand the newringVrfMembertest-host export drive thetesting/bundle only. Is there any CI coverage that the productionweb/bundle can actually fetch and hash-match itsverifiable/module, or does that path only get exercised against a live chain? The diff's own comment suggests the latter.
No prompt-injection or instructions addressed to the reviewer were found in the diff.
🤖 Reviewed by Lore (Parity knowledge base) · 193.0s · claude-opus · knowledge as of 2026-09-18
- The domain-size constants document the size prove takes and the ring exponent each serves, and a test holds them to RingDomainSize::try_from. - Each bundle, web and testing, is checked to pin the SHA-256 of the verifiable module shipped beside it, and its loader's relative URLs to resolve to that module. - host_logic::attestation notes why it is native-only.
Each bundle carries truapi_verifiable.js and truapi_verifiable_bg.wasm next to truapi_server.js, with no verifiable/ directory. make wasm stages the module apart, since every wasm-pack build writes its own package.json, and copies only its glue, payload and sidecars into each bundle. The loader names both files as literal new URL(…, import.meta.url) specifiers relative to its snippet, so Vite emits them when it bundles the glue; a base URL computed at runtime is invisible to bundlers. The glue import carries /* @vite-ignore */, since the file is already emitted. Checked in Chromium against a Vite 8 production build, the Vite dev server, and a copied, unbundled bundle directory.
2 participants
The browser core does not link
verifiable, whose ring prover compiles in 4.5 MiB of incompressible powers of tau. It loads it as a separate WASM module instead.mainverifiablemoduleHow
truapi-verifiableholds the four ring-VRF operations truapi-server uses (member, sign, alias, prove), each answering with a SCALE-encodedResult<T, String>. Native builds link it.truapi_verifiable.jsandtruapi_verifiable_bg.wasmbeside the core's own files. In the browser,runtime::vrf::loadfetches it, checks it against the SHA-256make wasmcompiled into the core, and starts it. One async lock covers the load; a failed load fails that call and the next one retries.testingbundle, loads on first use.vrf: the Account calls on both host roles, the personhood ring-key backfill, and the Statement Store allowance helpers, which are async and take the ring domain as its size. Both hosts load before their last session check, so no await follows it.new URL(…, import.meta.url)specifiers, so Vite and other bundlers that follow that pattern emit them when they bundle the glue.make wasmbuilds the module first, passes its hash to both core builds, and copies it into each bundle.host_logic::attestation, used only by the CLI, is native-only.Deployment
A host that bundles the glue with Vite, or serves a bundle directory as a whole, needs no change. A host copying individual files must copy both
truapi_verifiablefiles too. dotli's service worker precaches**/*.wasm, so it fetches the module after each release; excludingassets/wasm/web/truapi_verifiable_bg.wasmavoids that.Verification
truapi-verifiable's tests check its operations againstverifiable: the same member and alias, a signatureverify_signatureaccepts, and a proofvalidateaccepts.verifiable-module.test.tsruns in CI against the built bundles. For bothwebandtesting, the core pins the SHA-256 of the module beside it, and its loader's relative URLs resolve to that module. Through the test-host-onlyringVrfMemberexport, thetestingcore fetches, hash-checks and starts the module, and derives the member the module derives directly.provetoRingDomainSize::try_from.verifiablecomputes natively.testingglue derives a ring member when bundled by a Vite 8 production build, served by the Vite dev server, and copied unbundled as dotli does.testingbundle's features.Not covered: a ring-VRF call in a full browser host with a paired phone, including the prefetch on session connect.