Skip to content

runtime: lazily initialize imported-table function references - #14974

Open
leventov wants to merge 1 commit into
bytecodealliance:mainfrom
convex-in-prod:upstream-lazy-imported-table-funcrefs
Open

leventov wants to merge 1 commit into
bytecodealliance:mainfrom
convex-in-prod:upstream-lazy-imported-table-funcrefs

Conversation

@leventov

@leventov leventov commented Oct 23, 2026 •

Copy link
Copy Markdown

Active element segments targeting imported tables eagerly materialize native function-reference records, even when those entries are never used. This extends Wasmtime's existing lazy function-table initialization to eligible imported-table segments.

Store immutable function indices in shared module metadata and source-instance/range bindings on the table. The existing lazy-read path materializes a reference from its contributing instance on first access. Later segments override earlier ranges, and explicit table writes, including nulls, override deferred entries.

Per-table bindings are capped at 128 to bound metadata and first-access lookup costs. Unsupported segments, a full binding list or metadata allocation failure use eager initialization. Bounds checks, segment ordering and fuel charges remain at their existing initialization boundaries. The Store retains source instances after failed startup, and pooled table reuse clears deferred bindings.

The new fallible initializer suppresses host-call hooks to preserve the previous ref_func behavior; other calls keep their existing hooks.

Memory benefit

A local population comparison in the downstream convex-wasm-compiler integration, with approximately 47,000 side-module function references, saved 0.962 MiB of host resident memory per application instance:

MiB per instance Eager Deferred
Marginal host anonymous resident memory 2.862 1.900
Guest anonymous memory 7.109 7.109

Both variants used identical Core Wasm, inputs and collection policy. Host memory was estimated from the increase when adding four instances to four already warmed instances. This is a population estimate, including allocator effects. Allocated capacity was essentially unchanged; the saving comes from leaving unused function-reference pages untouched. Savings depend on how many references the workload uses.

Validation

All 16 tests in tests/all/table.rs and the call-hook instantiation regression passed in a focused harness against this branch with copying GC and pooling enabled. Coverage includes overlapping segments and table copies, explicit null writes, source lifetime after failed startup, partial initialization before a bounds trap, serialized reload, unchanged fuel charges, bounded-history fallback and pooled table reuse.

Keep eligible active-segment function indices in shared module metadata
and retain bounded source/range bindings on imported function tables.
Materialize VMFuncRef records through the existing lazy-read path instead
of touching all records at instantiation.

Apply bounds checks, segment ordering and fuel charges at their existing
startup boundaries. Tagged explicit writes override deferred entries;
unsupported segments and history overflow retain eager initialization.
The Store owns source instances even when later initialization traps.

Allow this internal fallible initializer to skip host-call hooks, matching
the previous infallible ref_func sequence without changing hooks for other
operations. Cover overlapping segments and copies, explicit null writes,
failed initialization, serialization, history bounds and pooled reuse.
@leventov
leventov requested review from a team as code owners October 23, 2026 22:54
@leventov
leventov requested review from cfallin and removed request for a team October 23, 2026 22:54
@github-actions github-actions Bot added the wasmtime:api Related to the API of the `wasmtime` crate itself label Oct 24, 2026
@alexcrichton

Copy link
Copy Markdown
Member

This is a pretty intense PR that's quite invasive to runtime internals in pretty subtle ways. In the end this is still bottoming out in an O(N) initialization per-table-segment, it's just with memset instead of libcalls. In that sense this is changing the constant factor overhead (in albeit a large way) but not changing the algorithmic complexity at all. I suspect the memory savings per-instance are probably the VMContext regions to store initialized VMFuncRefs, but I'm not certain.

Is the goal here speed or memory savings? If speed I'd feel better leaning on #14683 which should make ref.func significantly faster than it is today. If memory savings, however, that PR won't help and this'd still want to be considered.

@leventov

leventov commented Oct 26, 2026 •

Copy link
Copy Markdown
Author

The goal was memory savings.

Please forgive me an LLM reply (edited):

The memset and the saved memory concern different storage. The PR still clears every pointer-sized table slot. It avoids constructing the separate 32-byte VMFuncRef records inside VMContext until needed. Those record pages are not already initialized elsewhere. Wasmtime reserves their storage but deliberately does not zero or initialize the function-reference region during instance creation. Consequently, skipping record construction can leave whole pages without resident physical backing. If another initialization pass already wrote every page, this benefit would disappear.

My workload has 47k records. Empirical measurement shows that this change prevents > 900 KB of record pages from being allocated in physical memory. This saving is contingent on the specific access pattern, but if only a small fraction of all records are expected to be initialized, the saving should broadly generalise, save for specifically constructed artificial access patterns that touch every page.

If memory savings, however, that PR won't help and this'd still want to be considered.

Why wouldn't it help? From what I can gather, it helps me in practice.

@alexcrichton

Copy link
Copy Markdown
Member

Overall I know LLMs are good at whipping up an implementation, but at least personally I'm not comfortable landing this change. A feature like this I think needs to be extremely carefully designed because any possible bug can pretty easily lead to a security issue for example. I also feel that this is sort of a half-measures implementation where the table is still memset to 0 and it's just the VMFuncRef storage is lazily initialized. If we're going the route of full lazy initialization of all tables in all places it seems like we should shoot for a full-and-complete solution of lazily initializing everything, not just the VMFuncRef slots.

In general though I think this is going to require some careful design and review. For example this probably shouldn't end up sprinkling a flag to disable call-hooks which seems pretty unrelated. Similarly this ideally would centralize "apply lazy init" at compile-time in one location as it is today, as opposed to spreading it across a few locations. The testing here is also quite light comparatively with the weight of this change, and that would probably want to be improved, ideally with *.wast, and possibly with better fuzzer integration.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

wasmtime:api Related to the API of the `wasmtime` crate itself

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants