embind: value_object/value_array arguments allocate on every call, and there is no fill-in-place read path

[dimokol]

value_object and value_array make bindings pleasant to use, but that pleasantness currently comes with a per-call allocation cost that pushes per-frame APIs off of them. I ran into this profiling a physics hot loop that reads a handful of poses, velocities and raycasts from wasm every frame. Two related gaps:

1. Argument marshaling allocates per call. The toWireType for both value arrays and value objects runs var ptr = rawConstructor() and destructors.push(rawDestructor, ptr) on every call (src/lib/libembind.js, currently around lines 967 and 1075 on main). Each call with a composite argument creates a temporary C++ object that is freed right after the call, plus JS garbage for the destructor bookkeeping. In practice a call taking one small 3-field value_object id costs ~96 B of JS garbage no matter what it returns, and each value_array vec3 argument adds ~112 B. Scalar-only calls measure 0.

2. Reads always materialize fresh objects. There is no supported way to say "fill this caller-owned array/object in place", so every getter that returns a value type allocates its result. (Out params came up from the C++ signature angle back in #611; the per-frame-JS angle is what bites today.)

Numbers from @isaac-mason's box3d.js (embind bindings for the box3d physics engine), measured with forced-GC heapUsed deltas over 20k-call rounds, node v24, emsdk 6.0.2 defaults (-O3, default DYNAMIC_EXECUTION, no EMBIND_AOT):

call shape	B/call
getter taking one value_object id	96
same, plus one value_array vec3 argument	208
raycast: two vec3s + a filter in, result object out	~180
scalar-only call	0

For a physics loop at 60 to 120 Hz that's 1 to 2.5 KB of steady garbage per step. box3d.js now works around embind entirely on these paths: raw writer functions that take loose scalars and fill static wasm scratch, plus generated monomorphic JS readers that unpack the id/vector objects into scalars at the call site and copy results into caller-owned arrays (isaac-mason/box3d.js#3 and isaac-mason/box3d.js#4 have the details and a repro script). That took every reader and the raycast to 0 B/call, and made the raycast about 4.7x faster than the value-returning form. It works, but it's custom machinery that any embind library with per-frame reads has to reinvent from the outside.

As far as I can tell EMBIND_AOT (#20673) doesn't change this, since the allocations live inside the registered types' toWireType rather than in how invokers get generated. The AOT pathway does feel like a natural place to generate specialized fast invokers though.

Would there be appetite for making this first-class? Rough shapes it could take, though you'd know best what fits embind's design:

• Allocation-free by-value marshaling: value types have a known fixed size, so the temporary could live in a per-call stack/arena slot released after the invoke, skipping both the rawConstructor heap round-trip and the destructors array.
• Opt-in out-param bindings: a way to declare that a bound function fills a caller-provided array/object instead of returning a fresh one, with the invoker writing fields in place.
• Or more modestly: a documented, supported low-level hook that libraries can build zero-alloc facades on without reaching into internals.

Happy to help however it's useful: more measurements, prototyping one of these, or testing against box3d.js as a real consumer.

[brendandahl]

"Allocation-free by-value marshaling" sounds reasonable as a starting point. I've never really thought of value_array and value_object being memory efficient, but I guess we could make them better. We could determine if the C++ type is trivial to construct/destruct, then use stackSave, stackAlloc, and stackRestore to avoid the new and destructors running. I can throw some LLM tokens at it, if you haven't already prototyped something.

[dimokol]

I threw in some tokens myself, and it works, uploaded the prototype here: #27610.

Short version: on box3d.js rebuilt with the patched toolchain, the one-id getter goes 96 B/call to 0, id+vec3 208 to 16, the raycast 179 to 37, the world step 125 to 61. A stock rebuild with an unpatched current toolchain reproduces the before column exactly, so the delta is all patch. test_embind, test_embind_aot_js and test_embind_no_dynamic pass locally.

Three things I learned along the way that shaped it. The dominant per-call garbage comes from the per-field-write destructors arrays inside value marshaling rather than the invoker path: V8 sinks them in microbenchmarks but not in megabyte-scale modules, which is why small tests look clean while real libraries measure ~32 B per field. Second, the stack temporaries have to be zero-filled: the heap path's new T() value-initializes, and bindings that register only a subset of fields rely on the rest being zero (box3d's world def keeps its task-callback pointers unset, and garbage there is a function-table crash). Third, Asyncify and JSPI-async invokers defer destructor work past the call, so they keep the heap path.

The PR is a draft pending tests and a ChangeLog entry, happy to finish those once the shape looks right to you.