Add a context::clock WASM intrinsic - #880
Open
maoueh wants to merge 2 commits into
Open
Conversation
Its generated `pb/mod.rs` declared four modules — `google.protobuf`, `sf.substreams.rpc.v2` and `sf.substreams.sink.service.v1` — whose `.rs` files are not in the directory, so the crate failed to build before any of its own code was reached. Drop the declarations that have no backing file. This blocks rebuilding the package's committed .spkg.
Modules could only reach the block clock by declaring
`source: sf.substreams.v1.Clock` as a module input, which forces the
dependency into the module graph even when the clock is only needed
occasionally. Add a `context` host module exposing it as a call instead:
#[link(wasm_import_module = "context")]
extern "C" {
fn clock(output_ptr: *mut u8);
}
`context::clock` writes the clock to the guest heap as an encoded
`sf.substreams.v1.Clock` and stores the `{ptr, len}` pair at `output_ptr`,
the same convention the `state` getters use. It returns nothing: unlike
`state::get_at` there is no "not found" case. A marshalling failure is
raised as non-deterministic so it is never cached.
Implemented for wasmtime (the default runtime) and wazero (which also
covers wasip1/tinygo-v1); the JavaScript/v8 runtime is not covered.
`context` joins `env`, `state` and `logger` as a namespace WASM extensions
cannot register into, keeping it available for what lands there next.
Ergonomic Rust bindings will follow separately in substreams-rs.
Verified end to end by a new `assert_context_intrinsics_0` module in the
complex_substreams testdata package, which asserts inside the WASM that
the intrinsic matches the Clock it received as a module input, exercised
over blocks 0-10 on both runtimes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds a
contextWASM host module with a single intrinsic,context::clock, which writes the current block clock into guest memory as an encodedsf.substreams.v1.Clock:It follows the same calling convention as the
stategetters — the host allocates the payload on the guest heap and writes a{ptr, len}pair atoutput_ptr. It returns nothing: unlikestate::get_atthere is no "not found" case, the clock always exists. A marshalling failure is raised as non-deterministic, so a transient failure is never cached into the store.Implemented for wasmtime (the default runtime) and wazero (which also covers
wasip1/tinygo-v1). The JavaScript/v8 runtime is not covered.contextalso joinsenv,stateandloggeras a namespace WASM extensions cannot register into, keeping it available for whatever belongs in it next.Why
Until now the only way for a module to reach the clock was declaring
source: sf.substreams.v1.Clockas a module input, which forces the dependency into the module graph even when the clock is needed only occasionally. An intrinsic is callable at any point during execution and costs nothing when unused.Scope note: network id was dropped
The branch name says
network-idbecause it started as "expose the deployment's network id to modules", with the clock added alongside. The network id half is not in this PR. No compelling use case turned up for it, and anetworks:block in the manifest already parameterizes a package per network, which covers the same ground.The investigation is worth recording, since it also shows the network id would have been awkward to deliver:
--advertise-chain-nameis looked up in the networks registry, canonicalized from aliases, and cross-checked against the genesis block id. It just never reaches the substreams tier1 app.InfoServer.Init, which needs the block stores to be reachable and so resolves well after tier1 config is built.advertise_chain_name, and the ones that don't include eth-mainnet, eth-hoodi, eth-sepolia, arb-one, eth-bsc-mainnet and eth-polygon-mainnet.So any tier1-side network id would have started out unset on the busiest deployments, ruling out failing fast on a missing value and leaving a degraded mode to design around. That's the constraint to solve first if the feature comes back.
Testing
Call.DoClockround-trips through protobuf.assert_context_intrinsics_0module in thecomplex_substreamstestdata package asserts, inside the WASM, that the intrinsic matches the Clock it received as a module input. A mismatch panics in the module, so atrueoutput for every block is the proof. Exercised over blocks 0-10 via the tier2 integration test, on both wasmtime and wazero.The committed
.spkgis rebuilt, which shifts every module hash in that package.Notes for reviewers
complex_substreamstestdata crate could not compile at all, because its generatedpb/mod.rsdeclared four modules whose.rsfiles are absent from the directory. That had to be fixed before the package could be rebuilt.substreams::context::clock()) belong in substreams-rs and will follow separately. Until then modules declare the extern by hand, as the test fixture does.