Make a packed .spkg self-describe every network it supports

[maoueh]

Why

A published .spkg does not let a consumer such as substreams.dev describe the networks a package supports. Three defects sit behind that.

Derived initial blocks were frozen at pack time. substreams pack runs the full Reader.Read() pipeline before marshalling, so computeInitialBlock had already replaced every UNSET marker with a value derived for the packing network, and that value went into the artifact. On re-read the damage was invisible: ApplyNetwork only touches modules named explicitly under networks:, and computeInitialBlock skips any module whose initial block is no longer UNSET. Every module inheriting its start block stayed pinned to the packing network. Reproduced on a two-module package where mod2 consumes mod1, with mainnet.mod1 = 200 and sepolia.mod1 = 400:

override=""        mod1=200  mod2=200   <- mainnet, correct
override="sepolia" mod1=400  mod2=200   <- mod2 wrong, should be 400

Params escaped this only by accident: validateNetworks forces every network to declare the same param keys, so no param was ever left to inherit.

The networks map was too sparse to describe anything. Package.networks is packaged (field 13, contrary to first impressions), but held only the authored overrides. A site rendering a package page could not answer "what is mod2's start block on base?" without reimplementing computeInitialBlock and the module graph.

Network names were unvalidated. Nothing checked network: or the networks: keys against the Firehose network registry, so a typo published silently and consumers had no reliable key to map a package onto a real chain.

What changed

Densified networks map

densifyNetworks expands every network entry so each module carries an explicit initialBlock, and each module accepting params carries an explicit params value. No proto change — a dense map is just more explicit overrides than old readers used to see.

Per network: restore the authored initial blocks, apply that network's overrides, run computeInitialBlock, record the result for every module. Modules are restored afterwards, so the resolution that follows is unaffected. Params record the network's override when it has one and the module's carried value otherwise; modules without a params input are skipped, since ApplyParams rejects them.

It runs inside ApplyPackageTransformations, between validateNetworks and ApplyNetwork, and always rather than only on pack — it needs each module's authored initial block, and the UNSET marker is destroyed by computeInitialBlock later in that same function. Running it in the shared pipeline also keeps pack, info, run and gui in agreement. Idempotent on an already-dense package.

The frozen-initial-block defect disappears as a consequence: once every module is named, ApplyNetwork overrides them all and nothing is left to inherit. Cost is roughly 3 KB for a 20-module package supporting 10 networks.

Network-name warnings

New warning in warnIncompletePackage, already wired into pack, registry publish and registry verify. Each network name is looked up in the Firehose network registry: no match warns it is not a known registry ID or alias, several matches warn the alias is ambiguous and names the candidates, exactly one match is silent even when the name is an alias (eth-mainnet → mainnet). Warning only, never an error, so private and unlisted chains keep publishing. The lookup is injected as a parameter so the behaviour is tested against a fixture rather than the registry's live content.

substreams info

• --network, to inspect a package as any network it declares. run and gui already had the flag; without it an .spkg could only ever be read as its default network.
• --expand-networks, to list the per-module values. The Networks section is otherwise summarized to one line per network, since the expanded form now grows with modules × networks.
• Network and module ordering is sorted — the section iterated Go maps directly, so its output was non-deterministic between runs.

ApplyNetwork dead code

Its found flag was set unconditionally inside the module loop, making the "did you mean?" suggestion unreachable, so an initialBlock naming a nonexistent module was silently ignored. Densification makes the check moot for our own writes, but the function is exported. Its message also said param for module while reporting an initial block.

Compatibility

New CLI, old .spkg — the defect persists for already-published packages. The authored-versus-derived distinction was destroyed at their pack time and is not recoverable from the artifact; repacking from source fixes them.

Old CLI, new .spkg — works. The sparse-map code path reads a dense map unmodified.

Worth a second look before merge

Densification derives initial blocks for every declared network, not just the active one, so a package consistent on mainnet and inconsistent on sepolia now fails at substreams pack.

For: substreams run --network sepolia already fails today with the same error, so packing surfaces it earlier, where it is cheap to fix. Against: computeInitialBlock walks all modules regardless of the requested output module, so a package whose sepolia inconsistency sits in a branch nobody streams on that network used to pack fine and now will not. The alternative is to skip and warn about a network that fails derivation instead of failing the whole pack.

Testing

Built test-first throughout. densifyNetworks is covered for derived blocks, authored blocks, effective params, modules without a params input, the synthesized default-network entry, idempotency, and the conflicting-blocks error. The reported bug has a round-trip regression test that packs a package, re-reads it under a network override and asserts the derived module followed. renderNetworks and the network-name warnings have table tests. Existing reader_test.go expectations were updated to the densified values.

Full suite green, go vet clean.