refactor!: ephemeral arrays by mverzilli · Pull Request #22162 · AztecProtocol/aztec-packages

mverzilli · 2026-03-30T17:29:10Z

Implements a new type of array especially designed to use in oracle interfaces between Aztec.nr and PXE. The memory space of of these arrays is isolated by contract call frame and lives in memory. This makes them faster to work with, but it also removes the need for a lot of boilerplate to instantiate them, use them as params, and ultimately dispose of them.

Closes F-136

mverzilli · 2026-03-31T14:25:56Z

noir-projects/aztec-nr/aztec/src/ephemeral/mod.nr

+/// Ephemeral arrays are designed for transient communication between PXE (TypeScript) and contracts (Noir) during
+/// simulation, for example, note validation requests or event validation responses.


I think they are useful for more than that, but I want to grow a bit more confidence on the API and oracle design before encouraging people to use it happily

benesjan

This is great!

Reviewing it has completely exhausted me though 🥵

noir-projects/aztec-nr/aztec/src/ephemeral/mod.nr

benesjan · 2026-04-09T04:38:15Z

noir-projects/aztec-nr/aztec/src/ephemeral/mod.nr

+/// Ephemeral arrays are designed for transient communication between PXE (TypeScript) and contracts (Noir) during
+/// simulation, for example, note validation requests or event validation responses.


Suggested change

/// Ephemeral arrays are designed for transient communication between PXE (TypeScript) and contracts (Noir) during

/// simulation, for example, note validation requests or event validation responses.

/// Ephemeral arrays are designed for passing data between PXE (TypeScript) and contracts (Noir) during simulation,

/// for example, note validation requests or event validation responses.

Same as above: "Transient" is overloaded here. In Aztec it already means "transient notes" (notes created and nullified in the same tx), so reusing it for ephemeral arrays risks suggesting a connection that doesn't exist. Worse, it's redundant — "ephemeral" already conveys the lifetime, and the sentence goes on to say "during simulation."

I'd drop it.

benesjan · 2026-04-09T04:48:29Z

noir-projects/aztec-nr/aztec/src/ephemeral/mod.nr

+    }
+
+    /// Initializes an empty ephemeral array at the given base slot, clearing any pre-existing data.
+    pub unconstrained fn initialize(base_slot: Field) -> Self {


This seems unsafe as silently clearing pre-existing data hides bugs. I would instead just assert here that there is no pre-existing data. If there is a callsite where this behavior was desirable then I would just wipe the array before calling initialize here.

Mmm, I see your point but at the same time it's a bit annoying to have to combine at+clear everywhere you just know you're working with disposable data. How about I nuke this one and make clear return Self? That way we can use clear to instantiate and the name makes it clear that you might be wiping stuff

I think I would just rename this to clear_and_init if this is the target use case of this function. Or do you think it's too long and ugly?

I see that you went with your approach. Fine with me 👍

benesjan · 2026-04-09T05:07:57Z

noir-projects/aztec-nr/aztec/src/ephemeral/mod.nr

+        });
+    }
+
+    #[test(should_fail)]


Why don't we test error messages here and below? This risks the test passing when it fails for some other unrelated reason (e.g. like TXE not running or smt.)

benesjan · 2026-04-09T05:23:21Z

yarn-project/pxe/src/contract_function_simulator/ephemeral_array_service.ts

@@ -0,0 +1,107 @@
+import { Fr } from '@aztec/foundation/curves/bn254';
+
+/** In-memory array service for transient data during a single contract call frame. */


Suggested change

/** In-memory array service for transient data during a single contract call frame. */

/** In-memory store for ephemeral arrays scoped to a single contract call frame. */

Dropping "transient"

benesjan · 2026-04-09T06:31:13Z

noir-projects/aztec-nr/aztec/src/messages/processing/mod.nr

-        LOG_RETRIEVAL_RESPONSES_ARRAY_BASE_SLOT,
-        scope,
-    )
+    log_retrieval_requests.clear();


This is also only defensive clearing in case get_pending_partial_notes_completion_logs was called multiple times, right?

Would put here a comment if yes.

correct and good idea on adding a comment

benesjan · 2026-04-09T06:33:38Z

noir-projects/aztec-nr/aztec/src/messages/processing/offchain.nr


-/// Capsule array slot used by [`sync_inbox`] to pass tx hash resolution requests to PXE.
+/// Ephemeral array slot used by [`sync_inbox`] to pass tx hash resolution requests to PXE.
 global OFFCHAIN_CONTEXT_REQUESTS_SLOT: Field = sha256_to_field("AZTEC_NR::OFFCHAIN_CONTEXT_REQUESTS_SLOT".as_bytes());


Unrelated but here we don't use the "base slot" terminology. So if you think we want to keep base slot would rename it here (I think I would just drop base). Can be for followup.

I'll drop the base slot thing jargon, you're right it is unneeded

benesjan · 2026-04-09T06:44:31Z

yarn-project/pxe/src/contract_function_simulator/oracle/interfaces.ts

  setContractSyncCacheInvalid(contractAddress: AztecAddress, scopes: AztecAddress[]): void;
  emitOffchainEffect(data: Fr[]): Promise<void>;
+
+  // Ephemeral array methods — in-memory per-call-frame arrays for transient data.


Suggested change

// Ephemeral array methods — in-memory per-call-frame arrays for transient data.

// Ephemeral array methods

Once again - to not overload transient

benesjan · 2026-04-09T06:47:36Z

yarn-project/pxe/src/contract_function_simulator/oracle/utility_execution_oracle.ts

@@ -494,30 +496,41 @@ export class UtilityExecutionOracle implements IMiscOracle, IUtilityExecutionOra
  }

  public async getPendingTaggedLogs(pendingTaggedLogArrayBaseSlot: Fr, scope: AztecAddress) {


Suggested change

public async getPendingTaggedLogs(pendingTaggedLogArrayBaseSlot: Fr, scope: AztecAddress) {

// Deprecated, only kept for backwards compatibility until Alpha v5 rolls out.

public async getPendingTaggedLogs(pendingTaggedLogArrayBaseSlot: Fr, scope: AztecAddress) {

Useful to have this well-tracked

benesjan · 2026-04-09T06:50:15Z

yarn-project/pxe/src/contract_function_simulator/oracle/utility_execution_oracle.ts

+    return this.ephemeralArrayService.newArray(maybeLogRetrievalResponses.map(LogRetrievalResponse.toSerializedOption));
+  }
+
  public async getMessageContextsByTxHash(


Suggested change

public async getMessageContextsByTxHash(

// Deprecated, only kept for backwards compatibility until Alpha v5 rolls out.

public async getMessageContextsByTxHash(

Co-authored-by: Jan Beneš <janbenes1234@gmail.com>

Thunkar · 2026-04-09T09:32:38Z

noir-projects/aztec-nr/aztec/src/ephemeral/mod.nr

+    /// callback.
+    ///
+    /// It is **not** safe to push new elements from inside the callback.
+    pub unconstrained fn for_each<Env>(self, f: unconstrained fn[Env](u32, T) -> ())


IDK if I'm a fan of for_each being used for mutation. Much like JS, I think it would be better to have dedicated methods (filter, map etc) but maybe Noir is not expressive enough

Agree. This is a bit of a cargo cult of the CapsuleArray interface, I was kind of assuming "Noir is not expressive enough" is why we ended up with this

noir-projects/aztec-nr/aztec/src/messages/discovery/mod.nr

benesjan

LGTM!

Implements a new type of array especially designed to use in oracle interfaces between Aztec.nr and PXE. The memory space of of these arrays is isolated by contract call frame and lives in memory. This makes them faster to work with, but it also removes the need for a lot of boilerplate to instantiate them, use them as params, and ultimately dispose of them. Closes F-136 --------- Co-authored-by: Jan Beneš <janbenes1234@gmail.com>

AztecBot · 2026-04-09T14:09:40Z

✅ Successfully backported to backport-to-v4-next-staging #22441.

BEGIN_COMMIT_OVERRIDE cherry-pick: fix(pxe): support custom PrivateKernelProver and unify EmbeddedWalletOptions (#22348) fix: update testnet compatibility test and bust cache on pinned artifact changes (#22429) fix(pxe): support custom PrivateKernelProver and unify EmbeddedWalletOptions (backport #22348) (#22391) refactor!: ephemeral arrays (#22162) END_COMMIT_OVERRIDE

BEGIN_COMMIT_OVERRIDE refactor!: ephemeral arrays (#22162) END_COMMIT_OVERRIDE

mverzilli added 8 commits March 30, 2026 12:44

initial exploration

2769578

replace fetch_pending_tagged_logs impl with volatile arrays

8839209

volatile -> ephemeral

0b5a2a8

use ephemeral for notes and events validation and storage

be3ff2c

port message context resolution to ephemeral arrays

7ae06bf

port pending partial notes request arrays to ephemeral

952813a

simplify signatures thanks to ephemeral arrays

5e031a8

minutiae

d5eac0b

mverzilli marked this pull request as ready for review March 31, 2026 13:08

mverzilli requested a review from nventuro as a code owner March 31, 2026 13:08

mverzilli changed the base branch from next to merge-train/fairies March 31, 2026 13:08

merge from fairies

6642a5b

mverzilli removed the request for review from nventuro March 31, 2026 13:17

mverzilli added 2 commits March 31, 2026 14:13

some initial tests of ephemeral array isolation

50073ec

minutiae

578e874

mverzilli commented Mar 31, 2026

View reviewed changes

mverzilli added 12 commits April 8, 2026 12:38

Merge branch 'merge-train/fairies' into martin/volatile-arrays

e0f78ae

add initialize to guarantee clean initialized arrays

da44a14

add unit tests for initialize

5d4a2f8

comply with oracle naming conventions

8f5ee5e

more renames

27776e6

compatibility of validateAndStoreEnqueuedNotesAndEvents

b0094ac

fix compilation errors

603748d

fix compilation error in TokenBlacklist contract

ed83bbf

fix compile error

d1bb695

update oracle version numbers

193fa56

temporarily bump protocol contract oracle version to get full ci

f844bd8

fix ephemeral array unit tests

f0cf0c2

mverzilli requested review from Thunkar and nventuro April 8, 2026 18:05

mverzilli requested a review from benesjan April 8, 2026 18:05

benesjan reviewed Apr 9, 2026

View reviewed changes

Apply suggestion from @benesjan

b04ecea

Co-authored-by: Jan Beneš <janbenes1234@gmail.com>

Thunkar reviewed Apr 9, 2026

View reviewed changes

address code review feedback

9a7d346

Thunkar reviewed Apr 9, 2026

View reviewed changes

noir-projects/aztec-nr/aztec/src/messages/discovery/mod.nr Outdated Show resolved Hide resolved

update oracle hash

0ba10b0

mverzilli requested a review from benesjan April 9, 2026 09:36

more pr feedback

9a717f7

mverzilli requested a review from Thunkar April 9, 2026 10:09

mverzilli added the backport-to-v4-next label Apr 9, 2026

mverzilli added 2 commits April 9, 2026 10:16

Merge branch 'merge-train/fairies' into martin/volatile-arrays

cea0ef2

fix test

8ccdbf0

Thunkar approved these changes Apr 9, 2026

View reviewed changes

benesjan approved these changes Apr 9, 2026

View reviewed changes

mverzilli enabled auto-merge (squash) April 9, 2026 14:00

AztecBot added ci-wakeup-pr-after-merge and removed ci-wakeup-pr-after-merge labels Apr 9, 2026

mverzilli merged commit d8fbc34 into merge-train/fairies Apr 9, 2026
25 checks passed

mverzilli deleted the martin/volatile-arrays branch April 9, 2026 14:07

AztecBot mentioned this pull request Apr 9, 2026

chore: Accumulated backports to v4-next #22441

Merged

AztecBot mentioned this pull request Apr 9, 2026

feat: merge-train/fairies #22440

Merged

github-merge-queue bot pushed a commit that referenced this pull request Apr 9, 2026

feat: merge-train/fairies (#22440)

cf27bac

BEGIN_COMMIT_OVERRIDE refactor!: ephemeral arrays (#22162) END_COMMIT_OVERRIDE

		/// Ephemeral arrays are designed for transient communication between PXE (TypeScript) and contracts (Noir) during
		/// simulation, for example, note validation requests or event validation responses.

		@@ -0,0 +1,107 @@
		import { Fr } from '@aztec/foundation/curves/bn254';

		/** In-memory array service for transient data during a single contract call frame. */

	/** In-memory array service for transient data during a single contract call frame. */
	/** In-memory store for ephemeral arrays scoped to a single contract call frame. */

	// Ephemeral array methods — in-memory per-call-frame arrays for transient data.
	// Ephemeral array methods

		@@ -494,30 +496,41 @@ export class UtilityExecutionOracle implements IMiscOracle, IUtilityExecutionOra
		}

		public async getPendingTaggedLogs(pendingTaggedLogArrayBaseSlot: Fr, scope: AztecAddress) {

	public async getPendingTaggedLogs(pendingTaggedLogArrayBaseSlot: Fr, scope: AztecAddress) {
	// Deprecated, only kept for backwards compatibility until Alpha v5 rolls out.
	public async getPendingTaggedLogs(pendingTaggedLogArrayBaseSlot: Fr, scope: AztecAddress) {

	public async getMessageContextsByTxHash(
	// Deprecated, only kept for backwards compatibility until Alpha v5 rolls out.
	public async getMessageContextsByTxHash(

Conversation

mverzilli commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

benesjan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

benesjan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

AztecBot commented Apr 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

mverzilli commented Mar 30, 2026 •

edited

Loading