Implement `GET beacon/pool/payload_attestations` endpoint by SamAg19 · Pull Request #10038 · sigp/lighthouse

SamAg19 · 2026-09-11T02:45:52Z

Issue Addressed

Addressed the following 2 checklist items in #9045

Add GET /eth/v1/beacon/pool/payload_attestations endpoint
Modify GET /eth/v1/beacon/pool/payload_attestations SSZ response to use ProgressiveList

Proposed Changes

Commit 28837d6 : Payload attestation messages are now aggregated on insert instead of storing the raw messages and aggregating them only when get_payload_attestations function is called at the time of block production. This further changed the pool which now holds PayloadAttestation instead of PayloadAttestationMessage, keyed by PayloadAttestationData. get_payload_attestations becomes a filter-and-clone function as well
Commit 779548e : Adds get_all_payload_attestations function which returns the aggregated payload attestations in the pool based on the optional slot filter. An omitted slot returns all of the aggregated payload attestations in the pool. The new function will be used by the GET beacon/pool/payload_attestations endpoint and kept separate from the existing get_payload_attestations function
Commit fe3fa54 : Adds GET beacon/pool/payload_attestations as per beacon API spec of the endpoint. It serves JSON or SSZ, depending on Accept header with the SSZ response returning a ProgressiveList as per beacon-APIs#635 changes
Commit dd93df7 : http_api and network tests now assert on the aggregation bits for the message's data instead of the pool's message/aggregate count

…d-attestations

chong-he · 2026-09-14T02:32:51Z

CI is failing. You can check for lints with make lint-full if it is involving tests

…d-attestations

SamAg19 · 2026-09-14T04:31:46Z

CI is failing. You can check for lints with make lint-full if it is involving tests

Addressed ef18127 @chong-he

## Description After a restart, a Lighthouse node can fail to resume syncing on a Gloas network. This can happen when the node persists a Gloas beacon block before it fully imports the payload envelope. A restart then clears the in-memory pending payload cache. Sync downloads the envelope and custody columns for the block again, but reprocessing the known block does not restore its signed execution payload bid. Lighthouse cannot join the downloaded components, and the lookup can run out of attempts. Dependent blocks then stop progressing. Custody-column and execution-proof processing now load the bid from the persisted block when the cache is empty. Envelope processing reuses the bid from the block that it already loaded.

## Issue Addressed Part of FOCIL (EIP-7805). This puts the inclusion list store from sigp#9744 onto `BeaconChain` and adds the read wrappers that will be used by gossip verification, fork-choice enforcement, and block production. ## Proposed Changes * Add an `inclusion_list_store` field to `BeaconChain`, initialise it from `ChainSpec` in `builder.rs`, and prune it in `per_slot_task`. * Add `inclusion_list_committee`, which resolves the ordered committee for a slot and returns it together with the dependent root used to key the store. * Add `get_inclusion_list_transactions`, `get_inclusion_list_bits`, and `is_inclusion_list_bits_inclusive` as wrappers around the store reads. * Add `BeaconChainError::InclusionListStoreError`. * Add tests covering each read path, the epoch boundary case, and pruning through `per_slot_task`. This is independent of sigp#9846, and nothing is calling these wrappers yet. Gossip verification will be the follow-up. ## Additional Info The wrappers take an anchor block root rather than using the head. `with_committee_cache` only resolves a block's own or next epoch shuffling, while inclusion list reads use `slot - 1`. Once per epoch, that falls into the previous epoch and fails when using the head root. There's a test covering this case. `with_committee_cache` already gives us the attester shuffling decision root, so we get both the committee and the store key from the same lookup. This is also the root that the producer puts in the message as `dependent_root`. `get_inclusion_list_transactions` only needs to resolve the shuffling for the dependent root and then discards the committee, since that root comes out of the same lookup. Co-authored-by: Eitan Seri- Levi <eserilev@gmail.com>

## Issue Addressed Part of FOCIL (EIP-7805). This adds `engine_newPayloadV6`, as specified in [Bogota](https://github.com/ethereum/execution-apis/blob/main/src/engine/bogota.md). ## Proposed Changes * Add `ENGINE_NEW_PAYLOAD_V6` and the `new_payload_v6` capability. * Add `inclusion_list_transactions` to `NewPayloadRequestHeze`. * Add `new_payload_v6_heze` and dispatch Heze requests to it. * Add `inclusion_list_satisfied` to `PayloadStatusV1` for the new `inclusionListSatisfied` response field. * Route the method in the mock EL, and fix its Heze fork check, which required V5. * Add tests for deserialising the response with and without `inclusionListSatisfied`. Nothing constructs a `NewPayloadRequestHeze` yet, so this doesn't change any behaviour. I also cleaned up the two now stale `TODO(heze)`s. ## Additional Info I kept `inclusion_list_satisfied` on `PayloadStatusV1` rather than introducing a `PayloadStatusV2`. The new field is `BOOLEAN|null`, and the existing fields already use `Option` for the same reason. The alternative would be to introduce a versioned superstruct like `PayloadAttributes`, but that would mean making changes outside `execution_layer`. I'm fine with going that way if that's preferred. The Heze arm in `payload_notifier.rs` will need the transactions from the store. That part follows once sigp#9960 merges.

…API 5/5) (sigp#9807) Final PR of the Gloas builder API stack: - sign builder request-auth and preferences (REQUEST_AUTH signing domain, web3signer message type) - add the builder configuration store (`builder_definitions.yml`) and book documentation - add the builder-preferences service and switch block production to `POST` produceBlockV4 with a `BuilderConfig` body, threading the `Eth-Builder-Url` header through block publication - remove the now-unused legacy `GET` produceBlockV4 client methods

## Description Adds the per-validator builder configuration endpoints from [ethereum/keymanager-APIs#88](ethereum/keymanager-APIs#88). - `GET /eth/v1/validator/{pubkey}/builder_config` returns the configuration in use for the validator. - `POST /eth/v1/validator/{pubkey}/builder_config` replaces and persists the full configuration on a per-validator basis. It does not merge with the previous entry. - `DELETE /eth/v1/validator/{pubkey}/builder_config` removes the stored entry, so the validator inherits the global configuration again. The API stores per-validator entries under `validator_configs` in `builder_definitions.yml`. Changes made by `POST` and `DELETE` apply to subsequent builder preference publication and Gloas block production without a restart. When `auth_data` is omitted, global and per-validator builder configurations use the builder URL's lowercase ASCII hostname, following [ethereum/builder-specs#168](ethereum/builder-specs#168), [ethereum/beacon-APIs#642](ethereum/beacon-APIs#642) and [ethereum/keymanager-APIs#94](ethereum/keymanager-APIs#94). URLs with and without a trailing slash produce the same auth data. Closes sigp#9796 Closes sigp#10076 Co-authored-by: Mark Mackey <mark@sigmaprime.io>

…d-attestations

chong-he · 2026-09-23T08:20:43Z

+             query: PayloadAttestationPoolQuery,
+             accept_header: Option<Accept>| {
+                task_spawner.blocking_response_task(Priority::P1, move || {
+                    let fork_name = ForkName::Gloas;


This will always hardcode the fork name as Gloas, which is not ideal. We should get the fork name from the current slot using: chain.spec.fork_name_at_slot, you can refer to other endpoint examples for this.

Addressed bc54365

chong-he · 2026-09-23T08:22:52Z

+    // GET beacon/pool/payload_attestations
+    let get_beacon_pool_payload_attestations =
+        get_beacon_pool_payload_attestations(&beacon_pool_path);
+


We can move this above/below a few lines so that it doesn't intervene in between the post endpoint

Addressed bc54365

chong-he · 2026-09-23T08:24:04Z

+             accept_header: Option<Accept>| {
+                task_spawner.blocking_response_task(Priority::P1, move || {
+                    let fork_name = ForkName::Gloas;
+                    let attestations = chain.op_pool.get_all_payload_attestations(query.slot);


nit: prefer to stick with the payload_attestation name, i.e., let payload_attestations so that it is clear and not confused with normal attestations. Same with other namings

Addressed bc54365

chong-he · 2026-09-23T08:25:56Z

-        self.payload_attestation_messages
+    /// Returns all known `PayloadAttestation` objects, optionally filtered by slot.
+    /// Unlike `get_payload_attestations` this applies no block-root filter and no cap
+    pub fn get_all_payload_attestations(


For now we have 2 functions: get_payload_attestations and this get_all_payload_attestations. We could combine two into one I think, as what is done in attestations here:

lighthouse/beacon_node/operation_pool/src/lib.rs

Lines 783 to 793 in e423a66

pub fn get_filtered_attestations<F>(&self, filter: F) -> Vec<Attestation<E>>

where

F: Fn(&AttestationData, HashSet<u64>) -> bool,

{

self.attestations

.read()

.iter()

.filter(|att| filter(&att.attestation_data(), att.get_committee_indices_map()))

.map(|att| att.clone_as_attestation())

.collect()

}

That's a bit cleaner imo

Yeah this is much more cleaner. Implemented bc54365

chong-he · 2026-09-23T08:43:14Z

+            "valid payload attestation should be sent to network"
+        );
+
+        self.harness.extend_slots(1).await;


Is this extend_slot required?

At the time, extend_slots was added so that the following cases would be checked in a slot after the head:

All payload attestations are returned when the slot query parameter is omitted

Only the attestations for the queried slot are returned when a slot is given

A slot with no payload attestations in the pool returns an empty list

but thinking about it now, this can be checked without the extra extend_slots so removed it in c5a4fb6

chong-he · 2026-09-23T08:43:50Z

+    pub async fn test_get_beacon_pool_payload_attestations_invalid_slot(self) -> Self {
+        // The typed client takes an `Option<Slot>`, so a malformed slot can only be sent raw.
+        let url = self
+            .client
+            .server()
+            .expose_full()
+            .join("/eth/v1/beacon/pool/payload_attestations?slot=abc")
+            .unwrap();
+
+        let response = reqwest::Client::new().get(url).send().await.unwrap();
+
+        assert_eq!(response.status(), StatusCode::BAD_REQUEST);
+
+        self
+    }


Maybe no need this test? because currently other endpoints also don't have this invalid slot. This looks a bit of trivial

I added this to test out the invalid slot revert but since none of the other endpoints have it, it makes sense to remove it. Removed c5a4fb6

chong-he · 2026-09-23T08:45:45Z

+        self
+    }
+
+    pub async fn test_get_beacon_pool_payload_attestations_after_reorg(mut self) -> Self {


Not sure about this test too. Is this necessary?

The test was added to ensure payload attestations can be fetched regardless of whether they have been incorporated into a block as mentioned in the spec but I now noticed that none of the other pool endpoints have such a test, so for uniformity it makes sense to remove this one. Removed c5a4fb6

chong-he · 2026-09-23T08:49:18Z

+    pub fn pooled_payload_attestation_bits(&self, data: &PayloadAttestationData) -> usize {
+        self.chain
+            .op_pool
+            .get_payload_attestations(data.slot, data.beacon_block_root)
+            .iter()
+            .filter(|attestation| &attestation.data == data)
+            .map(|attestation| attestation.aggregation_bits.num_set_bits())
+            .sum()
+    }
+
+    /// Number of PTC positions held by `validator_index`, which is how many bits its message sets.
+    pub fn ptc_seats(&self, slot: Slot, validator_index: u64) -> usize {
+        self.chain
+            .canonical_head
+            .cached_head()
+            .snapshot
+            .beacon_state
+            .get_ptc(slot, &self.chain.spec)
+            .expect("should get PTC")
+            .0
+            .iter()
+            .filter(|index| **index as u64 == validator_index)
+            .count()
+    }
+


These two functions are duplication of the one in http tests.rs. We should make things simpler and not have this duplication. We can put these functions in beacon_chain/src/test_utils.rs so that they only appear once

Done c5a4fb6

… address non-test related comments

…d-attestations

SamAg19 added 6 commits September 9, 2026 23:17

Aggregate payload attestations on insert instead of at block production

28837d6

Add op pool accessor for all pooled payload attestations

779548e

cargo fmt

8ae8efa

Add GET /eth/v1/beacon/pool/payload_attestations endpoint

fe3fa54

Merge remote-tracking branch 'upstream/unstable' into get-pool-payloa…

3f60282

…d-attestations

Assert pooled payload attestation bits instead of aggregate count

dd93df7

SamAg19 requested a review from jxs as a code owner September 11, 2026 02:45

Merge branch 'unstable' into get-pool-payload-attestations

0a06fd2

chong-he added waiting-on-author The reviewer has suggested changes and awaits thier implementation. HTTP-API labels Sep 14, 2026

SamAg19 added 2 commits September 14, 2026 09:59

clippy fixes

ef18127

Merge remote-tracking branch 'upstream/unstable' into get-pool-payloa…

a3eb40b

…d-attestations

mergify Bot added ready-for-review The code is ready for review and removed waiting-on-author The reviewer has suggested changes and awaits thier implementation. labels Sep 14, 2026

SamAg19 and others added 8 commits September 16, 2026 23:33

Merge branch 'unstable' into get-pool-payload-attestations

23c85e5

Merge branch 'unstable' into get-pool-payload-attestations

c1c1d49

Merge remote-tracking branch 'upstream/unstable' into get-pool-payloa…

3415581

…d-attestations

chong-he requested changes Sep 23, 2026

View reviewed changes

chong-he added waiting-on-author The reviewer has suggested changes and awaits thier implementation. and removed ready-for-review The code is ready for review labels Sep 23, 2026

SamAg19 added 3 commits September 24, 2026 16:44

combine get_payload_attestations and get_all_payload_attestations and…

bc54365

… address non-test related comments

address test-related comments

c5a4fb6

Merge remote-tracking branch 'upstream/unstable' into get-pool-payloa…

e21bcd0

…d-attestations

	pub fn get_filtered_attestations<F>(&self, filter: F) -> Vec<Attestation<E>>
	where
	F: Fn(&AttestationData, HashSet<u64>) -> bool,
	{
	self.attestations
	.read()
	.iter()
	.filter(\|att\| filter(&att.attestation_data(), att.get_committee_indices_map()))
	.map(\|att\| att.clone_as_attestation())
	.collect()
	}

Conversation

SamAg19 commented Sep 11, 2026

Issue Addressed

Proposed Changes

Uh oh!

chong-he commented Sep 14, 2026

Uh oh!

SamAg19 commented Sep 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

SamAg19 commented Sep 14, 2026 •

edited

Loading