Skip to content

Fix flaky MultiHostValidatorOperator topology serial race#5748

Merged
martinflorian-da merged 1 commit into
mainfrom
martinflorian-da/cntf-8444-fix-multihost-validator-topology-serial-race
Jun 1, 2026
Merged

Fix flaky MultiHostValidatorOperator topology serial race#5748
martinflorian-da merged 1 commit into
mainfrom
martinflorian-da/cntf-8444-fix-multihost-validator-topology-serial-race

Conversation

@martinflorian-da
Copy link
Copy Markdown
Contributor

@martinflorian-da martinflorian-da commented May 29, 2026

Fixes DACH-NY/cn-test-failures#8444

[ci]

Theory: After bobValidator reconnects, its participant's topology store can still lag behind the party-to-participant change aliceValidator authorized while bob was disconnected. "Clear the onboarding flags" possibly races with catching up to topology and therefore gets rejected with TOPOLOGY_SERIAL_MISMATCH.

The best smoking gun I have for this from the logs is that reconnect finishes at 2026-05-18T06:52:39.586Z , then we try the propose at 2026-05-18T06:52:39.631. This is very short time...

So either this really fixes it or we have better proof that somehow a topology transaction disappears again (if this is what happens).

Signed-off-by: Martin Florian martin.florian@digitalasset.com

Fixes DACH-NY/cn-test-failures#8444

[ci]

Theory: After bobValidator reconnects, its participant's topology store can still lag behind the party-to-participant change aliceValidator authorized while bob was disconnected. "Clear the onboarding flags" possibly races with catching up to topology and therefore gets rejected with TOPOLOGY_SERIAL_MISMATCH.

The best smoking gun I have for this from the logs is that reconnect finishes at `2026-05-18T06:52:39.586Z `, then we try the propose at `2026-05-18T06:52:39.631`. This is very short time...

So either this really fixes it or we have better proof that somehow a topology transaction disappears again (if this is what happens).

Signed-off-by: Martin Florian <martin.florian@digitalasset.com>
@martinflorian-da martinflorian-da force-pushed the martinflorian-da/cntf-8444-fix-multihost-validator-topology-serial-race branch from 597dd33 to 639ddab Compare May 29, 2026 17:38
@martinflorian-da martinflorian-da marked this pull request as ready for review June 1, 2026 10:03
@martinflorian-da martinflorian-da requested a review from nicu-da June 1, 2026 10:03
@martinflorian-da martinflorian-da merged commit bd98b81 into main Jun 1, 2026
65 checks passed
@martinflorian-da martinflorian-da deleted the martinflorian-da/cntf-8444-fix-multihost-validator-topology-serial-race branch June 1, 2026 13:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants