Skip to content

feat: migrate test infrastructure to xnet#98

Merged
neekolas merged 13 commits into
mainfrom
04-04-use_xnet_for_backend_startup
Apr 8, 2026
Merged

feat: migrate test infrastructure to xnet#98
neekolas merged 13 commits into
mainfrom
04-04-use_xnet_for_backend_startup

Conversation

@neekolas
Copy link
Copy Markdown
Collaborator

@neekolas neekolas commented Apr 5, 2026

Summary

Replace the docker-compose-managed XMTP node (node-go + validation + databases) with xnet-cli for test infrastructure. This simplifies the setup significantly and adds advanced integration test scenarios.

  • xnet.toml configures the XMTP network (xmtpd, gateway, node-go, validation images)
  • docker-compose.yml simplified to 3 services: db, notification_server, integration — all on the external xnet Docker network
  • dev/up starts xnet with --paused, adds a migrator node, starts the notification DB
  • dev/integration accepts v3-writes or v4-writes scenario, dynamically resolves CoreDNS IP for container DNS
  • gatewayUrl support for V4-writes scenario (client publishes directly to V4 gateway)
  • 3 new integration tests: selective unsubscribe, group message sender filtering, unregister stops notifications
  • waitForNotification filters by installationId to prevent cross-test interference
  • CI uses Nix + Cachix to install xnet-cli via a reusable setup-xnet composite action

Design: docs/plans/2026-04-04-xnet-integration-design.md

Test plan

  • go test -p 1 ./... passes with xnet providing XMTP nodes
  • ./dev/integration v3-writes — all 5 integration tests pass
  • ./dev/integration v4-writes — all 5 integration tests pass (requires d14n activation)
  • shellcheck dev/up dev/down dev/integration — no errors

🤖 Generated with Claude Code

Note

Migrate test infrastructure to xnet for integration testing

  • Replaces the static docker-compose v3 node with an xnet-based environment, provisioned via new dev scripts (dev/up, dev/down, dev/xnet, dev/xnet-env) and a pinned xnet-cli installed via Nix (dev/install-xnet).
  • Adds a CI matrix in test.yml to run integration tests against v3-direct and v4-with-migrator message sources using the new setup-xnet composite action.
  • Adds a /readyz endpoint to ApiServer that returns 200/503 based on listener stream readiness; both v3 and v4 listeners now expose a Ready() method tied to active stream state.
  • Integration tests are reworked to wait for per-installation notifications and cover new behaviors: selective unsubscribe, group message sender filtering, and unregister.
  • Behavioral Change: docker-compose no longer includes a v3 node service; the notification server now depends on an external xnet Docker network and exposes /readyz for healthchecks.

Macroscope summarized 1620534.

@neekolas neekolas mentioned this pull request Apr 5, 2026
3 tasks
@neekolas neekolas changed the title feat: add xnet.toml and update env files for xnet migration feat: migrate test infrastructure to xnet Apr 5, 2026
@neekolas neekolas marked this pull request as ready for review April 5, 2026 02:49
@neekolas neekolas requested a review from a team as a code owner April 5, 2026 02:49
macroscopeapp[bot]
macroscopeapp Bot previously approved these changes Apr 5, 2026
@macroscopeapp
Copy link
Copy Markdown

macroscopeapp Bot commented Apr 5, 2026

Approvability

Verdict: Needs human review

1 blocking correctness issue found. While primarily test infrastructure changes, this PR introduces runtime modifications including a new /readyz endpoint and listener Ready() methods. An unresolved review comment identifies a potential data race in the readyCheck field that should be addressed before merging.

No code changes detected at 1620534. Prior analysis still applies.

You can customize Macroscope's approvability policy. Learn more.

@neekolas neekolas force-pushed the 04-04-use_xnet_for_backend_startup branch from 9de66a7 to d8f57b9 Compare April 5, 2026 04:28
@neekolas neekolas force-pushed the 04-04-v4_listener_cc branch from 7d94778 to f685900 Compare April 5, 2026 06:33
@neekolas neekolas force-pushed the 04-04-use_xnet_for_backend_startup branch from d8f57b9 to 8d3a4f0 Compare April 5, 2026 06:33
@neekolas neekolas force-pushed the 04-04-v4_listener_cc branch from f685900 to 4ac3963 Compare April 5, 2026 06:37
@neekolas neekolas force-pushed the 04-04-use_xnet_for_backend_startup branch 2 times, most recently from aee0554 to d6d09c2 Compare April 5, 2026 17:54
@neekolas neekolas force-pushed the 04-04-v4_listener_cc branch from 4ac3963 to 24536d4 Compare April 5, 2026 17:54
@macroscopeapp macroscopeapp Bot dismissed their stale review April 5, 2026 17:55

Dismissing prior approval to re-evaluate d6d09c2

@neekolas neekolas force-pushed the 04-04-use_xnet_for_backend_startup branch from d6d09c2 to b55e624 Compare April 5, 2026 17:56
@neekolas neekolas force-pushed the 04-04-v4_listener_cc branch from 24536d4 to 5e43ef2 Compare April 5, 2026 17:56
@neekolas neekolas force-pushed the 04-04-use_xnet_for_backend_startup branch from b55e624 to 721fca6 Compare April 6, 2026 04:12
Comment thread .github/actions/setup-xnet/action.yml
@neekolas neekolas force-pushed the 04-04-use_xnet_for_backend_startup branch 7 times, most recently from ae44dbb to 8b4c587 Compare April 6, 2026 21:09
macroscopeapp[bot]
macroscopeapp Bot previously approved these changes Apr 6, 2026
@neekolas neekolas force-pushed the 04-04-use_xnet_for_backend_startup branch 2 times, most recently from 9b2a528 to 3b6f8c0 Compare April 7, 2026 20:43
Comment thread dev/xnet Outdated
@neekolas neekolas force-pushed the 04-04-use_xnet_for_backend_startup branch 3 times, most recently from 8f06a44 to dba32cc Compare April 8, 2026 14:44
@neekolas neekolas force-pushed the 04-04-v4_listener_cc branch from 5e43ef2 to b532c6f Compare April 8, 2026 14:44
@neekolas neekolas force-pushed the 04-04-use_xnet_for_backend_startup branch from dba32cc to b928bbb Compare April 8, 2026 16:44
Copy link
Copy Markdown
Collaborator Author

neekolas commented Apr 8, 2026

Merge activity

  • Apr 8, 5:03 PM UTC: A user started a stack merge that includes this pull request via Graphite.
  • Apr 8, 5:10 PM UTC: Graphite rebased this pull request as part of a merge.
  • Apr 8, 5:11 PM UTC: @neekolas merged this pull request with Graphite.

@neekolas neekolas changed the base branch from 04-04-v4_listener_cc to graphite-base/98 April 8, 2026 17:08
@neekolas neekolas changed the base branch from graphite-base/98 to main April 8, 2026 17:09
neekolas and others added 13 commits April 8, 2026 17:10
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- xnet uses port 5556 (not 4446) when use_standard_ports=true
- xnet-cli up needs --paused flag to add migrator nodes
- xnet-cli node add --migrator (not node --migrator)
- Drop custom xmtpd/gateway images (incompatible --config-file CLI flag)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Now works after fixing xnet-cli to:
1. Use --environment=anvil instead of --config-file=config://anvil
2. Not override CLI image from xmtpd server image config

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…5556 for xnet

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…cenarios

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Uses baptistearno/rust-cargo-install to build xnet-cli from the
push-mtruypwzqklu branch which includes --environment=anvil and
CLI image separation fixes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add .github/actions/setup-xnet composite action that:
- Installs Nix with cachix/install-nix-action
- Configures xmtp Cachix cache for pre-built binaries
- Installs xnet-cli via nix profile from the libxmtp branch
- Sets up DNS for *.xmtpd.local resolution

Replaces the previous cargo-install approach with proper Nix
caching, matching the pattern used in xmtp/libxmtp CI.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The @xmtp/node-sdk native bindings require GLIBC 2.38+ which is
not available in node:22-slim (Debian bookworm, GLIBC 2.36).
Ubuntu 24.04 (Noble) provides GLIBC 2.39.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace the global waitForNextRequest with waitForNotification that
takes an installationId parameter. Notifications are only resolved
to the matching pending promise, preventing cross-test interference
from late-arriving messages.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@neekolas neekolas force-pushed the 04-04-use_xnet_for_backend_startup branch from b928bbb to 1620534 Compare April 8, 2026 17:10
@neekolas neekolas merged commit 3a13506 into main Apr 8, 2026
9 checks passed
@neekolas neekolas deleted the 04-04-use_xnet_for_backend_startup branch April 8, 2026 17:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants