Skip to content

refactor: replace local upstream protos with xmtpd imports#97

Merged
neekolas merged 3 commits into
mainfrom
04-04-remove_local_protos
Apr 8, 2026
Merged

refactor: replace local upstream protos with xmtpd imports#97
neekolas merged 3 commits into
mainfrom
04-04-remove_local_protos

Conversation

@neekolas
Copy link
Copy Markdown
Collaborator

@neekolas neekolas commented Apr 4, 2026

Summary

  • Pin xmtpd to 04-01-implement_api_schema_changes branch for notification API proto types
  • Remove Go generation plugins from root buf.gen.yaml (keep Swift/Kotlin/Java)
  • Delete 9 locally generated upstream proto directories under pkg/proto/ (75 .pb.go files)
  • Update all Go source imports to use github.com/xmtp/xmtpd/pkg/proto/ instead of local paths
  • Only pkg/proto/notifications/ (our own service proto) remains locally generated

This is a prerequisite for the V4 listener — importing xmtpd/pkg/envelopes causes proto namespace conflicts with locally generated copies of the same types.

Test plan

  • go build ./... clean
  • go test -p 1 ./... — all 73 existing tests pass
  • No protobuf namespace conflicts when importing xmtpd envelope types
  • pkg/proto/ only contains notifications/

🤖 Generated with Claude Code

Note

Replace local upstream protos with xmtpd imports and migrate subscription topics to binary storage

  • Replaces all local proto imports (message_api, migration) with imports from github.com/xmtp/xmtpd, and updates topic parsing to use structured topic.Topic objects from xmtpd throughout the codebase.
  • Adds a database migration (00004_binary_topics.up.sql) that converts the subscriptions.topic column from TEXT to BYTEA, storing raw topic bytes (kind + identifier) instead of strings; non-conforming and duplicate rows are deleted.
  • Updates the Subscriptions interface and all implementations (Subscribe, Unsubscribe, GetSubscriptions) to accept and return *topic.Topic instead of plain strings.
  • Adds topic_bytes/topics_bytes fields to the notifications v1 protobuf (Subscription, SubscribeRequest, UnsubscribeRequest) and normalizes API handlers to accept both string and binary topics, with bytes taking precedence and deduplication across both forms.
  • Risk: the migration deletes rows with topics that cannot be parsed as valid V3 topics and collapses duplicates that normalize to the same bytes — existing data must be clean V3 topics or it will be dropped.
📊 Macroscope summarized bc82114. 28 files reviewed, 3 issues evaluated, 3 issues filtered, 0 comments posted

🗂️ Filtered Issues

pkg/db/migrations/00004_binary_topics.down.sql — 0 comments posted, 1 evaluated, 1 filtered
  • line 4: The down migration causes complete data loss of all subscription topic data. The migration adds an empty topic_legacy TEXT column (line 4), then immediately drops the topic BYTEA column containing the actual data (line 5), then renames the empty column to topic (line 6). There is no UPDATE statement to convert the BYTEA data back to TEXT format before dropping the BYTEA column. [ Posting failed ]
pkg/db/migrations/migrations.go — 0 comments posted, 1 evaluated, 1 filtered
  • line 67: The function MigrateUpTo is documented as running migrations "up to (and including) the given version", but m.Migrate(version) will migrate in either direction. If the database is currently at version 5 and MigrateUpTo(ctx, db, 3) is called, the code will migrate DOWN to version 3, contradicting the function's name and documentation which imply only upward migration. Consider using a check like m.Version() to verify current version and prevent downward migration, or rename/redocument the function to reflect bidirectional behavior. [ Posting failed ]
pkg/xmtp/listener.go — 0 comments posted, 1 evaluated, 1 filtered
  • line 192: In processEnvelope at line 192, return err returns the last error from l.deliver instead of nil. If the last iteration of the sendRequests loop calls l.deliver and it fails, err is set to that error. But if the last iteration hits the shouldDeliver skip branch (the continue), err retains the error from a previous iteration's failed l.deliver call, causing processEnvelope to return a stale error. Conversely, if the loop completes without entering l.deliver at all (all requests skipped), err still holds the value from the l.installations.GetInstallations call (which was nil), so that path is fine. The real bug: when the loop processes multiple requests and an earlier deliver fails but a later one succeeds, err is overwritten to nil, silently swallowing the earlier delivery error. And when an earlier deliver fails but later iterations are skipped via continue, the stale error is returned, potentially causing the caller to log a misleading error. The function should either accumulate errors or return nil since individual delivery errors are already logged. [ Out of scope ]

@neekolas neekolas requested a review from a team as a code owner April 4, 2026 17:26
@macroscopeapp
Copy link
Copy Markdown

macroscopeapp Bot commented Apr 4, 2026

Approvability

Verdict: Unable to determine

Macroscope's correctness review was unable to post its findings for this PR. Approvability cannot proceed without a successful correctness review.

You can customize Macroscope's approvability policy. Learn more.

@neekolas neekolas force-pushed the 04-04-remove_local_protos branch from 356f73e to 5aab47d Compare April 5, 2026 02:48
@neekolas neekolas mentioned this pull request Apr 5, 2026
3 tasks
@neekolas neekolas force-pushed the 04-04-remove_local_protos branch from 5aab47d to 9674141 Compare April 5, 2026 17:54
@neekolas neekolas force-pushed the 04-03-add_support_for_new_topic_formats branch 2 times, most recently from 8c776b6 to 70db0e2 Compare April 5, 2026 17:56
@neekolas neekolas force-pushed the 04-04-remove_local_protos branch from 9674141 to 7fe7faa Compare April 5, 2026 17:56
@neekolas neekolas force-pushed the 04-03-add_support_for_new_topic_formats branch from 70db0e2 to c7d7e68 Compare April 8, 2026 14:44
@neekolas neekolas force-pushed the 04-04-remove_local_protos branch from 7fe7faa to c4452ec Compare April 8, 2026 14:44
Copy link
Copy Markdown
Collaborator Author

neekolas commented Apr 8, 2026

Merge activity

  • Apr 8, 5:03 PM UTC: A user started a stack merge that includes this pull request via Graphite.
  • Apr 8, 5:07 PM UTC: Graphite rebased this pull request as part of a merge.
  • Apr 8, 5:07 PM UTC: @neekolas merged this pull request with Graphite.

@neekolas neekolas changed the base branch from 04-03-add_support_for_new_topic_formats to graphite-base/97 April 8, 2026 17:04
@neekolas neekolas changed the base branch from graphite-base/97 to main April 8, 2026 17:05
neekolas and others added 3 commits April 8, 2026 17:06
@neekolas neekolas force-pushed the 04-04-remove_local_protos branch from c4452ec to bc82114 Compare April 8, 2026 17:06
@neekolas neekolas merged commit 93202d8 into main Apr 8, 2026
8 checks passed
@neekolas neekolas deleted the 04-04-remove_local_protos branch April 8, 2026 17:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants