Skip to content

feat(ampsync): allow configuring grpc decode limit#1797

Merged
shiyasmohd merged 3 commits intoedgeandnode:mainfrom
abarmat:fix/ampsync-decoded-msg-length
Feb 19, 2026
Merged

feat(ampsync): allow configuring grpc decode limit#1797
shiyasmohd merged 3 commits intoedgeandnode:mainfrom
abarmat:fix/ampsync-decoded-msg-length

Conversation

@abarmat
Copy link
Contributor

@abarmat abarmat commented Feb 18, 2026

Add a grpc_max_decode_mb option to ampsync config and propagate it through a new AmpClient constructor so the Arrow Flight client can raise its max decoding message size when syncing large tables.

Close: #1796

@shiyasmohd shiyasmohd self-requested a review February 18, 2026 12:42
@abarmat abarmat force-pushed the fix/ampsync-decoded-msg-length branch 3 times, most recently from cf3152f to 48cc47d Compare February 18, 2026 14:26
Copy link
Contributor

@shiyasmohd shiyasmohd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM ✅
Could you update the docs too crates/bin/ampsync/README.md ?

@abarmat
Copy link
Contributor Author

abarmat commented Feb 18, 2026

@shiyasmohd done

@shiyasmohd
Copy link
Contributor

@abarmat Commits are not verified 🙂, Could you commit with verified signatures too?

@abarmat abarmat force-pushed the fix/ampsync-decoded-msg-length branch from 9addb1e to e47ecdc Compare February 18, 2026 19:19
@abarmat
Copy link
Contributor Author

abarmat commented Feb 18, 2026

@shiyasmohd added signatures

@shiyasmohd
Copy link
Contributor

@abarmat Commit e51ef58 is still not verified.

Add a grpc_max_decode_mb option to ampsync config and propagate it through a new AmpClient constructor so the Arrow Flight client can raise its max decoding message size when syncing large tables.
Align ampsync docs with the new Flight decode limit setting so operators can tune large table syncs without relying on source inspection.

- Add `--grpc-max-decode-mb` to CLI options with default and range
- Document `AMPSYNC_GRPC_MAX_DECODE_MB` in environment variable config
- Add a tuning note for large Arrow Flight response frames
@abarmat abarmat force-pushed the fix/ampsync-decoded-msg-length branch from e47ecdc to 97b3a5a Compare February 19, 2026 00:38
@abarmat
Copy link
Contributor Author

abarmat commented Feb 19, 2026

@shiyasmohd I forgot about that one, pushed again

@shiyasmohd
Copy link
Contributor

Thanks for the PR! @abarmat

@shiyasmohd shiyasmohd merged commit 3b34c55 into edgeandnode:main Feb 19, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ampsync transactions stream fails on large Flight frames (decoded message length too large), causing permanent lag

2 participants

Comments