Repository navigation
fix: preserve valid UTF-8 in captured trace text - #335
Merged
Merged
Conversation
added 3 commits
October 13, 2026 17:11
Trace capture sliced request fields, request/response bodies and tool errors by byte offsets, which could split Chinese characters or emoji and make OTLP protobuf serialization fail. Raw invalid bytes could also reach span attributes and status descriptions. Normalize captured text to valid UTF-8 and truncate at complete character boundaries while preserving the existing byte limits and unlimited mode. Apply the same handling to protocol error traces without changing tool request or response payloads. Validation: regression tests reproduced the original OTLP serialization failure; make lint, the full make test suite and the mcp-gateway build pass after the fix. Tests cover multibyte boundaries, invalid bytes, unlimited capture, status descriptions and unchanged tool payloads. No deployment validation was performed.
Contributor
Reviewer's GuideThe PR introduces a shared UTF-8 normalization and byte-limited truncation path for trace text, applies it to HTTP tool and protocol error telemetry, and adds regression coverage proving multibyte and invalid input remains valid for OTLP serialization while tool payloads remain unchanged. Flow diagram for UTF-8-safe trace captureflowchart LR
Sources[Captured trace text]
Normalize[TruncateUTF8]
Telemetry[OTLP span attributes and status]
Payloads[Tool request and response payloads]
Sources -->|normalize invalid UTF-8 and truncate at rune boundaries| Normalize
Normalize --> Telemetry
Payloads -->|unchanged| Payloads
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
Contributor
There was a problem hiding this comment.
Hey - I've found 2 issues
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location path="pkg/utils/str.go" line_range="31" />
<code_context>
+// TruncateUTF8 replaces invalid UTF-8 with U+FFFD and truncates at a complete
+// character boundary within maxBytes. Nonpositive limits leave the length unlimited.
+func TruncateUTF8(text string, maxBytes int) string {
+ text = strings.ToValidUTF8(text, "\uFFFD")
+ if maxBytes <= 0 || len(text) <= maxBytes {
+ return text
</code_context>
<issue_to_address>
**Bounded captures allocate per full body**
When a downstream response is very large, contains many invalid UTF-8 bytes, and trace capture has a positive byte limit, `readDecodedResponseBody` reads the entire response, then `TruncateUTF8` normalizes it before applying `maxBytes`; a large body with many invalid bytes therefore creates a normalized copy proportional to the full body even when only a small trace preview is retained, potentially exhausting gateway memory.
Normalize only enough input to produce the configured preview, or cap the response size before reading it.
</issue_to_address>
### Comment 2
<location path="internal/core/tool.go" line_range="501" />
<code_context>
- if maxLen > 0 && len(body) > maxLen {
- body = body[:maxLen]
- }
+ body := utils.TruncateUTF8(string(respBodyBytes), maxLen)
scope.Span.SetAttributes(
attribute.Int(cnst.AttrHTTPStatusCode, resp.StatusCode),
</code_context>
<issue_to_address>
**Response traces are not exported**
When a downstream response has an invalid UTF-8 Content-Type header and response or downstream-error capture is enabled, `scope.Span.SetAttributes` adds the raw header as `AttrHTTPRespType`, so OTLP protobuf serialization fails and the trace is not exported.
Normalize the Content-Type value before adding it as a span attribute.
</issue_to_address>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Trace capture sliced request fields, request/response bodies and tool errors by byte offsets, which could split Chinese characters or emoji and make OTLP protobuf serialization fail. Raw invalid bytes could also reach span attributes and status descriptions.
Normalize captured text to valid UTF-8 and truncate at complete character boundaries while preserving the existing byte limits and unlimited mode. Apply the same handling to protocol error traces without changing tool request or response payloads.
Validation: regression tests reproduced the original OTLP serialization failure; make lint, the full make test suite and the mcp-gateway build pass after the fix. Tests cover multibyte boundaries, invalid bytes, unlimited capture, status descriptions and unchanged tool payloads. No deployment validation was performed.
Summary by Sourcery
Ensure captured trace text remains valid UTF-8 and safely respects configured byte limits.
Bug Fixes:
Enhancements:
Tests: