Skip to content

fix: preserve valid UTF-8 in captured trace text - #335

Merged
iFurySt merged 4 commits into
AmoyLab:mainfrom
qiankunli:fix/trace-utf8
Oct 13, 2026
Merged

iFurySt merged 4 commits into
AmoyLab:mainfrom
qiankunli:fix/trace-utf8

Conversation

@qiankunli

@qiankunli qiankunli commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Trace capture sliced request fields, request/response bodies and tool errors by byte offsets, which could split Chinese characters or emoji and make OTLP protobuf serialization fail. Raw invalid bytes could also reach span attributes and status descriptions.

Normalize captured text to valid UTF-8 and truncate at complete character boundaries while preserving the existing byte limits and unlimited mode. Apply the same handling to protocol error traces without changing tool request or response payloads.

Validation: regression tests reproduced the original OTLP serialization failure; make lint, the full make test suite and the mcp-gateway build pass after the fix. Tests cover multibyte boundaries, invalid bytes, unlimited capture, status descriptions and unchanged tool payloads. No deployment validation was performed.

Summary by Sourcery

Ensure captured trace text remains valid UTF-8 and safely respects configured byte limits.

Bug Fixes:

  • Preserve valid UTF-8 when truncating captured trace fields, request and response bodies, protocol errors, and span status descriptions.
  • Prevent multibyte text and invalid byte sequences from causing OTLP trace serialization failures while retaining existing capture limits and unlimited mode.

Enhancements:

  • Centralize UTF-8-aware truncation for captured text and error descriptions without modifying tool request or response payloads.

Tests:

  • Add regression coverage for multibyte character boundaries, unlimited capture, error status descriptions, OTLP serialization, and unchanged tool payloads.

qiankunli added 3 commits October 13, 2026 17:11
Trace capture sliced request fields, request/response bodies and tool errors by byte offsets, which could split Chinese characters or emoji and make OTLP protobuf serialization fail. Raw invalid bytes could also reach span attributes and status descriptions.

Normalize captured text to valid UTF-8 and truncate at complete character boundaries while preserving the existing byte limits and unlimited mode. Apply the same handling to protocol error traces without changing tool request or response payloads.

Validation: regression tests reproduced the original OTLP serialization failure; make lint, the full make test suite and the mcp-gateway build pass after the fix. Tests cover multibyte boundaries, invalid bytes, unlimited capture, status descriptions and unchanged tool payloads. No deployment validation was performed.
@sourcery-ai

sourcery-ai Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Reviewer's Guide

The PR introduces a shared UTF-8 normalization and byte-limited truncation path for trace text, applies it to HTTP tool and protocol error telemetry, and adds regression coverage proving multibyte and invalid input remains valid for OTLP serialization while tool payloads remain unchanged.

Flow diagram for UTF-8-safe trace capture

flowchart LR
    Sources[Captured trace text]
    Normalize[TruncateUTF8]
    Telemetry[OTLP span attributes and status]
    Payloads[Tool request and response payloads]

    Sources -->|normalize invalid UTF-8 and truncate at rune boundaries| Normalize
    Normalize --> Telemetry
    Payloads -->|unchanged| Payloads
Loading

File-Level Changes

Change Details Files
Centralize UTF-8 normalization and byte-budget truncation for captured trace text.
  • Add a helper that replaces invalid UTF-8 and truncates only at complete rune boundaries.
  • Apply the helper to downstream argument fields, request bodies, response previews, and response bodies while retaining unlimited behavior for nonpositive limits.
  • Add utility-level coverage for ASCII, multibyte boundaries, invalid and incomplete input, replacement-byte budgeting, and unlimited mode.
pkg/utils/str.go
pkg/utils/str_test.go
internal/core/tool.go
Make tool execution error traces safe for OTLP serialization without altering tool payloads.
  • Normalize and safely truncate HTTP trace attributes and preserve the original request and response payloads.
  • Exercise Chinese, emoji, and invalid-byte capture cases across argument, request, error, and response trace attributes.
  • Verify captured spans can be marshaled as OTLP protobuf messages.
internal/core/execute_tool_test.go
internal/core/tool.go
Apply the same UTF-8-safe handling to protocol/tool error status descriptions.
  • Replace byte slicing of error messages with the shared UTF-8 truncation helper.
  • Validate multibyte boundary handling, invalid-byte replacement, and successful OTLP export of status descriptions.
internal/core/response.go
internal/core/response_test.go

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 2 issues

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="pkg/utils/str.go" line_range="31" />
<code_context>
+// TruncateUTF8 replaces invalid UTF-8 with U+FFFD and truncates at a complete
+// character boundary within maxBytes. Nonpositive limits leave the length unlimited.
+func TruncateUTF8(text string, maxBytes int) string {
+	text = strings.ToValidUTF8(text, "\uFFFD")
+	if maxBytes <= 0 || len(text) <= maxBytes {
+		return text
</code_context>
<issue_to_address>
**Bounded captures allocate per full body**

When a downstream response is very large, contains many invalid UTF-8 bytes, and trace capture has a positive byte limit, `readDecodedResponseBody` reads the entire response, then `TruncateUTF8` normalizes it before applying `maxBytes`; a large body with many invalid bytes therefore creates a normalized copy proportional to the full body even when only a small trace preview is retained, potentially exhausting gateway memory.

Normalize only enough input to produce the configured preview, or cap the response size before reading it.
</issue_to_address>

### Comment 2
<location path="internal/core/tool.go" line_range="501" />
<code_context>
-		if maxLen > 0 && len(body) > maxLen {
-			body = body[:maxLen]
-		}
+		body := utils.TruncateUTF8(string(respBodyBytes), maxLen)
 		scope.Span.SetAttributes(
 			attribute.Int(cnst.AttrHTTPStatusCode, resp.StatusCode),
</code_context>
<issue_to_address>
**Response traces are not exported**

When a downstream response has an invalid UTF-8 Content-Type header and response or downstream-error capture is enabled, `scope.Span.SetAttributes` adds the raw header as `AttrHTTPRespType`, so OTLP protobuf serialization fails and the trace is not exported.

Normalize the Content-Type value before adding it as a span attribute.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

Comment thread pkg/utils/str.go Outdated
Comment thread internal/core/tool.go

@iFurySt iFurySt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks!

@iFurySt
iFurySt merged commit e6defc2 into AmoyLab:main Oct 13, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants