fix(openai): parse raw API responses and add flag to skip tracing them

[hassiebp]

Problem

Calls made via the OpenAI SDK's .with_raw_response API return a LegacyAPIResponse instead of the parsed model. The wrapper's data extraction found no choices/usage on it and silently exported generations without output and usage, causing the server to fall back to tokenizer-based estimation (input-only, no cached-token details, wrong cost).

This matters beyond direct raw-response users: LiteLLM calls the OpenAI SDK exclusively through .with_raw_response (unconditionally, to read rate-limit headers for its router). Any process that imports langfuse.openai and also uses LiteLLM gets one of these broken OpenAI-generation observations for every LiteLLM call — on top of the observation produced by LiteLLM's own langfuse_otel callback, double-counting observations and cost.

Changes

• Parse raw responses before extraction: LegacyAPIResponse/APIResponse results are unwrapped via .parse() so output, usage (incl. prompt_tokens_details.cached_tokens) and model are captured. .parse() caches its result on the response object, so callers that parse later (e.g. LiteLLM) are unaffected — verified against LiteLLM 1.83.7.
• Raw streaming calls pass through untraced (stream=True via .with_raw_response, or .with_streaming_response): instrumenting them would require consuming the caller's stream or raw body. These previously produced broken input-only generations.
• New env flag LANGFUSE_OPENAI_SKIP_RAW_RESPONSES (default False): when set, all raw-response calls are passed through untraced. This is the supported way to combine the langfuse.openai wrapper (for direct OpenAI calls) with another instrumented library that calls the OpenAI SDK internally via raw responses (e.g. LiteLLM + langfuse_otel callback) without duplicate observations per LLM call.

Testing

• 4 new unit tests using httpx.MockTransport so the SDK's real raw-response machinery runs end-to-end: sync + async raw calls capture output/usage incl. cached tokens; skip flag bypasses instrumentation while normal calls stay traced; raw streaming passes through with the stream contract intact.
• Full unit suite passes (tests/unit, 607 passed; pre-existing test_prompt.py fixture errors also occur on main).
• Verified end-to-end against live OpenAI + LiteLLM 1.83.7 with the langfuse_otel callback:
  • default: OpenAI-generation now carries output and full usage incl. cached_tokens
  • with flag: no OpenAI-generation spans for LiteLLM-internal calls; direct calls unaffected

🤖 Generated with Claude Code

Greptile Summary

This PR fixes a silent data-loss bug in the OpenAI wrapper where calls made via .with_raw_response (used unconditionally by LiteLLM) returned a LegacyAPIResponse/APIResponse object rather than the parsed model, causing output, usage, and cost to be missing from exported generations. It also adds LANGFUSE_OPENAI_SKIP_RAW_RESPONSES as an opt-in env flag to bypass instrumentation of raw-response calls entirely.

• Core fix (_unwrap_raw_response): calls .parse() on raw response objects before data extraction; .parse() caches its result so downstream callers (e.g. LiteLLM) that parse later are unaffected.
• Streaming raw responses always bypass tracing: instrumenting them would require consuming the caller's stream body, so they are passed through untraced; non-streaming raw responses are now fully traced.
• New LANGFUSE_OPENAI_SKIP_RAW_RESPONSES flag: when set, all raw-response calls skip the wrapper; four new unit tests using httpx.MockTransport cover the sync/async capture, the skip flag, and streaming pass-through.

Confidence Score: 4/5

The core fix is sound and well-tested; the only issue is a style rule violation that does not affect runtime correctness.

The fix correctly unwraps raw API responses before data extraction, preserves the original response object for the caller, and guards streaming cases that cannot be instrumented. The inline imports inside _unwrap_raw_response are the only concern — they run on every non-streaming call and swallow import errors inside a broad except, but have no functional impact.

langfuse/openai.py — the inline imports inside _unwrap_raw_response should be moved to module level alongside the existing RAW_RESPONSE_HEADER guard.

Prompt To Fix All With AI

Fix the following 2 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 2
langfuse/openai.py:58-61
Imports placed inside `_unwrap_raw_response` violate the project rule requiring all imports at module level. Each call to this function (i.e. every non-streaming, non-skipped OpenAI call) reimports these modules, and the broad `except Exception` here also swallows any import error silently. Moving them to module level matches how `RAW_RESPONSE_HEADER` is already imported (top-level `try/except ImportError`), and lets the `.parse()` failure be caught independently.

```suggestion
try:
    from openai._constants import RAW_RESPONSE_HEADER
except ImportError:
    RAW_RESPONSE_HEADER = "X-Stainless-Raw-Response"

try:
    from openai._legacy_response import LegacyAPIResponse as _LegacyAPIResponse
    from openai._response import APIResponse as _APIResponse

    _RAW_RESPONSE_TYPES: tuple = (_LegacyAPIResponse, _APIResponse)
except ImportError:
    _RAW_RESPONSE_TYPES = ()
```

### Issue 2 of 2
langfuse/openai.py:1192-1199
If the imports above are moved to module level, the body of `_unwrap_raw_response` can be simplified to avoid re-importing and to isolate the `.parse()` failure from the type-check itself.

```suggestion
    if _RAW_RESPONSE_TYPES and isinstance(openai_response, _RAW_RESPONSE_TYPES):
        try:
            return openai_response.parse()
        except Exception as e:
            logger.debug(f"Failed to parse raw OpenAI response for tracing: {e}")
```

_{Reviews (1): Last reviewed commit: "fix(openai): parse raw API responses and..." | Re-trigger Greptile}

Context used:

• Rule used - Move imports to the top of the module instead of p... (source)

Learned From
langfuse/langfuse-python#1387

[github-actions]

@claude review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f8b0e6ec31

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".