fix(hook): stop silent export skips from state-lock contention - #49
Open
kamikaze011001 wants to merge 1 commit into
Open
fix(hook): stop silent export skips from state-lock contention#49kamikaze011001 wants to merge 1 commit into
kamikaze011001 wants to merge 1 commit into
Conversation
The state-file lock is held for the whole of emit_new_turns_from_transcript, including the network emit to Langfuse, so hold time scales with how much a session has to flush. A real 82-turn flush took 2.16s, which already exceeds the 2.0s acquisition timeout, so any session reaching Stop during a large flush loses the race. On timeout main() swallowed the TimeoutError at debug level and returned 0, reporting success. With CC_LANGFUSE_DEBUG defaulting to false this left no trace anywhere: the turn is not exported until the session's next Stop, and never at all if the session goes idle first. It surfaces as Langfuse collecting only one of several concurrent sessions. Two changes: - Raise the default lock timeout 2.0s -> 30.0s. Waiting is strictly better than skipping: the hook is async and the work is idempotent, so a slow acquire costs latency while a failed acquire costs data. - Log the skip at info instead of debug, so the cause is visible without opting into debug logging. This mitigates the contention rather than removing it. The underlying issue is that the lock spans network I/O; narrowing the critical section or moving to per-session state files would remove the contention entirely, since state is already keyed per session. Refs langfuse#48 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the two immediate items from #48 (items 3 and 4 in that issue's list).
Problem
emit_new_turns_from_transcript()holds one globalflockfor its entire duration — reading the transcript, parsing turns, emitting to Langfuse over the network, and saving state. Hold time therefore scales with how much a session has to flush, and a real 82-turn flush from my own log took 2.16s:That already exceeds the 2.0s acquisition timeout, so any concurrent session reaching
Stopduring a large flush loses the race. On timeout,main()caught theTimeoutError, logged it atdebug, and returned0— reporting success. SinceCC_LANGFUSE_DEBUGdefaults tofalse, this left no trace at all: the turn isn't exported until that session's nextStop, and never if the session goes idle first. The symptom is Langfuse collecting only one of several concurrent sessions, with the apparent fault server-side and the actual cause entirely client-side.Changes
infoinstead ofdebug, so the cause is visible without opting into debug logging.What this does not fix
This mitigates the contention; it doesn't remove it. The root problem is that the lock spans network I/O, so hold time stays unbounded and data-dependent — a large enough backfill could still exhaust any fixed timeout. The structural fixes suggested in #48 still stand:
stateis already keyed byget_session_state_key(session_id, transcript_path)and concurrent sessions share nothing that needs mutual exclusion.Happy to follow up with either if you have a preference on direction.
Testing
uv run -m pytest→ 123 passed. No existing test referencesFileLockortimeout_s, so nothing needed updating; I didn't add a test for the new default since asserting on a timing constant would mostly pin the number rather than the behaviour, but glad to add one if you'd like it.🤖 Generated with Claude Code