Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions electron/ai-edition/agent-tools.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -205,6 +205,36 @@ describe("executeAgentTool", () => {
expect(payload.segments[1].kind).toBe("silence");
});

it("getTranscript returns a long transcript whole, word for word", () => {
// The regression test for a `.slice(0, 800)` that used to sit here. On the
// production path a segment is one WORD, so the cap cut a half-hour
// recording at roughly its fifth minute and reported nothing — the model
// trimmed the silences it could see and called the job done. 4000 words is
// about half an hour of speech.
const base = fixtureDocument();
const segments = Array.from({ length: 4000 }, (_, i) => ({
id: `seg_${i}`,
kind: "speech" as const,
startSec: i * 0.45,
endSec: i * 0.45 + 0.4,
text: `mot${i}`,
wordIds: [],
}));
Comment thread
coderabbitai[bot] marked this conversation as resolved.
const doc = {
...base,
transcript: null,
transcripts: [{ ...base.transcripts[0], segments }],
};

const result = executeAgentTool(doc, "getTranscript", "{}");
expect(result.ok).toBe(true);
const payload = JSON.parse(result.resultJson);
expect(payload.segments).toHaveLength(4000);
// The last word matters more than the count: a cap keeps the head and
// drops the tail, so the tail is what proves it is gone.
expect(payload.segments.at(-1).text).toBe("mot3999");
});

it("getTranscript fails cleanly when no transcript exists", () => {
const doc = { ...fixtureDocument(), transcripts: [], transcript: null };
const result = executeAgentTool(doc, "getTranscript", "{}");
Expand Down
18 changes: 15 additions & 3 deletions electron/ai-edition/agent-tools.ts
Original file line number Diff line number Diff line change
Expand Up @@ -854,9 +854,21 @@ export function executeAgentTool(
if (!transcript) {
return failure(`No transcript for asset ${assetId ?? "(none)"}.`);
}
// ponytail: segments only — words would blow the context for long
// recordings and the segment text already carries the content.
const segments = transcript.segments.slice(0, 800).map((s) => ({
// ponytail: no cap. There used to be a `.slice(0, 800)` here, guarded by
// "words would blow the context" — written believing a segment was a
// phrase. On the production path a segment IS one word
// (src/lib/captioning/transcribe.ts: whisper's word timings are mapped
// one-to-one), so the cap cut the transcript at the 800th WORD — around
// five minutes of speech — and said nothing about it. The model read a
// fifth of a half-hour recording, cut the silences it could see, and
// reported the job done, because nothing in the payload told it otherwise.
//
// A whole 30-minute transcript is ~285k characters, ~70k tokens: large,
// and well inside every model this app talks to. If a recording ever does
// get near a window, the honest fix is to know the window — the app has no
// per-model context budget today — not to guess a number here and drop the
// rest in silence.
const segments = transcript.segments.map((s) => ({
id: s.id,
kind: s.kind,
startSec: s.startSec,
Expand Down
2 changes: 1 addition & 1 deletion technical-documentation/architecture/ai-agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ The model never free-writes the project document. It can only call the fixed set
| Tool | What it does | What it mutates |
|---|---|---|
| `getCurrentDocument` | Reads a compact project, asset, clip, trim, and modifier snapshot with explicit time bases. Each asset reports `hasCameraTrack` / `cameraVisible` / `hasCursorTelemetry` beside `hasTranscript` (`hasCursorTelemetry` is three-valued: `true`, `false` when the asset was checked and has none, `null` when it was not checked — never `false` for something we failed to look at), the document reports `hasAnyCamera` and `autoFocusAll`, and each zoom reports the `renderedScale` the viewer will see plus `customScale` / `depthIsOverridden` when a custom scale makes its `depth` inert. | Nothing. |
| `getTranscript` | Reads up to 800 transcript segments for an asset or the primary asset. | Nothing. |
| `getTranscript` | Reads the transcript segments for an asset, or the primary asset, in full. On the production path a segment is one word, so a half-hour recording is a few thousand of them — there is no cap, and no per-model context budget to derive one from. | Nothing. |
| `getCursorTrack` | Reads the recorded pointer telemetry for an asset as a DIGEST: the moments the cursor sat still or clicked, each with its hold, its average position, its click count, its source time and the `virtualSec` that `addZoom` takes — never the raw samples. Answers `available:false` with `reason:"no-sidecar"` (checked, this asset has none) or `reason:"unavailable"` (could not be read from here), and the two are never conflated. | Nothing. |
| `addTrim` | Adds a source-time cut inside a clip. | `timeline.trimRanges`. |
| `setTrim` | Moves or resizes an existing source-time trim. | The matching `timeline.trimRanges` entry. |
Expand Down
5 changes: 3 additions & 2 deletions workbench/lib/fixtures.ts
Original file line number Diff line number Diff line change
Expand Up @@ -356,8 +356,9 @@ export function multipleModifiers(options?: { projectId?: string }): AxcutDocume
});
}

/** More than 800 transcript segments — `agent-tools.ts:625` slices at 800 and
* says nothing about it (DSL-6). */
/** More than 800 transcript segments. `getTranscript` used to slice at 800 and
* say nothing about it; it no longer caps, and this fixture is what keeps that
* honest. */
export function longTranscript(options?: { segments?: number; projectId?: string }): AxcutDocument {
const count = options?.segments ?? 900;
const durationSec = count * 2;
Expand Down
Loading