First-class external transcript provider for AIChatAgent

[duz052]

From our perspective, AIChatAgent gives us a better level of base control for this kind of product than @cloudflare/think. We already have our own context management, search, memory, persistence model, and product-specific chat workflow. AIChatAgent feels closer to the right abstraction for us.

The main issue is transcript ownership.

@cloudflare/think supports a custom storage/session provider, so we can make Postgres the canonical transcript store. However, Think also brings a higher-level session/context lifecycle that we may not want to adopt for every part of our system.

On the other hand, AIChatAgent gives us more direct control over the chat flow, but as far as we can tell it does not currently provide a first-class way to replace its internal transcript storage with an external provider.

That leaves us with an awkward tradeoff:

• Use Think to get external storage provider support, but accept a higher-level lifecycle model
• Use AIChatAgent to keep more direct control, but double-write transcripts into both DO SQLite and Postgres

For our use case, double-writing transcripts is the core problem. It creates consistency risks, complicates reconnect/retry/partial-generation behavior, and makes cross-chat search depend on a second canonical copy in Postgres anyway.

We are fine with Durable Objects storing transient runtime state, including resumable stream buffers and recovery state. We just do not want the Durable Object to also be the long-term canonical transcript database.

Would Cloudflare consider supporting a first-class external transcript provider for AIChatAgent, similar in spirit to Think’s storage provider model, where message history can be owned by an external database such as Postgres?

The important part for us is that the external provider would be the authoritative transcript store used by the chat protocol lifecycle, rather than just a secondary sync target.

This would let us keep the simpler AIChatAgent control model while avoiding permanent double storage for chat history.

[threepointone]

It's on my mind. I'm in the middle of a giant refactor atm that should make this tractable. I'll keep thinking about it, it'll probably happen.

That said, I'm be curious about the perceived risks you're talking about. Have you seen any actual drift, etc? A lot of the perf benefits/programming model come because we have this incredibly fast storage solution available in the agent itself.

[duz052]

The actual risk we see is not random corruption in DO SQLite.

In our current design, Postgres is the source of truth for finished messages, while the DO/runtime stream is the source of truth while a response is actively streaming.

The difficult case is when a user sends a message, switches tabs or the browser suspends the page while the LLM is streaming, and comes back after the stream already finished.

At that point the finished assistant message may already be in Postgres, but the client-side chat state may have missed the final websocket message / final transcript broadcast from the agent. Then we need custom reconciliation logic to move the UI from “DO stream truth” to “Postgres finished transcript truth”.

The risk is the source-of-truth handoff between live streaming
state and our product transcript store.