feat(files): add durable project uploads (#61)

## Summary
- Persist uploaded documents, images, data, and code as durable project
files instead of embedding file contents in prompts.
- Materialize uploads into each project's Daytona workspace so agents
and later chats can reuse them.
- Add composer upload state, reusable project-file references, sandbox
recovery, observability, and complete local setup documentation.

## What's Included

### Project file API and storage
- Adds authenticated upload and listing contracts with extension, size,
batch, and file-signature validation.
- Stores durable copies in R2 and materializes them under
`/workspace/<project>/uploads/`.
- Restores files when a sandbox is provisioned, replaced, or recovered.

### Composer integration
- Adds upload progress and failure states to the home and thread
composers.
- Replaces prompt-embedded file contents with stable project-relative
references.
- Makes existing project files available through the composer reference
menu.

### Sandbox reliability
- Detects unhealthy workspace mounts and replaces affected sandboxes.
- Preserves uploads across chats, reloads, and sandbox lifecycle
transitions.
- Improves structured upstream HTTP error reporting.

### Local development
- Documents the supported Docker Compose stack, required credentials,
troubleshooting, and direct browser QA workflow.
- Moves the local web endpoint to `http://localhost:3001`.

## Architecture
The browser uploads a validated file through the gateway to the
project-scoped sandbox Durable Object. The Durable Object writes a
durable R2 copy and materializes the same bytes into the active Daytona
project workspace. Composer references point to that workspace path,
while provisioning and recovery rematerialize durable files when
necessary.

## Decisions Made
| Decision | Choice | Reasoning |
|---|---|---|
| Durable source of truth | R2 with workspace materialization | Files
survive sandbox replacement without storing binary data in Postgres. |
| Prompt representation | Project-relative paths | Avoids prompt bloat
and lets agents use native document tooling. |
| Validation | Browser allowlist plus server signature checks | Provides
fast feedback without trusting client MIME metadata. |
| Local port | 3001 | Matches the supported local environment and avoids
the existing port collision. |

## Edge Cases Handled
| Scenario | Handling |
|---|---|
| Empty, oversized, unsupported, or spoofed file | Reject before durable
persistence. |
| Duplicate filename | Safely normalize and resolve the project path. |
| Missing or unhealthy workspace mount | Replace the sandbox and
rematerialize durable project files. |
| Reload or later chat | Reuse the same stored project file without
re-uploading. |
| Partial batch failure | Surface per-file progress and error state in
the composer. |

## Verification
- [x] `pnpm typecheck`
- [x] `pnpm lint`
- [x] `pnpm turbo build --force` under Node 22.22.2
- [x] Direct browser upload and agent-read verification for TXT, CSV,
PNG, Markdown, DOCX, PDF, PPTX, and XLSX
- [x] SHA-256 equality between local fixtures and Daytona materialized
files
- [x] Files tree and rich preview verification for DOCX, PDF, PPTX, and
XLSX
- [x] Hard-reload persistence and clean browser console/service-log
review

Author: iamjr15

README.md

@@ -4,36 +4,182 @@ Cheatcode is a TypeScript-first generalist AI agent platform with a Vercel-hoste
 
 The live source, package READMEs, migrations, and deployment configuration define the current system. The deleted `plan.md` is intentionally not authoritative and must not be restored.
 
-## Local setup
+## Run Cheatcode locally
+
+`pnpm dev` is the only supported full-stack local entrypoint. It builds a
+reproducible Docker image and starts:
+
+- the Next.js web app;
+- the gateway, agent, webhooks, and preview-proxy Workers in one chained local
+  Wrangler process;
+- local Durable Object, KV, R2, Workflow, and Wrangler state; and
+- the shared package build watcher.
+
+The application processes run locally, but a fully functional stack still uses
+real remote services. In particular, local Workers connect to the production
+Supabase database through its public session pooler and three isolated runtime
+roles, and agents create development sandboxes in Daytona. Local startup never
+starts Postgres, applies migrations, or deploys anything to Cloudflare or
+Vercel.
+
+### Prerequisites
+
+Install:
+
+- Docker Desktop or Docker Engine with a recent Docker Compose release that
+  supports `docker compose up --watch`;
+- NVM (or another version manager capable of selecting the exact Node version
+  in `.nvmrc`); and
+- Corepack, which supplies the exact pnpm version declared in `package.json`.
+
+Prepare and verify the host toolchain:
 
 ```bash
+nvm install
 nvm use
+corepack enable
+corepack prepare pnpm@11.15.0 --activate
+
+node --version
+pnpm --version
+docker compose version
+docker info
+```
+
+The expected Node and pnpm versions are `v22.22.2` and `11.15.0`. Do not ignore
+an engine warning: select or install Node 22.22.2 before installing packages or
+running repository commands. Docker must be running before `pnpm dev`.
+
+### Configure local credentials
+
+Create the one local application environment file:
+
+```bash
 cp .env.example .env.local
-# Fill the production Supabase runtime-role URLs plus the Clerk development,
-# Daytona, Polar sandbox, and integration values.
+chmod 600 .env.local
+```
+
+Fill every required value in `.env.local`. Keep the following boundaries:
+
+- `SUPABASE_GATEWAY_DATABASE_URL`, `SUPABASE_AGENT_DATABASE_URL`, and
+  `SUPABASE_WEBHOOKS_DATABASE_URL` are the production Supabase session-pooler
+  URLs for `app_gateway`, `app_agent`, and `app_webhooks`. Do not use a direct
+  database URL, an administrative role, `service_role`, or one role's password
+  for another role.
+- `NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY` and `CLERK_SECRET_KEY` must come from the
+  Clerk development instance and must begin with `pk_test_` and `sk_test_`.
+  Production Clerk keys are intentionally rejected locally.
+- `DAYTONA_API_KEY`, `DAYTONA_SANDBOX_SNAPSHOT`, `DAYTONA_TARGET`, and
+  `DAYTONA_WORKSPACE_VOLUME` select the development Daytona environment.
+  `DAYTONA_WORKSPACE_VOLUME` must remain
+  `cheatcode-workspaces-development`; never point local runs at the production
+  workspace volume.
+- `POLAR_SERVER` must remain `sandbox`. Add the Polar sandbox access token,
+  webhook secret, and sandbox product IDs to exercise billing locally.
+- Each signing secret group in `.env.example` must contain non-placeholder
+  values of at least 32 UTF-8 bytes. Secrets within a group must be distinct.
+  The startup runner checks these requirements before launching a Worker.
+- Keep `NEXT_PUBLIC_GATEWAY_URL=http://127.0.0.1:8787`,
+  `NEXT_PUBLIC_PREVIEW_HOSTNAME=localhost`, and
+  `NEXT_PUBLIC_VERCEL_GIT_COMMIT_SHA=development` for the standard local
+  topology.
+- `COMPOSIO_API_KEY`, `COMPOSIO_AUTH_CONFIGS`, and
+  `COMPOSIO_WEBHOOK_SECRET` are required for connected-tool flows.
+  `COMPOSIO_AUTH_CONFIGS` is the JSON object that maps each supported toolkit
+  name to its Composio auth-config ID.
+- `DEEPSEEK_PLATFORM_API_KEY` is optional because users may rely entirely on
+  BYOK. `DAYTONA_ORG_ID`, Clerk webhook verification, and internal alert
+  delivery are optional only when the corresponding account or callback flow
+  is not being exercised.
+
+Do not copy `.env.production` into `.env.local`. Do not put database migration
+credentials in this file; authorized operators keep those only in the ignored
+`.env.migrate` file. The full variable list and safe local defaults live in
+[`.env.example`](./.env.example).
+
+### Start the stack
+
+From the repository root:
+
+```bash
 pnpm dev
 ```
 
-`pnpm dev` is the complete local entrypoint. Docker Compose builds the pinned
-Node and pnpm development image and starts Next.js plus the chained Workers.
-Local Workers use the production Supabase database through its session pooler
-and the same three isolated runtime roles as production Hyperdrive. Local
-startup never applies database migrations. Stop the stack with:
+The first run builds the pinned Node 22.22.2/pnpm 11.15.0 image, installs the
+locked workspace dependencies inside it, builds shared packages, validates
+`.env.local`, generates permission-restricted local Wrangler configs, and then
+starts the watchers. Subsequent source edits are synchronized into the
+container. Changes to package manifests or the lockfile trigger an image
+rebuild.
+
+Wait for the Compose service to report `healthy`. In another terminal:
 
 ```bash
-pnpm dev:down
+docker compose --env-file .env.local ps
+docker compose --env-file .env.local logs -f app
 ```
 
 Expected local endpoints:
 
-- Web: `http://127.0.0.1:3000`
+- Web app: `http://localhost:3001`
 - Gateway and chained Workers: `http://127.0.0.1:8787`
-- Wrangler inspector: `http://localhost:9239`
+- Gateway health: `http://127.0.0.1:8787/health`
+- Wrangler inspector: `http://127.0.0.1:9239`
+
+The Compose health check verifies both the web app's Cheatcode symbol asset and
+the gateway's JSON health response. A healthy gateway also proves that the
+service-bound agent and webhooks Workers are reachable.
+
+### Stop or reset the stack
+
+Stop all local services cleanly:
+
+```bash
+pnpm dev:down
+```
+
+The normal shutdown keeps the local Next and Wrangler cache volumes so the next
+start is faster. If those generated caches become corrupt, remove only those
+local volumes and rebuild:
+
+```bash
+docker compose --env-file .env.local down --volumes --remove-orphans
+pnpm dev
+```
+
+This does not delete production Supabase data or Daytona workspaces. Project and
+account deletion must still go through the application so its durable cleanup
+workflow can remove remote resources correctly.
+
+### Troubleshooting
+
+- **Node engine mismatch:** run `nvm install 22.22.2 && nvm use 22.22.2`, then
+  confirm `node --version` before retrying.
+- **Docker cannot connect:** start Docker Desktop or the Docker daemon and
+  confirm `docker info` succeeds.
+- **A required environment value is missing:** read the startup error, update
+  the named value in `.env.local`, and rerun `pnpm dev`. The runner also rejects
+  production Clerk keys, unsafe database targets, reused signing secrets, and
+  cloud-only credentials in the local file.
+- **Port already in use:** release ports `3001`, `8787`, and `9239`; the
+  supported Compose topology binds all three to loopback.
+- **A dependency changed but the image did not rebuild:** run
+  `docker compose --env-file .env.local build --no-cache app`, then
+  `pnpm dev`.
+- **The UI loads but an external feature fails:** confirm the relevant remote
+  service credential is populated and active. Supabase, Clerk, Daytona, Polar,
+  Composio, and provider APIs are not emulated by Compose.
+- **A provider webhook is being tested:** the provider must be configured to
+  reach the local webhooks Worker through a trusted public ingress, and its
+  signing secret must match `.env.local`. Loopback URLs cannot receive
+  internet-originated callbacks by themselves.
+
+### Verify the product
 
 Product QA is direct browser operation only:
 
 ```bash
-agent-browser --auto-connect --session cheatcode-debug open http://127.0.0.1:3000
+agent-browser --auto-connect --session cheatcode-debug open http://localhost:3001
 agent-browser --auto-connect --session cheatcode-debug snapshot -i
 ```
 

apps/agent-worker/README.md

@@ -32,6 +32,18 @@ existence before minting a one-hour HMAC capability; the public signed download
 streaming second hop. Expiring capabilities and internal R2 keys are never stored in transcripts or
 returned by artifact tools.
 
+User uploads are durable project files rather than prompt text. The authenticated project-file
+route accepts one bounded raw file at a time, validates its filename, extension, UTF-8 or binary
+signature, and tenant/project write state, then derives deterministic file and version UUIDs from
+the project path and content digest. R2 stores immutable bytes under the existing
+`user/project/` lifecycle prefix with create-only checksum verification. ProjectSandbox stores the
+small current/version namespace records and mirrors the current version to
+`/workspace/<workspaceSlug>/uploads/` on the user's persistent Daytona volume before exposing it.
+An exact replay is idempotent; uploading new bytes at the same path creates a retained version and
+updates the working copy. Project deletion removes the namespace during fenced workspace cleanup
+and the existing resource-deletion prefix sweep removes every immutable object. Account deletion
+clears both through the existing account state and R2 lifecycle phases.
+
 Run creation validates the gateway payload with the shared `CreateRunSchema` from
 `packages/types` before selecting the run-scoped `AgentRun` Durable Object. The
 database binds a gateway-hashed idempotency key to the exact body and thread. After the

apps/agent-worker/src/agent-api-system-routes.ts

@@ -175,7 +175,7 @@ async function deleteInternalUserState(c: AgentContext): Promise<Response> {
     await sandbox.deleteAccountState();
     return deletedStateResponse(c);
   }
-  const sandbox = await sandboxForUser(c.env, userId);
+  const sandbox = await sandboxStubForUser(c.env, userId);
   await sandbox.cleanupProjectWorkspace({
     projectId: body.projectId,
     workspaceSlug: body.workspaceSlug,

apps/agent-worker/src/agent-routing.ts

@@ -9,6 +9,7 @@ import {
   findActiveAgentRunForThread,
   findAgentEntitlementByUserId,
   findAgentRunForUser,
+  getProject,
   getProjectWriteState,
   getThread,
   type RunPersonalization,
@@ -20,7 +21,14 @@ import {
   emitUserEvent,
   readBoundedResponseJson,
 } from "@cheatcode/observability";
-import { AgentRunId, type CreateRun, ThreadId, UserId } from "@cheatcode/types";
+import {
+  AgentRunId,
+  type CreateRun,
+  ProjectId,
+  type ProjectSummary,
+  ThreadId,
+  UserId,
+} from "@cheatcode/types";
 import {
   QUOTA_FEATURES,
   QUOTA_TRACKER_MAX_RESPONSE_BYTES,
@@ -120,6 +128,53 @@ export async function requireWritableThreadProject(
   }
 }
 
+export async function requireProjectAccess(
+  env: AgentEnv,
+  userId: string,
+  projectId: string,
+  writable: boolean,
+): Promise<ProjectSummary & { workspaceSlug: string }> {
+  const parsedUserId = UserId(userId);
+  const { db, close } = createDb(env.HYPERDRIVE, {
+    audience: "app_agent",
+    signingSecret: env.DATABASE_CONTEXT_SIGNING_SECRET_AGENT,
+  });
+  try {
+    return await withUserContext(db, parsedUserId, async (tx) => {
+      const project = await getProject(tx, {
+        projectId: ProjectId(projectId),
+        userId: parsedUserId,
+      });
+      if (!project) {
+        throw new APIError(404, "not_found_project", "Project not found", { retriable: false });
+      }
+      if (writable && project.readOnly) {
+        throw new APIError(
+          403,
+          "permission_plan_required",
+          "Project is read-only after downgrade",
+          {
+            details: {
+              archiveAfter: project.archiveAfter?.toISOString() ?? null,
+              overQuota: project.overQuota,
+            },
+            hint: "Delete or archive over-limit projects, or upgrade your plan to continue editing this project.",
+            retriable: false,
+          },
+        );
+      }
+      return {
+        ...project,
+        archiveAfter: project.archiveAfter?.toISOString() ?? null,
+        createdAt: project.createdAt.toISOString(),
+        updatedAt: project.updatedAt.toISOString(),
+      };
+    });
+  } finally {
+    await close();
+  }
+}
+
 export function agentRunForRunId(env: AgentEnv, runId: string): DurableObjectStub<AgentRun> {
   return env.AGENT_RUN.get(env.AGENT_RUN.idFromName(runId));
 }

apps/agent-worker/src/durable-objects/project-sandbox-content.ts

@@ -49,6 +49,7 @@ import {
   shellQuote,
   timeoutSeconds,
 } from "./project-sandbox-process-support";
+import { ProjectSandboxProjectFiles } from "./project-sandbox-project-files";
 import {
   type ProjectArchiveInput,
   ProjectArchiveInputSchema,
@@ -81,7 +82,6 @@ import {
   type ProjectWriteFileInput,
   ProjectWriteFileInputSchema,
 } from "./project-sandbox-runtime";
-import { ProjectSandboxWorkspaceTransition } from "./project-sandbox-workspace-transition";
 
 const PREVIEW_STATUS_PROBE_TIMEOUT_MS = 3_000;
 const PREVIEW_WAKE_TIMEOUT_MS = 90_000;
@@ -91,7 +91,7 @@ const BROWSER_TAKEOVER_PORT_MIN = 60_000;
 const BROWSER_TAKEOVER_PORT_MAX = 60_999;
 const BROWSER_TAKEOVER_SCRIPT = "/opt/cheatcode/start-browser-takeover.sh";
 
-export abstract class ProjectSandboxContent extends ProjectSandboxWorkspaceTransition {
+export abstract class ProjectSandboxContent extends ProjectSandboxProjectFiles {
   public downloadProjectArchive(input: ProjectArchiveInput): Promise<Response> {
     return this.downloadProjectArchiveForRpc(input, () => undefined);
   }
@@ -194,9 +194,7 @@ export abstract class ProjectSandboxContent extends ProjectSandboxWorkspaceTrans
   public async writeFile(input: ProjectWriteFileInput): Promise<SandboxWriteFileResult> {
     const parsed = ProjectWriteFileInputSchema.parse(input);
     const id = await this.ensureSandbox();
-    await this.client()
-      .createFolder(id, dirname(parsed.path))
-      .catch(() => undefined);
+    await this.client().createFolder(id, dirname(parsed.path));
     const bytes =
       parsed.encoding === "base64"
         ? decodeBase64(parsed.content)
@@ -393,18 +391,22 @@ export abstract class ProjectSandboxContent extends ProjectSandboxWorkspaceTrans
   public cleanupProjectWorkspace(input: ProjectCleanupWorkspaceInput): Promise<void> {
     const parsed = ProjectCleanupWorkspaceInputSchema.parse(input);
     return this.deleteProjectWorkspace(parsed, () =>
-      this.performProjectWorkspaceCleanup(parsed.workspaceSlug),
+      this.performProjectWorkspaceCleanup(parsed.projectId, parsed.workspaceSlug),
     );
   }
 
-  private async performProjectWorkspaceCleanup(workspaceSlug: string): Promise<void> {
+  private async performProjectWorkspaceCleanup(
+    projectId: string,
+    workspaceSlug: string,
+  ): Promise<void> {
     const id = await this.ensureExistingSandboxStarted();
     await super.killAllProcesses();
     if (id) {
       await this.terminateUntrackedSandboxProcesses(id);
       await this.removeWorkspaceFolder(id, workspaceSlug);
     }
     await this.freeProjectPort(workspaceSlug);
+    await this.deleteUploadedFileMetadata(projectId);
   }
 
   private async mobileExpoProxy(

apps/agent-worker/src/durable-objects/project-sandbox-lifecycle-support.ts

@@ -19,6 +19,7 @@ export interface ProjectSandboxEnv {
   PREVIEW_HOSTNAME: string;
   QUOTA_TRACKER: DurableObjectNamespace;
   R2_AUDIT: R2Bucket;
+  R2_OUTPUTS: R2Bucket;
 }
 
 export const ACCOUNT_DELETION_TOMBSTONE_KEY = "account_deletion_tombstone";