Skip to content

Commit 470e337

Browse files
authored
refactor: wave b structural hardening (#97)
Wave B of the approved architecture-hardening plan (rev 5.3). Three commits, each adversarially reviewed (two DIVERGENT verdicts fully remediated before commit, one SOUND). - **B1/B2**: ~2.9k LOC of worker policy evicted from packages/db to its owners with every invariant still enforced inside its original locked transaction; db→billing dependency inverted. Includes a systemic fix: the dependency-cruiser gate was blind to ALL resolved cross-package edges (dist-exclude); now visible (1,218→1,656 edges) with both ban rules proven firing. - **B3/B4/B5**: one route manifest kills the 3× path declarations (all 21 forwarded routes, costs verified 1:1); create-run runs one handle + two short transactions with no pg transaction held across DO RPCs, thread read deduplicated, deleted-thread race returns typed 404; compile-time-branded UserContextDatabase; types ./api + ./internal subpaths (root barrel −117 lines, web provably cannot reach internal contracts). Ultra/Max tiers fully pruned (types, env, billing catalog, worker branches) — fixes the half-configured checkout bug; tier CHECK narrowed via appended migration 0001. - **B6**: analytics watchdog deleted (+its 2 CF env keys); OpsMaintenanceWorkflow split into per-job workflows with precise envs; daily maintenance is tableless — one deterministic instance per UTC day, immutable payload cutoff, idempotent R2 delete + identity-rechecked DB delete (timestamp(3) identity + disjoint-run-state proofs reviewed), accepted next-day recovery; daily_maintenance_jobs dropped via appended migration 0002; byok-revalidation demoted to cron+lease (per-target failure isolation + loud error events added per review). **Operator note post-merge:** the old `cheatcode-ops-maintenance` Workflow resource is orphaned in the Cloudflare account by the rename — delete it manually. Gates per commit: biome, tsc, turbo build --force, knip zero, dependency-cruiser clean, drizzle ledger append-only.
1 parent 5b45ec4 commit 470e337

209 files changed

Lines changed: 7745 additions & 4167 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.dependency-cruiser.cjs

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,12 @@ module.exports = {
2525
from: { path: "^packages/(tools-[^/]+)/" },
2626
to: { path: "^packages/tools-[^/]+/", pathNot: "^packages/$1/" },
2727
},
28+
{
29+
name: "database-must-not-import-billing-policy",
30+
severity: "error",
31+
from: { path: "^packages/db/" },
32+
to: { path: "^packages/billing/" },
33+
},
2834
{
2935
name: "vercel-web-must-not-import-worker-runtime-packages",
3036
severity: "error",
@@ -53,6 +59,6 @@ module.exports = {
5359
},
5460
],
5561
options: {
56-
doNotFollow: { path: "node_modules" },
62+
doNotFollow: { path: "(^|/)(dist|node_modules)/" },
5763
},
5864
};

.env.example

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -47,8 +47,6 @@ POLAR_SERVER=sandbox
4747
POLAR_WEBHOOK_SECRET=
4848
POLAR_PRODUCT_ID_PRO=
4949
POLAR_PRODUCT_ID_PREMIUM=
50-
POLAR_PRODUCT_ID_ULTRA=
51-
POLAR_PRODUCT_ID_MAX=
5250

5351
# Internal local contracts.
5452
DATABASE_CONTEXT_SIGNING_SECRET_AGENT=replace_with_a_distinct_32_byte_secret

.github/workflows/static-checks.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -252,7 +252,7 @@ jobs:
252252
pnpm exec dependency-cruise \
253253
--config .dependency-cruiser.cjs \
254254
--ts-config tsconfig.base.json \
255-
--exclude '(^|/)(dist|\.next|\.turbo|node_modules)/' \
255+
--exclude '(^|/)(\.next|\.turbo|node_modules)/' \
256256
"${directories[@]}"
257257
258258
- name: Check dead code

apps/agent-worker/README.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -132,6 +132,8 @@ instructions, source, schemas, templates, and assets in Files. A hidden mirror
132132
manifest avoids rewriting unchanged packages and limits cleanup to files previously
133133
managed by that package, preserving local dependencies and generated output. Curated
134134
default skills are immutable snapshot files under `/home/node/.cheatcode/default-skills/`.
135+
Agent-worker owns the custom-skill capacity decision and applies it inside the database's
136+
per-user locked catalog transaction before inserting a new skill.
135137

136138
Managed processes use required stable IDs and a maximum of 32 live metadata slots per user
137139
sandbox. Reusing an ID atomically replaces that slot. At capacity, ProjectSandbox reconciles the
@@ -168,7 +170,7 @@ and it does not apply per-run or daily dollar caps. Provider usage remains an
168170
opaque SDK concern.
169171

170172
AgentRun writes Workers Analytics Engine agent-run metrics on terminal statuses and emits
171-
a first-visible-chunk TTFT performance metric for the analytics watchdog. Run
173+
a first-visible-chunk TTFT performance metric. Run
172174
admission events carry the planned logical model, while stream-attempt/completion events carry
173175
the resolved logical model. A failure before any stream attempt keeps planned attribution instead;
174176
provider-local transport IDs remain structured-log context. R2-backed artifact
@@ -192,7 +194,7 @@ truncation, and there is no transcript-length, step, token, or cost ceiling.
192194
Mastra tool-call chunks also emit `step_started`, `step_completed`,
193195
`tool_invoked`, and `skill_invoked` events when those chunks are present in the
194196
live stream. If the last stream subscriber disconnects while a run is still
195-
running, AgentRun emits `run_abandoned` for the watchdog/funnel trail.
197+
running, AgentRun emits `run_abandoned` for the funnel trail.
196198

197199
Project deletion first fences project/thread mutations, refuses an active run, records a
198200
durable cleanup request, then removes that project's workspace folder. The database marks

apps/agent-worker/src/agent-api-run-routes.ts

Lines changed: 99 additions & 101 deletions
Original file line numberDiff line numberDiff line change
@@ -1,25 +1,30 @@
11
import {
22
type AgentRunHandle,
3+
type AgentRunThreadContext,
34
createAgentRunForThread,
4-
createDb,
55
createThreadMessage,
6-
getThread,
6+
type Database,
7+
loadAgentRunThreadContext,
78
type RunPersonalization,
89
reconcileAbsentAgentRunStart,
9-
withUserContext,
10+
type UserDatabaseSession,
11+
withUserDb,
1012
} from "@cheatcode/db";
1113
import { APIError, readJsonRequest } from "@cheatcode/observability";
1214
import {
1315
type AgentRunId,
14-
BrowserTakeoverResumeResultSchema,
15-
BrowserTakeoverResumeSchema,
16-
BrowserTakeoverStatusSchema,
17-
type CreateRun,
1816
ThreadId,
1917
UserId as toUserId,
2018
type UIMessagePart,
2119
type UserId,
2220
} from "@cheatcode/types";
21+
import {
22+
BrowserTakeoverResumeResultSchema,
23+
BrowserTakeoverResumeSchema,
24+
BrowserTakeoverStatusSchema,
25+
type CreateRun,
26+
} from "@cheatcode/types/api";
27+
import { AGENT_FORWARD_ROUTES } from "@cheatcode/types/internal";
2328
import type { Context, Hono } from "hono";
2429
import { z } from "zod";
2530
import type { AgentEnv } from "./agent-env";
@@ -28,9 +33,10 @@ import {
2833
activeRunForThreadRoute,
2934
agentRunForRunId,
3035
callAgentRun,
36+
enforceRunEntitlementPolicy,
3137
fetchAgentRun,
38+
loadRunEntitlementPolicy,
3239
reconcileAgentRunAdmission,
33-
runEntitlementPolicy,
3440
runForRoute,
3541
sandboxForUser,
3642
startAgentRun,
@@ -63,11 +69,24 @@ type RejectedRunResult = Exclude<
6369

6470
export function registerAgentRunHttpRoutes(app: Hono<{ Bindings: AgentEnv }>): void {
6571
app.post("/v1/threads/:threadId/runs", createRun);
66-
app.get("/v1/threads/:threadId/runs/stream", streamActiveRun);
67-
app.post("/v1/runs/:runId/cancel", cancelRun);
68-
app.get("/v1/threads/:threadId/browser-takeover", browserTakeoverStatus);
69-
app.post("/v1/threads/:threadId/browser-takeover/start", startBrowserTakeover);
70-
app.post("/v1/threads/:threadId/browser-takeover/resume", resumeBrowserTakeover);
72+
const routes = AGENT_FORWARD_ROUTES.piped;
73+
app.on(routes.streamRun.method, routes.streamRun.path, streamActiveRun);
74+
app.on(routes.cancelRun.method, routes.cancelRun.path, cancelRun);
75+
app.on(
76+
routes.browserTakeoverStatus.method,
77+
routes.browserTakeoverStatus.path,
78+
browserTakeoverStatus,
79+
);
80+
app.on(
81+
routes.browserTakeoverStart.method,
82+
routes.browserTakeoverStart.path,
83+
startBrowserTakeover,
84+
);
85+
app.on(
86+
routes.browserTakeoverResume.method,
87+
routes.browserTakeoverResume.path,
88+
resumeBrowserTakeover,
89+
);
7190
}
7291

7392
async function createRun(c: AgentContext): Promise<Response> {
@@ -78,61 +97,56 @@ async function createRun(c: AgentContext): Promise<Response> {
7897
await readJsonRequest(c.req.raw, MAX_CREATE_RUN_BODY_BYTES, "Create run request"),
7998
);
8099
const requestIdentity = readRunRequestIdentity(c.req.raw.headers);
81-
const personalization = await loadRequestPersonalization(c.env, parsedUserId, threadId, body);
82-
const policy = await runEntitlementPolicy(c.env, userId);
83-
const sandboxName = await userSandboxName(userId);
84-
const sandbox = await sandboxForUser(c.env, userId);
85-
await syncSandboxQuotaPeriod(sandbox, policy.quotaPeriodEnd);
86-
const result = await persistRunRequest(c.env, {
87-
body,
88-
personalization,
89-
requestIdentity,
90-
threadId,
91-
userId: parsedUserId,
92-
});
93-
if (result.type === "created") {
94-
const outcome = await startAgentRun(c.env, {
95-
body,
96-
modelExplicit: result.modelExplicit,
97-
personalization,
98-
run: result.run,
99-
sandboxName,
100-
userId,
101-
});
102-
return resolveRunAdmission(c.env, parsedUserId, result.run, outcome);
103-
}
104-
if (result.type === "active-run-exists" || result.type === "idempotent-replay") {
105-
const outcome = await reconcileAgentRunAdmission(c.env, userId, result.run.runId);
106-
return resolveRunAdmission(c.env, parsedUserId, result.run, outcome);
107-
}
108-
throw rejectedRunError(result);
109-
}
110-
111-
async function loadRequestPersonalization(
112-
env: AgentEnv,
113-
userId: UserId,
114-
threadId: string,
115-
body: CreateRun,
116-
): Promise<RunPersonalization> {
117-
const { db, close } = createDb(env.HYPERDRIVE, {
118-
audience: "app_agent",
119-
signingSecret: env.DATABASE_CONTEXT_SIGNING_SECRET_AGENT,
120-
});
121-
try {
122-
return await withUserContext(db, userId, async (tx) => {
123-
const thread = await getThread(tx, { threadId: ThreadId(threadId), userId });
100+
return withUserDb(c.env, parsedUserId, async ({ transaction }) => {
101+
const prepared = await transaction(async (tx) => {
102+
const thread = await loadAgentRunThreadContext(tx, {
103+
threadId: ThreadId(threadId),
104+
userId: parsedUserId,
105+
});
124106
if (!thread) {
125107
throw new APIError(404, "not_found_thread", "Thread not found", { retriable: false });
126108
}
127-
return loadRunPersonalization(tx, userId, body.model);
109+
return {
110+
entitlement: await loadRunEntitlementPolicy(tx, parsedUserId),
111+
personalization: await loadRunPersonalization(tx, parsedUserId, body.model),
112+
thread,
113+
};
128114
});
129-
} finally {
130-
await close();
131-
}
115+
const policy = await enforceRunEntitlementPolicy(c.env, userId, prepared.entitlement);
116+
const sandboxName = await userSandboxName(userId);
117+
const sandbox = await sandboxForUser(c.env, userId);
118+
await syncSandboxQuotaPeriod(sandbox, policy.quotaPeriodEnd);
119+
const result = await transaction((tx) =>
120+
persistRunRequest(tx, prepared.thread, {
121+
body,
122+
personalization: prepared.personalization,
123+
requestIdentity,
124+
threadId,
125+
userId: parsedUserId,
126+
}),
127+
);
128+
if (result.type === "created") {
129+
const outcome = await startAgentRun(c.env, {
130+
body,
131+
modelExplicit: result.modelExplicit,
132+
personalization: prepared.personalization,
133+
run: result.run,
134+
sandboxName,
135+
userId,
136+
});
137+
return resolveRunAdmission(parsedUserId, result.run, outcome, transaction);
138+
}
139+
if (result.type === "active-run-exists" || result.type === "idempotent-replay") {
140+
const outcome = await reconcileAgentRunAdmission(c.env, userId, result.run.runId);
141+
return resolveRunAdmission(parsedUserId, result.run, outcome, transaction);
142+
}
143+
throw rejectedRunError(result);
144+
});
132145
}
133146

134147
async function persistRunRequest(
135-
env: AgentEnv,
148+
tx: Database,
149+
thread: AgentRunThreadContext,
136150
input: {
137151
body: CreateRun;
138152
personalization: RunPersonalization;
@@ -141,34 +155,28 @@ async function persistRunRequest(
141155
userId: UserId;
142156
},
143157
): Promise<CreateRunResult> {
144-
const { db, close } = createDb(env.HYPERDRIVE, {
145-
audience: "app_agent",
146-
signingSecret: env.DATABASE_CONTEXT_SIGNING_SECRET_AGENT,
147-
});
148-
try {
149-
return await withUserContext(db, input.userId, async (tx) => {
150-
const created = await createAgentRunForThread(tx, {
151-
idempotencyKeyHash: input.requestIdentity.keyHash,
152-
personalization: input.personalization,
153-
requestBodyHash: input.requestIdentity.bodyHash,
154-
threadId: ThreadId(input.threadId),
155-
userId: input.userId,
156-
...(input.body.model === undefined ? {} : { modelId: input.body.model }),
157-
});
158-
if (created.type === "created") {
159-
await createThreadMessage(tx, {
160-
agentRunId: created.run.runId,
161-
parts: persistedUserMessageParts(input.body),
162-
role: "user",
163-
threadId: created.run.threadId,
164-
userId: input.userId,
165-
});
166-
}
167-
return created;
158+
const created = await createAgentRunForThread(
159+
tx,
160+
{
161+
idempotencyKeyHash: input.requestIdentity.keyHash,
162+
personalization: input.personalization,
163+
requestBodyHash: input.requestIdentity.bodyHash,
164+
threadId: ThreadId(input.threadId),
165+
userId: input.userId,
166+
...(input.body.model === undefined ? {} : { modelId: input.body.model }),
167+
},
168+
thread,
169+
);
170+
if (created.type === "created") {
171+
await createThreadMessage(tx, {
172+
agentRunId: created.run.runId,
173+
parts: persistedUserMessageParts(input.body),
174+
role: "user",
175+
threadId: created.run.threadId,
176+
userId: input.userId,
168177
});
169-
} finally {
170-
await close();
171178
}
179+
return created;
172180
}
173181

174182
function persistedUserMessageParts(body: CreateRun): UIMessagePart[] {
@@ -209,40 +217,30 @@ function rejectedRunError(result: RejectedRunResult): APIError {
209217
}
210218

211219
async function resolveRunAdmission(
212-
env: AgentEnv,
213220
userId: UserId,
214221
run: AgentRunHandle,
215222
outcome: AgentRunAdmissionOutcome,
223+
transaction: UserDatabaseSession["transaction"],
216224
): Promise<Response> {
217225
if (outcome.type === "confirmed") {
218226
return withRunLocation(outcome.response, run.runId);
219227
}
220228
if (outcome.type === "ambiguous") {
221229
throw runAdmissionAmbiguousError(run.runId);
222230
}
223-
const reconciliation = await reconcileAbsentRunRow(env, userId, run.runId);
231+
const reconciliation = await reconcileAbsentRunRow(transaction, userId, run.runId);
224232
if (reconciliation === "not-found") {
225233
throw runAdmissionAmbiguousError(run.runId);
226234
}
227235
throw runAdmissionAbsentError(run.runId);
228236
}
229237

230238
async function reconcileAbsentRunRow(
231-
env: AgentEnv,
239+
transaction: UserDatabaseSession["transaction"],
232240
userId: UserId,
233241
runId: AgentRunId,
234242
): Promise<"failed" | "not-found" | "terminal"> {
235-
const { db, close } = createDb(env.HYPERDRIVE, {
236-
audience: "app_agent",
237-
signingSecret: env.DATABASE_CONTEXT_SIGNING_SECRET_AGENT,
238-
});
239-
try {
240-
return await withUserContext(db, userId, (tx) =>
241-
reconcileAbsentAgentRunStart(tx, { runId, userId }),
242-
);
243-
} finally {
244-
await close();
245-
}
243+
return transaction((tx) => reconcileAbsentAgentRunStart(tx, { runId, userId }));
246244
}
247245

248246
function runAdmissionAbsentError(runId: AgentRunId): APIError {

0 commit comments

Comments
 (0)