Skip to content

refactor(server): optimize injected agent instructions#2977

Draft
reachjalil wants to merge 3 commits into
different-ai:devfrom
reachjalil:feature/prompt-search-guidance
Draft

refactor(server): optimize injected agent instructions#2977
reachjalil wants to merge 3 commits into
different-ai:devfrom
reachjalil:feature/prompt-search-guidance

Conversation

@reachjalil

@reachjalil reachjalil commented Jul 22, 2026

Copy link
Copy Markdown
Collaborator

Intent

Reduce always-on desktop-agent instructions to the OpenWork-specific decisions the model cannot safely infer: workspace output paths, private-data boundaries, confirmed Memory Bank writes, product-doc routing, live Connect state, and boundaries between UI, browser, session, extension, and Cloud tools.

The goal is not brevity by itself. It is less duplicated context, fewer conflicting instructions, and fewer unnecessary tool calls while preserving product and safety invariants.

Methodology

  1. Inventory every OpenWork-owned system instruction injected through the default agent prompt and both experimental.chat.system.transform plugins.
  2. Classify each statement as:
    • stable product invariant — keep in the base instruction
    • live workspace/model state — keep in conditional runtime steering
    • tool-local usage detail — keep in the tool description
    • retrievable product detail — move out of always-on context and use bundled docs
    • common agent behavior — remove unless it protects a measured OpenWork requirement
  3. State each retained decision once, under a short named scope, using direct action-oriented wording.
  4. Replace mandatory search counts with an outcome-based stop condition: start with one focused search and refine only when no result is relevant.
  5. Lock the design with behavior assertions and an aggregate instruction-size budget.

This follows current guidance to use lean prompts, avoid repeated instructions/examples, keep tool descriptions concise, prioritize critical constraints, and validate changes on representative behavior:

Reassessment

Instruction layer Retained Removed or relocated
Default OpenWork agent identity, workspace-relative outputs, private-data boundary, explicit/confirmed Memory Bank operations generic coding style, file-format recipes, repeated memory examples, guessed capability names
Product guidance docs-first product answers, docs-vs-action boundary, live steering authority, Connect-vs-local-MCP routing embedded product manual, provider recipes, OAuth/JWT protocol detail, static client support matrix, hard-coded docs catalog
Ready Connect steering one focused search, conditional keyword refinement, exact returned-name execution, connection-status relay, connector-vs-Cloud auth distinction mandatory 2-4 searches, service lists, repeated settings examples, equivalent retries
Signed-out/disabled Connect steering current state and one repair path concatenated generic extension policy and repeated explanation
Extension fallback list before declaring unavailable, then call the match duplicated tool names and schemas already described by the tools
Past sessions search, read, clarify ambiguity, answer from evidence repeated descriptions of transcript matching and UI navigation
Browser open URL first, reuse returned target, external-site boundary repeated browser command list and implementation-specific target heuristics
OpenWork UI app-vs-browser boundary, snapshot/list-before-unknown-action hard-coded action examples already discoverable through the action catalog
Tool descriptions concise tool-local purpose and sequencing copies of global system instructions appended to multiple descriptions

Measured change

  • Before (e967b255d): 18,380 characters across the nine base and conditional instruction variants.
  • After: 3,518 characters across the same inventory.
  • Reduction: 80.9%.

This is an instruction-inventory comparison, not a claim that every request previously included every conditional variant.

Checks

  • pnpm --dir apps/server exec bun test src/opencode-plugins/openwork-system-instructions.test.ts src/opencode-plugins/openwork-capabilities-knowledge.test.ts src/opencode-plugins/openwork-extensions-preview-connect-steering.test.ts src/opencode-plugins/openwork-extensions-preview.test.ts src/openwork-runtime-config.test.ts — 39 passed
  • pnpm --filter openwork-server typecheck — passed
  • git diff --check — passed

The new instruction-design tests inventory all variants, require scoped headings, reject shouty ALWAYS / CRITICAL / FIRST wording, enforce separation between retrievable product detail and live state, and cap the combined inventory at 4,000 characters.

Impact and gaps

  • Prompt-only server/runtime-plugin change; no API, persistence, permission, or rendered UI contract changes.
  • Bundled documentation remains the source for the product and protocol detail removed from always-on context.
  • No visual evidence applies because this changes model instructions rather than rendered UI.
  • Full server test suite and model-provider behavioral evals were not run; the PR uses deterministic instruction and routing contracts.

@vercel

vercel Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
openwork-landing Ready Ready Preview, Comment, Open in v0 Jul 22, 2026 6:30am
1 Skipped Deployment
Project Deployment Actions Updated (UTC)
openwork-diagnostics Skipped Skipped Jul 22, 2026 6:30am

@vercel
vercel Bot temporarily deployed to Preview – openwork-diagnostics July 22, 2026 06:08 Inactive
@vercel

vercel Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

@reachjalil is attempting to deploy a commit to the Different AI Team on Vercel.

A member of the Team first needs to authorize it.

@vercel
vercel Bot temporarily deployed to Preview – openwork-diagnostics July 22, 2026 06:29 Inactive
@reachjalil reachjalil changed the title refactor(server): simplify capability search steering refactor(server): optimize injected agent instructions Jul 22, 2026
@reachjalil

Copy link
Copy Markdown
Collaborator Author

Experiment status — do not merge yet

This PR is intentionally an experiment. It is not intended to be merged, marked ready, or used as a standalone final prompt decision yet. Keep it in draft while we evaluate the three related changes together and collect model-behavior evidence.

Combined experiment branch: reachjalil/openwork:feature/capability-search-experiment

The branch is based on current dev and merges these exact PR heads without modifying the source branches:

Combined validation checklist

Composition and environment

  • Merge all three exact PR heads onto current dev with no conflicts.
  • Install locked dependencies and pass the isolated Hub readiness check.
  • Confirm the experiment branch still contains only the union of the three PRs plus merge commits.

Automated contracts

  • Run the lexical ranking, capability-search, Google Workspace, external-cache, external-search divergence, timeout, prompt-instruction, capabilities-knowledge, extension-steering, extension-preview, and runtime-config focused tests together.
  • Run Den API and OpenWork server typechecks from the combined branch.
  • Verify the warm external-search benchmark performs zero additional remote catalog requests while returning the same schema-bearing matches as the cold search.
  • Verify cache failures are not retained, concurrent misses coalesce, TTL expiry reloads, credential/connection version changes invalidate entries, and memory bounds evict safely.

Agent search behavior

  • With a clearly relevant request, confirm the agent starts with one concise capability search rather than a fixed batch of variants.
  • When the first results are irrelevant or empty, confirm the agent retries with meaningfully broader or different keywords.
  • Confirm it stops searching once a relevant exact capability is found and executes only an exact returned name.
  • Confirm lexical ranking puts the intended capability first for list/create, Gmail, Calendar, Drive, memory, and representative external MCP queries.
  • Confirm repeated or synonymous queries do not trigger redundant remote external-MCP catalog discovery during the cache window.
  • Compare tool-call count and elapsed time with current dev on the same representative tasks.

Contract and safety preservation

  • Confirm native bodySchema and external argumentsSchema, schemaDigest, invocation metadata, marketplace requirements, and connection actions are unchanged.
  • Confirm connection-status results remain actionable and outrank unrelated callable tools without distorting relevant tool ranking.
  • Confirm a successful Cloud search is not misreported as a need to reconnect OpenWork Cloud when the downstream connector is the failing boundary.
  • Confirm authorization, tenant/member scope, credential checks, and capability execution remain live and are never bypassed by the search cache.
  • Confirm unchanged failing searches are not retried until the user completes the requested connector fix.

Prompt-layer behavior

  • Test at least one OpenAI model, one Anthropic model, and one supported local/other provider to catch provider-specific instruction-following differences.
  • Confirm product questions use bundled docs, while connected-service action requests use live capability tools rather than documentation as a substitute.
  • Confirm OpenWork UI requests use openwork_ui_*, external websites use the built-in browser flow, and past-session questions use session search/read.
  • Confirm Memory Bank writes require explicit user intent and confirmation, exclude secrets/sensitive personal data, and do not auto-recall.
  • Check that removed static recipes remain discoverable through docs or tool schemas when actually needed.

Decision gate

  • Record failures, model/provider differences, search-call counts, latency measurements, and any lost behavior.
  • Decide which prompt rules need restoring, shortening, or moving into tool descriptions/docs.
  • Only after the evidence is reviewed, decide whether any source PR should be revised and considered for readiness. Do not merge the combined experiment branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant