Skip to content

Add file and screenshot input for local agents - #39

Open
lvwerra wants to merge 4 commits into
mainfrom
feat/screenshot-input
Open

Add file and screenshot input for local agents#39
lvwerra wants to merge 4 commits into
mainfrom
feat/screenshot-input

Conversation

@lvwerra

@lvwerra lvwerra commented Aug 5, 2026

Copy link
Copy Markdown
Member

Summary

  • accept up to five regular files per turn at 100 MiB each, including PDF, DOCX and other Office files, archives, source files, and unknown formats
  • keep validated PNG, JPEG, GIF, and WebP files on the native image path while delivering other files to the agent by their managed session-scoped paths
  • support paste, drop, and file picking in active sessions; keep the quick-creation dialog uncluttered by omitting its redundant file picker while retaining paste and drop
  • use one structured, race-safe delivery path with image-only/file-only turns, native first-turn Codex images, Hermes image commands, and retryable terminal insertion
  • prevent duplicate clipboard images by treating the canonical clipboard file list as the single source for synchronous paste events
  • stream uploads with per-file and per-session limits, sanitize bounded filenames, detect raster formats from bytes, and force non-raster files such as SVG and HTML to download instead of rendering inline
  • retain attachments through stopped-session retries, make insertion idempotent, lock chips for the full send transaction, and protect shared viewers from injection

Remote agents remain explicitly disabled because they cannot read files stored on the Space.

Limits

  • 100 MiB per file
  • 5 files per turn
  • 500 MiB and 200 files per session

Verification

  • npm test in server
  • node test/attachments.test.mjs in server
  • npm run typecheck in web
  • npm run test:screenshots in server
  • node terminal-ui.test.mjs in server
  • git diff --cached --check

Design

The architecture, security model, failure behavior, limits, and manual PDF/DOCX test matrix are documented in docs/screenshot-input.md.

@lvwerra lvwerra changed the title Add screenshot input for local agents Add file and screenshot input for local agents Aug 6, 2026
@lvwerra lvwerra closed this Aug 6, 2026
@lvwerra lvwerra reopened this Aug 6, 2026
@lvwerra
lvwerra marked this pull request as ready for review August 6, 2026 22:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant