Add claude_code_env (Claude Code coding-agent environment) - #1030
Draft
sergiopaniego wants to merge 12 commits into
Draft
Add claude_code_env (Claude Code coding-agent environment)#1030sergiopaniego wants to merge 12 commits into
sergiopaniego wants to merge 12 commits into
Conversation
The sandbox protocol (SandboxBackend/SandboxHandle/BgJob/ExecResult), the E2B and Hugging Face backends, the interception proxy, and build_template move from envs/opencode_env/sandbox/ to src/openenv/core/sandbox/, so environments import them from core rather than from opencode_env. opencode_env now imports from openenv.core.sandbox. interception.py is no longer baked into the opencode pre-baked image (the harness uploads it at rollout start from core), and the opencode hf_image dir moves out of sandbox/ to opencode_env/hf_image/. e2b stays a lazy import so core keeps no hard e2b dependency. No backward-compat shim, opencode_env is experimental. AI-assisted.
The sandbox module moved to openenv.core, so the relative-import fallback must keep importing the backend from the installed package, not the removed local sandbox module.
Mirrors opencode_env/pi_env for the Claude Code CLI. Claude Code speaks the Anthropic Messages API only, so transparent_proxy mode runs an in-sandbox Anthropic->OpenAI translation shim in front of the shared interception proxy; token_ids + logprobs are captured on the OpenAI/vLLM side exactly as for the other envs (validated locally). Sandbox layer comes from openenv.core.sandbox. Includes the pi_env robustness (start_agent=False setup-before-agent, non-zero agent-exit guard, pipefail run command). Smoke 0 (headless install + run) and Smoke 1 (shim translation + capture) pass. Real-vLLM E2E, run_rollout and training smokes still pending.
…mat) vLLM with --return-tokens-as-token-ids encodes the id in the token field as "token_id:<int>" with no separate token_id key, so the capture read 0 token ids on current vLLM. Accept both the prefixed token string and a token_id key. Affects every env that trains on captured token ids (opencode/pi too).
--allowedTools is variadic and swallowed the trailing "$(cat instruction)" prompt argument, so Claude Code got no prompt and exited with 'Input must be provided...' before making a single model call. Pipe the instruction on stdin.
…dead token_id-key path
The empty completion_token_ids is recovered downstream by TRL's rollout worker from completion_tokens, so populating it in the proxy just duplicates that parse. Keeps claude_code_env from forking the shared core proxy. See closed OpenEnv#1029.
- docs/source/environments/claude_code.md + toctree entry + environments.md card - tests/envs/test_claude_code_factory_lifecycle.py (create() teardown/retry, 4 tests) - hf_image/Dockerfile pre-bakes Node + Claude Code + proxy/shim deps - rewrite README from the actual code (drop the pi_env leftovers)
…idance - examples/claude_code_env_simple.py: inference demo hitting the deployed Space - docs tutorial claude-agent-grpo.md: black-box AsyncGRPO training recipe, links to TRL - docker-build.yml: build the pre-baked claude-code-sandbox image, named in the docs - config: cap proxy max_tokens at 8192 and document a generous vLLM max-model-len
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
claude_code_env, a Claude Code coding-agent environment mirroringopencode_envandpi_envfor loop-owning GRPO training. Claude Code speaks the Anthropic Messages API, so an in-sandbox Anthropic-to-OpenAI shim (anthropic_shim.py) sits in front of the shared interception proxy. Capture still happens at the vLLM seam, so training correctness is unchanged.Type of Change
Alignment Checklist
opencode_env/pi_env)RFC Status
Test Plan
tests/envs/test_claude_code_factory_lifecycle.py: 4 factory lifecycle tests passexamples/claude_code_env_simple.pyClaude Code Review
AI-assisted.
Merge order: stacked on #1022 (imports
openenv.core.sandbox). Merge #1022 first, then rebase this. The combined diff cleans up once #1022 lands. The TRL trainer example (claude.py/claude_code_hf_sandbox.py) ships as a separate PR againsthuggingface/trl.