Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ As an AI agent operating via non-interactive shell tools (`run_shell_command`),
- Specially crafted mock scripts that simulate timeouts or API calls.
- **I CANNOT Run (Requires User Assistance):**
- **`colab auth`**: This command relies on the traditional Gcloud fallback `input_request` (via `USE_AUTH_EPHEM='0'`), which prompts the user via Python's `input()` to click a URL, sign in, and paste back an authorization code. My shell tool will hang indefinitely on this `input()`.
- **`colab drivemount`**: This command prompts the user via `sys.stdin.readline()` (specifically querying `/dev/tty` to ensure input is captured) to press `Enter` after granting OAuth consent in the browser. My shell tool will timeout/hang waiting for `Enter`.
- **`colab drivemount`**: This command prompts the user to press `Enter` after granting OAuth consent in the browser. The prompt reads the controlling terminal (`/dev/tty` on POSIX, `CONIN$` on Windows), falling back to stdin. My shell tool will timeout/hang waiting for `Enter`. The consent URL is also opened in a GUI browser when one is available.
- **Interactive (TTY) `colab repl` / `colab console`**: When stdin is a real terminal these commands drop into interactive raw-TTY modes that require real-time keystroke streaming. My shell tools cannot support this. (Piped stdin is fine — see above.)

Whenever working on interactive commands, I must build the core logic, write mock tests, and explicitly ask the user to run the live test in their terminal to verify success.
7 changes: 4 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Designed to support seamless developer productivity, headless automation, and AI
[Demo](https://github.com/user-attachments/assets/656226a9-af13-4fdb-8eda-d7de747336a2)

> [!NOTE]
> **Platform support:** the Colab CLI currently supports **Linux and macOS** only. Windows is not supported at this time.
> **Platform support:** Linux, macOS, and Windows (PowerShell and cmd.exe). Interactive `colab console` uses the Windows console API on Windows (raw keystrokes and ANSI color). `colab ssh --proxy-mode` reads stdin with a blocking read so it works as an OpenSSH ProxyCommand under Windows OpenSSH.

> [!TIP]
> Looking for in-notebook, interactive agent-assisted coding instead of a terminal workflow? See the [Colab MCP Server](https://github.com/googlecolab/colab-mcp).
Expand Down Expand Up @@ -97,7 +97,7 @@ Run `colab <command> --help` to view specific options, defaults, and detailed he
| Command | Description |
| --- | --- |
| `colab auth [-s NAME]` | Authenticate the VM for GCP services (BigQuery, GCS, etc.) |
| `colab drivemount [-s NAME] [PATH]` | Mount Google Drive on the VM (default: `/content/drive`) |
| `colab drivemount [-s NAME] [--authuser N] [PATH]` | Mount Google Drive on the VM (default: `/content/drive`) |
| `colab install [-s NAME] [-r FILE \| PKG...]` | Install packages on the VM using `uv` (falls back to `pip`) |
| `colab log [-s NAME] [-n N] [-o FILE]` | View or export session history (`.ipynb`, `.md`, `.txt`, `.jsonl`) |
| `colab usage` | Show account compute-unit usage rate and balance |
Expand Down Expand Up @@ -144,7 +144,8 @@ colab stop -s analysis
## Usage Notes

* **Machine shape:** Use `--high-mem` with `colab new`, `colab run`, or `colab ssh` (when auto-creating a runtime) to request a high-RAM machine shape. Requires Colab Pro or Pro+ entitlement for supported accelerators (CPU, T4, A100, etc.). L4 and TPU runtimes ignore this flag because they only offer one shape. Machine shape is shown in `colab sessions` and `colab status`.
* **TTY Requirements:** The interactive commands `repl` and `console` require a local TTY. When running inside automated scripts or pipelines, make sure to pipe stdin (e.g., `echo "print(1)" | colab repl`) to trigger non-interactive execution modes.
* **TTY Requirements:** The interactive commands `repl` and `console` require a local TTY. When running inside automated scripts or pipelines, make sure to pipe stdin (e.g., `echo "print(1)" | colab repl`) to trigger non-interactive execution modes. On Windows, `colab console` switches the console out of line-editing mode so arrow keys and Ctrl-C reach the remote shell; colors need virtual-terminal processing, which the CLI turns on for cmd.exe and Windows PowerShell.
* **Drive mount and multiple Google accounts:** `colab drivemount` prints a consent URL (and opens it in the browser when it can) and waits up to 10 minutes. On the account chooser, select the same account the CLI authenticated as (`colab whoami` / the `login_hint` in the URL). A browser that is signed into several accounts otherwise attributes the Drive session to the active account, and Colab rejects the mount with HTTP 400. `--authuser N` appends that index to the consent URL when the chooser keeps opening the wrong account.
* **Transparent Code Execution:** When calling `colab exec -f file.py`, the CLI reads the file locally and transmits its content to the remote kernel. You do not need to manually upload files before execution.
* **Storage & State Paths:** Session tokens and metadata are stored at `~/.config/colab-cli/sessions.json`. Global CLI settings are located at `~/.config/colab-cli/settings.json`. These can be customized or isolated via the global `--config` flag.

Expand Down
6 changes: 5 additions & 1 deletion docs/02_execution_and_interactive.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
---
log:
2026-10-07: Windows console support for `colab console`. cmd.exe and PowerShell have no `termios`, `/dev/tty`, or `SIGWINCH`, so interactive console used to stay line-buffered (arrow keys never left the local line editor, Ctrl-C killed the CLI, ANSI from the remote PTY rendered as garbage in conhost). `console.py` now enables virtual-terminal processing and UTF-8, switches the input handle to raw mode via `SetConsoleMode`, reads keys with `msvcrt.getwch`, translates scan codes to ANSI, and polls the window size. POSIX raw mode and piped stdin are unchanged. `colab ssh --proxy-mode` reads stdin with a blocking `os.read` (Windows `select()` cannot wait on a pipe) and quotes the ProxyCommand for `cmd.exe`.

2026-05-07: Fixed `colab console` piped-stdin handling. Previously a piped invocation (e.g. `echo 'cmd' | colab console -s s`) sent the command and then hung indefinitely because the previous EOF handler emitted a bare `\x04` (Ctrl-D), which the remote `tmux`-wrapped bash treats as a literal character rather than a session terminator. The new handler sends `exit\n` (which bash actually exits on) and then closes the websocket from the client side after a short grace period (`PIPED_EOF_GRACE_SECONDS = 0.5s`) so any tail output (bash `logout`, tmux `[exited]`) makes it back to the user. TTY mode is unchanged: real-terminal EOF is left to the remote shell. Verified live: `echo 'echo HELLO' | colab console -s s` now exits in ~1.2s instead of hanging.

2026-05-07: Fixed `print_kitty` (used by `colab exec --output-image` and any image-producing exec) to no-op when `sys.stdout.isatty()` is false. The Kitty Graphics Protocol escape sequence is meaningless when stdout is a file or pipe and was visually corrupting captured output (a multi-KB base64 PNG blob would land in log files, grep targets, or showboat captures). Image bytes are still saved to disk via `handle_image`'s file-write path; only the inline-render attempt is suppressed.
Expand Down Expand Up @@ -33,7 +35,7 @@ Execution involves sending Python code (or shell commands) to the Jupyter kernel
### 3. Console (`colab console`)
- **Implementation**: Connects directly to the backend terminal endpoint (`/colab/tty`) via WebSockets using `websocket-client`.
- **Interactive**: Bypasses the Jupyter kernel entirely to provide a raw, PTY-backed bash session on the Colab VM.
- **Terminal Management**: Configures `sys.stdin` to raw mode using `termios` and `tty`, passing single characters to the socket and writing raw ANSI escape sequences directly to `sys.stdout.buffer`. Hooks into `SIGWINCH` to communicate local terminal dimensions (`cols`/`rows`) to the remote bash environment so output rendering works perfectly during resizing.
- **Terminal Management**: On Linux and macOS, configures `sys.stdin` to raw mode using `termios` and `tty`, passing single characters to the socket and writing raw ANSI escape sequences directly to `sys.stdout.buffer`. Hooks into `SIGWINCH` to communicate local terminal dimensions (`cols`/`rows`) to the remote bash environment so output rendering works perfectly during resizing. On Windows (cmd.exe and PowerShell), `termios` does not exist: the CLI enables `ENABLE_VIRTUAL_TERMINAL_PROCESSING` and UTF-8 on the console, sets raw input mode with `SetConsoleMode` (so the local line editor and Ctrl-C handler get out of the way), reads keys with `msvcrt.getwch`, maps scan codes to ANSI sequences, and polls `os.get_terminal_size` because there is no `SIGWINCH`.
- **Piped stdin**: Detected via `sys.stdin.isatty()`. When piped, the input characters are forwarded one at a time to the remote pty, and on EOF the client sends `exit\n` and then closes the websocket itself after `PIPED_EOF_GRACE_SECONDS` (0.5s) so the user's shell goodbye text drains back. The remote `/colab/tty` endpoint wraps bash in tmux, which intercepts a bare `\x04` as a literal character — that is why we send `exit\n` rather than Ctrl-D.

## Implementation Details
Expand All @@ -54,4 +56,6 @@ TDD is mandatory for all execution features.
- **Test Case**: Verify large piped inputs are handled without buffer overflow or truncation.
- **Test Case**: `colab console` with piped stdin sends `exit\n` and calls `ws.close()` on EOF (regression: previously sent `\x04` only and hung).
- **Test Case**: `colab console` in TTY mode does not synthesize an exit on EOF (the user owns the session lifecycle).
- **Test Case**: With `termios` unavailable and `os.name == "nt"`, `connect_console` enters the Windows raw-console context manager and does not call `tty.setraw`.
- **Test Case**: Windows key translation maps scan codes to ANSI and the stdin thread forwards those characters one at a time.
- **Test Case**: `print_kitty` is a no-op when `sys.stdout.isatty()` is false (regression: previously emitted ANSI/base64 into pipes and files).
35 changes: 31 additions & 4 deletions docs/04_automation_and_utility.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
---
log:
2026-10-07: Drive mount fixes for upstream issues #113, #103, and #108. `drive.mount` is now called with `timeout_ms=600000` so the kernel's `blocking_request` does not give up at the 120s default while the user is still in the browser (that produced `ValueError: mount failed` with an empty reason). The propagation handshake runs on a daemon thread so it does not block the websocket recv loop. The consent URL is opened in the system browser when one is available (`os.startfile` on Windows, `open` on macOS, `xdg-open` elsewhere) and the prompt names the `login_hint` account. HTTP 400 is explained as a multi-account browser mismatch instead of dumping Google's HTML error page, and an error `colab_reply` unblocks the kernel immediately. Optional `--authuser N` rewrites the consent URL only when the user asks; the propagation request's `authuser` parameter was shown in #103 not to fix the 400 by itself, and the URL is not forced to `authuser=0`. The Enter prompt reads `/dev/tty` on POSIX and `CONIN$` on Windows.

2026-09-25: Cleaned up references to keep-alive background daemon and pre-flight pings. VM liveness is automatically maintained by the Colab backend based on active kernel activity.
2026-08-10: Added `colab usage` for account-level compute-unit rate/balance via `GET /tun/m/ccu-info` on the session backend (same bearer token as `colab new`).
2026-06-11: Replaced the `oauth2` provider's `run_local_server()` (localhost redirect) with a remote copy-paste flow (`_run_remote_flow` in `auth.py`). The CLI now prints an authorization URL built with `redirect_uri=https://sdk.cloud.google.com/applicationdefaultauthcode.html` and `token_usage=remote`, then reads the pasted authorization code via `input()` and exchanges it with `flow.fetch_token(code=...)`. This is the same flow `gcloud auth application-default login` uses and works identically in local and remote/headless/container environments, removing the heuristic of whether to auto-open a browser. Confirmed server-side acceptance with a live GET-only check against the bundled cloud-SDK client (`764086051850-...`); the OOB redirect and a non-bundled client id were both verified to be rejected (`OOB flow has been blocked` / `redirect_uri_mismatch`). Unit tests in `tests/test_auth.py` assert no localhost server is started, the redirect URI + `token_usage=remote` are set, and the pasted code is exchanged.
Expand Down Expand Up @@ -116,7 +118,7 @@ How each provider supplies the scope:
- **Action**: Execute `drive.mount()` and transparently proxy Colab's
proprietary credential propagation flow.
- **Code**: `python from google.colab import drive
drive.mount('/content/drive')`
drive.mount('/content/drive', timeout_ms=600000)`
- **Handling**: Because `drivefs` enforces the ephemeral side-channel
propagation (`colab_request` over websocket), the CLI intercepts these
messages using `ColabRuntime.colab_request_hook`. When intercepted, the CLI
Expand All @@ -128,7 +130,27 @@ How each provider supplies the scope:
user is OAuthing in their browser. To avoid the upstream 10s
`jupyter_kernel_client` default raising `TimeoutError` mid-flow, this
subcommand passes `timeout=INTERACTIVE_AUTOMATION_TIMEOUT_SEC` (600s) to
`ColabRuntime.execute_code`. Same applies to `colab auth`.
`ColabRuntime.execute_code`. Same applies to `colab auth`. Separately,
`google.colab.drive.mount` defaults `timeout_ms` to 120000 and that budget
is the `blocking_request` wait, which includes the browser step. The
injected call passes `timeout_ms=600000` so the kernel does not abandon
the mount at two minutes and then fail with `ValueError: mount failed`.
- **Where the prompt runs**: The `colab_request` hook returns immediately
and finishes propagation on a daemon thread. The websocket recv loop stays
free to deliver the kernel's later output.
- **Consent UX**: The CLI prints the `unauthorized_redirect_uri`, highlights
the `login_hint` account, and tries to open that URL in the GUI browser.
The user presses Enter on the controlling terminal (`/dev/tty` or, on
Windows, `CONIN$`).
- **Multi-account 400**: Propagation HTTP 400 means the browser's active
Google account was not the CLI account. The CLI says so and replies to
the kernel with an error `colab_reply` instead of dumping the HTML body
and leaving `drive.mount` to time out. `--authuser N` is an opt-in that
appends `authuser` to the consent URL; it is not applied by default.
- **What still depends on Google**: The CLI cannot make the browser
attribute the gsession to `login_hint` when a different account is active.
The user has to pick the matching account on the chooser. There is no
fully non-interactive Drive mount; the ephemeral consent page is required.

### 4. Logging and Notebook Capture (`colab log`)

Expand Down Expand Up @@ -291,8 +313,13 @@ TDD is mandatory for all automation features.
- **Test Case**: Verify `colab install` correctly injects `pip install` or `uv
install` commands to the remote VM kernel.
- **Test Case**: Verify `colab drivemount` correctly injects `drive.mount()`
commands and registers the `colab_request_hook` to intercept credential
propagation events.
commands with `timeout_ms` of at least 10 minutes and registers the
`colab_request_hook` to intercept credential propagation events.
- **Test Case**: A propagation HTTP 400 produces an account-mismatch message
(no raw HTML dump) and a `colab_reply` that carries `error`, so the kernel
does not sit until `timeout_ms`.
- **Test Case**: `--authuser` rewrites the consent URL; the default path
leaves the server-built URL unchanged.

### 2. History Capture

Expand Down
2 changes: 2 additions & 0 deletions docs/06_ssh_access.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
---
log:
2026-10-07: Windows ProxyCommand. `select()` on the stdin fd raises on Windows (only sockets are selectable), which killed the `--proxy-mode` input pump. The pump now blocks in `os.read`. The ProxyCommand string is quoted with `subprocess.list2cmdline` on Windows because OpenSSH there re-parses it with cmd.exe, and `UserKnownHostsFile` uses `os.devnull` (`nul`) instead of `/dev/null`. `--rm` signal installation skips `SIGHUP` when the interpreter does not define it.

2026-09-25: Cleaned up keep-alive daemon references in session creation and teardown documentation.
2026-08-09: Added `--high-mem` passthrough when `colab ssh` auto-creates a runtime (forwards to `colab new --high-mem`).
2026-07-17: Initial design and implementation of `colab ssh` — client side of SSH-over-WebSocket runtime access. Adds three modes (interactive shell, `-s SESSION`, and `--proxy-mode` OpenSSH ProxyCommand bridge), `--identity/-i` key selection, and per-HTTP-status handshake error messages. Server side is out of scope for this repo; the subcommand is a no-op against runtimes that do not expose the `/colab/ssh` endpoint (surfaces an actionable HTTP 404 message).
Expand Down
2 changes: 1 addition & 1 deletion skills/colab-operator/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ by running `uv tool install google-colab-cli` or `pip install google-colab-cli`.

### Automate
- `colab auth -s <name>` — VM-side GCP creds, needed before in-VM GCS/BigQuery calls (interactive; not agent-runnable).
- `colab drivemount -s <name> [PATH]` — mounts Drive at `/content/drive` by default (interactive; not agent-runnable).
- `colab drivemount -s <name> [--authuser N] [PATH]` — mounts Drive at `/content/drive` by default (interactive; not agent-runnable). The user must approve the printed consent URL in a browser as the same account `colab whoami` shows; a multi-account browser rejects the mount with HTTP 400 unless that account is the one selected on the chooser. The kernel waits 10 minutes, not 2.
- `colab install -s <name> pkg1 pkg2` — installs via `uv pip install --system`, falling back to `pip`. Also `colab install -s <name> -r requirements.txt`.

### Inspect & report
Expand Down
17 changes: 16 additions & 1 deletion src/colab_cli/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,16 @@
from colab_cli import auto_update
from colab_cli.auth import AuthProvider
from colab_cli.common import state, setup_logging
from colab_cli.commands import session, execution, files, automation, run, ssh, utility, usage
from colab_cli.commands import (
session,
execution,
files,
automation,
run,
ssh,
utility,
usage,
)


class AlphabeticalGroup(TyperGroup):
Expand Down Expand Up @@ -80,6 +89,12 @@ def callback(
"""
Colab CLI global configuration.
"""
# ANSI colors and cursor control in cmd.exe / Windows PowerShell 5.1.
# No-op on Linux, macOS, and consoles that already have virtual-terminal
# processing enabled.
from colab_cli.terminal import enable_windows_virtual_terminal

enable_windows_virtual_terminal()
state.client_oauth_config = client_oauth_config
state.config_path = config
state.logtostderr = logtostderr
Expand Down
Loading