---

🏗️ Architecture

flowchart TD 
 subgraph "Browser (Next.js 14)" 
 UI["Canvas UI\n(React + Tailwind)"] 
 VC["VoiceContext\n(WebSocket + WebAudio)"] 
 GC["GraphContext\n(SSE streaming)"] 
 AC["AssistantContext\n(CRUD)"] 
 end 
 
 subgraph "Next.js API Layer" 
 PROXY["Catch-all proxy\n/api/[..._path]"] 
 SSE_ROUTE["POST /api/run_sse"] 
 end 
 
 subgraph "Ice Backend (FastAPI)" 
 WS["WebSocket /ws/{user}/{session}\nrun_live() voice_runner"] 
 SSE["POST /run_sse\nrun_async() text_runner"] 
 REST["Session / Assistant / Store REST"] 
 OAUTH["GitHub OAuth\n/connect/github"] 
 HEALTH["GET /health"] 
 end 
 
 subgraph "Agent Tree (ADK)" 
 ROOT["ice_agent (root)\nGemini 2.5 Flash Native Audio\nor Gemini 2.5 Flash"] 
 RESEARCHER["researcher\nGemini 2.5 Flash\nVoice: Kore"] 
 COMPUTE["compute\nGemini 2.5 Flash\nVoice: Charon"] 
 end 
 
 subgraph "Tools" 
 ART["Artifact Tools\ngenerate / rewrite / highlight"] 
 SEARCH["Google Search\nAgent Tool"] 
 TU["ToolUniverse MCP\n2000+ remote tools"] 
 COGNEE["Cognee\nKnowledge Graph"] 
 GITHUB["GitHub MCP\n70+ tools"] 
 VISION["monitor_camera_stream\nVision Streaming"] 
 SKILLS["151 Scientific Skills\n(SkillToolset)"] 
 end 
 
 subgraph "Storage" 
 SESSION_DB["Session Service\nInMemory or SQLite/Postgres"] 
 ARTIFACT_SVC["Artifact Service\nInMemory or GCS"] 
 FIRESTORE["Firestore\nassistants / store / shares"] 
 end 
 
 UI --> VC 
 UI --> GC 
 UI --> AC 
 VC -->|"binary PCM + JSON"| WS 
 GC --> SSE_ROUTE --> PROXY --> SSE 
 AC --> PROXY --> REST 
 PROXY --> REST 
 
 WS --> ROOT 
 SSE --> ROOT 
 ROOT --> RESEARCHER 
 ROOT --> COMPUTE 
 ROOT --> ART 
 ROOT --> SEARCH 
 ROOT --> TU 
 ROOT --> COGNEE 
 ROOT --> GITHUB 
 ROOT --> VISION 
 COMPUTE --> SKILLS 
 
 ROOT --> SESSION_DB 
 ROOT --> ARTIFACT_SVC 
 REST --> FIRESTORE 
 OAUTH --> FIRESTORE 

Loading

---

🚀 Local Development Setup

Prerequisites

Tool	Version
Python	3.13+
Node.js	18+
npm / yarn / pnpm	Latest
uvicorn	via pip

---

Backend

1. Clone and enter the backend directory

git clone <repo-url> 
cd ice/backend 

2. Create a virtual environment and install dependencies

python -m venv .venv 
source .venv/bin/activate # Windows: .venv\Scripts\activate 
pip install -r requirements.txt 

Optional extras for full capability:

pip install tooluniverse mcp # ToolUniverse MCP 
pip install cognee cognee-integration-google-adk # Knowledge graph 

3. Configure environment variables

Create a .env file in backend/ (or export them in your shell):

# Required [header-1](#header-1)
GOOGLE_API_KEY=your_google_api_key_here 
 
# Optional — Google Cloud [header-2](#header-2)
GOOGLE_CLOUD_PROJECT=your-gcp-project-id 
GOOGLE_CLOUD_LOCATION=us-central1 
 
# Optional — Model overrides (all default to gemini-2.5-flash) [header-3](#header-3)
ICE_LIVE_MODEL=gemini-2.5-flash-native-audio-latest 
ICE_TEXT_MODEL=gemini-2.5-flash 
ICE_META_MODEL=gemini-2.5-flash 
ICE_REFLECTION_MODEL=gemini-2.5-flash 
ICE_VISION_MODEL=gemini-2.5-flash 
 
# Optional — Session persistence (default: in-memory) [header-4](#header-4)
ICE_SESSION_DB_URI=sqlite+aiosqlite:///sessions.db 
 
# Optional — Artifact persistence (default: in-memory) [header-5](#header-5)
ICE_ARTIFACT_BUCKET=your-gcs-bucket-name 
 
# Optional — Firestore persistence (default: false) [header-6](#header-6)
FIRESTORE_ENABLED=false 
GCP_PROJECT=your-gcp-project-id 
FIRESTORE_EMULATOR_HOST=127.0.0.1:8080 # For local Firestore emulator 
 
# Optional — GitHub OAuth [header-7](#header-7)
GITHUB_CLIENT_ID=your_github_client_id 
GITHUB_CLIENT_SECRET=your_github_client_secret 
GITHUB_REDIRECT_URI=http://localhost:8000/connect/github/callback 
ICE_FRONTEND_URL=http://localhost:3000 
 
# Optional — Production code execution sandbox [header-8](#header-8)
ICE_SANDBOX_RESOURCE_NAME=projects/.../sandboxEnvironments/... 
 
# Optional — CORS (default: "*") [header-9](#header-9)
ICE_CORS_ORIGINS=http://localhost:3000 

4. Start the dev server

make dev 
# or directly: [header-10](#header-10)
uvicorn main:app --host 0.0.0.0 --port 8000 --reload 

The backend will be available at http://localhost:8000.

Verify with:

make health 
# or: [header-11](#header-11)
curl http://localhost:8000/health 

5. Run tests

make test # All tests 
make test-quick # Skip slow integration tests 

---

Frontend

1. Enter the frontend directory

cd ice/frontend 

2. Install dependencies

npm install 
# or: yarn install / pnpm install [header-12](#header-12)

3. Configure environment variables

Create a .env.local file in frontend/:

# Required — Ice backend URL [header-13](#header-13)
NEXT_PUBLIC_ICE_BACKEND_URL=http://localhost:8000 
 
# Optional — API proxy base (default: /api) [header-14](#header-14)
NEXT_PUBLIC_API_URL=/api 

4. Start the dev server

npm run dev 
# or: yarn dev [header-15](#header-15)

The frontend will be available at http://localhost:3000.

---

🔌 API Endpoints

Method	Path	Description
GET	/health	Health check / readiness probe
WS	/ws/{user_id}/{session_id}?modality=audio|both	Bidi voice streaming (run_live)
POST	/run_sse	Text-mode SSE streaming (run_async)
POST	/apps/{app}/users/{uid}/sessions	Create session (with optional assistant_id)
GET	/apps/{app}/users/{uid}/sessions	List sessions with state snapshot
GET	/apps/{app}/users/{uid}/sessions/{sid}	Get session with events
PATCH	/apps/{app}/users/{uid}/sessions/{sid}	Update session state (EventActions)
DELETE	/apps/{app}/users/{uid}/sessions/{sid}	Delete session
GET	/apps/{app}/users/{uid}/sessions/{sid}/messages	Reconstruct messages from events
POST	/apps/{app}/users/{uid}/sessions/{sid}/share	Generate share link (short slug)
GET	/s/{slug}	Get shared session info
POST	/apps/{app}/users/{uid}/sessions/{sid}/fork	Fork session to a new user
POST/GET/DELETE	/apps/{app}/users/{uid}/sessions/{sid}/artifacts[/{name}]	Artifact CRUD
POST	/assistants	Create assistant
POST	/assistants/search	Search assistants
PATCH	/assistants/{id}	Update assistant
DELETE	/assistants/{id}	Delete assistant
POST/DELETE	/store/get, /store/put, /store/delete	KV store (reflections, context docs)
GET	/connect/github	Start GitHub OAuth flow (PKCE)
GET	/connect/github/callback	GitHub OAuth callback
GET	/integrations/github/status	Check GitHub connection status
DELETE	/integrations/github/disconnect	Disconnect GitHub, shut down MCP

---

👏 Acknowledgments

ice is built on these open-source projects:

• Agent Development Kit (ADK) — multi-agent orchestration, live streaming, session management (Apache 2.0)
• ToolUniverse — 2,064 API tools via MCP server (Apache 2.0)
• K-Dense Scientific Skills — 151 skills for rdkit, biopython, scipy (MIT)
• RDKit — cheminformatics and molecule rendering (BSD)
• 3Dmol.js — WebGL molecular visualization (BSD)
• BlockNote — rich text editor (MPL 2.0)
• assistant-ui — chat interface components
• Open Canvas — artifact editor and reflection patterns (MIT)
• TalkingHead — 3D avatar with lip-sync and morph targets (MIT)
• Cognee — knowledge graph with Gemini embeddings (MIT)
• GitHub MCP Server — GitHub tools via Model Context Protocol (MIT)

---

📄 License

This project is licensed under the MIT - see the LICENSE file for details.