Skip to content

feat: add Apple Intelligence as a local model provider (text only) - #2157

Open
Douglasymlai wants to merge 3 commits into
mainfrom
feat/apple-intelligence-models
Open

Douglasymlai wants to merge 3 commits into
mainfrom
feat/apple-intelligence-models

Conversation

@Douglasymlai

@Douglasymlai Douglasymlai commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Pull Request

Related Issue

N/A — maintainer-directed work; no linked issue required for this PR.

Description

First working version of the plan in docs/apple-intelligence-models-plan.md: Apple Intelligence as a local model provider on Apple silicon Macs, using the on-device model with no endpoint, key or download. Text only for now.

What it does

  • Swift bridge (native/apple-bridge/). A small helper that the backend starts on demand. It serves the on-device model through Apple's Foundation Models framework as an OpenAI-compatible API (/v1/health, /v1/models, /v1/chat/completions, streaming or not) on 127.0.0.1 only. Every request needs the token the backend passes on stdin; requests with an Origin header or another Host are rejected. It exits when the backend goes away or after 10 idle minutes. Requests with tools get 400 tools_unsupported. Foundation Models is weak-linked, so on older macOS the bridge still starts and reports unsupported_os.
  • Backend (backend/app/apple_bridge/). A supervisor that starts the bridge, reads its port, restarts it after a crash with backoff, and gives up after three crashes in five minutes until Try again. GET /model/apple-intelligence/status and POST /model/apple-intelligence/test serve Settings. The provider id apple-intelligence aliases to the OpenAI-compatible client, and the live bridge URL and token are filled in when a model is built, so no port is ever stored. If the bridge can't run, the Run fails with the reason. There is no fallback to another model.
  • Desktop. Electron passes the bridge path to the backend as EIGENT_APPLE_BRIDGE on Apple silicon Macs only, and has a fixed IPC call that opens the Apple Intelligence pane of System Settings.
  • Settings → Models. An Apple Intelligence entry in Add provider, offered only by the macOS desktop app. Opening it starts the bridge and shows the status: ready, or why not (Apple Intelligence off, with Open System Settings; model downloading; macOS too old; Mac not supported; Intel Mac; not in this build). Test runs a short generation and shows how long it took. Add model saves apple-on-device with an empty endpoint. Copy is in all 11 locales.
  • CI. A new Apple Bridge workflow builds and tests the Swift package on a macOS runner and checks that Foundation Models is weak-linked.

Platform gating

Platform Settings Backend
Windows, Linux, web client Not offered Status returns not_macos without starting anything
Intel Mac Offered, shows "Not available on this Mac" not_apple_silicon, nothing started
Apple silicon, build without the bridge Offered, shows "Not available in this build" bridge_missing
Apple silicon, macOS < 26 Offered, shows "Requires macOS 26 or later" bridge reports unsupported_os
Apple silicon, macOS 26+ Status from Apple's availability API bridge started on demand

A failed status call is shown as "helper isn't responding" and never breaks Settings or model selection. Startup is unchanged: nothing about Apple Intelligence runs until the provider is opened or an Apple model is used.

Changes to the plan

  • §4.4 named a provider card (ConfigModelCard) and a "local providers group". Neither exists in Settings → Models any more: providers are added from the Add provider dialog. The section now follows the Codex entry, which also has no fields.
  • §4.4 now says why Apple Intelligence records are classified as custom, not local, for now: four places in the renderer classify providers from LOCAL_MODEL_OPTIONS, and default-model selection needs them to agree.
  • §4.2: the first version writes earlier turns into the prompt instead of rebuilding a Transcript. A Transcript is only needed to replay tool calls, so it comes with tool calling.
  • New §8 lists what is in this version and what is not.

Not done yet

  • Tool calling and the native Transcript (plan PR 3). Until then only Sessions that need no tools work. A simple question gets an answer; a Task that needs tools fails with tools_unsupported.
  • The context check before a Run, setting the token limit to the context size, and disabled picker entries with a reason.
  • Reading the context size on macOS 27 (the bridge reports 4,096).
  • Private Cloud Compute, and the spike for the open questions in §5.
  • Bridge status in /health?detail=true.
  • Building, signing and notarizing the bridge in the release workflow. Today it is built with npm run build:apple-bridge.
  • The saved model shows as apple-on-device in model lists, without a friendlier name.

Testing

On Windows:

  • Backend: 47 new tests in backend/tests/app/apple_bridge/. They cover platform gating, a fake bridge process that follows the real start-up protocol (handshake, token, idle exit, crash limit, backoff, start timeout, stop), status for each unavailable reason, the test route, URL and token substitution through agent_model, no fallback, and an OpenAI SDK round trip. Ruff, ruff format and Bandit pass. The neighbouring test_agent_model.py, tests/app/model and test_local_control.py suites pass (445 tests).
  • Frontend: 43 new tests (AppleIntelligenceProvider.test.tsx, appleIntelligence.test.ts, appleBridge.test.ts) cover gating on each platform, each status, Add model, Test, Open System Settings, Try again and an unreadable status. ModelsNavigation, SettingsPage, configuredModels and bundledGitPackaging tests pass. npm run type-check, ESLint, check:design-tokens, the Electron access guard and i18n locale parity pass.
  • Swift: compiled and tested only by the new Apple Bridge workflow on a macOS runner. Nothing ran on a Mac with Apple Intelligence.

macOS checks for a tester

Build the bridge with npm run build:apple-bridge, then npm run dev.

  1. Apple silicon, macOS 26+, Apple Intelligence on: Settings → Models → Add model shows Apple Intelligence. Opening it shows ready, "Text only" and "Context 4K". Test passes and shows a time. Add model saves it.
  2. Make it the default and ask a simple question in a new Session. The answer streams. Stop it mid-answer: the generation stops.
  3. Ask for a Task that needs tools: the Run fails with a clear "can't use tools" message and does not switch to another model.
  4. Turn Apple Intelligence off: the status says so, Open System Settings opens the Apple Intelligence & Siri pane, and Try again updates the status after turning it back on.
  5. While the model is still downloading (fresh setup), the status says so.
  6. Intel Mac, and an Apple silicon Mac on macOS 15: the entry shows the matching "not available" reason and nothing else breaks.
  7. Quit Eigent: no eigent-apple-bridge process is left (pgrep eigent-apple-bridge). Leave it idle for 10 minutes: the bridge exits, and the next request starts it again.
  8. Kill the bridge during a generation: the Run fails with a reason, and the next request starts a new bridge.
  9. From a browser tab, fetch('http://127.0.0.1:<port>/v1/models') is rejected (port from lsof -iTCP -sTCP:LISTEN | grep eigent-ap).
  10. A packaged, signed build: the bridge starts under the hardened runtime. Expect this to need the release-workflow work listed above.

Testing Evidence (REQUIRED)

  • I have included human-verified testing evidence in this PR.
  • This PR includes frontend/UI changes, and I attached screenshot(s) or screen recording(s).
  • No frontend/UI changes in this PR.

Not yet available: the UI only appears in the macOS desktop app, and this was built on Windows. Screenshots and human testing need a Mac; see the checks above.

What is the purpose of this pull request?

  • Bug fix
  • New Feature
  • Documentation update
  • Other

Contribution Guidelines Acknowledgement

🤖 Generated with Claude Code

Douglasymlai and others added 2 commits November 18, 2026 23:47
First working version of docs/apple-intelligence-models-plan.md.

- native/apple-bridge: Swift helper that serves the on-device model over
  an OpenAI-compatible API on 127.0.0.1 with a per-start token, Origin and
  Host checks, error mapping, streaming and an idle stop. Tools are
  rejected with tools_unsupported for now.
- Backend: supervisor that starts the bridge on demand, a status and a
  test route, the apple-intelligence alias, and the live bridge URL and
  token filled in when a model is built, with no fallback.
- Desktop: passes the bridge path on Apple silicon Macs only, and opens
  the Apple Intelligence pane of System Settings.
- Settings: an Apple Intelligence entry in Add provider, offered only in
  the macOS desktop app, with status, Test and Add model.
- CI: builds and tests the bridge on a macOS runner.
- Plan: corrected the Settings section against current code and added
  the implementation status.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Douglasymlai Douglasymlai changed the title docs: plan Apple Intelligence as a local model provider feat: add Apple Intelligence as a local model provider (text only) Nov 19, 2026
@Douglasymlai
Douglasymlai marked this pull request as ready for review November 19, 2026 00:23
Comment on lines +50 to +55
return {
"ok": False,
"elapsed_ms": None,
"code": exc.reason,
"message": str(exc),
}

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants