Skip to content

Add local model connector options for local server#90

Open
seagpt wants to merge 1 commit into
supermemoryai:mainfrom
seagpt:feat/local-model-connectors
Open

Add local model connector options for local server#90
seagpt wants to merge 1 commit into
supermemoryai:mainfrom
seagpt:feat/local-model-connectors

Conversation

@seagpt

@seagpt seagpt commented Jun 15, 2026

Copy link
Copy Markdown

Summary

Adds first-class local model connector options for the local Supermemory launcher and Supermemory.local() startup path.

This keeps the local server's existing OpenAI-compatible model-provider interface, but makes common local runtimes explicit:

  • CLI shortcuts for Ollama, LM Studio, llama.cpp, and generic OpenAI-compatible endpoints
  • Supermemory.local({ localModel: ... }) TypeScript options for programmatic startup
  • loopback-friendly dummy OPENAI_API_KEY fallback for local endpoints that ignore bearer tokens
  • README examples and a client test covering the new local-model option shape

Why

The local server already supports OpenAI-compatible model routing through env vars such as OPENAI_BASE_URL, OPENAI_MODEL, OPENAI_FAST_MODEL, and OPENAI_TEXT_MODEL. In practice, users running fully local memory stacks need a documented, typed way to point Supermemory at Ollama / LM Studio / llama.cpp / vLLM without falling back to hosted LLMs or manual env glue.

Validation

Ran locally:

  • node --check bin/cli
  • env -u SUPERMEMORY_API_KEY corepack yarn test --runInBand tests/index.test.ts → 52 passed
  • env -u SUPERMEMORY_API_KEY ./node_modules/typescript/bin/tsc
  • env -u SUPERMEMORY_API_KEY corepack yarn build → success
  • git diff --check

I also smoke-tested the patched CLI against a local llama.cpp/OpenAI-compatible endpoint via --llamacpp-model + --llamacpp-url; the local server started, accepted a document, used local embeddings, issued a /v1/responses request to the configured local endpoint, and search found the inserted test document.

Notes / known limitation

This PR only adds the connector/configuration surface. During local smoke testing, the self-hosted memory-agent path still failed after receiving a local Responses API function-call response, even though document add/list/search and local embeddings worked. That appears to be a deeper local-server/model-agent compatibility issue rather than SDK routing, and likely belongs in the server implementation rather than this SDK launcher patch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant