Skip to content

feat: first-class Foundation Models with native tools and dynamic profiles#112

Open
christopherkarani wants to merge 1 commit into
mainfrom
codex/check-frameworks-issues-20260613-timeout
Open

feat: first-class Foundation Models with native tools and dynamic profiles#112
christopherkarani wants to merge 1 commit into
mainfrom
codex/check-frameworks-issues-20260613-timeout

Conversation

@christopherkarani

Copy link
Copy Markdown
Owner

Summary

Makes Apple Foundation Models a first-class Swarm path for macOS/iOS and unblocks engineers who hit Conduit coupling, prompt-emulated tools, and Tool name clashes when mixing Swarm with Apple Intelligence.

Foundation Models (main change)

  • Add FoundationModelsInferenceProvider that talks to LanguageModelSession without Conduit
  • Bridge Swarm ToolSchema β†’ Apple's FoundationModels.Tool using guided generation (DynamicGenerationSchema / GenerationSchema)
  • Capture structured tool calls and return them to Swarm's agent loop (guardrails/observers still own execution)
  • Default provider factory prefers the native FM path when the system model is available
  • Deprecate Conduit-backed .foundationModels() / conduitFoundationModels() for migration only
  • Document Swarm.Tool vs FoundationModels.Tool clash guidance

Dynamic Profiles (WWDC 2026–aligned)

  • Add Swarm Profile, DynamicProfile, DynamicInstructions, ProfileMode, and ModeSwitchingDynamicProfile
  • Re-resolve instructions, tool filters, generation overrides, and history policy every turn
  • Wire via .foundationModels(profile:)
  • Note: Apple's native LanguageModelSession.DynamicProfile is not in the macOS 26.2 SDK yet; this is the portable Swarm layer ready to bridge when the SDK ships

Also on this branch

  • Harden workflow timeout race
  • Remove stale MembraneHive import
  • Version bump to 0.6.0

Test plan

  • swift build
  • swift test --filter DynamicProfile
  • swift test --filter FoundationModelsNativeToolBridgeTests|FoundationModelsInferenceProviderTests|FoundationModelsToolCallingTests|LanguageModelSessionTool|ConduitProviderSelection|AgentDefault
  • Live FM tests with SWARM_RUN_LIVE_FOUNDATION_MODELS_TESTS=1 (native tool capture + profile mode switch)
  • CI green on macOS / Linux
  • Manual smoke: Agent("…", inferenceProvider: .foundationModels()) { MyTool() }

Example

// First-class on-device (no Conduit)
let agent = try Agent("Be helpful.", inferenceProvider: .foundationModels()) {
    LookupTool()
}

// Dynamic profiles / baton-pass phases
enum Phase { case brainstorm, review }
let mode = ProfileMode(Phase.brainstorm)
let profile = ModeSwitchingDynamicProfile(mode: mode) { phase in
    switch phase {
    case .brainstorm:
        Profile(id: "brainstorm", instructions: "Ideate freely.",
                generation: .init(temperature: 1.0))
    case .review:
        Profile(id: "review", instructions: "Be precise.",
                history: .dropToolTranscriptAndKeepLast(count: 12),
                generation: .init(temperature: 0.2))
    }
}
let profiled = try Agent("…", inferenceProvider: .foundationModels(profile: profile))
mode.current = .review

…namic profiles

Make Apple Foundation Models a first-class Swarm path that no longer depends
on Conduit for on-device inference. Bridge Swarm tool schemas to
FoundationModels.Tool with guided generation, capture tool calls for the
agent loop, and add WWDC 2026-aligned Dynamic Profiles for phase-based
instructions, tool filters, generation overrides, and history policy.

Deprecate the Conduit FM selection path, document Tool naming clashes, and
cover the new provider/bridge/profile surface with unit and live tests.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

πŸ’‘ Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fff133e593

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with πŸ‘.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +380 to +383
if let first = messages.first, first.role == .system {
// Replace / strengthen leading system message for this phase.
var copy = messages
copy[0] = .system(instructions)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve the existing system prompt

When a dynamic profile is used through Agent, the first message is already the agent's system prompt, but this replacement discards it and keeps only the profile/configuration instructions. In calls like Agent("You are a billing bot", inferenceProvider: .foundationModels(profile: ...)), the Foundation Models turn no longer receives the agent instructions unless the caller duplicates them in the provider configuration; merge the new instructions with the existing system content instead of replacing it.

Useful? React with πŸ‘Β / πŸ‘Ž.

Comment on lines +219 to +224
let effectiveTools: [ToolSchema]
if resolved.options.toolChoice == ToolChoice.none {
effectiveTools = []
} else {
effectiveTools = resolved.tools
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Honor specific tool choice before bridging

This only removes tools for .none; for ToolChoice.specific(toolName:) it still bridges every resolved tool to FoundationModels. Because the later prompt text is only a hint, requests that are supposed to force search can still let the native model call calc or any other provided tool; filter effectiveTools down to the named tool for this case.

Useful? React with πŸ‘Β / πŸ‘Ž.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant