Connect generic provider image input to Responses-to-Chat pipeline - #86
Open
octo-patch wants to merge 1 commit into
Open
Connect generic provider image input to Responses-to-Chat pipeline#86octo-patch wants to merge 1 commit into
octo-patch wants to merge 1 commit into
Conversation
Generic OpenAI-compatible providers (e.g. MiniMax) declare supportsImages per model in providers.json, but reqToChat only consulted the hardcoded MiMo-only modelSupportsImages() check, so image parts were stripped for any non-MiMo model id even when the upstream actually accepts image input. Forward the declared model's supportsImages metadata through ReqToChatOpts so generic providers keep image parts for vision-capable models.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reason: Connect MiniMax image input to the existing Responses-to-Chat image pipeline; generic supportsImages metadata was UI-only, so MiniMax images were stripped.
Problem
Generic OpenAI-compatible providers (e.g. MiniMax) declare
supportsImagesper model inproviders.json, butreqToChatonly consulted the hardcoded MiMo-onlymodelSupportsImages()check when deciding whether to keepinput_imageparts. As a result, image parts were stripped for any non-MiMo model id — even when the upstream actually accepts image input — making the declaredsupportsImagesmetadata UI-only.Change
src/translate/reqToChat.ts: addsupportsImages?: booleantoReqToChatOpts. When provided, it overrides the hardcodedmodelSupportsImages(effectiveModel)check; otherwise behavior is unchanged (MiMo keeps its existing routing).src/providers/generic.ts: inpreprocessResponses, look up the resolved upstream model'ssupportsImagesmetadata from the declared model catalog and forward it toreqToChat. Open-catalog providers (no declared models) and unknown ids keep falling back to the hardcoded check.Checks
npx tsc -p .— typecheck passes.npx vitest run— full suite (50 files, 681 tests) passes, including newcreateGenericProvider — supportsImages metadata (input-capability)tests covering image forwarding whensupportsImages: true, stripping when omitted, and fallback for open-catalog providers.