feat: add default runner configuration if none exists

[ilopezluna]

Fixes #407

In Docker Desktop the context size field is expected to be found when showing request/responses:

const baseRecord = {
            id: request.id,
            model: requestData.model,
            contextSize: response.config?.['context-size'] || 4096,
            timestamp: parseInt(request.timestamp, 10),
            statusCode: request.status_code,
            request: requestData,
          };

When no explicit configuration call has been made, there is no runner configuration, so the field response.config.context-size is not populated.
This PR fixes that.

[gemini-code-assist]

Summary of Changes

Hello @ilopezluna, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request resolves an issue where the context-size field was not consistently present in the runner configuration, which is crucial for Docker Desktop's request/response display. It implements a mechanism to automatically generate a default runner configuration, populating it with the model's inherent context size, whenever an explicit configuration is absent. This ensures that all necessary configuration details are consistently available for reporting and user interface purposes.

Highlights

• Default Runner Configuration: Introduced logic to create a default runner configuration if no explicit configuration is provided, ensuring that essential model parameters are always available.
• Context Size Population: The default configuration now automatically retrieves and populates the context-size from the model's bundle runtime configuration, addressing an issue where this field was missing for display in Docker Desktop.
• Code Refactoring: Performed a minor refactoring in the BundleForModel function within pkg/distribution/internal/store/bundles.go to simplify the error handling and return logic.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[sourcery-ai]

Hey - I've left some high level feedback:

• The nested conditionals when building the default runnerConfig in loader.load are quite deep; consider extracting this logic into a small helper (e.g., defaultBackendConfigFromBundle(modelManager, modelID)) to make the flow easier to follow and to keep load() focused on orchestration.
• Calling modelManager.GetBundle(modelID) inside load for the default config may introduce extra I/O or parsing; if a bundle is already available in this code path, consider reusing it or caching to avoid redundant bundle loading.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- The nested conditionals when building the default runnerConfig in loader.load are quite deep; consider extracting this logic into a small helper (e.g., `defaultBackendConfigFromBundle(modelManager, modelID)`) to make the flow easier to follow and to keep load() focused on orchestration.
- Calling `modelManager.GetBundle(modelID)` inside `load` for the default config may introduce extra I/O or parsing; if a bundle is already available in this code path, consider reusing it or caching to avoid redundant bundle loading.

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[gemini-code-assist]

Code Review

This pull request introduces a feature to create a default runner configuration if one doesn't exist, primarily to ensure the context-size is available. The change in pkg/distribution/internal/store/bundles.go is a good stylistic refactoring. My main feedback is on pkg/inference/scheduling/loader.go, where the new logic can be improved by adding error logging and reducing code nesting for better readability and maintainability.