Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,15 @@

## Next


### Fixed

- Fixed `BaseGeminiLLM`/`GeminiLLM` silently dropping token usage on every call: `invoke`/`ainvoke` (both the string and message-list paths, sync and async) built `LLMResponse` from `response.text` alone, ignoring `response.usage_metadata` entirely. `LLMResponse.usage` is now populated from it, matching `AnthropicLLM`/`OpenAILLM`/`VertexAILLM`.

### Changed

- (**breaking**) `BaseGeminiLLM`/`GeminiLLM.supports_structured_output` is now `True` — it already supported structured output via `response_schema`/`response_mime_type`, but never declared the capability flag, so pipeline components checking it fell back to prompt-based JSON. `SimpleKGPipeline`, `SchemaFromTextExtractor`, and `LLMEntityRelationExtractor` now enable structured output by default with `GeminiLLM`. To keep the previous behavior, construct the extractors with `use_structured_output=False`.

## 1.19.0

### Added
Expand Down
2 changes: 2 additions & 0 deletions src/neo4j_graphrag/llm/google_genai_llm.py
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,8 @@ class BaseGeminiLLM(LLMBase, abc.ABC):
constructing the ``client`` SDK instance.
"""

supports_structured_output: bool = True

client: "genai.Client"

def __init__(
Expand Down
10 changes: 10 additions & 0 deletions tests/unit/llm/test_google_genai_llm.py
Original file line number Diff line number Diff line change
Expand Up @@ -208,6 +208,16 @@ def test_gemini_llm_is_base_gemini_llm_subclass() -> None:
assert issubclass(GeminiLLM, BaseGeminiLLM)


def test_gemini_llm_supports_structured_output(
mock_genai: Tuple[MagicMock, MagicMock],
) -> None:
"""GeminiLLM already supports response_schema/response_mime_type; the
capability flag must say so, or SimpleKGPipeline/SchemaFromTextExtractor/
LLMEntityRelationExtractor silently fall back to prompt-based JSON."""
llm = GeminiLLM("gemini-2.0-flash")
assert llm.supports_structured_output is True

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the most LLM test ever... set a default... test the default is default :D



def test_gemini_llm_init_only_constructs_client(
mock_genai: Tuple[MagicMock, MagicMock],
) -> None:
Expand Down
Loading