fix: fix mypy

[parthea]

Towards #15104

This PR addresses the following mypy failure for google-cloud-bigquery-storage:

(py3142) partheniou@partheniou-vm-3:~/git/google-cloud-python/packages/google-cloud-bigquery-storage$ nox -s mypy-3.14
nox > Running session mypy-3.14
nox > Creating virtual environment (virtualenv) using python3.14 in .nox/mypy-3-14
nox > python -m pip install 'mypy<1.16.0' types-requests types-protobuf
nox > python -m pip install .
nox > mypy -p google
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1/gapic_types.py:25: error: Skipping analyzing "proto": module is installed, but missing library stubs or py.typed marker  [import-untyped]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1/gapic_types.py:25: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1beta/__init__.py:87: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1alpha/__init__.py:87: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1/reader.py:23: error: Cannot find implementation or library stub for module named "fastavro"  [import-not-found]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1/reader.py:27: error: Skipping analyzing "google.rpc.error_details_pb2": module is installed, but missing library stubs or py.typed marker  [import-untyped]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1/reader.py:27: error: Skipping analyzing "google.rpc": module is installed, but missing library stubs or py.typed marker  [import-untyped]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1/reader.py:30: error: Library stubs not installed for "pandas"  [import-untyped]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1/reader.py:30: note: Hint: "python3 -m pip install pandas-stubs"
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1/reader.py:30: note: (or run "mypy --install-types" to install all missing stub packages)
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1/reader.py:34: error: Cannot find implementation or library stub for module named "pyarrow"  [import-not-found]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1/__init__.py:98: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1beta2/__init__.py:98: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1/writer.py:28: error: Skipping analyzing "grpc": module is installed, but missing library stubs or py.typed marker  [import-untyped]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1/writer.py:58: error: Incompatible return value type (got "BaseException", expected "Exception")  [return-value]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1/writer.py:112: error: Need type annotation for "_close_callbacks" (hint: "_close_callbacks: list[<type>] = ...")  [var-annotated]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1/writer.py:115: error: Need type annotation for "_closed_connection" (hint: "_closed_connection: dict[<type>, <type>] = ...")  [var-annotated]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1/writer.py:160: error: Incompatible types in assignment (expression has type "str", variable has type "None")  [assignment]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1/writer.py:260: error: Need type annotation for "_queue"  [var-annotated]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1/writer.py:319: error: Incompatible types in assignment (expression has type "BidiRpc", variable has type "None")  [assignment]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1/writer.py:333: error: "None" has no attribute "add_done_callback"  [attr-defined]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1/writer.py:335: error: Incompatible types in assignment (expression has type "BackgroundConsumer", variable has type "None")  [assignment]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1/writer.py:336: error: "None" has no attribute "start"  [attr-defined]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1/writer.py:345: error: "None" has no attribute "is_active"  [attr-defined]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1/writer.py:364: error: "None" has no attribute "is_active"  [attr-defined]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1/writer.py:395: error: Incompatible types in assignment (expression has type "str", variable has type "None")  [assignment]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1/writer.py:432: error: "None" has no attribute "send"  [attr-defined]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1/writer.py:450: error: "None" has no attribute "stop"  [attr-defined]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1/writer.py:470: error: Incompatible types in assignment (expression has type "Exception", variable has type "StreamClosedError")  [assignment]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1beta2/writer.py:28: error: Skipping analyzing "grpc": module is installed, but missing library stubs or py.typed marker  [import-untyped]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1beta2/writer.py:57: error: Incompatible return value type (got "BaseException", expected "Exception")  [return-value]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1beta2/writer.py:88: error: Need type annotation for "_close_callbacks" (hint: "_close_callbacks: list[<type>] = ...")  [var-annotated]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1beta2/writer.py:89: error: Need type annotation for "_futures_queue"  [var-annotated]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1beta2/writer.py:154: error: Incompatible types in assignment (expression has type "str", variable has type "None")  [assignment]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1beta2/writer.py:159: error: Incompatible types in assignment (expression has type "BidiRpc", variable has type "None")  [assignment]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1beta2/writer.py:173: error: "None" has no attribute "add_done_callback"  [attr-defined]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1beta2/writer.py:175: error: Incompatible types in assignment (expression has type "BackgroundConsumer", variable has type "None")  [assignment]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1beta2/writer.py:176: error: "None" has no attribute "start"  [attr-defined]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1beta2/writer.py:185: error: "None" has no attribute "is_active"  [attr-defined]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1beta2/writer.py:204: error: "None" has no attribute "is_active"  [attr-defined]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1beta2/writer.py:254: error: "None" has no attribute "send"  [attr-defined]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1beta2/writer.py:305: error: "None" has no attribute "stop"  [attr-defined]
.nox/mypy-3-14/lib/python3.14/site-packages/google/cloud/bigquery_storage_v1beta2/writer.py:326: error: Incompatible types in assignment (expression has type "Exception", variable has type "StreamClosedError")  [assignment]
Found 36 errors in 4 files (checked 87 source files)
nox > Command mypy -p google failed with exit code 1
nox > Session mypy-3.14 failed.

[gemini-code-assist]

Summary of Changes

Hello @parthea, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request focuses on resolving a comprehensive set of mypy type-checking errors within the google-cloud-bigquery-storage library. The changes primarily involve enhancing type annotations, adding type: ignore directives for external libraries lacking stubs, and refining the handling of optional dependencies. The overall impact is a significant improvement in the library's type-checking compliance, leading to more robust and maintainable code.

Highlights

• Mypy Type Checking Fixes: Addressed numerous mypy errors related to missing library stubs and untyped imports by introducing type: ignore comments for proto, google.rpc.error_details_pb2, pyarrow, and grpc imports.
• Optional Dependency Handling: Refactored checks for optional dependencies (fastavro, pandas, pyarrow) by introducing dedicated boolean flags (_FASTAVRO_INSTALLED, _PANDAS_INSTALLED, _PYARROW_INSTALLED) and using these flags instead of is None checks, improving clarity and type-checking accuracy.
• Improved Type Annotations: Introduced explicit type annotations for various class members and function parameters in writer.py files (e.g., _close_callbacks, _closed_connection, _stream_name, _rpc, _consumer, _queue, _wrap_as_exception, _on_rpc_done) to resolve var-annotated and return-value mypy errors.
• Nullability Checks: Added explicit None checks before attempting to call methods on potentially None objects (self._rpc.send, self._consumer.stop) to prevent attr-defined mypy errors.
• Noxfile Updates: Modified noxfile.py to include pandas-stubs and ensure fastavro is installed via .[fastavro] during the mypy session, providing necessary type information for these libraries.

Changelog

• packages/google-cloud-bigquery-storage/google/cloud/bigquery_storage_v1/gapic_types.py
  • Added type: ignore to the import proto statement to suppress import-untyped mypy errors.
• packages/google-cloud-bigquery-storage/google/cloud/bigquery_storage_v1/reader.py
  • Introduced _FASTAVRO_INSTALLED, _PANDAS_INSTALLED, and _PYARROW_INSTALLED boolean flags to track optional dependency availability.
  • Replaced if <module> is None: checks with if not _<MODULE>_INSTALLED: for pandas, fastavro, and pyarrow.
  • Added type: ignore to import google.rpc.error_details_pb2 and import pyarrow to resolve import-untyped and import-not-found errors.
• packages/google-cloud-bigquery-storage/google/cloud/bigquery_storage_v1/writer.py
  • Added type: ignore to import grpc to suppress import-untyped errors.
  • Updated type hints for _wrap_as_exception to include BaseException.
  • Added explicit type annotations for _close_callbacks, _closed_connection, _stream_name, _rpc, _consumer, and _queue.
  • Changed if self._stream_name is None: to if not self._stream_name: for string emptiness check.
  • Updated type hints for _on_rpc_done to include BaseException.
  • Added if self._rpc is not None: and if self._consumer is not None: checks before calling methods on these objects.
  • Added type annotation for exc variable in _shutdown method.
• packages/google-cloud-bigquery-storage/google/cloud/bigquery_storage_v1beta2/writer.py
  • Added type: ignore to import grpc to suppress import-untyped errors.
  • Updated type hints for _wrap_as_exception to include BaseException.
  • Added explicit type annotations for _close_callbacks, _futures_queue, _rpc, _stream_name, and _consumer.
  • Added if self._rpc is not None: and if self._consumer is not None: checks before calling methods on these objects.
  • Added type annotation for exc variable in _shutdown method.
• packages/google-cloud-bigquery-storage/noxfile.py
  • Added pandas-stubs to the mypy session installation.
  • Changed session.install('.') to session.install('.[fastavro]') to ensure fastavro is installed for type checking.

Activity

• The pull request was initiated by parthea to address a series of mypy type-checking failures in the google-cloud-bigquery-storage library.
• The author has implemented changes across multiple Python files, primarily focusing on adding type annotations, type: ignore comments for problematic imports, and improving conditional import logic.
• The noxfile.py was updated to ensure the mypy testing environment correctly installs necessary type stubs and optional dependencies.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request successfully addresses a large number of mypy errors in google-cloud-bigquery-storage. The changes are well-targeted and correctly resolve the typing issues by adding type: ignore comments, improving optional dependency handling, adding missing type annotations, and updating the nox session configuration. These changes significantly improve the type safety and maintainability of the code. I have added a few suggestions to simplify some of the new type hints for better clarity.