Make babel_validation a top-level import; add ruff and offline unit tests to CI - #106
Open
gaurav wants to merge 10 commits into
Open
Make babel_validation a top-level import; add ruff and offline unit tests to CI#106gaurav wants to merge 10 commits into
gaurav wants to merge 10 commits into
Conversation
Change pyproject.toml to package src/babel_validation directly so babel_validation is importable as a top-level name after install. Update all test imports from src.babel_validation.* to babel_validation.*. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 tasks
Contributor
There was a problem hiding this comment.
Pull request overview
Makes babel_validation a conventional top-level installed package.
Changes:
- Updates Hatchling packaging configuration.
- Migrates four test imports away from the
src.prefix. - Updates contributor documentation.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
pyproject.toml |
Packages src/babel_validation directly. |
tests/test_environment/test_env.py |
Uses the top-level package import. |
tests/nodenorm/test_nodenorm_from_gsheet.py |
Updates the Google Sheet import. |
tests/nameres/test_nameres_from_gsheet.py |
Updates the Google Sheet import. |
tests/nameres/test_blocklist.py |
Updates the blocklist import. |
CLAUDE.md |
Documents the new import convention. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…stall Moving the wheel target from `packages = ["src"]` to `packages = ["src/babel_validation"]` meant `babel_validation` was only importable via the installed distribution. Anyone with a venv built before that change -- or following the bare `pytest --target dev` in README.md and CLAUDE.md -- hit `ModuleNotFoundError: No module named 'babel_validation'` at collection until they re-ran `uv sync`. `pythonpath = ["src"]` (built into pytest >= 7) puts the source tree on sys.path during collection, so the test suite no longer depends on install state. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Nothing imports `src.*` any more, but the file still marked `src` as a package, so editors and linters kept offering `src.babel_validation` as a completion -- and such an import resolves to a second, distinct module object with its own caches rather than failing loudly. Deleting it does not fully close that path (PEP 420 namespace packages keep `src.` importable while the repo root is on sys.path, which the root conftest.py guarantees under pytest's default prepend import mode), but it removes the marker that invites the mistake. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Settings are copied from NCATSTranslator/Babel so the two repositories lint identically: line-length 120, `E`/`F`/`I`/`UP` selected, `E501` ignored because `ruff format` already owns wrapping. ruff format supersedes black rather than joining it -- two formatters with different opinions produce a diff every time whichever one ran last disagrees with the other, so black is dropped from the dependencies and from CLAUDE.md. Also registers the `unit` marker, again matching Babel's wording. Every test in this repository currently talks to a live service or downloads the Google Sheet, so `pytest -m unit` is empty today; the following commits give it something to run and wire it up to CI. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Nineteen were mechanical and applied with `ruff check --fix`: import blocks sorted into stdlib/third-party/first-party groups (I001), the eight `Optional[str]` annotations in BlocklistEntry rewritten as `str | None` now that the floor is Python 3.11 (UP045), and one f-string with no placeholders (F541). The remaining three were dead locals (F841). `input_results`, `source_info` (with the `source`/`source_url` reads that fed only it), and `first_biolink_type` were each assigned and never read, so removing them changes no behaviour -- but `source_info` and `first_biolink_type` sit next to assertions that could reasonably have used them, so they may be worth reinstating deliberately rather than by accident. log-analysis/ is excluded: those are exploratory notebooks, and one cell does not parse, so linting them would gate CI on scratch work. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Mechanical only: `uv run ruff format .` over the repository, no behavioural changes. Isolated in its own commit so the substantive commits on either side stay readable. 19 files were reformatted. Most of the diff is the two quote styles (ruff format normalises to double quotes, as black does) and long argument lists exploded one-per-line. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Until now every test in this repository needed a live NodeNorm, NameRes, or Google Sheet, so a pull request could not be checked at all without network access -- and `babel_validation.services` had no tests whatsoever, despite being the part other repositories are meant to import. Two areas are covered, chosen because both fail quietly: - CachedNodeNorm/CachedNameRes caching. A cache that stops caching makes a run slower rather than failing it, so nothing would report it. The tests pin the cache-warming pattern the module docstrings promise, that params are part of the cache key, that negative results are cached, and that invalidation clears every params variant of one query without flushing the rest. - TestRow.from_data_row and GoogleSheetTestCases.test_rows. A renamed sheet column yields an empty string rather than an error, and a numbering slip yields a test ID pointing a maintainer at the wrong line of the sheet. The tests pin the column headings, the strict xfail marking, and that dropping a blank row does not renumber the rows after it. One test documents rather than endorses: lookup() and bulk_lookup() share a single cache namespace keyed on (query, params), so a lookup() result is served to a later bulk_lookup(). That is harmless today only because NameRes returns a list of hits from /lookup and one such list per string from /bulk-lookup -- a coincidence the module docstring does not mention, and the test fails if it ever stops holding. deselected_by_markexpr is covered too, since it is what keeps `pytest -m unit` from downloading the Google Sheet during collection. Verified offline: with HTTPS_PROXY pointed at a dead port, `pytest -m unit` passes 43 tests in 0.28s. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two jobs, following NCATSTranslator/Babel's workflow: `ruff check` plus `ruff format --check`, and `pytest -m unit`. Neither needs a NodeNorm or NameRes deployment, so both are safe to run on every PR -- the existing service tests stay a manual `pytest --target dev`, since they fail on upstream outages rather than on the change under review. The unit job also runs `pytest --collect-only` first. That does need the network, but it is the only check that exercises the pytest_generate_tests hooks in the gsheet modules, where an import error or a broken parametrize would otherwise surface only at the start of a live run. README.md and CLAUDE.md document the split, including the trap that `-m unit` and `--category "Unit Tests"` are unrelated: the marker selects tests/unit/, while the category selects Google Sheet rows that still call a live service. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The two methods key the cache on (query, params) with no record of which endpoint produced an entry, so either can serve the other with no HTTP call. That is deliberate and correct: for equal parameters the endpoints are interchangeable, since /lookup returns a list of hits and /bulk-lookup returns that same list under the string it was asked about. Nothing said so, and the module docstring pointed the other way by describing the endpoints as having "different response shapes" -- true of the return types (a mapping against a bare list), but not of the hits themselves, which is what the cache stores. Reworded to separate the two, and the caching model now states the premise the shared keyspace rests on. No behaviour change; the existing test that pins the shared cache is reframed to match. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Started as the
src.prefix removal; grew a CI story, because the packaging change exposed that this repository had no automated checks at all.Packaging
pyproject.tomlpackagessrc/babel_validationdirectly instead of the wholesrc/directory, sobabel_validationis a proper top-level package after installfrom src.babel_validation.X import Ytofrom babel_validation.X import Ypythonpath = ["src"]under[tool.pytest.ini_options], so the suite imports from the source tree rather than depending on install statesrc/__init__.pyis deletedLinting
E/F/I/UP,E501ignoredruff checkreported are fixed, andruff formatis applied repo-wideTesting
tests/unit/— 43 offline tests forsrc/babel_validation/, marked@pytest.mark.unit.github/workflows/test.ymlrunsruff check,ruff format --check, andpytest -m uniton every PRDocs
CachedNameRes's docstring records whylookup()andbulk_lookup()may share one cache (see below)Read commit by commit; the
ruff formatsweep is isolated inefb3e01so it can be skipped or dropped.Why
pythonpathNarrowing the wheel target to
src/babel_validationalso narrows what an editable install puts onsys.path. Before this PR,from src.babel_validation…resolved off the repo root that pytest prepends during collection, so the suite ran with no install at all. Afterwards it resolves only through the installed distribution — and any venv built from the oldpackages = ["src"]still points its.pthat the repo root, sopytestfails collection withModuleNotFoundError: No module named 'babel_validation'untiluv syncis re-run. That would have hit every existing contributor venv, plus anyone following the barepytest --target devin README.md and CLAUDE.md.pythonpath = ["src"](built into pytest ≥ 7, no plugin) puts the source tree onsys.pathduring collection, so the suite no longer depends on install state either way.What the unit tests cover, and why those things
Every test here needed a live NodeNorm, NameRes, or Google Sheet, so a PR could not be checked without network access — and
babel_validation.serviceshad no tests at all, despite being the part other repositories are meant to import. Both areas covered were chosen because they fail quietly:CachedNodeNorm/CachedNameRescaching. A cache that stops caching makes a run slower rather than failing it, so nothing reports it. Pinned: the cache-warming pattern the module docstrings promise, params being part of the cache key, negative results being cached, and invalidation clearing every params variant of one query without flushing the rest.TestRow.from_data_rowandGoogleSheetTestCases.test_rows. A renamed sheet column yields an empty string rather than an error, and a numbering slip yields a test ID pointing a maintainer at the wrong line of the sheet. Pinned: the column headings, the strict-xfail marking, and that dropping a blank row does not renumber the rows after it.deselected_by_markexpr, since it is what keepspytest -m unitfrom downloading the Google Sheet during collection.Verified offline — with
HTTPS_PROXYpointed at a dead port,pytest -m unitpasses 43 tests in 0.28s.A note on the NameRes cache
CachedNameRes.lookup()andbulk_lookup()share a single cache namespace keyed on(query, params), with no record of which endpoint produced the entry. So alookup()result is handed straight back to a laterbulk_lookup()for the same query, with no request made:That is intended and correct: given the same parameters the two endpoints are interchangeable, since
/lookupreturns a list of hits and/bulk-lookupreturns that same list under the string it was asked about. Nothing recorded it, though, and the module docstring pointed the other way by calling the endpoints "different response shapes" — true of the return types, not of the hits the cache stores. The docstring now states the premise the shared keyspace rests on, andtest_lookup_and_bulk_lookup_share_one_cache_namespacefails if NameRes ever makes the endpoints disagree.Notes
-m unitand--category "Unit Tests"are unrelated despite the names. The marker selectstests/unit/; the category selects Google Sheet rows that still call a live service. Documented in CLAUDE.md.src/__init__.pydoes not fully close the shadow-import path:src/is still a directory, so PEP 420 namespace packages keepimport src.babel_validation…resolving while the repo root is onsys.path, which the rootconftest.pyguarantees. Closing it entirely needsimportmode = importlib, which changes collection semantics suite-wide — not worth it for a hazard that only fires on a hand-writtensrc.-prefixed import.log-analysis/is excluded from ruff: those are exploratory notebooks, and one cell does not parse.source_infoandfirst_biolink_typesit next to assertions that could reasonably have used them, so they may be worth reinstating deliberately.Test plan
pytest --collect-onlyagainst a venv with a stale editable.pth(nouv sync) — 4485 tests collected, previouslyModuleNotFoundErrorpytest -m unitwith all network blocked — 43 passed, 25 deselected, 0.28suv sync --frozen,ruff check .,ruff format --check .— all cleangrepforsrc.babel_validation— no imports remain; only filesystem paths in CLAUDE.mdpytest --target dev -x— smoke-test against dev environment🤖 Generated with Claude Code