Skip to content

Make babel_validation a top-level import; add ruff and offline unit tests to CI - #106

Open
gaurav wants to merge 10 commits into
mainfrom
split/2-top-level-package
Open

Make babel_validation a top-level import; add ruff and offline unit tests to CI#106
gaurav wants to merge 10 commits into
mainfrom
split/2-top-level-package

Conversation

@gaurav

@gaurav gaurav commented Jun 26, 2026

Copy link
Copy Markdown
Collaborator

Summary

Started as the src. prefix removal; grew a CI story, because the packaging change exposed that this repository had no automated checks at all.

Packaging

  • pyproject.toml packages src/babel_validation directly instead of the whole src/ directory, so babel_validation is a proper top-level package after install
  • All 4 test-file imports move from from src.babel_validation.X import Y to from babel_validation.X import Y
  • pythonpath = ["src"] under [tool.pytest.ini_options], so the suite imports from the source tree rather than depending on install state
  • The now-dead src/__init__.py is deleted

Linting

  • ruff replaces black, with Babel's settings: line-length 120, E/F/I/UP, E501 ignored
  • The 22 issues ruff check reported are fixed, and ruff format is applied repo-wide

Testing

  • tests/unit/ — 43 offline tests for src/babel_validation/, marked @pytest.mark.unit
  • .github/workflows/test.yml runs ruff check, ruff format --check, and pytest -m unit on every PR

Docs

  • README.md and CLAUDE.md cover the offline/live split, how to mark a new test, and the ruff commands
  • CachedNameRes's docstring records why lookup() and bulk_lookup() may share one cache (see below)

Read commit by commit; the ruff format sweep is isolated in efb3e01 so it can be skipped or dropped.

Why pythonpath

Narrowing the wheel target to src/babel_validation also narrows what an editable install puts on sys.path. Before this PR, from src.babel_validation… resolved off the repo root that pytest prepends during collection, so the suite ran with no install at all. Afterwards it resolves only through the installed distribution — and any venv built from the old packages = ["src"] still points its .pth at the repo root, so pytest fails collection with ModuleNotFoundError: No module named 'babel_validation' until uv sync is re-run. That would have hit every existing contributor venv, plus anyone following the bare pytest --target dev in README.md and CLAUDE.md.

pythonpath = ["src"] (built into pytest ≥ 7, no plugin) puts the source tree on sys.path during collection, so the suite no longer depends on install state either way.

What the unit tests cover, and why those things

Every test here needed a live NodeNorm, NameRes, or Google Sheet, so a PR could not be checked without network access — and babel_validation.services had no tests at all, despite being the part other repositories are meant to import. Both areas covered were chosen because they fail quietly:

  • CachedNodeNorm / CachedNameRes caching. A cache that stops caching makes a run slower rather than failing it, so nothing reports it. Pinned: the cache-warming pattern the module docstrings promise, params being part of the cache key, negative results being cached, and invalidation clearing every params variant of one query without flushing the rest.
  • TestRow.from_data_row and GoogleSheetTestCases.test_rows. A renamed sheet column yields an empty string rather than an error, and a numbering slip yields a test ID pointing a maintainer at the wrong line of the sheet. Pinned: the column headings, the strict-xfail marking, and that dropping a blank row does not renumber the rows after it.
  • deselected_by_markexpr, since it is what keeps pytest -m unit from downloading the Google Sheet during collection.

Verified offline — with HTTPS_PROXY pointed at a dead port, pytest -m unit passes 43 tests in 0.28s.

A note on the NameRes cache

CachedNameRes.lookup() and bulk_lookup() share a single cache namespace keyed on (query, params), with no record of which endpoint produced the entry. So a lookup() result is handed straight back to a later bulk_lookup() for the same query, with no request made:

nr.lookup("diabetes")            # -> [{'curie': 'MONDO:1'}]
nr.bulk_lookup(["diabetes"])     # -> {'diabetes': [{'curie': 'MONDO:1'}]}, no HTTP call

That is intended and correct: given the same parameters the two endpoints are interchangeable, since /lookup returns a list of hits and /bulk-lookup returns that same list under the string it was asked about. Nothing recorded it, though, and the module docstring pointed the other way by calling the endpoints "different response shapes" — true of the return types, not of the hits the cache stores. The docstring now states the premise the shared keyspace rests on, and test_lookup_and_bulk_lookup_share_one_cache_namespace fails if NameRes ever makes the endpoints disagree.

Notes

  • -m unit and --category "Unit Tests" are unrelated despite the names. The marker selects tests/unit/; the category selects Google Sheet rows that still call a live service. Documented in CLAUDE.md.
  • Deleting src/__init__.py does not fully close the shadow-import path: src/ is still a directory, so PEP 420 namespace packages keep import src.babel_validation… resolving while the repo root is on sys.path, which the root conftest.py guarantees. Closing it entirely needs importmode = importlib, which changes collection semantics suite-wide — not worth it for a hazard that only fires on a hand-written src.-prefixed import.
  • log-analysis/ is excluded from ruff: those are exploratory notebooks, and one cell does not parse.
  • Three dead locals were removed for F841. source_info and first_biolink_type sit next to assertions that could reasonably have used them, so they may be worth reinstating deliberately.

Test plan

  • pytest --collect-only against a venv with a stale editable .pth (no uv sync) — 4485 tests collected, previously ModuleNotFoundError
  • pytest -m unit with all network blocked — 43 passed, 25 deselected, 0.28s
  • uv sync --frozen, ruff check ., ruff format --check . — all clean
  • grep for src.babel_validation — no imports remain; only filesystem paths in CLAUDE.md
  • pytest --target dev -x — smoke-test against dev environment

🤖 Generated with Claude Code

Change pyproject.toml to package src/babel_validation directly so
babel_validation is importable as a top-level name after install.
Update all test imports from src.babel_validation.* to babel_validation.*.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Base automatically changed from split/1-library-reorg to main June 26, 2026 18:26

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Makes babel_validation a conventional top-level installed package.

Changes:

  • Updates Hatchling packaging configuration.
  • Migrates four test imports away from the src. prefix.
  • Updates contributor documentation.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated no comments.

Show a summary per file
File Description
pyproject.toml Packages src/babel_validation directly.
tests/test_environment/test_env.py Uses the top-level package import.
tests/nodenorm/test_nodenorm_from_gsheet.py Updates the Google Sheet import.
tests/nameres/test_nameres_from_gsheet.py Updates the Google Sheet import.
tests/nameres/test_blocklist.py Updates the blocklist import.
CLAUDE.md Documents the new import convention.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

gaurav and others added 7 commits August 18, 2026 15:31
…stall

Moving the wheel target from `packages = ["src"]` to
`packages = ["src/babel_validation"]` meant `babel_validation` was only
importable via the installed distribution. Anyone with a venv built
before that change -- or following the bare `pytest --target dev` in
README.md and CLAUDE.md -- hit `ModuleNotFoundError: No module named
'babel_validation'` at collection until they re-ran `uv sync`.

`pythonpath = ["src"]` (built into pytest >= 7) puts the source tree on
sys.path during collection, so the test suite no longer depends on
install state.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Nothing imports `src.*` any more, but the file still marked `src` as a
package, so editors and linters kept offering `src.babel_validation` as
a completion -- and such an import resolves to a second, distinct module
object with its own caches rather than failing loudly.

Deleting it does not fully close that path (PEP 420 namespace packages
keep `src.` importable while the repo root is on sys.path, which the
root conftest.py guarantees under pytest's default prepend import mode),
but it removes the marker that invites the mistake.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Settings are copied from NCATSTranslator/Babel so the two repositories
lint identically: line-length 120, `E`/`F`/`I`/`UP` selected, `E501`
ignored because `ruff format` already owns wrapping.

ruff format supersedes black rather than joining it -- two formatters
with different opinions produce a diff every time whichever one ran last
disagrees with the other, so black is dropped from the dependencies and
from CLAUDE.md.

Also registers the `unit` marker, again matching Babel's wording. Every
test in this repository currently talks to a live service or downloads
the Google Sheet, so `pytest -m unit` is empty today; the following
commits give it something to run and wire it up to CI.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Nineteen were mechanical and applied with `ruff check --fix`: import
blocks sorted into stdlib/third-party/first-party groups (I001), the
eight `Optional[str]` annotations in BlocklistEntry rewritten as
`str | None` now that the floor is Python 3.11 (UP045), and one f-string
with no placeholders (F541).

The remaining three were dead locals (F841). `input_results`,
`source_info` (with the `source`/`source_url` reads that fed only it),
and `first_biolink_type` were each assigned and never read, so removing
them changes no behaviour -- but `source_info` and `first_biolink_type`
sit next to assertions that could reasonably have used them, so they may
be worth reinstating deliberately rather than by accident.

log-analysis/ is excluded: those are exploratory notebooks, and one cell
does not parse, so linting them would gate CI on scratch work.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Mechanical only: `uv run ruff format .` over the repository, no
behavioural changes. Isolated in its own commit so the substantive
commits on either side stay readable.

19 files were reformatted. Most of the diff is the two quote styles
(ruff format normalises to double quotes, as black does) and long
argument lists exploded one-per-line.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Until now every test in this repository needed a live NodeNorm, NameRes,
or Google Sheet, so a pull request could not be checked at all without
network access -- and `babel_validation.services` had no tests whatsoever,
despite being the part other repositories are meant to import.

Two areas are covered, chosen because both fail quietly:

- CachedNodeNorm/CachedNameRes caching. A cache that stops caching makes
  a run slower rather than failing it, so nothing would report it. The
  tests pin the cache-warming pattern the module docstrings promise, that
  params are part of the cache key, that negative results are cached, and
  that invalidation clears every params variant of one query without
  flushing the rest.

- TestRow.from_data_row and GoogleSheetTestCases.test_rows. A renamed
  sheet column yields an empty string rather than an error, and a
  numbering slip yields a test ID pointing a maintainer at the wrong line
  of the sheet. The tests pin the column headings, the strict xfail
  marking, and that dropping a blank row does not renumber the rows after
  it.

One test documents rather than endorses: lookup() and bulk_lookup() share
a single cache namespace keyed on (query, params), so a lookup() result
is served to a later bulk_lookup(). That is harmless today only because
NameRes returns a list of hits from /lookup and one such list per string
from /bulk-lookup -- a coincidence the module docstring does not mention,
and the test fails if it ever stops holding.

deselected_by_markexpr is covered too, since it is what keeps `pytest -m
unit` from downloading the Google Sheet during collection.

Verified offline: with HTTPS_PROXY pointed at a dead port, `pytest -m
unit` passes 43 tests in 0.28s.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two jobs, following NCATSTranslator/Babel's workflow: `ruff check` plus
`ruff format --check`, and `pytest -m unit`. Neither needs a NodeNorm or
NameRes deployment, so both are safe to run on every PR -- the existing
service tests stay a manual `pytest --target dev`, since they fail on
upstream outages rather than on the change under review.

The unit job also runs `pytest --collect-only` first. That does need the
network, but it is the only check that exercises the pytest_generate_tests
hooks in the gsheet modules, where an import error or a broken
parametrize would otherwise surface only at the start of a live run.

README.md and CLAUDE.md document the split, including the trap that
`-m unit` and `--category "Unit Tests"` are unrelated: the marker selects
tests/unit/, while the category selects Google Sheet rows that still call
a live service.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@gaurav gaurav changed the title Make babel_validation a top-level import (drop src. prefix) Make babel_validation a top-level import; add ruff and offline unit tests to CI Aug 18, 2026
The two methods key the cache on (query, params) with no record of which
endpoint produced an entry, so either can serve the other with no HTTP
call. That is deliberate and correct: for equal parameters the endpoints
are interchangeable, since /lookup returns a list of hits and
/bulk-lookup returns that same list under the string it was asked about.

Nothing said so, and the module docstring pointed the other way by
describing the endpoints as having "different response shapes" -- true of
the return types (a mapping against a bare list), but not of the hits
themselves, which is what the cache stores. Reworded to separate the two,
and the caching model now states the premise the shared keyspace rests on.

No behaviour change; the existing test that pins the shared cache is
reframed to match.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants