[Alessandro100]

Summary:

Backs the feed location directory on mobilitydatabase.org (/locations). It is built on the OSM geopolygon read model rather than the feeds' declared Location rows: places, hierarchy and names come from GeopolygonLocationSearch, and feeds link to places through their reverse-geolocated stops (FeedOsmLocationGroup -> OsmLocationGroupGeopolygon). A GTFS Realtime feed, which has no stops, takes the places of the schedule feeds it references.

Each locations endpoint has one job: /v1/locations searches, /v1/locations/{location_id} and /v1/locations/lookup return one place, and /v1/locations/{location_id}/children lists a place's children.

• GET /v1/locations (search and filter, list only):
  • New parameters: continent (slug or code; the continent row included), has_feeds, include=feed_counts and sort (admin_level, the current order, or feed_count).
  • location_type=continent returns the seven continents.
• GET /v1/locations/{location_id}: one place, or 404.
• GET /v1/locations/lookup?path=...: the same place, addressed by its path (e.g. north-america/canada/quebec), or 404. The path is case-insensitive and leading or trailing slashes are ignored.
• include: on the two single-place endpoints, include=feed_counts,children adds the place's counts and the first page of its direct children. has_feeds, sort, limit and offset then apply to the children.
• GET /v1/locations/{location_id}/children: a place's direct children, paged, with the same has_feeds, include=feed_counts and sort. 404 for an unknown id.
• Every location has a non-null location_id and a path:
  • Continents are locations with fixed negative ids: -1 Africa, -2 Antarctica, -3 Asia, -4 Europe, -5 North America, -6 Oceania, -7 South America. Each continent is the parent of its countries.
  • Paths: path, path_slugs and path_names start at the continent, so a location's path is its web URL after /locations/.
  • Result fields: new continent_code, slug, path, path_slugs and feed_counts (GTFS / GTFS-RT / GBFS / total). A place returned by /{location_id} or lookup (LocationDetail) also has children, filled only when include=children is requested.
  • Required fields: location_id, location_type, slug, path, path_slugs and path_names are marked required in the spec.
• GET /v1/search:
  • location_id (continents included) or location_path restricts results to feeds with stops in that place, sub-areas included. Pass one, not both (400). An unknown path returns no feed.
  • sort=location ranks feeds by the share of their stops inside the place (a city bus feed before a regional train), then by provider, feed name and stable id. It requires location_id or location_path.
• Continents: new continent and country_continent tables.
  • Seed: liquibase/changes/feat_location_directory.sql (250 countries). The mapping is taken as is from country_converter's seven-continent classification, which follows Our World in Data's region definitions, with no overrides. scripts/country_continent_seed.py regenerates the seed; the library is a dev-only dependency.
  • Ids: feat_location_directory_continent_ids.sql adds the continents' fixed ids. It is a separate changeset because dev already ran the first one.
• GeopolygonLocationSearch view:
  • Continent rows: adds the continents as rows; each country's parent is its continent.
  • Paths: prefixes every path with the continent.
  • New columns: continent_code, slug, path_slugs and an indexed path.
  • One row per country: it keeps one country row per ISO code, the lowest admin level; the other boundaries resolve to it as a parent and in paths, so their sub-areas are children of the kept country. On dev, French Guiana, Guadeloupe, Martinique, Réunion and Clipperton Island each have several boundaries.
• Route order: the generated router registers routes by operationId, which put /{location_id} before /lookup. utils/route_order.py moves fixed paths first for the locations router only.
• Removed before release:
  • the /v1/locations/directory endpoint;
  • the country / subdivision / municipality and continent search parameters;
  • the parent_location_id, path and include_feed_counts parameters of /v1/locations;
  • the Location-based directory code.

Breaking changes:

For existing users of /v1/locations (released since v1.16.0). Nothing released is removed or renamed, but these results change:

1. Continents appear in results. A search without location_type can return the seven continents (e.g. search_query=america returns North and South America), sorted first.
2. New location_type value: continent. Clients generated with a strict enum can reject it.
3. Negative ids. Continents have location_id -1 to -7. Code that treats location_id as an OSM id (for example to build openstreetmap.org links) breaks on them.
4. A country's parent_location_id is its continent's id instead of null. Code that finds countries by "no parent" stops working.
5. path_names starts with the continent: ["North America", "Canada", "Québec"] instead of ["Canada", "Québec"]. Code that reads path_names[0] as the country gets the continent. display_name is unchanged.
6. Duplicate country boundaries are gone (see the view above): about 7 rows on dev (Martinique, Guadeloupe, French Guiana, Réunion, Clipperton). Stored ids of the dropped boundaries no longer resolve.
7. Unsupported sort and include values return 400 on /v1/locations and /v1/search. Before, a sort parameter was ignored, so a client that already sent one gets an error now.

Not breaking: the new optional parameters, the new response fields, the new endpoints, and the fields newly marked required in the spec. Everything else removed (/v1/locations/directory, parent_location_id, include_feed_counts, continent on /v1/search) was added and removed within this PR.

Deployment order:

• Migrate first: run Liquibase (the continent changesets and the GeopolygonLocationSearch rebuild) before deploying the API in each environment; the new code fails without them. Done on dev; QA and prod still need it.
• API before web: feat: add feed location directory mobilitydatabase-web#259 only works against the new endpoints, so merge and deploy it after this PR.

Design notes:

• Feed counts: distinct feeds with status active, inactive, development or future, with the same operational_status rule as /v1/search. A feed counts in every place where it has stops, so a regional feed appears under each municipality it serves. Continent counts are distinct feeds, never sums of country counts. Counts are only computed when asked for (include=feed_counts, has_feeds, sort=feed_count).
• Slugs: the name lowercased, without accents, with other characters replaced by -.
  • Collisions: siblings that would share a slug each get --<location_id> appended (springfield--1234). A name never produces --, so a suffixed slug cannot equal another sibling's natural slug. On dev, 1,274 parent and slug pairs collide, mostly same-named US townships directly under their state.
  • Stability: collisions are counted over all places, so a slug doesn't change when feeds do.
  • No ASCII letters or digits: the name's slug is its location id.
  • Paths change when places are renamed in OSM. Store location_id, not paths.
• Realtime feeds: a GTFS Realtime feed without a location group of its own takes the groups of the schedule feeds it references (feedreference), with their stop counts. A realtime feed with its own groups keeps only those.
• Coverage: other feeds that were never reverse-geolocated have no location group, so they appear neither in the directory nor in the location filters. On dev, among the feeds with a directory status:
  • GTFS: 1,655 of 3,949 are located (3,155 have a dataset).
  • GBFS: 420 of 1,268 are located.
  • GTFS Realtime: 427 of 1,485 are located, up from 17, through their schedule feeds.
• Performance: counts are computed per request; the web pages are cached for 6 hours.
  • Filtering inside the union: a feed's location groups are the union of its own groups and the groups a realtime feed inherits from its schedule feeds. Every count, has_feeds check and location search applies the location filter inside each branch of that union, and joins the feed there too. PostgreSQL cannot push a filter into the inherited branch (a join) and estimates it at about one row, so filtering the union afterwards made it rebuild the inherited groups once per location group: continent requests took minutes on dev, some hitting the 300 s timeout.
  • One counting pass per page: with has_feeds or sort=feed_count, one query counts the feeds of the matching locations and uses the counts to filter, sort, page and return the total (a window count). It used to count them up to four times.
  • Measured on dev (Cloud Run request log, pages nobody had opened): a lookup with children and counts takes 0.21 to 0.62 s, a search by location 0.05 to 0.45 s, and the whole web page 0.56 to 0.93 s. Continent lookups (Asia, Africa, South America) took 0.8 to 1.7 s, and the Europe search 4.3 s, before the single-pass change.

Expected behavior:

• /v1/locations?location_type=continent&include=feed_counts lists the seven continents (ids -1 to -7) with counts.
• /v1/locations/lookup?path=north-america/canada/quebec&include=feed_counts,children&has_feeds=true&sort=feed_count returns Québec with its children that have feeds, most feeds first. /v1/locations/lookup?path=atlantis returns 404.
• /v1/locations/-5/children?has_feeds=true&include=feed_counts&sort=feed_count lists the North American countries that have feeds; each country's parent_location_id is -5.
• /v1/search?location_path=north-america/canada/quebec&status=active,inactive,development,future&sort=location lists Québec's feeds, as does location_id=<Québec's id>.
• 400 for:
  • both location_id and location_path;
  • sort=location without either of them;
  • include=children on the list or children endpoint;
  • an unknown sort.

Testing tips:

• Tests:

  • api/tests/integration/location_directory/: a geopolygon fixture covering continents, duplicate country boundaries, slug collisions, feed counts, realtime feeds located through their schedule feed, and every endpoint and filter above.
  • api/tests/unittest/test_locations_api_impl.py.

• Full suite: the full api/ suite, run with the pytest command scripts/api-tests.sh uses: 882 passed. 6 user_service early-access tests fail locally because my test database lacks the early_access_* tables; this PR doesn't touch user_service. flake8 and black are clean.

• Migration:

  • Order: the new changesets run before the materialized views are rebuilt (liquibase/changelog.xml).
  • Upgrade path: Liquibase was run on an empty database at the previous commit, then on this one. Only the continent-ids changeset and the view rebuild ran, and REFRESH MATERIALIZED VIEW CONCURRENTLY works on the new view.

• Dev: the schema is migrated (the continent ids changeset and the view rebuild) and the API is deployed to dev; the web PR's preview uses it.

• Before merging: this PR redefines GeopolygonLocationSearch, which the geopolygon loader refreshes and /v1/locations already reads; it's worth a review from whoever owns it.

• Web client: feat: add feed location directory mobilitydatabase-web#259

• Run the unit tests with ./scripts/api-tests.sh to make sure you didn't break anything (suite run with the script's pytest command directly, see above)

• Add or update any needed documentation to the repo (OpenAPI spec)

• Format the title like "feat: [new feature short description]". Title must follow the Conventional Commit Specification(https://www.conventionalcommits.org/en/v1.0.0/).

• Linked all relevant issues

• Include screenshot(s) showing how this pull request works and fixes the issue(s)

Follow up tickets

• Depending on how these pages are found / indexed by crawlers, might have to include them in our site map (Stronger opportunity to appear in searches like "transit data montreal")

Generated with Claude Code

[CLAassistant]

All committers have signed the CLA.

[Copilot]

🟡 Changes recommended

Unresolved issues can orphan subdivisions after country deduplication, return incorrect totals for zero-limit queries, and generate slug collisions that make locations unreachable by their advertised paths.

3 open issues

• Preserve total counts when limit is zero · New
• Remap subdivisions to the retained country boundary · New
• Prevent slug collisions with natural numeric names · New

What changed in this PR

Adds a geopolygon-backed location directory for mobilitydatabase.org, with place lookup, child navigation, feed counts and location-based feed search. It also introduces documented breaking changes to existing location results, including continent rows and continent-prefixed paths.

Changes:

• Adds location detail, path lookup and children endpoints, plus feed-count filtering and sorting.
• Introduces continent reference data and extends the location read model with hierarchy paths and slugs.
• Adds shared feed-location queries, realtime-feed inheritance and endpoint tests.

File	Description
scripts/​country_continent_seed.py	Generates country-to-continent seed data.
liquibase/​materialized_views/​geopolygon_location_search.sql	Adds continents, paths, slugs and country deduplication.
liquibase/​changes/​feat_location_directory.sql	Creates and seeds continent reference tables.
liquibase/​changes/​feat_location_directory_continent_ids.sql	Assigns stable negative continent IDs.
liquibase/​changelog.xml	Orders migrations before view rebuilding.
functions-python/​test_utils/​database_utils.py	Preserves reference data during test cleanup.
docs/​DatabaseCatalogAPI.yaml	Defines new endpoints, parameters and response fields.
api/​tests/​unittest/​test_locations_api_impl.py	Tests validation, mapping and location responses.
api/​tests/​test_utils/​db_utils.py	Preserves continent data during API test cleanup.
api/​tests/​integration/​location_directory/​test_location_directory_api.py	Tests directory endpoints and location-based feed search.
api/​tests/​integration/​location_directory/​conftest.py	Supplies hierarchy and feed-location fixtures.
api/​src/​utils/​route_order.py	Prioritizes fixed routes over parameterized routes.
api/​src/​shared/​db_models/​location_search_result_impl.py	Maps expanded location response fields.
api/​src/​shared/​db_models/​location_detail_impl.py	Maps location details with optional children.
api/​src/​main.py	Registers lookup before the location-ID route.
api/​src/​feeds/​impl/​search_api_impl.py	Adds location filtering and stop-share sorting.
api/​src/​feeds/​impl/​locations_api_impl.py	Implements lookup, children and feed-count queries.
api/​src/​feeds/​impl/​location_feeds.py	Shares location membership, inheritance and counting logic.
api/​src/​feeds/​impl/​error_handling.py	Adds location and search validation messages.
api/​.openapi-generator/​FILES	Records the new generated response models.

_{Review effort: Balanced}