Index library.alternate_artist_name in search_doc at weight C (BS#3733)

apps/backend/services/library.service.ts

@@ -3332,10 +3332,11 @@ function toLibraryViewRow<T extends AlbumRankHelperColumns>(row: T): Omit<T, key
 
 /**
  * Tsvector + plays ranker for the dj-site Both-mode default. Reads
- * `library.search_doc` (the STORED generated tsvector from migration 0058)
- * against a last-token prefix `tsquery` (`buildPrefixTsquery`,
- * WXYC/Backend-Service#670) so multi-term queries get AND-semantics and the
- * token the DJ is still typing gets prefix behavior.
+ * `library.search_doc` (the STORED generated tsvector from migration 0058:
+ * artist name at weight A, album title at B and, since migration 0208 /
+ * BS#3733, alternate artist name at C) against a last-token prefix `tsquery`
+ * (`buildPrefixTsquery`, WXYC/Backend-Service#670) so multi-term queries get
+ * AND-semantics and the token the DJ is still typing gets prefix behavior.
  *
  * Ranking is an explicit tier — not a `ts_rank * plays` product (BS#2725).
  * `plays` spans roughly 10x on the real catalog, wider than the typical
@@ -3632,9 +3633,12 @@ async function searchLibraryByTrigramBoth(
  * Alias-only probe that runs after a non-empty stage 1 on the catalog route
  * (BS#3552): the releases of every artist whose `artist_search_alias` variant
  * matches `query`, minus every release the tsvector tier itself matches.
- * `search_doc` indexes only `artist_name` and `album_title`, so without this
- * a surname query such as `Giddens` stops at Jerry Giddens and never reaches
- * the Carolina Chocolate Drops card filed as "Rhiannon Giddens".
+ * `search_doc` indexes a release's own `artist_name`, `album_title` and
+ * `alternate_artist_name` (BS#3733), not the alias variants of its artist, so
+ * without this a surname query such as `Giddens` stops at Jerry Giddens and
+ * never reaches the Carolina Chocolate Drops card filed as "Rhiannon
+ * Giddens". A release whose own alternate name matches is a stage-1 row, so
+ * the exclusion below keeps it out of the alias rows.
  *
  * It is the stage-2 query without branch (a), excluding on
  * `search_doc @@ tsquery` instead of the trigram predicate. That excludes

docs/playlist-search/README.md

@@ -129,7 +129,7 @@ Use `'simple'` (no stemming) — music titles are full of proper nouns, foreign
 
 **The segments touch.** `tsvector || tsvector` shifts the right operand's positions to continue from the left's with **no gap**, so each segment's first lexeme is adjacent to the previous segment's last one and a phrase query can match across a field boundary — matching `artist <-> track`, say, where those words were never adjacent in any real text. As shipped, `flowsheet.search_doc` has **five** segments (migration `0054` added `dj_name` to `0052`'s original four, and `0065` replayed it), so there are **four** such seams.
 
-`library.search_doc` had the same defect across its single seam, and migration `0178` fixed it at the column: concatenate a sentinel between the segments to buy a position shift, then remove it with `ts_delete`, which deletes a lexeme's entry without renumbering the survivors. See `docs/catalog-search/README.md` → **Why the segments do not touch** for the mechanism and the numbers.
+`library.search_doc` had the same defect across its seam, and migration `0178` fixed it at the column (`0208` later added a third segment behind a second gap of the same kind): concatenate a sentinel between the segments to buy a position shift, then remove it with `ts_delete`, which deletes a lexeme's entry without renumbering the survivors. See `docs/catalog-search/README.md` → **Why the segments do not touch** for the mechanism and the numbers.
 
 **Flowsheet cannot take that column fix, and closes the same four seams at the reader instead (BS#2726).** Redefining a `STORED GENERATED` column requires `DROP COLUMN` + `ADD COLUMN` (Postgres cannot change a generation expression in place), which rewrites every row under `ACCESS EXCLUSIVE`. Migration `0178`'s rewrite cost about 1.71 s on the 64K-row catalog; the equivalent measured on a restored production flowsheet snapshot (db.t4g.small, PG 14.22, 2.65M rows) held ACCESS EXCLUSIVE for **~8 m 06 s**, and `migrate()` applies every pending migration in one transaction, so that lock — which blocks every flowsheet read and write — would be held for the whole deploy's migration batch, not just this rewrite. There is no quiet hour that makes an 8-minute outage of the flowsheet acceptable, so this stays a reader-side fix rather than a column rewrite; a planned flowsheet downtime (a PG major-version upgrade, say) is the point at which the column rewrite becomes affordable and the guard below can retire.
 

shared/database/src/migrations/0208_3733-library-search-doc-alternate-artist-name.sql

@@ -0,0 +1,98 @@
+-- 0208 (BS#3733): add library.alternate_artist_name to library.search_doc, at
+-- weight C, behind its own position gap.
+--
+-- The station librarian filed a CD under Kath Bloom with the alternate artist
+-- name `kath bloom & david Shapiro`, and classic catalog search for `shapiro`
+-- did not return it. Stage 1 of Both-mode search (`searchLibraryByTsvector`)
+-- is a prefix tsquery over this column, which indexed only `artist_name` (A)
+-- and `album_title` (B). The only other route to an alternate name is the
+-- `artist_search_alias` path, and it cannot reach this case: it scores the
+-- whole credit with pg_trgm (`shapiro` against `Kath Bloom & David Shapiro`
+-- is 0.32, under its 0.40 floor), the #3554 splitter splits on commas only,
+-- and the daily job can take up to 7 days to pick a new name up. /wxycdb's
+-- Lucene index put every word of the alternate name into the release's
+-- tokenized contents when the release was saved, and that is what the
+-- librarian expects. Because the column is generated, a release is now
+-- findable by any word of its alternate name as soon as it is saved or
+-- edited. The alias path is unchanged and still adds the owning artist's card.
+--
+-- WEIGHT C keeps an alternate-name-only hit below an artist-name or title hit
+-- on the same word: `ts_rank` weights C at 0.2 against A's 1.0 and B's 0.4.
+-- `match_tier` (an exact whole-lexeme hit above a prefix-only one) still sorts
+-- first, as it already does for title hits. BS#871 plans to add `label` to this
+-- column; whichever of the two lands second chooses its weight relative to the
+-- other and rebases.
+--
+-- THE SECOND GAP. Migration 0178's header explains the mechanism: a sentinel
+-- concatenated between two segments buys a position shift, and `ts_delete`
+-- removes the sentinel without renumbering the survivors. The new segment gets
+-- its own gap for the same reason the title got one: without it the title's
+-- last lexeme and the alternate name's first are adjacent, and a phrase query
+-- (which `to_tsquery` builds from an apostrophe or hyphen token) matches across
+-- the seam. With it, `Cold Krush Cuts` / `DJ Krush & Coldcut` stores
+-- `'cuts':8B` and `'dj':1A,12C`, a distance of 4. On the 64,193-row production
+-- clone, zero rows contain the sentinel in `alternate_artist_name` either, so
+-- 0178's sentinel-collision note still holds.
+--
+-- WHY DROP-AND-RE-ADD, and every other deviation from drizzle-kit's output,
+-- is as 0178's header records it. Postgres cannot change a generated column's
+-- expression in place before PG 17, and prod RDS is PG 14. `DROP COLUMN` takes
+-- `library_search_doc_idx` with it, so the CREATE INDEX is hand-added and not
+-- optional. The lock and statement timeouts and the IF EXISTS / IF NOT EXISTS
+-- guards are hand-added too; the guards make every partial-apply state
+-- re-runnable (the generated-column exception in docs/migrations.md's
+-- `if-not-exists-index` rule). The index is not built CONCURRENTLY, because
+-- `migrate()` runs inside a transaction and the DROP destroys the index in that
+-- same transaction, so there is nothing ops could pre-build.
+--
+-- LOCK BUDGET. Measured on the full 64,193-row clone (PG 18.6, three runs,
+-- each in a rolled-back transaction): DROP COLUMN about 1 ms, the ADD COLUMN
+-- rewrite 1,061-1,213 ms, CREATE INDEX 115-148 ms, ANALYZE 450-509 ms. The same
+-- statements with 0178's two-segment expression took 994-1,103 ms for the
+-- rewrite and 125-144 ms for the index on the same clone. The GIN index grows
+-- 3,944 kB -> 4,096 kB on a fresh build, from 4,844 alternate names. As in
+-- 0178, `migrate()` applies every pending migration in ONE transaction, so the
+-- ACCESS EXCLUSIVE lock DROP COLUMN takes on `library` is held until the whole
+-- batch commits. This migration must therefore deploy with no other migration
+-- pending, during a quiet hour. The timeouts bound the wait for the lock and
+-- the rewrite, so contention fails the deploy with a clear error instead of
+-- wedging it. This is the same knowing exception to the `ddl-only` rule that
+-- 0065 and 0178 took.
+--
+-- Re-checked on the schema at 0207: no view or materialized view depends on
+-- library.search_doc (pg_depend/pg_rewrite: 0 rows; `library_artist_view` and
+-- `rotation_library_view` read `library` but not this column), so the DROP
+-- needs no CASCADE. The only dependents are the column's own generation
+-- expression and `library_search_doc_idx`. The column's attnum moves from 31
+-- to the end of the table. `restoreCatalogDeleteSnapshot` replays exactly the
+-- keys `captureCatalogDeleteSnapshot` stored, and capture skips generated
+-- columns (`!column.generated`), so the move has no effect there and a restored
+-- row gets the new vector on insert.
+--
+-- The trailing ANALYZE is required: a table rewrite discards the column's
+-- pg_stats row and bumps no counters, so autoanalyze never repairs it (0150 and
+-- 0178 record the same hazard), and the catalog search could fall back to a
+-- bitmap-heap or sequential scan in front of on-air DJs (the BS#934 shape).
+
+SET LOCAL lock_timeout = '30s';--> statement-breakpoint
+SET LOCAL statement_timeout = '10min';--> statement-breakpoint
+
+ALTER TABLE "wxyc_schema"."library" DROP COLUMN IF EXISTS "search_doc";--> statement-breakpoint
+
+ALTER TABLE "wxyc_schema"."library"
+  ADD COLUMN "search_doc" tsvector
+  GENERATED ALWAYS AS (
+    ts_delete(
+      setweight(to_tsvector('simple', coalesce("artist_name", '')), 'A')
+      || to_tsvector('simple', 'wxycsearchdocgap wxycsearchdocgap wxycsearchdocgap')
+      || setweight(to_tsvector('simple', coalesce("album_title", '')), 'B')
+      || to_tsvector('simple', 'wxycsearchdocgap wxycsearchdocgap wxycsearchdocgap')
+      || setweight(to_tsvector('simple', coalesce("alternate_artist_name", '')), 'C'),
+      'wxycsearchdocgap'
+    )
+  ) STORED;--> statement-breakpoint
+
+CREATE INDEX IF NOT EXISTS "library_search_doc_idx"
+  ON "wxyc_schema"."library" USING gin ("search_doc");--> statement-breakpoint
+
+ANALYZE "wxyc_schema"."library";