feat!: upgrade sqlparser to 0.62 by takaebato · Pull Request #55 · takaebato/sql-insight

takaebato · 2026-07-04T07:15:46Z

Summary

Upgrades sqlparser 0.61 → 0.62 and adapts the resolver/normalizer to its AST
changes. Beyond keeping the build green, this makes sql-insight analyze several
constructs 0.62 can now parse:

ClickHouse ARRAY JOIN / LEFT ARRAY JOIN / INNER ARRAY JOIN
Redshift ALTER SORTKEY (cols) (schema-level; no column writes)
Spark multi-column aliases expr AS (a, b) (best-effort: reads counted once,
lineage to the first alias)
Oracle subquery INSERT targets INSERT INTO (SELECT …) … (flagged as an
unsupported write target, dropped)

Internal adaptations: Parens-wrapped VALUES rows, ObjectName INSERT columns,
TableAliasWithoutColumns, LambdaFunctionParameter, and the
pre_visit_value(&mut ValueWithSpan) visitor signature.

New tests: ARRAY JOIN arm coverage (×3), Spark multi-alias lineage, ALTER SORTKEY
no-column-writes, Oracle subquery-INSERT diagnostic, typed lambda parameter,
case-insensitive / quoted INSERT-column alphabetization, aliased-INSERT target.
Lambda tests moved from GenericDialect to DuckDbDialect (see the breaking note
below).

Breaking changes (reference for the release-PR CHANGELOG note)

The re-exported sqlparser moves 0.61 → 0.62. It's part of the public API
(pub use sqlparser; its AST types appear in TableReference, extractor
inputs, and &dyn Dialect), so consumers must upgrade too. AST types you may
touch changed — e.g. Insert::columns is now Vec<ObjectName> (was
Vec<Ident>), Values::rows is Vec<Parens<Vec<Expr>>>, literals are
ValueWithSpan, lambda params are LambdaFunctionParameter.
Under GenericDialect, -> is now a binary operator, not a lambda arrow
(upstream revert of an accidental enablement). Lambda syntax (x -> …) needs a
lambda-aware dialect such as DuckDbDialect / DatabricksDialect.

Deferred to follow-up PRs

Fan-out lineage for expr AS (a, b, …) — emit an edge to every alias, not just
the first (needs a one-expr → N-outputs model change).
Resolve Oracle INSERT INTO (subquery) through to its base table as the write
target, instead of dropping it.
Stop surfacing a ClickHouse ARRAY JOIN array operand as a (spurious) table read.

🤖 Generated with Claude Code

Adapt the resolver and normalizer to sqlparser 0.62's AST changes, and newly analyze the constructs it can now parse: ClickHouse ARRAY JOIN, Redshift ALTER SORTKEY, Spark multi-column aliases (`expr AS (a, b)`), and Oracle subquery INSERT targets. The re-exported `sqlparser` dependency moves 0.61 -> 0.62; because its AST types are part of the public API (`pub use sqlparser`), consumers must upgrade too. Under `GenericDialect`, `->` is now a binary operator, not a lambda arrow (upstream revert of an accidental enablement), so lambda tests move to `DuckDbDialect`. Follow-ups deferred to separate PRs: fan-out lineage for `expr AS (a, b)`, resolving Oracle subquery INSERT targets to their base table, and not surfacing an ARRAY JOIN array operand as a table read. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AteCEF9XeviLCkfa3EQMfc

github-actions · 2026-07-04T07:15:54Z

✅ PR title follows the Conventional Commits spec.

codecov · 2026-07-04T07:17:06Z

Codecov Report

❌ Patch coverage is 91.07143% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 94.76%. Comparing base (3d782bb) to head (a5cee46).

Files with missing lines	Patch %	Lines
sql-insight/src/reference.rs	66.66%	4 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master      #55      +/-   ##
==========================================
- Coverage   94.80%   94.76%   -0.04%     
==========================================
  Files          27       27              
  Lines        5500     5542      +42     
  Branches     5500     5542      +42     
==========================================
+ Hits         5214     5252      +38     
- Misses        208      212       +4     
  Partials       78       78

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

sqlparser parses the operand of `ARRAY JOIN` / `LEFT ARRAY JOIN` / `INNER ARRAY JOIN` as a `TableFactor::Table`, indistinguishable from a real table, so `SELECT s FROM t ARRAY JOIN arr` used to surface a phantom read of a table `arr`. Bind the operand as an opaque unnest instead — never a table read: a plain name reads as a column of the left relation (`t.arr`), and an array expression (`arrayConcat(t.a, t.b)`) reads its argument columns. The unnested output is exposed as a synthetic column, so a reference to the alias no longer leaks as a phantom `t.<alias>` read either. Follow-up from #55. Left as-is (both exotic): the analogous multi-table DELETE-target path (`DELETE … USING t ARRAY JOIN arr`), and a multi-array `ARRAY JOIN a, b` — which sqlparser parses as a comma-join, so `b` still looks like a table. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AteCEF9XeviLCkfa3EQMfc

sqlparser parses the operand of `ARRAY JOIN` / `LEFT ARRAY JOIN` / `INNER ARRAY JOIN` as a `TableFactor::Table`, indistinguishable from a real table, so `SELECT s FROM t ARRAY JOIN arr` used to surface a phantom read of a table `arr`. Bind the operand as an opaque unnest instead — never a table read: a plain name reads as a column of the left relation (`t.arr`), and an array expression (`arrayConcat(t.a, t.b)`) reads its argument columns. The unnested output is exposed as a synthetic column, so a reference to the alias no longer leaks as a phantom `t.<alias>` read either. The same special case applies on the UPDATE target's join clause (`UPDATE t ARRAY JOIN arr SET …`), and a non-Table operand shape (a derived subquery — invalid ClickHouse but parseable) falls back to the normal factor path so its reads survive. Follow-up from #55. Left as-is (both exotic): the analogous multi-table DELETE-target path (`DELETE … USING t ARRAY JOIN arr`), and a multi-array `ARRAY JOIN a, b` — which sqlparser parses as a comma-join, so `b` still looks like a table. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AteCEF9XeviLCkfa3EQMfc

## Summary Fixes a follow-up noted in #55: a ClickHouse `ARRAY JOIN` operand was surfaced as a (spurious) table read. sqlparser parses the operand of `ARRAY JOIN` / `LEFT ARRAY JOIN` / `INNER ARRAY JOIN` as a `TableFactor::Table`, indistinguishable from a real table — so `SELECT s FROM t ARRAY JOIN arr` reported a phantom read of a table `arr`. (ClickHouse's ARRAY JOIN operand is always an array — a column or an array expression — never a table.) This binds the ARRAY JOIN operand as an opaque unnest (a `TableFunction`, so no table read): - A plain array **column** (`arr` / `t.arr`) reads as a column of the left relation: `read_tables` no longer includes it (`SELECT s FROM t ARRAY JOIN arr` → `[t]`, not `[t, arr]`), and `t.arr` surfaces as a column read. - An array **expression** (`arrayConcat(t.a, t.b)`) reads its argument columns (`t.a`, `t.b`) — not the function name. - The unnested output (`AS x`, else the operand's own name) is exposed as a synthetic column, so a reference to it no longer leaks as a phantom `t.x` read. Edge cases covered on review: - The **UPDATE target's join clause** takes the same special case: `UPDATE t ARRAY JOIN arr SET a = 1` no longer reads a phantom `arr` table (and `t` correctly surfaces as a read — the unnest references the write target's own data). - A **non-Table operand** (a derived subquery — invalid ClickHouse but sqlparser parses it) falls back to the normal factor path, so its reads survive instead of being silently dropped. - Pipe joins (`|> JOIN`) cannot parse an ARRAY JOIN operator — unreachable. Tests: updated the ARRAY JOIN arm coverage (now expects the operand column read), plus table-level "operand is not a read table" (SELECT and UPDATE), "alias does not leak as a phantom column", "expression operand reads its arguments", and "derived operand keeps its reads" tests. All gates green (fmt / clippy / `test --all` / doc). Left as-is (both exotic): the analogous multi-table DELETE-target path (`DELETE … USING t ARRAY JOIN arr`), and a multi-array `ARRAY JOIN a, b` — which sqlparser parses as a comma-join, so `b` still looks like a table. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

`INSERT INTO (SELECT a, b FROM emp WHERE dept = 10) VALUES (…)` writes the view's single base table — resolve through to it instead of dropping the whole statement as an unsupported target. The view projection names the target columns (an explicit list: plain columns only, `a AS x` writes base `a`; a wildcard / expression falls back to the column-less catalog-fill / diagnostic path), the WHERE predicate is filter reads against the target (a new `Insert::target_predicate`), and a SELECT source traces through the view (`s.x → emp.a`). Only the minimal insertable-view shape (projection + FROM + WHERE) resolves through — enforced by exhaustive `Query` / `Select` destructures, so a new clause forces a decision. Everything else keeps the previous drop + `UnsupportedStatement` flag: a join / set operation / nested WITH names no single base table SQL text can determine (key-preserved rules need a catalog), and any other clause (GROUP BY / DISTINCT / ORDER BY / FETCH / …) makes the view non-insertable and could carry silently-dropped column refs. An aliased base table (`FROM emp e`) resolves the predicate's `e.dept` through the alias. `TableReference`'s `TryFrom<&Insert>` sees through the same shapes. Follow-up from #55; only `OracleDialect` parses this syntax. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AteCEF9XeviLCkfa3EQMfc

) ## Summary Implements a follow-up noted in #55: Oracle's inline-view INSERT (`INSERT INTO (SELECT … FROM emp) …`) now resolves through to the view's single base table as the write target, instead of dropping the whole statement as an unsupported target. Semantically the row lands in the base table — the subquery is an unnamed updatable view that restricts columns / enforces a predicate. What resolves (single-table view only): - **Write target** = the view's base table: `INSERT INTO (SELECT a FROM emp) …` writes `emp` (table level: `create_tables=[emp]`). - **Target columns** = the view projection, as an explicit column list — plain columns only; `a AS x` writes base `a`. A wildcard / expression projection makes positions indeterminate → the usual column-less path (catalog fill, or `InsertColumnsUnresolved`). The explicit list also drives the arity check (`… (SELECT a FROM emp) VALUES (1, 2)` → `InsertColumnsArityMismatch`). - **View WHERE** = filter reads against the target (what a `WITH CHECK OPTION` would enforce; sqlparser cannot parse the `WITH CHECK OPTION` keywords inside the target, so the plain predicate is what surfaces): reads `emp.dept`, never a lineage source. Carried on a new internal `Insert::target_predicate`. - **Column lineage** traces through the view: `INSERT INTO (SELECT a FROM emp) SELECT x FROM s` → `s.x → emp.a`. What stays flagged (`UnsupportedStatement`, as before): everything but the minimal insertable-view shape (projection + FROM + WHERE). A join (Oracle's key-preserved rules need a catalog), a set operation, a nested `WITH` / pipe chain, or a non-table factor names no single base table — and any other clause (GROUP BY / HAVING / DISTINCT / ORDER BY / FETCH / CONNECT BY / …) makes the view non-insertable (ORA-01732) and could carry column refs that would otherwise drop silently. Enforced by exhaustive `Query` / `Select` destructures, so a new sqlparser clause forces a keep-or-reject decision. Edge cases covered on review: - An **aliased base table** (`INSERT INTO (SELECT e.a FROM emp e WHERE e.dept = 10) …`): the predicate's `e.dept` resolves through the alias to `emp.dept` (and, as in SQL, the alias shadows the bare table name). - **Clause-bearing views** (`GROUP BY` / `ORDER BY` / `DISTINCT` / `FETCH`) previously slipped through the shape check with their column refs silently dropped — now rejected to the flag+drop path. - `INSERT OVERWRITE` / `REPLACE INTO` bucketing (`peel_to_insert`) only reads the flags — unaffected. `TableReference`'s `TryFrom<&Insert>` sees through the same shapes (and errors on the rest), so the public target-identity parse agrees with the analysis. Tests: 7 column-level cases (values / WHERE-reads / lineage-through-view / alias / wildcard / expression / arity), a table-level CRUD case, a `TryFrom<&Insert>` unit test, and the existing subquery-target diagnostic test narrowed to a join view. All gates green (fmt / clippy / `test --all` / doc). Only `OracleDialect` parses this syntax (`supports_insert_table_query`). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

## Summary Implements the last follow-up noted in #55 (design agreed in discussion): lineage for Spark's multi-column alias `expr AS (a, b, …)` now **fans out to every alias** instead of stopping at the first. ```sql SELECT explode(t.arr) AS (k, v) FROM t -- before: reads [t.arr], lineage [arr → k] -- after: reads [t.arr], lineage [arr → k, arr → v] ``` ## Design: the fan is one item (no references) `NamedExpr` gains `names: OutputNames` — `Single(Option<Ident>)` for an ordinary item, `Fan(Vec<Ident>)` for a multi-column alias: ```text SELECT id, explode(t.arr) AS (k, v), name FROM t Projection.exprs = [ ← 3 items, 4 positions NamedExpr { names: Single(id), expr: Column(t.id) } // position 0 NamedExpr { names: Fan([k, v]), expr: Call(explode(arr)) } // positions 1, 2 NamedExpr { names: Single(name), expr: Column(t.name) } // position 3 ] ``` - **The expression exists exactly once** → `reads` count it once by construction (occurrence-based, unchanged), and item walks (`own_exprs`) stay a plain map. - **Names can never separate from their expression** — there is no back-reference to dangle under any reorder / filter / splice (the design review rejected a positional `Fanout { back }` reference and a `debug_assert`-guarded fallback for exactly this reason). - **Item index ≠ output position**: every positional / named consumer goes through one slot view (`output_slots` / `slot_count` / `resolved_output_expr`), where a fan yields its shared expression once per name — query-output lineage, INSERT positional pairing, RETURNING, CTAS / CREATE VIEW, the shared positional trace (set-op branches, scalar subqueries, `EXCLUDED`), derived-table name traces, output-column scopes (`OutputCol`, fan outputs are never identity), `AS d(x, y)` positional renames, and the created-columns / insert arity checks. - Edge kind = the head expression's kind: `explode(arr)` fans out `Transformation`s; a bare column `t.a AS (x, y)` fans out `Passthrough`s. Non-breaking: output shapes are unchanged; lineage gains edges (the first-alias-only behavior was a documented best-effort in #55, not a contract). Verified end-to-end: fan-out for 2 aliases, `Passthrough` fan-out for a bare column, the tail alias traced through a derived table, INSERT pairing and CTAS / CREATE VIEW feeding every target column, positional fan renames through a derived alias list (`AS d(x, y)`) and a CTE column list, UNION branches tracing positionally (existing reads-once pins unchanged). All gates green (fmt / clippy / `test --all` / doc); 766 tests, **100% patch coverage** (llvm-cov ∩ diff). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

## Summary The changelog body template credits ` by @author` on every normal group entry but not on the "Breaking Changes" `####` headings. This release will be the first to render that section (#55 / #59 are `feat!`), so add the same guarded attribution there: ```jinja #### … {{ commit.message }}{% if commit.remote.username %} by @{{ commit.remote.username }}{% endif %} ``` Also drops CLAUDE.md's stale breaking-changes bullet: it still described the PR-description changelog block dropped in #56, the `!` flag is already covered by the PR-title bullet, and the rendering / hand-written-migration- notes facts live at their source in release-plz.toml's comments. ## Verification `commit.remote.username` is the same variable the normal-entry line two lines below already uses, under the same guard. Rendering can't be verified locally without GitHub-integrated release-plz — after this merges, release-plz will regenerate the open release PR (#48), where the Breaking Changes headings can be eyeballed before releasing. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

The CLI gained a real feature (#53: binstall, completions, man pages), so promote release-plz's 0.x feat→patch default to a minor bump. Add concise migration notes under the library's Breaking Changes headings (#59 SET attribution, #55 sqlparser 0.62). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@takaebato

## 🤖 New release * `sql-insight`: 0.3.0 -> 0.4.0 (✓ API compatible changes) * `sql-insight-cli`: 0.2.1 -> 0.2.2 <details><summary>Changelog</summary> ## `sql-insight` <blockquote> ## [0.4.0](sql-insight-v0.3.0...sql-insight-v0.4.0) - 2026-07-05 ### ⚠️ Breaking Changes #### attribute unqualified SET targets with the read-side rules ([#59](#59)) by @takaebato #### upgrade sqlparser to 0.62 ([#55](#55)) by @takaebato ### Added - resolve Oracle join-view INSERT targets by column attribution ([#61](#61)) by @takaebato - fan out multi-column-alias lineage to every alias ([#60](#60)) by @takaebato - resolve Oracle inline-view INSERT targets to their base table ([#58](#58)) by @takaebato ### Fixed - don't surface a ClickHouse ARRAY JOIN operand as a table read ([#57](#57)) by @takaebato ### Other Changes - changelog breaking-change workflow, version-bump docs, and keywords ([#51](#51)) by @takaebato - tidy keywords, README versions, and add a version-sync check ([#46](#46)) by @takaebato </blockquote> ## `sql-insight-cli` <blockquote> ## [0.2.2](sql-insight-cli-v0.2.1...sql-insight-cli-v0.2.2) - 2026-07-05 ### Added - *(cli)* prebuilt binary distribution — cargo binstall, completions, man, and provenance ([#53](#53)) by @takaebato ### Fixed - *(deps)* update rust crate clap_mangen to 0.3 ([#54](#54)) by @renovate[bot] ### Other Changes - changelog breaking-change workflow, version-bump docs, and keywords ([#51](#51)) by @takaebato - tidy keywords, README versions, and add a version-sync check ([#46](#46)) by @takaebato </blockquote> </details> --- This PR was generated with [release-plz](https://github.com/release-plz/release-plz/). --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Takahiro Ebato <takahiro.ebato@gmail.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

takaebato merged commit b0eb0dd into master Jul 4, 2026
15 checks passed

takaebato deleted the deps/update-sqlparser-0.62 branch July 4, 2026 08:07

github-actions Bot mentioned this pull request Jul 4, 2026

chore: release #48

Merged

takaebato mentioned this pull request Jul 4, 2026

fix: don't surface a ClickHouse ARRAY JOIN operand as a table read #57

Merged

takaebato mentioned this pull request Jul 4, 2026

feat: resolve Oracle inline-view INSERT targets to their base table #58

Merged

takaebato mentioned this pull request Jul 5, 2026

feat: fan out multi-column-alias lineage to every alias #60

Merged

takaebato mentioned this pull request Jul 5, 2026

ci: credit the author on breaking-change changelog headings #62

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat!: upgrade sqlparser to 0.62#55

feat!: upgrade sqlparser to 0.62#55
takaebato merged 1 commit into
masterfrom
deps/update-sqlparser-0.62

takaebato commented Jul 4, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 4, 2026

Uh oh!

codecov Bot commented Jul 4, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

takaebato commented Jul 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Breaking changes (reference for the release-PR CHANGELOG note)

Deferred to follow-up PRs

Uh oh!

github-actions Bot commented Jul 4, 2026

Uh oh!

codecov Bot commented Jul 4, 2026

Codecov Report

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

takaebato commented Jul 4, 2026 •

edited

Loading