feat(generator)!: Fix the json path with single quote issue

[fivetran-amrutabhimsenayachit]

fixes #8251

Problem
When a JSON key contains an apostrophe (e.g. Customer's dept), sqlglot generated broken SQL — the apostrophe closed the SQL string early, producing text that couldn't even be parsed back, let alone run on a real database. On top of that, BigQuery's older JSON_EXTRACT/JSON_EXTRACT_SCALAR functions reject such keys even when properly escaped — only its JSON_VALUE/JSON_QUERY functions actually work.

Fix
Escape the apostrophe correctly wherever a JSON key gets wrapped in a SQL string, across all affected dialects (Postgres, SQLite, MySQL, BigQuery, Databricks, etc.) — so the generated SQL is always valid and parses back cleanly.
For BigQuery specifically, automatically switch JSON_EXTRACT/JSON_EXTRACT_SCALAR/JSON_EXTRACT_ARRAY to JSON_QUERY/JSON_VALUE/JSON_QUERY_ARRAY whenever the key has an apostrophe, since real BigQuery rejects the old functions for that case regardless of escaping.

Test Summary:

Queries / Commands	Dialect	Result	Matches?	Details
sqlite3 ':memory:' "SELECT json_extract('{"it''s":"simple","Customer''s department":"Dept A"}', '$."it''s"') AS short_key, json_extract('{"it''s":"simple","Customer''s department":"Dept A"}', '$."Customer''s department"') AS long_key;"	SQLite	`simple	Dept A`	Yes
duckdb -c "SELECT json_extract('{"it''s":"simple","Customer''s department":"Dept A"}', '$."it''s"') AS short_key, json_extract('{"it''s":"simple","Customer''s department":"Dept A"}', '$."Customer''s department"') AS long_key;"	DuckDB	short_key = "simple", long_key = "Dept A"	Yes	Confirms DuckDB accepts the same double-quoted key form and returns JSON values.
`psql -X -A -F '	' -t -c "SELECT jsonb_extract_path_text('{"it''s":"simple","Customer''s department":"Dept A"}'::jsonb, 'it''s') AS short_key, jsonb_extract_path_text('{"it''s":"simple","Customer''s department":"Dept A"}'::jsonb, 'Customer''s department') AS long_key;"`	Postgres	`simple	Dept A`
mysql -N -e "SELECT JSON_UNQUOTE(JSON_EXTRACT('{"it''s":"simple","Customer''s department":"Dept A"}', '$."it''s"')) AS short_key, JSON_UNQUOTE(JSON_EXTRACT('{"it''s":"simple","Customer''s department":"Dept A"}', '$."Customer''s department"')) AS long_key;"	MySQL	simple Dept A	Yes	Confirms MySQL accepts the emitted JSONPath literal and returns the expected scalar values.
snow sql -q "SELECT CAST(GET_PATH(PARSE_JSON('{"it''s":"simple","Customer''s department":"Dept A"}'), '["it''s"]') AS VARCHAR) AS short_key, CAST(GET_PATH(PARSE_JSON('{"it''s":"simple","Customer''s department":"Dept A"}'), '["Customer''s department"]') AS VARCHAR) AS long_key"	Snowflake	simple, Dept A	Yes	Confirms Snowflake accepts the bracketed double-quoted path segments for apostrophe-bearing keys.
bq query --nouse_legacy_sql "WITH data AS (SELECT JSON '{"it\'s": "simple", "Customer\'s department": "Dept A"}' AS a) SELECT JSON_VALUE(a, '$."it\'s"') AS short_key, JSON_VALUE(a, '$."Customer\'s department"') AS long_key FROM data"	BigQuery	short_key = simple, long_key = Dept A	Yes	Confirms the BigQuery fallback to the standard JSON_VALUE family works on the real engine for apostrophe-bearing keys.
GET_JSON_OBJECT('{"a": 42}', '$.a')	Databricks	42	Yes	Baseline sanity check: plain key extraction works.
GET_JSON_OBJECT(concat(...), concat(...)) with chr(39)	Databricks	42	Yes	Confirms apostrophe-key matching itself works when the key is constructed at runtime.
PARSE_JSON('{"a": 42}'):a	Databricks	42	Yes	Confirms PARSE_JSON and colon-path support exist on the runtime, separate from GET_JSON_OBJECT.
'{"it\'s": 42}'	Databricks	{"it's": 42}	Yes	Confirms backslash is the correct string-literal escape in Databricks SQL.
GET_JSON_OBJECT('{"it\'s": 42}', '$["it\'s"]')	Databricks	42	Yes	Decisive validation: this is the exact SQLGlot Databricks output shape, and it succeeds on the real engine.

[github-actions]

SQLGlot Integration Test Results

✅ All tests passed

Comparing:

• this branch (sqlglot:fix_json_path_with_single_quote @ sqlglot ce7cb94)
• baseline (main @ sqlglot d7dd2cd)

Overall

main: 182937 total, 163862 passed (pass rate: 89.6%)

sqlglot:fix_json_path_with_single_quote: 182937 total, 163862 passed (pass rate: 89.6%)

Transitions:
No change

✅ All tests passed

[georgesittas]

@fivetran-amrutabhimsenayachit same feedback re: semver as I shared here. Also, let's simplify descriptions even more; it still feels wall-of-text-ish due to the long sentences. Doesn't help with review, I just skip it instinctively.

[geooo109]

This should be escaped = f"'{escaped}'", right?

[fivetran-amrutabhimsenayachit]

If we remove the backslashes there, we stop escaping the inner single quotes at the jsonpath layer. That changes the generated sql text.
Eg:
$[\'it\'s\'] will be changed to $['it\'s']

[geooo109]

Yeap. we can also remove entirly the https://github.com/tobymao/sqlglot/pull/8309/changes#diff-362498362ae539e216cd83ebfe510977107648742f1a1bee7b2db7b5b7d27974R5404-R5407

[fivetran-amrutabhimsenayachit]

Removing this entirely would cause other dialects like Postgres, SQLite, MySQL, Presto, Trino, Snowflake, DuckDB, ClickHouse, and Redshift to break. As this is literally the fix provided for the issue reported in #8251

[fivetran-amrutabhimsenayachit]

@fivetran-amrutabhimsenayachit same feedback re: semver as I shared here. Also, let's simplify descriptions even more; it still feels wall-of-text-ish due to the long sentences. Doesn't help with review, I just skip it instinctively.

Sure, changed the scope to generator. For this particular PR, I just summarized the description from the actual ticket:#8251, which has all the details about the issue.