Generate evolution scripts from which Play runs Ebean's statements

[mkurz]

• Fixes [2.7.x][5.0.x] Evolution fails for auto-generated script contains create procedures script #166
• Fixes Evolution can't be run due to "delimiter $$" DDL parsing bug - Play 3, Ebean 8 #496

Some of the statements in Ebean's DDL contain semicolons:

• the stored procedures that Ebean creates on MySQL, MariaDB and SQL Server
• the triggers for @History (Postgres, MySQL)
• the blocks with which Oracle and DB2 drop sequences in the drop-all DDL
• column comments with a semicolon

Ebean separates such statements with its own conventions, like delimiter $$ or GO. Play's evolutions split on every semicolon instead, so the generated 1.sql failed to apply, e.g. with "near 'delimiter $$'" on MySQL and MariaDB, "Incorrect syntax near 'GO'" on SQL Server or "Unterminated dollar quote" on Postgres.

Play now has a !split-semicolon directive for evolution scripts:

• Add a !split-semicolon directive to evolution scripts playframework#14346)

so Play is updated to 3.1.0-M10-3968a052-SNAPSHOT, which has it.

Play Ebean now splits the DDL with Ebean's own DdlParser, which Ebean's DdlRunner uses as well, and writes each statement with a semicolon between comments, like this (from MySQL, shortened):

-- !split-semicolon: never
CREATE PROCEDURE usp_ebean_drop_column(IN p_table_name VARCHAR(255), IN p_column_name VARCHAR(255))
BEGIN
CALL usp_ebean_drop_foreign_keys(p_table_name, p_column_name);
SET @sql = CONCAT('ALTER TABLE `', p_table_name, '` DROP COLUMN `', p_column_name, '`');
PREPARE stmt FROM @sql;
EXECUTE stmt;
END
-- !split-semicolon: always
;

• If Play already runs Ebean's statements from the DDL, the DDL stays exactly as it is. So a script only changes where Play would have split Ebean's statements differently.
• The DDL header (Ebean's ddl.header property), which can contain anything, always stays as it is.

The docs explain this. They also say that once a production database uses the generated script, the script should stop being regenerated: remove the -- Created by Ebean DDL comment or turn off play.ebean.generateEvolutionsScripts, especially before upgrading to Play Ebean 9, whose scripts differ where the DDL has such statements. Upgrading Play alone doesn't regenerate anything.

Tests:

• EbeanEvolutionScriptTest generates the DDL offline for all of Ebean's platforms, with a sequence, @History and a comment with semicolons. It checks that Play runs the statements Ebean would run, for the Ups and the Downs. Without this change, this fails on MySQL, MariaDB, SQL Server, Postgres, HANA, NuoDB, YugabyteDB, Oracle and DB2. It also covers the DDL header and DDL that stays as it is.
• EbeanEvolutionScriptDatabasesTest applies the generated script with Play's evolutions, uses the models (including @History versions and a stored procedure), and reverts the script, twice. It runs on H2 and SQLite always, and on Postgres, MySQL, MariaDB, SQL Server and Oracle when PLAY_EBEAN_TEST_<DB>_URL, _USER and _PASSWORD (or _PASSWORD_FILE) are set.
• Locally, all of these passed on Postgres 17, MySQL 8.4, MariaDB 11.4, SQL Server 2022 and Oracle 23ai Free, and testFull passed on Scala 2.13.x, 3.3.x, 3.9.x and 3.next. CI only runs the H2 and SQLite variants.

How to run the database tests (also in the README, with Docker commands for each database):

• Set these variables for each database: PLAY_EBEAN_TEST_<DB>_URL, _USER, and _PASSWORD or _PASSWORD_FILE, where <DB> is POSTGRES, MYSQL, MARIADB, SQLSERVER or ORACLE. A database without them is skipped.
• Use an empty database you can throw away. The test creates and drops its tables, and Ebean's stored procedures and history tables.

docker run -d --name play-ebean-test-postgres -e POSTGRES_PASSWORD=test -e POSTGRES_DB=play_ebean_test -p 127.0.0.1:15432:5432 postgres:17
export PLAY_EBEAN_TEST_POSTGRES_URL='jdbc:postgresql://127.0.0.1:15432/play_ebean_test' PLAY_EBEAN_TEST_POSTGRES_USER=postgres PLAY_EBEAN_TEST_POSTGRES_PASSWORD=test
sbt --server 'core/testOnly play.db.ebean.EbeanEvolutionScriptDatabasesTest'

With --server, sbt starts on its own and sees the variables. Without it, sbt can connect to an sbt server that's already running, which doesn't see them.

[mergify]

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

• Queue this pull request