0.10.4

Notable Changes

• Added rate limiting support for kafka-safe and upsert-kafka-safe sources via the new scan.rate-limit.records-per-second table option.
• Switched the bundled HTTP connector from com.getindata:flink-http-connector to the Apache variant org.apache.flink:flink-connector-http.
• Updated the HTTP connector version from 0.25.0 to 1.0.0-2.2.

Dependency Updates

• Bumped org.sonatype.central:central-publishing-maven-plugin from 0.10.0 to 0.11.0.
• Bumped org.junit:junit-bom from 6.1.0 to 6.1.1.

Full Changelog

0.10.3...0.10.4

0.10.3

Highlights

• Added adaptive Kafka source watermarks for kafka-safe and upser-kafka-safe sources using Kafka record timestamps and SOURCE_WATERMARK().
• Added idle source-watermark advancement so event-time windows can close after Kafka traffic stops.
• Added standalone env-utils module with reusable environment variable resolution utilities.
• Improved CliRunner fatal error logging so uncaught exceptions are visible in JobManager pod logs.

Kafka Watermarks

New configurable Kafka source watermark options:

• scan.source-watermark.min-records
• scan.source-watermark.min-out-of-orderness
• scan.source-watermark.max-out-of-orderness
• scan.source-watermark.out-of-orderness-quantile
• scan.source-watermark.idle-advance-timeout
• scan.source-watermark.idle-advance-safety-margin
• scan.source-watermark.idle-advance-broker-check-timeout
• scan.source-watermark.idle-advance-broker-check-ttl

Example usage:

ts TIMESTAMP_LTZ(3) METADATA FROM 'timestamp',
WATERMARK FOR ts AS SOURCE_WATERMARK()

Fixes

• Logged uncaught CliRunner exceptions before exit, improving diagnostics for Kubernetes/Flink JobManager deployments.

Dependency Updates

• Kafka connector: 4.0.1-2.0 to 5.0.0-2.2
• Iceberg: 1.10.2 to 1.11.0
• Jackson BOM: 2.21.3 to 2.22.0
• AWS MSK IAM Auth: 2.3.6 to 2.3.7
• JUnit BOM: 6.0.3 to 6.1.0
• Maven Surefire Plugin: 3.5.5 to 3.5.6
• Maven Dependency Plugin: 3.10.0 to 3.11.0
• JaCoCo: 0.8.14 to 0.8.15

Full Changelog

0.10.2...0.10.3

0.10.2

What's Changed

• feat: Option to skip replaying events after failure recovery in metronome source by @ferenc-csaky in #337
• feat: Rework metronome source to emit AT MOST one event per sec by @ferenc-csaky in #338
• Bump org.apache.maven.plugins:maven-enforcer-plugin from 3.6.2 to 3.6.3 by @dependabot[bot] in #340
• Bump iceberg.version from 1.10.1 to 1.10.2 by @dependabot[bot] in #339
• chore: Bump Flink version to 2.2.1 by @ferenc-csaky in #341

Full Changelog: 0.10.1...0.10.2

0.10.1

What's Changed

• Bump jdbi3.version from 3.52.0 to 3.52.1 by @dependabot[bot] in #315
• Bump io.sundr:sundr-maven-plugin from 0.230.2 to 0.240.1 by @dependabot[bot] in #314
• Bump io.sundr:sundr-maven-plugin from 0.240.1 to 0.300.0 by @dependabot[bot] in #316
• feat: Configurable Postgres ON CONFLICT behavior by @ferenc-csaky in #318
• Bump testcontainers.version from 2.0.4 to 2.0.5 by @dependabot[bot] in #319
• Bump org.projectlombok:lombok from 1.18.44 to 1.18.46 by @dependabot[bot] in #320
• feat!: Ship runner uber JAR in Flink lib/ by @ferenc-csaky in #317
• feat: Add sql-runner wrapper script to simplify flink run by @ferenc-csaky in #321
• feat: Add metronome source implementation by @ferenc-csaky in #324
• fix: Wire SafeUpsertKafkaDynamicTableFactory by @ferenc-csaky in #325
• Bump org.postgresql:postgresql from 42.7.10 to 42.7.11 by @dependabot[bot] in #322
• Bump jdbi3.version from 3.52.1 to 3.53.0 by @dependabot[bot] in #323
• feat: Improve metronome connector no more split signaling by @ferenc-csaky in #327
• Bump software.amazon.msk:aws-msk-iam-auth from 2.3.5 to 2.3.6 by @dependabot[bot] in #330
• Bump log4j.version from 2.25.4 to 2.26.0 by @dependabot[bot] in #329
• feat: Ship async-profiler in the Docker image by @ferenc-csaky in #326
• feat: Use current timestamp when metronome replays or catches up by @ferenc-csaky in #332
• feat: use full JDK by @ferenc-csaky in #335
• test: Validate proper dynamic table factory wiring by @ferenc-csaky in #334
• chore: Minor Dockerfile optimizations by @ferenc-csaky in #336

Full Changelog: 0.10.0...0.10.1

0.10.0

What's Changed

• Bump org.apache.maven.plugins:maven-resources-plugin from 3.4.0 to 3.5.0 by @dependabot[bot] in #303
• Bump org.apache.maven.plugins:maven-shade-plugin from 3.6.1 to 3.6.2 by @dependabot[bot] in #302
• Bump org.projectlombok:lombok from 1.18.42 to 1.18.44 by @dependabot[bot] in #306
• Bump mockito.version from 5.22.0 to 5.23.0 by @dependabot[bot] in #305
• fix: Correct identifier for upsert-kafka-safe connector by @ferenc-csaky in #307
• Bump jdbi3.version from 3.51.0 to 3.52.0 by @dependabot[bot] in #311
• Bump testcontainers.version from 2.0.3 to 2.0.4 by @dependabot[bot] in #310
• Bump log4j.version from 2.25.3 to 2.25.4 by @dependabot[bot] in #312
• feat: Add officially released Flink 2.x compatible CDC connectors by @ferenc-csaky in #313

Full Changelog: 0.9.5...0.10.0

0.9.5

What's Changed

• Add Hadoop core-site.xml for S3A IRSA credential support by @velo in #290
• Bump org.junit:junit-bom from 6.0.2 to 6.0.3 by @dependabot[bot] in #291
• Bump org.postgresql:postgresql from 42.7.9 to 42.7.10 by @dependabot[bot] in #292
• Bump maven-surefire-plugin.version from 3.5.4 to 3.5.5 by @dependabot[bot] in #293
• Bump com.jayway.jsonpath:json-path from 2.10.0 to 3.0.0 by @dependabot[bot] in #294
• Bump mockito.version from 5.21.0 to 5.22.0 by @dependabot[bot] in #297
• Support Bash-style default values in EnvVarResolver by @ferenc-csaky in #299

Full Changelog: 0.9.4...0.9.5

0.9.4

What's Changed

• Include a custom SQL Server CDC connector that is patched to Flink 2.x by @ferenc-csaky in #280
• Pin Netty version to match with Flink by @velo in #281
• Bump org.assertj:assertj-core from 3.27.6 to 3.27.7 by @dependabot[bot] in #279
• Handle Row objects without named fields properly in to_jsonb by @ferenc-csaky in #284
• Add pre-commit code formatting hook by @ferenc-csaky in #285
• Do not show REST not found errors in log by @ferenc-csaky in #286
• Bump org.apache.maven.plugins:maven-dependency-plugin from 3.9.0 to 3.10.0 by @dependabot[bot] in #289

Full Changelog: 0.9.3...0.9.4

0.9.3

What's Changed

• Bump org.postgresql:postgresql from 42.7.8 to 42.7.9 by @dependabot[bot] in #274
• Override RexNodeJsonDeserializer until loading PTFs via compiled plan is patched in Flink by @ferenc-csaky in #278

Full Changelog: 0.9.2...0.9.3

0.9.2

What's Changed

• Ship hadoop-common and hadoop-mapreduce-client-core to Flink lib by @velo in #266
• Bump jdbi3.version from 3.50.0 to 3.51.0 by @dependabot[bot] in #261
• Bump mockito.version from 5.20.0 to 5.21.0 by @dependabot[bot] in #262
• Bump testcontainers.version from 2.0.2 to 2.0.3 by @dependabot[bot] in #263
• Bump log4j.version from 2.25.2 to 2.25.3 by @dependabot[bot] in #264
• Bump iceberg.version from 1.10.0 to 1.10.1 by @dependabot[bot] in #268
• Bump org.codehaus.mojo:exec-maven-plugin from 3.6.2 to 3.6.3 by @dependabot[bot] in #265
• Bump spotless plugin to 3.1.0 by @ferenc-csaky in #269
• Bump org.sonatype.central:central-publishing-maven-plugin from 0.9.0 to 0.10.0 by @dependabot[bot] in #270
• Bump org.junit:junit-bom from 6.0.1 to 6.0.2 by @dependabot[bot] in #271
• Shade aws-msk-iam-auth to the runner uber JAR by @ferenc-csaky in #272
• Package flink-parquet into the flink-sql-runner JAR by @ferenc-csaky in #273
• Package flink-connector-http into the flink-sql-runner JAR by @ferenc-csaky in #275

Full Changelog: 0.9.1...0.9.2

0.9.1

What's Changed

• Add to_append_stream PTF by @ferenc-csaky in #259
• Ship aws-msk-iam-auth to Flink lib by @ferenc-csaky in #260

Full Changelog: 0.9.0...0.9.1