Skip to content

ci: build only the Linux native image per PR#997

Merged
nielspardon merged 1 commit into
substrait-io:mainfrom
nielspardon:ci/speed-up-native-image-builds
Jul 9, 2026
Merged

ci: build only the Linux native image per PR#997
nielspardon merged 1 commit into
substrait-io:mainfrom
nielspardon:ci/speed-up-native-image-builds

Conversation

@nielspardon

Copy link
Copy Markdown
Member

What

Speeds up PR builds by taking the slow, 10x-billed macOS native-image build off the PR critical path, without losing macOS coverage.

The native image built on PRs is only used to run the smoke tests (smoke.sh / tpch_smoke.sh) and is uploaded as a throwaway PR artifact — the shipped release binaries are compiled separately in release.yml. That means the runtime performance of the PR binary is irrelevant, which unlocks the cheapest win below.

Changes

  1. Quick build mode on PRs. The Linux native image is now built with --quick-build-native (GraalVM -Ob), which skips the expensive optimizing compiles and typically cuts native-image build time ~30–40%. The tradeoff — a slower/larger binary — doesn't matter because the PR binary is only smoke-tested.
  2. Linux-only per PR. The macOS-latest matrix entry is dropped from the PR native-image job. The Linux build already validates the GraalVM configuration and runs the full smoke suite; macOS-specific native regressions are rare and now covered out-of-band (see below). macOS runners are the long pole and bill at 10x.
  3. Runs in parallel with tests. Removed needs: [java, integration] so the native-image job starts immediately instead of waiting for the Java + integration jobs to finish.

To keep macOS native coverage, a new native-image-macos.yml workflow builds and smoke-tests the macOS native image (full optimization, mirroring the release build) on push to main (per-merge signal) plus a weekly backstop and workflow_dispatch. Regressions are still caught promptly, and before the Sunday release build — just not on the per-PR critical path.

Notes for reviewers

  • Required status checks: the per-PR macOS native check (Build Isthmus Native Image (macOS-latest)) no longer runs. If branch protection requires it, that setting needs updating; the Linux check (Build Isthmus Native Image (ubuntu-latest)) is unchanged.
  • --quick-build-native was verified to be a recognized option on the nativeCompile task (gradle help --task nativeCompile → "Enables quick build mode").
  • release.yml is intentionally left untouched — release binaries stay fully optimized on both Linux and macOS.

Closes #768

🤖 Generated with AI

Take the slow, 10x-billed macOS native-image build off the PR critical
path to speed up PR builds:

- Build only the Linux native image on PRs, in quick-build mode
  (--quick-build-native / -Ob). The PR binary is only smoke-tested and
  never shipped, so trading binary runtime performance for a much faster
  build is free.
- Run the native-image job in parallel with the java/integration jobs
  instead of after them (dropped the `needs`).
- Move macOS native-image coverage to a dedicated workflow that runs on
  push to main plus a weekly backstop, so macOS-specific native
  regressions are still caught before the Sunday release.

Closes substrait-io#768

@benbellick benbellick left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀 this has been a long time coming. Thanks for taking a look!

@nielspardon nielspardon merged commit c69e10d into substrait-io:main Jul 9, 2026
13 checks passed
@nielspardon nielspardon deleted the ci/speed-up-native-image-builds branch July 9, 2026 18:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci: building isthmus native image takes long time

2 participants