Skip to content

CI build and toolchain for recent versions of gcc/llvm - #645

Draft
lgrz wants to merge 11 commits into
pisa-engine:mainfrom
lgrz:644-ci-toolchain
Draft

lgrz wants to merge 11 commits into
pisa-engine:mainfrom
lgrz:644-ci-toolchain

Conversation

@lgrz

@lgrz lgrz commented Sep 28, 2026

Copy link
Copy Markdown
Collaborator

This is a draft. This patch updates a number of submodules, modifies spdlog build config and consolidates some of the build tasks towards a minimal number of Dockerfiles; one for gcc and one clang++.

The trunk.yml and pr.yml "test" tasks should build. Although, I am still in the process of testing other build configurations from the various Pisa options.

Part of #644

- Build three versions of GCC and LLVM (clang++).

- LLVM dockerfile includes (mostly) all of the toolchain. That is, it
  includes `clang-tidy`, `clang-format`. This allows for future
  consolidation of other build tasks that use these tools.

- Remove duplicate docker files where the directory name was being used
  as the compiler version to build; compiler version is now an argument
  to `test/docker/Dockerfile.{gcc,llvm}`.

- Build images are trixie; package changes were needed (autotools
  and gumbo submodule)

- ASAN and release builds are disabled to simplify initial review and
  build testing.
    - use `*_INIT` in toolchain file

    - remove `-std=c++20` from toolchain file, already defined in main build
      file

    - libfmt: disable c++20 module libary

    - rapidcheck: inject header includes [1]; may be wise to consider
      updating rapidcheck in the near future

    [1]: https://libcxx.llvm.org/DesignDocs/HeaderRemovalPolicy.html
Migrate to catch2 v3.0.1, upgrade to minimum major release version.
More work is needed to update Catch2 to the latest version in the v3.x
series.

Fix explicit header includes (clang/llvm)

ref: https://github.com/catchorg/Catch2/blob/v3.0.1/docs/migrate-v2-to-v3.md
@elshize

elshize commented Oct 1, 2026

Copy link
Copy Markdown
Member

@lgrz Did you have a chance to look at the compilation errors? I only had a quick look and it seems like maybe related to catch upgrade, looks as if some standard headers must be included in some tests - but I didn't have much time to have a closer look.


coverage:
runs-on: ubuntu-22.04
runs-on: ubuntu-latest

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a specific reason to use latest? This will eventually be a different version, and our tests will setup will change and likely break. A good practice is to fix the version, unless we have a compelling reason not to?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, no reason to changes this and I will roll this back. I was looking at conflicting docs/examples on gh actions. It also currently triggers a build warning.

Comment thread .github/workflows/pr.yml
fail-fast: true
matrix:
compiler: [gcc14, clang19]
compiler: [gcc]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you know if it's possible to move this to include section, meaning just fully define each pair of compiler/version rather than use the product generation?

The asymmetry is non-intuitive, we have two different ways of defining the same thing.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried logical variations that I thought should make sense, but they were
invalid yaml configurations as far a gh-actions was concerned.

This was rejected by gh-actions...

matrix:
  include:
    - compiler: llvm
      version: 23
    - compiler: gcc
      version: 16

And, also this one...

matrix:
  compiler: []
  version: []
  include:
    - compiler: llvm
      version: 23
    - compiler: gcc
      version: 16

So, now I am beginning to understand that maybe the directory based approach as
the argument to the compiler to build may have made understanding the build
configuration easier.

I agree in that having multiple ways to define the same thing here is ripe for
mis-understanding and error. I will take another and see what can be done for
this. Thanks.

@lgrz lgrz Oct 3, 2026 •

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can define them all in the matrix and use exclude to remove the invalid combinations. (I prefer whitelist over blacklist though...)

Comment thread .github/workflows/pr.yml
shell: bash
run: docker run pisa ctest -VV -j 4
docker build \
-f $DOCKER_FILE \

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think bash variables should be in double quotes: -f "$DOCKER_FILE". This should technically not happen here, but it's possible for a variable to evaluate to multiple arguments if there's a space inside. If nothing else, it makes it more difficult to debug when a space is added somewhere by mistake, plus it's a general good practice to do this.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, good catch. I will include a fix for this in the next round of changes.

@lgrz

lgrz commented Oct 3, 2026

Copy link
Copy Markdown
Collaborator Author

@lgrz Did you have a chance to look at the compilation errors? I only had a quick look and it seems like maybe related to catch upgrade, looks as if some standard headers must be included in some tests - but I didn't have much time to have a closer look.

Thanks, and sorry I was not very clear on the intention of the status of
the current state of the PR. After submitting the PR I should have added a
detailed comment on the status and intention of the changes.

They are (and were) very much a work-in-progress. For some reason I thought
that getting them onto a PR on this repo would help identify additional
build problems. Sorry for not explaining things properly; and thank you for
taking the time to have a look at the current PR.

Apologies, the PR is half-baked. There are still changes for me to come back and
make another pass on. And, as per some of your review comments, I will wind
back versions of the submodules.

Part of the reason there is a half-baked PR in draft mode, is that there
were quite a few hoops to jump through to get LLVM 23 to build the project.
This started in libfmt, then spdlog, then rapidcheck, then catch2. To
eventually get it working I updated these submodules and then fiddled with
the build config and missing includes.

I have since patched the explicit header includes that are required by recent
versions of LLVM. And, now that I am at this stage, I can see a different
set of changes that can be made to fix the build system while rolling back
the submodules.

I am still working on this and I'm not sure how long it will take and when
I will have blocks of time to get through it. Hopefully in the next week or
two I imagine.

I will take your initial review feedback into account as well, thanks for this.

And, I will try not to trigger any more builds on this PR until it is ready.

@lgrz

lgrz commented Oct 12, 2026

Copy link
Copy Markdown
Collaborator Author

FYI, still working on this one and are closer to having a workable fix for recent versions of clang/llvm.

The work around for getting spdlog to build is to use set(SPDLOG_USE_STD_FORMAT ON). This avoids compile errors in the bundled libfmt that is included with spdlog.

We may need to do something similar for the second version of libfmt which is a direct dependency in pisa.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants