cleanup + Libc for gpu - #161565
Open
ZuseZ4 wants to merge 5 commits into
Open
Conversation
Collaborator
|
This PR changes how LLVM is built. Consider updating src/bootstrap/download-ci-llvm-stamp.
|
rustbot
added
A-rustc-dev-guide
Collaborator
|
|
Member
Author
|
@bors try jobs=dist-x86_64-linux |
This comment has been minimized.
This comment has been minimized.
rust-bors Bot
pushed a commit
that referenced
this pull request
Aug 22, 2026
cleanup + Libc for gpu try-job: dist-x86_64-linux
rust-bors
Bot
added
S-waiting-on-author
Contributor
|
💔 Test for 8052db3 failed: CI. Failed job:
|
This comment has been minimized.
This comment has been minimized.
Member
Author
cc @jhuber6 @Kobzol It seems like libc is using yaml for code generation. How acceptable is it to add that dependency? Once a build works, I'll measure the build times, but I'd expect libc-for-gpu to compile very quickly compared to the other components. |
Yeah, it's used to generate the headers from config files instead of tons of |
Kobzol
reviewed
Aug 24, 2026
Member
There was a problem hiding this comment.
Where should the yaml be installed, into the system Python? We do already use some Python dependencies for tidy, but that requires a non-trivial system for using a lockfile, managing a virtualenv, etc. That's probably overkill for this one. So I guess that just installing pyyaml into the job where you want to run this on CI would be enough for now.
| } | ||
| } | ||
|
|
||
| #[derive(Debug, Copy, Clone, Hash, PartialEq, Eq)] |
Member
There was a problem hiding this comment.
Could you please add a comment that briefly explains what this step does, or rather what it produces?
|
|
||
| let out_dir = builder.offload_out(target).join(gpu_target); | ||
| let lib_dir = out_dir.join("lib").join(gpu_target); | ||
| let files = vec![lib_dir.join("libc.a"), lib_dir.join("libm.a"), lib_dir.join("crt1.o")]; |
Member
There was a problem hiding this comment.
This probably doesn't work on Windows? Is the whole thing Linux-only? If yes, then we should maybe print a warning and skip it on other OSes. Not sure if Offload is also Linux only.
Member
Author
|
@bors try jobs=dist-x86_64-linux |
This comment has been minimized.
This comment has been minimized.
Originally introduced in the Enzyme build and coppied around since, but without effect for a few years by now.
rustbot
added
A-CI
Collaborator
|
This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed. Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers. |
Member
Author
|
@bors try jobs=dist-x86_64-linux |
This comment has been minimized.
This comment has been minimized.
rust-bors Bot
pushed a commit
that referenced
this pull request
Aug 24, 2026
cleanup + Libc for gpu try-job: dist-x86_64-linux
This comment has been minimized.
This comment has been minimized.
rustbot
added
the
A-LLVM
This comment has been minimized.
This comment has been minimized.
Contributor
|
💔 Test for 77addc6 failed: CI. Failed job:
|
Collaborator
|
The job Click to see the possible cause of the failure (guessed by this bot) |
Member
Author
|
@bors try jobs=dist-x86_64-linux |
This comment has been minimized.
This comment has been minimized.
rust-bors Bot
pushed a commit
that referenced
this pull request
Aug 24, 2026
cleanup + Libc for gpu try-job: dist-x86_64-linux
Contributor
|
☔ The latest upstream changes made this pull request unmergeable. Please resolve the merge conflicts by rebasing. |
Member
Author
|
@Kobzol you're going too fast with your bootstrap cleanups 😂 |
Member
|
The last one wasn't me 😆 But yeah, a lot of refactoring going on in bootstrap right now 😅 |
Contributor
|
💔 Test for 351bec1 failed: CI. Failed job:
|
Collaborator
|
The job Click to see the possible cause of the failure (guessed by this bot) |
Member
Author
|
oh fun. hdrogen seems to use a @jhuber6 since it only seems to affect 3 lines, I guess we have a good chance of changing it in libc? Their official baseline is 3.8, but I hope for 10 lines total they wouldn't mind. Since the total PR would be 10 lines, it should also be easy enough to backport. LLM found of the 3 uses: Details``` 3. The only incompatibility is PEP 584 dict union, at exactly three sites, all in hdrgen/header.py: - :142 — self.extra_standards |= other.extra_standards - :177 — descriptions = LIBRARY_DESCRIPTIONS | self.extra_standards - :219 — same expression as :177 ``` |
Sounds reasonable to me, I don't think it makes a big difference either way |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
View all comments
Disclaimer: I used an llm to implement the fixes, since I still don't like cmake.
I confirm that on a high-level these are the right fixes as far as I can tell, and I reviewed/refactored it.
After #161431 I rebuilt on another server, and got
ld.lld: error: Invalid summary version 14. Version should be in the range [1-12]., since my lld was still on 22, but our submodule now is 23. The issue only shows up when we build the twolibomptarget-nvptxandlibomptarget-amdgpubitcode files (or their libompdevice versions), which we have never been using, and results from https://arxiv.org/abs/2608.13759 also show that we don't need them for now. They are meant to be called from within GPU kernels, so their build was always a bit mroe complicated. The first commit gets rid of them.In theory, we could now even get rid of our clang requirements, but that'd be too easy. So I instead used the second commit to add a new libc-for-gpu component, which allows running (most of?) the standard library on the GPU. Similar to the just removed libompdevice it's meant to be called from kernels, but this time not for optimized algorithms, but for libc functionality. I've tried that previously after our last update to LLVM 22 in #152011, but back then I never got it to work, only to build. This time I also managed to print hello-world from a GPU, so I'm more comfortable merging it. We are not wiring it up yet, and we're also not distributing libc-for-gpu yet, I leave those for a follow-up pr.
closes: #152011
cc @sgasho @Sa4dUs
r? kobzol