Repository navigation
gh-156049: Add support for building with HWASAN - #156721
Merged
Merged
Conversation
Co-authored-by: Florian Mayer <fmayer@google.com> Co-authored-by: Anna <araslanova.anna.a@gmail.com>
StanFromIreland
requested review from
AA-Turner,
corona10,
emmatyping,
erlend-aasland and
itamaro
as code owners
August 31, 2026 15:21
This was referenced Aug 31, 2026
Documentation build overview
132 files changed ·
|
fmayer
reviewed
Aug 31, 2026
Contributor
|
lgtm |
Member
|
Hi. You asked me to review this change, but I don't know how to test it manually. Can you provide a way to introduce a bug on purpose and explain how to test that HWASAN detects it properly? |
vstinner
reviewed
Sep 22, 2026
vstinner
reviewed
Sep 22, 2026
| Note that on x86-64 this uses `page aliasing | ||
| <https://clang.llvm.org/docs/HardwareAssistedAddressSanitizerDesign.html#supported-architectures>`_, | ||
| which only tags heap allocations and is unsafe for programs that ``fork()``, | ||
| including much of the test suite. |
Member
There was a problem hiding this comment.
Do you know which GCC and clang versions added support for this sanitizer?
Member
Author
There was a problem hiding this comment.
LLVM (the reference implementation) supports it on AArch64, and additionally on x86-64 since LLVM 13 with page aliasing. GCC supports it only on AArch64, since GCC 11.
Member
Author
Hi, thanks for the review! If you mean just a trivial example, try for example applying this patch which adds a little heap-use-after-free: diff --git a/Modules/zlibmodule.c b/Modules/zlibmodule.c
index a7fefb1fec5..e7fe3131052 100644
--- a/Modules/zlibmodule.c
+++ b/Modules/zlibmodule.c
@@ -1987,6 +1987,9 @@ static unsigned int
zlib_crc32_impl(PyObject *module, Py_buffer *data, unsigned int value)
/*[clinic end generated code: output=b217562e4fe6d6a6 input=1229cb2fb5ea948a]*/
{
+ volatile char *probe = PyMem_RawMalloc(1);
+ PyMem_RawFree((void *)probe);
+ *probe = '\0';
/* Releasing the GIL for very small buffers is inefficient
and may lower performance */
if (data->len > 1024*5) {Then compile Python with the flag: And run: Which for me results in: |
Co-authored-by: Victor Stinner <vstinner@python.org>
Member
Author
|
I pushed a commit which incorporates the fix in #157934 for MSan/ASan. |
Member
Author
|
@emmatyping can you please take a look? |
Member
|
Would it be crazy to add an unit test for the test? Add a function in _testcapi which triggers HWASAN (as you shown), and make sure that it fails if HWASAN is used. |
vstinner
reviewed
Oct 5, 2026
| [x86_64|amd64], [hwasan_flags="$hwasan_flags -fsanitize-hwaddress-experimental-aliasing"] | ||
| ) | ||
| AX_CHECK_COMPILE_FLAG([$hwasan_flags],[ | ||
| BASECFLAGS="$hwasan_flags -fno-omit-frame-pointer $BASECFLAGS" |
Member
There was a problem hiding this comment.
What's the rationale for adding -fno-omit-frame-pointer? Is it required by -fsanitize=hwaddress? Why not putting it in hwasan_flags?
Member
Author
There was a problem hiding this comment.
It isn't required, it's just there for usable stack traces in the reports as the sanitizer runtime uses its frame-pointer-based unwinder for the allocation/free stacks. The --with-address-sanitizer and --with-memory-sanitizer blocks add it for the same reason.
Member
There was a problem hiding this comment.
Oh ok, I didn't notice that it's also used by other sanitizers.
Since PEP 831 in Python 3.15, Python is built with -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer by default, so it should not make a big difference to add -fno-omit-frame-pointer anyway :-)
Member
Author
Sure, I added such a test. It also works under ASan. |
vstinner
reviewed
Oct 5, 2026
vstinner
left a comment
vstinner
left a comment
Member
There was a problem hiding this comment.
LGTM. The change looks complete and it's well documented. Thanks to updating libregrtest ;-)
| def check_sanitizer(*, address=False, memory=False, ub=False, thread=False, | ||
| function=True): | ||
| def check_sanitizer(*, address=False, hwaddress=False, memory=False, ub=False, | ||
| thread=False, function=True): |
Member
There was a problem hiding this comment.
That's a long list of sanitizers! 😃 It's cool that we supported all of them!
vstinner
approved these changes
Oct 5, 2026
Member
|
Oh, multiple GitHub Action jobs failed with "Cancelled after 15m". I'm not sure what happened. Anyway, I clicked on re-run failed jobs. |
Member
Author
|
There was a GitHub incident, thank you! |
|
Member
Author
|
Aargh, I wrote #158932 to fix the test. |
Member
|
@StanFromIreland: Ah, test_capi.AddressSanitizerTests.test_buffer_overflow() failed on x86-64 MacOS Intel ASAN NoGIL 3.x because Problem: the buffer overflow is not detected using mimalloc: `PYTHONMALLOC=mimalloc ./python -c 'import _testcapi; _testcapi.pymem_buffer_overflow()' exits successfully. I suppose that skipping the test on Free Threading is an acceptable trade-off for now. Note: the overflow is detected using |
Member
Author
However, that is CPython's own debug-hooks checker, not {HW}ASan. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This supersedes GH-149166 and GH-156050, pulling in their changes and adding a
--with-hwaddress-sanitizerconfigure option as well as a few little miscellaneous changes for better support. Note that x86-64 has to use clang's page aliasingmode, which shares the heap across
fork(), so the test suite is only reliableon AArch64.
@AnnaAr321 and @fmayer I'd appreciate your reviews :-)