Skip to content

fix(macos): make matching python3-config available in build/test venvs#2922

Draft
henryiii wants to merge 3 commits into
pypa:mainfrom
henryiii:fix/macos-python3-config
Draft

fix(macos): make matching python3-config available in build/test venvs#2922
henryiii wants to merge 3 commits into
pypa:mainfrom
henryiii:fix/macos-python3-config

Conversation

@henryiii

Copy link
Copy Markdown
Contributor

🤖 AI text below 🤖

Problem

On macOS, cibuildwheel builds and tests inside a virtualenv and only adds the venv's bin directory to PATH. Virtualenvs do not contain a python3-config (or versioned python{X.Y}-config) script. The python.org framework installs (/Library/Frameworks/Python.framework/Versions/{version}/bin/) do ship python3-config, but that directory is never placed on PATH.

As a result, complex C/C++ extension builds that invoke python3-config (to obtain include dirs / link flags) fall through to an unrelated python3-config found elsewhere on PATH — typically the system or Homebrew Python (e.g. /opt/homebrew/bin/python3-config). This produces broken wheels or ABI mismatches.

Root cause

cibuildwheel/venv.py creates the venv and activate_virtualenv only prepends the venv bin to PATH. No python*-config exists in that bin, and the matching one next to the base interpreter is not exposed.

Fix

After the venv is created (POSIX only), symlink the base interpreter's python*-config scripts into the venv's bin directory so the correct one is found first on PATH. This is done in a small helper _symlink_python_config_scripts(base_python, venv_bin) called from virtualenv().

  • Uses a python*-config glob next to the base interpreter, so both the unversioned python3-config and the versioned python3.X-config are linked when present.
  • Pre-existing entries in the venv bin are never overwritten.
  • OSError during symlink creation is suppressed, so the build is never failed by this best-effort step.

Edge cases / limitations

  • Windows: skipped entirely (config scripts are a POSIX concept; guarded by _IS_WIN).
  • PyPy / GraalPy: these ship pypy3-config / similar, which the python*-config glob does not match, so nothing is linked — by design. The build is never failed when no config script exists.
  • uv-managed interpreters: may not ship a config script; handled gracefully (nothing linked).
  • This helper runs for all POSIX virtualenv() callers (macOS, iOS, Android, pyodide, audit), but only acts when a matching python*-config actually exists next to the base interpreter, so non-macOS flows are effectively unaffected.

Testing

Added unit_test/venv_test.py covering the helper directly (the full virtualenv() path requires network/installs and a real framework Python, so a true end-to-end unit test is impractical):

  • a base interpreter with python3-config + python3.X-config gets both linked into the venv bin;
  • a PyPy-style base (only pypy3-config) links nothing;
  • a pre-existing real python3-config in the venv bin is left untouched.

All three pass; the full unit_test suite passes (791 passed). ruff, ruff-format, codespell, and flake8-lazy pass on the changed files; mypy is clean on venv.py apart from the pre-existing uv import-stub note that the project config already ignores.

Note for reviewers: the helper is best-effort (uses symlinks, suppresses OSError). Open question: should it prefer copying over symlinking, or restrict the behavior to the CPython/macOS path only rather than all POSIX virtualenv() callers? Happy to narrow the scope if preferred.

Closes #2021

henryiii added 2 commits June 16, 2026 22:24
Virtualenvs do not contain a python3-config (or versioned
python{X.Y}-config) script, and cibuildwheel only puts the venv bin
directory on PATH. The python.org framework builds ship these scripts
next to the base interpreter, but that directory is never added to PATH,
so C/C++ extensions that invoke python3-config picked up an unrelated
(system/Homebrew) python3-config, producing broken or ABI-mismatched
wheels.

After creating the venv on POSIX platforms, symlink the base
interpreter's python*-config scripts into the venv bin so the correct
one is found first on PATH. Interpreters without such a script (PyPy,
GraalPy, some uv-managed builds) are handled gracefully and never fail
the build.

Assisted-by: ClaudeCode:claude-opus-4.8
Fixes ruff (ANN/TC) and mypy (no-untyped-def) failures in CI.

Assisted-by: ClaudeCode:claude-opus-4.8
@henryiii henryiii force-pushed the fix/macos-python3-config branch from 4b8d3a5 to 968ce48 Compare June 17, 2026 02:25
Refactor _symlink_python_config_scripts docstring to clarify behavior regarding linking python-config scripts.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

macOS: venvs lacking python3-config causes some builds to fail

1 participant