Repository navigation
Conversation
1 task
Byron
force-pushed
the
gix-backend
branch
4 times, most recently
from
October 7, 2026 13:25
594e7dd to
105114d
Compare
Byron
pushed a commit
that referenced
this pull request
Oct 7, 2026
PR #2274's Python 3.12 CI jobs spent 57m 25s in Windows `pytest` versus 5m 2s on Ubuntu, with 85,777 and 82,683 `Git.execute` launches respectively. A local profile of the file-handle and recursive submodule regressions spent about 65 of 77 seconds executing Git, including 44 seconds in repository construction. Repeated process startup makes the separate metadata probes expensive. Query reference format, object format, bare status and the absolute common directory in one `git rev-parse` call on the CLI construction path. Keep the path last and split only the three fixed scalar fields so embedded newlines remain intact. Git still computes every value. Supported Gix discovery keeps its native metadata and existing CLI format-validation query, with unchanged native launch ceilings and fallbacks. The focused selection drops from 2,447 to 2,009 launches: three saved for each of 146 repository opens. Three paired Windows measurements of 20 opens each reduce the median from 300.74 ms to 221.18 ms per open, with matching metadata and 11 versus eight launches. Add `--durations=30` to CI so future slow tests are visible, and record the profiling evidence and reproduction requirements in `doc/gix-backend.md`. Cover both object formats, both reference formats, bare and non-bare repos, spaces and embedded newlines. The eight applicable Windows cases fail before the fix and pass afterward; all 16 cases pass on Ubuntu WSL. The optional Windows Gix selection reproduces the same 17 failures on the original and patched sources. Document the demonstrated path-formatting and undecodable `commondir` differences under `GIX-14`, distinguishing them from Windows test assumptions. Keep these bugs open pending compatible upstream behavior. Validation: the full Windows CLI suite passes with 1,662 passed, 60 skipped, nine expected failures, two unexpected passes and 40 passing subtests. It records 77,732 launches in 3,715.16s. The initial baseline records 86,032 in 3,990.28s; its only two failures pass on unchanged code after restoring CI's `core.autocrlf=true` and placing `pytest`'s temporary directory outside a Git checkout. The overlapping full runs are not a controlled elapsed-time comparison. Repository-wide Ruff lint/format, all pinned pre-commit hooks, `mypy` (47 files), `basedpyright` (zero errors or warnings), the Sphinx HTML build and `git diff --check` pass.
Byron
pushed a commit
that referenced
this pull request
Oct 8, 2026
PR #2274's Python 3.12 CI jobs spent 57m 25s in Windows `pytest` versus 5m 2s on Ubuntu, with 85,777 and 82,683 `Git.execute` launches respectively. A local profile of the file-handle and recursive submodule regressions spent about 65 of 77 seconds executing Git, including 44 seconds in repository construction. Repeated process startup makes the separate metadata probes expensive. Query reference format, object format, bare status and the absolute common directory in one `git rev-parse` call on the CLI construction path. Keep the path last and split only the three fixed scalar fields so embedded newlines remain intact. Git still computes every value. Supported Gix discovery keeps its native metadata and existing CLI format-validation query, with unchanged native launch ceilings and fallbacks. The focused selection drops from 2,447 to 2,009 launches: three saved for each of 146 repository opens. Three paired Windows measurements of 20 opens each reduce the median from 300.74 ms to 221.18 ms per open, with matching metadata and 11 versus eight launches. Add `--durations=30` to CI so future slow tests are visible, and record the profiling evidence and reproduction requirements in `doc/gix-backend.md`. Cover both object formats, both reference formats, bare and non-bare repos, spaces and embedded newlines. The eight applicable Windows cases fail before the fix and pass afterward; all 16 cases pass on Ubuntu WSL. The optional Windows Gix selection reproduces the same 17 failures on the original and patched sources. Document the demonstrated path-formatting and undecodable `commondir` differences under `GIX-14`, distinguishing them from Windows test assumptions. Keep these bugs open pending compatible upstream behavior. Validation: the full Windows CLI suite passes with 1,662 passed, 60 skipped, nine expected failures, two unexpected passes and 40 passing subtests. It records 77,732 launches in 3,715.16s. The initial baseline records 86,032 in 3,990.28s; its only two failures pass on unchanged code after restoring CI's `core.autocrlf=true` and placing `pytest`'s temporary directory outside a Git checkout. The overlapping full runs are not a controlled elapsed-time comparison. Repository-wide Ruff lint/format, all pinned pre-commit hooks, `mypy` (47 files), `basedpyright` (zero errors or warnings), the Sphinx HTML build and `git diff --check` pass.
GitPython's Python implementations of repository storage couple its behavior to on-disk formats and object-ID widths. Delegate repository discovery, configuration, references, reflogs, object storage, tree construction, index operations, and revision parsing to `git.cmd.Git` so the default backend works with SHA-1/SHA-256 objects and files/reftable references. Require Git 2.52 or newer, retain the deprecated `GitDB` as an explicit choice, and preserve raw `repo.git` access. Use existing unsafe option/protocol primitives together with operand validation, NUL-framed records, and protected option ordering. Suppress implicit hooks, filesystem monitors, maintenance, lazy fetching, external diffs, and default text conversion in managed plumbing. Explicit commit hooks remain supported; signing and custom archive commands require opt-in. Preserve the established clean/smudge-filter behavior of existing worktree operations. Keep common workflows and map recognizable native failures to established exceptions. Remove or restrict low-level binary index/tree, raw reflog, precompressed object, and direct storage mutation APIs that cannot be exposed faithfully through Git. Preserve semantic index edits through private native indexes, discover worktree storage through Git, and reconnect retained submodule metadata without assuming its object or reference backend. Document the API and CLI changes in `doc/source/changes.rst`, add format and injection coverage, update minimum-Git CI and fuzz tooling, and bound the existing throughput benchmarks for subprocess-based operations. This work is planned for GitPython 3.4 some weeks before Git 3's official release, allowing users to test this branch and report compatibility feedback beforehand. Validation: the full pytest run produced 1,328 passes, 79 skips, one expected failure, and three submodule failures that were resolved and passed reruns. A fresh submodule/offline run had 224 passes; its two metadata-alias failures were fixed and retested, followed by eight passing retained-metadata checks. The Git 2.52 matrix passed 127 tests, plus four later quoted-branch checks. Ruff, mypy, basedpyright, Sphinx with warnings as errors, Python 3.8 package smoke tests, and deterministic fuzz-harness/version-guard checks pass. The updated fuzzing Docker image was not built and no long fuzz campaign was run. After rebasing onto the 3.2.1 security fixes, Alpine and Ubuntu CI failed `TestSubmodule.test_update_rejects_parent_component_in_name`: a NUL-bearing name reached `Git._check_operand()` first and raised `UnsafeOptionError` instead of the existing `ValueError` contract. Run the established submodule name and Windows filename validation before the CLI operand guard, retaining both exception compatibility and protection against command injection. The 22 focused name/Windows-path regressions, Ruff lint and formatting, and `git diff --check` pass locally. The `test (windows, 3.12, gix)` CI job at `57cc270` failed the nested submodule tutorial with `unable to move ... to ...: Directory not empty`. `_connect_module()` unconditionally ran `git init --separate-git-dir` after cloning had already connected the checkout. Git's initialization attempts to rename the existing metadata directory onto itself, which can fail on Windows even when both paths identify the same repository. Resolve an existing gitfile with `find_submodule_git_dir()` and compare its target by filesystem identity before reinitializing. This uses the selected Git/Gix discovery implementation and handles path aliases without decoding Git metadata in Python. Preserve a valid connection while still updating `core.worktree`; Git continues to create missing connections. The shared helper covers clones, retained metadata and moved submodule checkouts. Keep Git's object/reference-format validation before this guard, including rejection of unknown repository extensions before configuration changes. A regression models the Windows self-rename failure and verifies that a moved checkout retains its gitfile, object identity and contents while its worktree setting is repaired. It fails before the guard and passes after it. A second regression rejects unknown repository extensions without modifying configuration, preserving the original initialization precondition. Validation: both regressions and the four storage-format combinations passed at the introducing commit (six tests); this ancestor predates the Gix backend. On the restored branch, the affected tests, metadata-symlink cases and original `Tutorials.test_submodules` passed with Gix first (49 tests in 18.53 seconds) and CLI afterward (49 tests in 36.87 seconds). Mypy passed for all 47 source files; the pinned `pre-commit` hooks and `git diff --check` also passed. The original tutorial retains coverage in Windows Gix CI.
The Cygwin performance job failed all six setup phases because the Cygwin package currently supplies Git 2.51.0, below the new Git 2.52 minimum. The same mismatch prevents the regular Cygwin suite from opening repositories. Build upstream Git `v2.52.0` with Cygwin tools and install it in `/usr/local` before preparing the fixtures. Include the HTTPS development dependencies so the existing clone and remote tests retain network transport support. Verify that the selected `git` is the built version before continuing. Validation: the workflow parses as YAML, the added shell block passes `bash -n` and ShellCheck, and `git diff --check` passes. The native Cygwin build and test suites require the Windows CI runner.
The macOS Python 3.8 and 3.14 CI jobs each passed 1,348 tests but failed `test_refresh_with_good_relative_git_path_arg`. Installing the supported Homebrew Git exposes `/opt/homebrew/opt/git/bin` on `PATH`; changing into that directory resolves it to the versioned Cellar directory. Compute the expected relative executable path from the current directory after changing into it. This preserves the documented `Git.refresh()` behavior and executable symlinks while removing the test's assumption that `shutil.which()` and `os.getcwd()` retain the same directory spelling. Validation: reproduced the failure using a symlinked directory on `PATH`. All 36 refresh tests pass with both ordinary and symlinked `PATH` values. Ruff lint, Ruff format, and `git diff --check` pass.
The Windows Python 3.8 CI job reported 42 failures and 179 setup errors. Most submodule failures shared a discovery bug: Git resolves relative gitfile targets using forward slashes even when the caller supplies a native Windows path. Normalize Git-facing discovery operands with the existing platform helper, and account for native separators in Git's worktree registry output. Use matching `surrogateescape` codecs for Git protocol paths so undecodable tree names round-trip without Windows filesystem encoding changing their bytes. Verify path/stage, mode, and object ID after materializing a private index: `git update-index --index-info` can exit successfully while dropping Windows-incompatible names. Raise `ValueError` before publishing such an index and document the platform restriction in `changes.rst`. Keep unusual names in object-only tests when the host cannot represent them in a checkout. Use native-valid paths for worktree tests, assert rejection of unsupported index names and quoted file references, and keep full quoted reference coverage with reftable. Fix separator and LF assumptions in config, URL, and packed-reference fixtures. Close test-owned repositories before submodule removal and make the fake Windows Git executable discoverable without shell execution. Also restore root paths for `Repo.tree()` results resolved directly from tree IDs while preserving explicit subtree paths. Validation: 116 repository tests and 10 focused submodule tests passed; index/helper tests passed 93 with 2 platform skips; the focused format/safety run passed 72; config/reference/remote tests passed 57 with 24 subtests; 36 refresh tests and 2 revision regressions passed. The Git 2.52 targeted index matrix passed 19 tests. Ruff, mypy, basedpyright, Sphinx with warnings as errors, and `git diff --check` pass. Native Windows validation awaits CI. The rebased Windows Python 3.14 job then failed only `test_quoted_remote_and_submodule_names`: registering an existing checkout still validates the submodule name as a possible metadata path, so `quoted"module` correctly raises `ValueError`. Create that fixture using a filesystem-safe name, then rename its `.gitmodules` section through the config writer. This exercises quoted config parsing without requesting a Windows-invalid metadata name or weakening filename validation. The focused remote and Windows destination-name tests passed 24 cases on macOS; native Windows confirmation awaits CI.
The partial Cygwin fast-suite log exposed a failure in `test_valid_unusual_index_names_round_trip`: native Git omitted a literal backslash filename. Cygwin Git recognizes Windows separators and applies NTFS path protection, even though Python reports a POSIX platform. Move that case into the existing unsupported-name assertions for Cygwin. Check that GitPython raises `ValueError`, preserves the published index, and removes its lock. Other POSIX systems retain the round-trip case; Windows retains the control-character and colon rejection cases. Document the Cygwin restriction in `changes.rst`. Validation: three focused index tests pass locally. The Cygwin and Windows test branches pass with only Git's ignored-record behavior simulated; native Cygwin verification awaits CI. Ruff lint, formatting, and `git diff --check` pass.
The Windows Python 3.8 partial CI log showed two failures in `test_submodule_allows_existing_metadata_symlinks`: preparing update and move aliases raised `PermissionError` before invoking GitPython. Git marks the submodule's `.git` file hidden; Python's `write_text()` attempts to recreate it, which Windows rejects for an existing hidden file. Open that fixture file with `r+`, write the replacement target, and truncate it. This preserves the hidden attribute while replacing the full contents. The separate fixture that creates a previously absent gitfile is unchanged. Validation: all six native/windows37 update, move, and remove alias modes pass locally. Ruff lint, formatting, and `git diff --check` pass. The Windows-specific file-attribute behavior will be verified by CI.
The Windows Python 3.15 CI job failed to set up twelve missing-submodule cases because `shutil.rmtree()` cannot remove read-only loose Git objects. The fixture deliberately removes retained metadata to model an absent submodule, so use the existing `git.util.rmtree()` helper, which clears read-only attributes when retrying Windows deletions. All twelve affected cases pass locally, along with Ruff lint and format checks. Production behavior and test coverage are unchanged.
The Windows Python 3.15 CI job failed to remove submodule checkouts because persistent `cat-file` processes still used them as working directories. `Submodule.update()` relied on collection of its temporary `Repo`, but captured log records retained a `Head` argument and therefore the repository and its process. Recursive updates also opened an extra unbounded repository. Close the owned repository after updates and on errors, and reuse it for recursion with final cleanup. This preserves `keep_going` behavior while releasing processes even when logs or callbacks retain repository objects. The compatibility test now scopes its own repository and closes it before removal; allocation tracing identified those separate caller-owned handles. Four regressions retain real logging arguments and verify process cleanup for normal, failing, recursive, and recursive `keep_going` updates. Both previously failing tests pass with tracing asserting no live checkout processes at each removal. Ruff, mypy, basedpyright, and `git diff --check` pass locally. Native Windows validation will run in CI.
The Cygwin full suite reached 1,372 passing tests but failed its native-Git detection check. The new source build installed Git into `/usr/local`, while GitPython's existing detector expects `uname` beside the selected Git executable. Cygwin installs `uname` in its normal `/usr/bin` directory. Install Git 2.52 under `/usr`, replacing the older packaged Git and preserving that standard layout. Verify `Git.is_cygwin()` immediately after installing Python dependencies so a setup regression fails before the long test suite. The detector and its missing-`uname` behavior remain unchanged. YAML parsing, extracted Bash syntax, ShellCheck, and `git diff --check` pass locally. The preceding Cygwin performance suite also passed all six tests; native validation of the corrected installation runs in CI.
…ckout Exercise a current high-download GitPython consumer with its unchanged upstream tests. Add a shared `uv` runner that resolves the latest PyPI release, retrieves verified source, installs a private environment, and replaces the released GitPython dependency with this editable checkout. Verify the imported `git` module before testing and retain source provenance, frozen requirements, and JUnit results for diagnosis and reproduction. Clear inherited Git repository/configuration settings and Python import paths so upstream commits and resets use their own fixtures. Use pytest's long `--override-ini` spelling because Bandit's CLI tests interpret `-o` as a forbidden Bandit output option. Reject successful runs with no passing tests, including entirely skipped suites. Add a CI job for the latest Bandit release, with Git 2.52 or newer, and local usage and download-ranking documentation. Bandit 1.9.4 passes all 12 selected tests against this checkout on Python 3.12 and Git 2.54, even with deliberately invalid inherited Git directory, index, and config settings. Ruff, workflow YAML parsing, and whitespace checks also pass. No GitPython compatibility changes were needed.
Add MLflow's released Git project and context tests to the shared local
runner and CI matrix. Rank the project by `mlflow-skinny` downloads without
summing overlapping distributions, resolve its current PyPI release, and
check out the matching `v{version}` source tag for unchanged upstream tests.
Install the matching full `mlflow` package because upstream global fixtures
need its server and SQLite support. Clear `CI` and `GITHUB_ACTIONS` inside
the isolated run to avoid unrelated upstream wheel builds and conda
cleanup. Disable telemetry and keep the venv first on `PATH` so project
subprocesses use the same editable GitPython checkout.
The shared runner passes all 47 selected tests for MLflow 3.16.1 on Python
3.12 and Git 2.54, including 31 repository/project/model-versioning cases and 16 Git
context or credential-redaction contract cases. Validation started with
both CI variables set, exercising the CI isolation. Public example clones
and a localhost HTTP server are required; no cloud services or models are
needed. The model-versioning cases also cover staged/unstaged diffs and
dirty-state handling. No GitPython compatibility changes were necessary.
Include `langchain-community` as a current runtime GitPython user: its
published `GitLoader` requests a manual GitPython installation even though
it is absent from `Requires-Dist`. Its September download count ranks
above the other selected consumers.
Resolve the latest PyPI release and run unchanged tests from the matching
`libs/community/v{version}` tag against the editable GitPython checkout.
Use upstream test-plugin ranges and `--only-extended` so missing integration
dependencies fail collection instead of silently skipping the tests.
Add the same profile to CI and document the runtime-use selection.
The shared `uv` runner passes both GitLoader tests for release 0.4.2 on
Python 3.12 and Git 2.54. They exercise real local clones, commits,
checkout, tree traversal, ignored files, repeated loads, and remote URL
validation. No network services are used during testing and no GitPython
compatibility changes were required.
SWE-bench is a current high-download GitPython user, but its latest 5.0.2 release has no tests for the GitPython inference helpers and no matching Git release tag. Retrieve the verified PyPI source and explicitly document that this profile runs a GitPython-authored supplemental integration test, rather than misrepresent unrelated upstream tests as compatibility coverage. Import the real `AutoContextManager` with only its required `chardet` and GitPython dependencies. Exercise clone, commit checkout, reset, untracked cleanup, directory restoration, and clone reuse for SHA-1/SHA-256 with files/reftable. Route its normal URL to a local fixture and permit only file transport during the test. Do not patch production modules or mock GitPython. BM25's Java/Pyserini helpers remain outside this focused check. Add the profile to the local runner and CI. Expand pytest-option paths, cut off unrelated ancestor conftests, and use importlib mode so the supplemental filename cannot shadow the installed upstream package. All four cases pass through the shared runner on Python 3.12 and Git 2.54; they also passed separately on minimum Git 2.52. Ruff and whitespace checks pass. No GitPython compatibility changes were required.
Complete the five-current-user compatibility matrix with `acryl-datahub`. Resolve the latest PyPI release and retrieve its tag from `acryldata/datahub`, which publishes patch tags missing from the repository named by package metadata. Run the unchanged Git integration file against the installed release and this editable GitPython checkout. Exclude unrelated SQL/docker conftests, disable telemetry, and clear the private SSH test credential variable. The selected tests still exercise a real public GitLab clone and fixed-commit checkout, a localhost SSH timeout, GitCommandError handling, password redaction, and source configuration. The upstream private-clone test retains its credential-dependent skip. Document all five current users and their September 2026 download ranking, including optional runtime integrations, distribution deduplication, and projects whose latest releases dropped GitPython. Add an all-project local command and the fifth CI matrix entry. The shared runner passes 7 tests with 1 upstream skip for DataHub 1.7.0.14 on Python 3.12 and Git 2.54. All five profiles have now passed locally, with 68 upstream passes plus 4 SWE-bench supplemental cases. Ruff, Python syntax, workflow YAML/matrix consistency, Bash syntax, ShellCheck, and whitespace checks pass. No GitPython compatibility fixes were required. Also clear inherited pytest options and plugins: a caller's `-k` filter could otherwise leave only unrelated contract tests and yield a misleading pass. All four SWE-bench cases still pass with a deliberately nonmatching inherited filter and a nonexistent plugin, verifying their removal.
Add `GitPython[gix]` with the published `GixPython==0.1.0` dependency and select it when the `gix` module is installed. Start with an empty native dispatch table so library-managed commands retain their CLI behavior. Keep the existing safety boundary and distinguish unsupported calls from errors after a native write. Add per-operation reporting and a test runner that creates its historical fixture in a disposable local clone. The installation test verifies backend selection, and the tutorial fixture no longer clones GitHub. Let the optional tox environment resolve its dependencies from the package index rather than requiring an unpublished local wheel. Tests themselves retain offline package installation using cached wheels. Pin the official initial release so the optional backend has a reproducible API baseline. Restrict that dependency to Python 3.11 or newer, matching its published interpreter requirement while allowing universal resolution of GitPython extras on the existing Python 3.8+ support range. The published Apple Silicon wheel was downloaded from PyPI and verified against its SHA-256 digest; it replaces the previous local artifact with the same version number. Validation on CPython 3.12/macOS: the SHA-1/SHA-256 backend smoke checks and fresh extra-installation check passed (3 tests). Ruff lint and formatting pass. Universal `uv sync --all-extras --all-groups --dry-run` initially rejected the unmarked dependency for Python 3.8–3.10; the Python-version marker fixes that resolution failure when dynamic package metadata is refreshed.
Use GixPython's header lookup for `Git.get_object_header` and ODB metadata. Preserve the existing object-ID, kind and size tuple, missing-object behavior, and CLI diagnostics for unsupported storage or revision syntax. Keep the persistent-process lifecycle regressions explicitly exercising the CLI fallback. The staged version passes Ruff and Python syntax checks. Its SHA-1/SHA-256 native smoke and focused pytest selection pass: 10 passed in 4.34s.
Read object contents through GixPython and return independent byte streams from `Git.stream_object_data`. Partially reading one object must not corrupt another read. Retain CLI streaming above 8 MiB because the current native lookup buffers an entire object (GIX-2). The staged version passes Ruff and Python syntax checks. Its SHA-1/SHA-256 native smoke and focused pytest selection pass: 9 passed in 2.40s.
Handle verified revisions and repository metadata exposed by GixPython. Keep Git's initial repository validation and unsupported discovery options. Message searches, dirty suffixes and describe-shaped names fall back because the native grammar differs in regex handling and exact-tag precedence (`GIX-14`/`GIX-17`). GixPython's native configuration identifies `extensions.refStorage`; accessing `HEAD` in either a SHA-1 or SHA-256 reftable repository raises `gix.Error` with an unsupported-storage cause. The bindings currently flatten the native error kind. Add a direct regression for this signal, without manufacturing an `Unsupported` exception in Python. Keep `rev-parse --show-ref-format` on Git because it also validates repository extensions. Even with strict configuration and trust options, Gix accepts an unknown version-1 extension and successfully reads `HEAD` where Git rejects the repository. Add that regression too. A reference-format getter is not the prerequisite: compatible extension validation is. The remaining query costs one CLI invocation after wrapper initialization; this amendment does not reduce it. Validation: affected backend and revision tests passed with Gix first (30 passed in 20.22s), then CLI (22 passed, 1 skipped in 21.26s). Ruff lint/format and `git diff --check` passed.
Implement the managed full-tree NUL listing with native tree entries. Preserve Git's octal modes, object kinds, IDs and raw filename bytes for the existing tree parser. Other listing options continue through Git. The staged version passes Ruff and Python syntax checks. Its SHA-1/SHA-256 native smoke and focused pytest selection pass: 26 passed in 3.86s.
Use native reference lookup for quiet, nonrecursive symbolic-target reads. Preserve detached-reference exit status and let Git supply missing-reference diagnostics. Symbolic mutations retain their existing reflog and validation behavior through the CLI. The staged version passes Ruff and Python syntax checks. Its SHA-1/SHA-256 native smoke and focused pytest selection pass: 13 passed, 24 subtests passed in 3.81s.
List native reference names in Git's order for the managed `for-each-ref` format and literal prefixes. Keep glob patterns, pseudo refs and other formats on the CLI rather than interpreting them as plain prefixes. The staged version passes Ruff and Python syntax checks. Its SHA-1/SHA-256 native smoke and focused pytest selection pass: 6 passed in 1.88s.
Serve simple managed `config --get` requests from the native configuration snapshot, including implicit booleans and missing-key status. Open configuration queries without synthetic safety settings so they report the user's actual values. Files, streams, enumeration and mutations stay on Git (GIX-12). The staged version passes Ruff and Python syntax checks. Its SHA-1/SHA-256 native smoke and focused pytest selection pass: 23 passed in 1.11s.
Full-suite timings mix repository operations with repeated fixture setup and coverage. Add a benchmark-only `pyperf` harness against a pinned existing GitPython checkout, with one warm `Repo` per worker and fresh high-level wrappers per invocation. Measure the complete read-only journey and eight named public-API operations, including a patch operation that uses fallback. Separately time direct `Repo` construction and discovery from `git/objects`, creating and closing a repository per invocation. Keep those lifecycle costs outside the already-open journey so future native-handle reuse has a visible effect on operation timings. `MEASUREMENTS` makes additions join individual timings, the journey and result parity checks. Retain environment/revision metadata, result digests and one invocation's native/fallback decisions in raw `pyperf` results. The comparison reports both backend means and standard deviations and rejects mismatched results, installations or workload metadata. Document reproducible local setup without downloading another interpreter or modifying the measured repository. All eleven workload paths passed GixPython first and CLI second on the fixed fixture, with matching result digests. Four comparison regression cases passed with both installations. Ruff lint/format and `git diff --check` passed.
Add a separate `Backend benchmark` job using one CPython 3.12 interpreter and two installations, with official `GixPython==0.1.0` only in the native one. Prepare a fixed SHA-1/files checkout of GitPython 3.1.45 with one branch, an untracked file and an ignored directory before timing. Keep local/global Git configuration and optional locks from changing the measured workload. Run the calibrated `pyperf` suite with GixPython first, then CLI on the same runner. Include separate direct repository opening and nested discovery while keeping the operation journey on a retained `Repo`. Validate result digests and workload metadata, publish all means and standard deviations plus `pyperf` significance reporting in the job summary, and retain raw JSON and comparison artifacts even on failure. Timings are observational on shared runners; execution and parity errors fail the job, without a noisy performance threshold. New `MEASUREMENTS` entries automatically participate in this job. The final fixed workload passed all eleven GixPython-first and CLI-second smoke measurements locally with matching result digests. Four comparison regression cases passed both installations. Workflow YAML parsing, Ruff lint/format and `git diff --check` passed. Hosted execution will follow the branch push.
Document the new `pyperf` workload and local measurements independently of full-suite setup/coverage timings. Record all eleven means and standard deviations, fixture/source revisions and environment details: direct opening and nested discovery favor GixPython, but the already-open journey is about 13% slower because history metadata and commit statistics offset other gains. Explain retained `Repo` ownership during operation measurements, separate open/discovery lifecycle costs, fresh high-level wrappers, parity checks and the extensible CI artifact workflow. Native handle reuse remains future work; these measurements expose its potential benefit without implementing it. Both complete calibrated runs finished on existing CPython 3.12.14 with GixPython first and CLI second. All eleven result digests match; the comparison and `pyperf` significance table completed successfully. Record the stability warnings rather than treating one machine's warm-cache values as universal. The initial dedicated CI benchmark job also passed. `git diff --check` passed.
Count successful `Git.execute` launches separately from native/fallback decisions, including raw commands and persistent-process starts. Report pytest setup/call/teardown totals and optional ceilings, and put warm per-measurement launch counts and ceilings into the existing `pyperf` CI comparison. Commands that fail after spawning count; failed creation and Git's own children do not. Require Gix-backed implementations in `AGENTS.md` from this commit onward. Python may adapt inputs/results and manage native handles, but cannot provide missing Git semantics or fabricate success. Missing or unclear Gix capabilities retain CLI fallback with actionable upstream documentation. For the pinned fixture, warm CLI/Gix launch counts are 23/1 for the journey, 11/5 for opening and 13/7 for discovery. The opening/discovery Gix ceilings include the restored reference-format query (`GIX-1`), one more call than the previous fabricated answer. This commit itself reduces CLI calls by zero. Keep the earlier 30,898-launch full-suite report explicitly historical; it does not validate the rewritten implementation. Validation: 13 counter/ceiling/comparison tests passed with Gix first (0.06s) then CLI (0.05s). All 11 existing benchmark operations passed an untimed Gix-then-CLI result-digest comparison and the corrected CLI ceilings. `git diff --check` passed. Only existing CPython 3.12.14 was used. Clarify the native-location boundary following the metadata review: adapting `Repository.git_dir()` with the fixed `modules` and `COMMIT_EDITMSG` leaf names is permitted, as is canonicalizing an input path before reopening through Gix. Both require Git parity tests; neither permits a general Python Git-path resolver. This follow-up changes policy text only and passes `git diff --check`. Define `native` explicitly as Gitoxide execution through GixPython, never a Python implementation. Require each removed CLI call to map to identified Gix calls. Treat observed Git/Gix divergences as compatibility bugs, including intentional differences, and require equivalent behavior or a mode as strict as Git. Record expected/actual behavior and reproducible evidence, and keep an upstream bug open when a Gix-based adapter workaround restores local parity. Missing bindings and unverified guards remain distinguishable from reproduced bugs. This clarification changes documentation only; `git diff --check` passes.
The native adapter previously called `gix.open_opts()` for every operation, repeating configuration parsing and losing shared index/object-store state. Each `Repo` now owns a native handle, associated with `Git` through a weak reference. `close()` releases it and pickling excludes native resources. Reuse preserves storage/environment guards. Metadata and environment changes, and successful raw CLI launches, invalidate the configuration view and trigger `reload()` on the retained handle. Config queries keep a separate fresh view without synthetic safety settings. Includes reload conservatively because GixPython does not expose their source paths. A per-repository lock prevents concurrent refresh races; Gix handles read sharing and index/ODB refresh. Validated Gix first: 260 affected tests and 14 subtests passed, with 3 skips; 38 native regressions passed again after the metadata guard was completed. CLI compatibility: 222 affected tests and 14 subtests passed, with 3 skips. Ruff lint/format, mypy and basedpyright passed.
Use `gix.open_opts()` and native accessors for storage, worktree, object-format and empty-tree metadata. Retain the native handle and leave parent traversal under `search_parent_directories`. Every candidate reaches Gix before a discovery decision; do not fabricate rejection from Python `HEAD` checks. Gix already supplies worktree metadata for linked worktrees of bare main repositories. Its `is_bare()` reflects inherited configuration, so combine it with the native worktree's presence for GitPython's bare flag and managed `--is-bare-repository`. Remove the unnecessary linked-bare fallback. Cover opening through the worktree, its gitfile, and its administrative directory. Keep the real format query for Git-compatible repository-extension validation (`GIX-1`), not because storage cannot be identified. Force native HEAD decoding and retain common-directory/layout guards and Git diagnostics (`GIX-14`). In particular, strict native opening still ignores dangling `commondir` symlinks. Each linked-worktree bare query now drops from one CLI call to zero. Opening those worktrees uses the normal native path: only the one format-validation query and executable-version preflight remain. Nested invalid candidates still need Git diagnostics because GixPython flattens the native error kinds. Validation: affected opening/discovery/worktree/format/lifetime tests passed with Gix first (25 passed, 1 skipped, 14 subtests in 4.86s), then CLI (12 passed, 2 skipped, 14 subtests in 3.20s). Ruff lint/format and `git diff --check` passed. Replay preserves all opening and safety changes.
Record the original retained-handle/opening benchmark, full-suite timings, launch counts and validation with their measured revision and local artifacts. These observations predate the Gix-only audit and include the Python reference-format, linked-worktree and negative-discovery shortcuts that the rewritten opening commit removes. Label the zero-call opening/discovery rows and their 69x/91x ratios as historical, superseded measurements. They must not be read as current Gix coverage or current performance. Keep the already-open journey and test observations alongside their original context rather than silently replacing old data with new claims. This documentation amendment reduces CLI calls by zero and changes no runtime behavior. Validation: `git diff --check` passed; the corrected opening commit already passed its focused Gix tests followed by CLI tests.
Native operations already retain a `gix.Repository`, but the choice to reuse or recreate it was hidden inside backend dispatch. Route bound access through `Repo._get_gix_repository()` and provide `recreate=True` as the control point for a future configurable policy. Preserve individual command options and the existing refresh, locking, pickling and resource-release behavior. Document retained native caches as an intentional deviation from a fresh Git process per command. Configuration queries still use a separate fresh handle so the execution handle's safety overrides do not affect queried values. CLI-call reduction: 0. The fixed probe of ten native metadata queries starts 0 CLI processes before and after this ownership refactor. Explicit recreation also starts none; later commits can use this accessor for their conversions. Validation: Gix first, 304 tests and 14 subtests passed (3 skipped), then CLI, 264 tests and 14 subtests passed (4 skipped). Four targeted Gix cases also passed after checking preservation of a separate bound command's options. Ruff lint/format, mypy and basedpyright passed.
GitPython asks `--git-path` for `modules` and `COMMIT_EDITMSG`. Git's `path.c` keeps these fixed leaves in the private Git directory, including linked worktrees; the location is already available from Gix. Resolve the repository's canonical Git directory through Gix and append only those two leaf names. Use the existing `to_native_path_linux()` formatter for Git's forward-slash output on Windows. This is the native-location adaptation permitted by `AGENTS.md`. Keep arbitrary metadata names on Git, since objects, indexes, hooks and other paths have separate configuration/environment or common-directory rules. Symlinked metadata leaves also retain CLI canonicalization, including dangling targets. Do not implement the general Git path resolver in Python. Each ordinary fixed-path lookup drops from one CLI launch to zero. Regression tests forbid launches for both names across normal, bare and linked worktrees, including a bare main repository, and compare all returned paths with Git. Additional cases require CLI for metadata symlinks and an unconverted path. Validation: affected metadata/submodule/commit-message tests passed with Gix first (87 passed in 27.87s), plus 2 fallback cases in 0.16s, then CLI (81 passed, 1 skipped in 62.10s). After applying the shared path formatter, the focused checks passed again with Gix first (9 passed in 1.58s), then CLI (3 passed, 1 skipped in 0.94s). Ruff lint/format and `git diff --check` passed. The ledger now identifies only the remaining general-resolution requirements.
Commit and tree serialization wrote native objects, then launched a fresh `git cat-file` process to read their bytes back. Read through `Repo.odb.stream` instead so Gix can serve the object and the CLI backend can reuse its batch reader. Existing large-object and unsupported-storage fallbacks still apply. The fixed probe serializing one commit and one tree drops from 2 CLI launches to 0 with Gix, excluding fixture setup. Regression assertions compare the serialized bytes with Git and prohibit CLI launches during native readback. The earlier whole-suite inventory contained 779 matching readbacks; that is an optimization target, not a newly measured suite-wide reduction. Validation: the affected commit, tree, object/index, merge and serialization selection passed with Gix first (28 tests, 8.04s), then CLI (26 tests, one skipped, 19.05s). Ruff lint/format and `git diff --check` passed.
Reference reads previously tried `symbolic-ref` and then `rev-parse`, falling back to Git twice for an ordinary missing reference. Use one native exact-name lookup for `HEAD` and full `refs/` names, returning the existing direct or symbolic target or the same missing-reference `ValueError`. Keep filesystem containment checks on both the requested name and symbolic target. Existing traversal limits still reject cycles and overly deep chains. Partial-name lookups, unsupported storage/environment, and native decoding errors retain the CLI path and its diagnostics. The fixed missing-reference probe drops from 2 CLI launches to 0. A regression compares direct, packed and dangling symbolic reference reads with CLI results and asserts no CLI launches for native reads. The earlier full inventory had 473 adapter-identified missing-reference fallbacks, before secondary probes. Validation: Gix reference, reflog and backend tests passed first (107 tests and 24 subtests, 26.90s), followed by CLI (63 tests, one skipped and 24 subtests, 37.18s). Ruff lint/format, mypy, basedpyright and `git diff --check` passed.
Use `gix.Target.Symbolic` for the native full-name grammar before creating any repository or CLI wrapper. Keep the existing operand guard, successful-name cache and per-operation filesystem containment checks. GixPython rejects standalone names that Git accepts with `--allow-onelevel`, including lowercase names, digits, punctuation and some Unicode. Defer native standalone rejections to Git rather than duplicating either grammar in Python. Document GIX-20 as an upstream binding/behavior shortcoming: four audited calls differed (`refs` twice, `hellothere`, `valid_one_level_refname`), and expanded probes include `1`, `A1`, `HEAD_1`, `A-B`, `A.B`, and `Ä`. A fresh supported name now costs 0 CLI launches instead of 2, including the previous wrapper's version probe. Regression assertions cover accepted and rejected native names without a repository and the broader standalone fallback. Fallback decisions are not also counted as completed native validations. Validation: the reference, backend and command-guard selection passed with Gix first (220 tests and 24 subtests, 22.73s), then CLI (175 tests, one skipped and 24 subtests, 44.51s). Ruff lint/format and `git diff --check` passed.
Use the shared native discovery path for standalone storage-directory and gitfile helpers. Preserve native validation, storage/environment guards, and Git fallback for rejected candidates or unsupported layouts. After Gix opens a gitfile, reopen its Git directory through Gix with a canonical input path and the same strict trust/configuration options. Return the second handle's actual metadata. This resolves symlink targets and also avoids treating an arbitrary gitfile's location as the worktree. The extra native open is an accepted temporary cost; no Git storage files or worktree locations are interpreted in Python. Each supported gitfile helper now needs zero CLI launches, including symlink targets that previously needed one. `Repo` retains the reopened handle and its separate format-validation query. Failed reopening selects fallback and is never recorded as native success. Preserve native worktree classification from the preceding correction; do not restore the linked-bare fallback. Tests compare Git-directory and worktree locations for ordinary, `..` and symlink targets, verify that both opens keep the same options, and inject a failure into the second open. Discovery/worktree/submodule cases passed with Gix first: 61 other cases plus all 8 gitfile cases, 14 subtests and 2 skips across the focused runs. The matching CLI selection passed 54 cases and 14 subtests, with 3 skips, in 38.48s. Ruff lint/format and `git diff --check` passed. Direct helper-output comparisons use Git's raw path spelling, preserving Windows separators independently of `Repo`'s PathLike properties. The final path-format selection passed with Gix first (9 passed in 1.22s), then CLI (1 passed, 1 skipped).
`Repo.rev_parse` resolved an object natively but still launched `cat-file` to find its tree/index mode, then probed revision prefixes to locate its path. Use `Repository.rev_parse` to obtain the ID and path/mode together. Preserve path normalization and the absent metadata for root-tree specifications. Share the existing revision-grammar guard. Index revisions also check the existing custom/sparse-index capability guard before native parsing, including OID lookup on the fallback path, so an alternate index is never silently replaced by the repository's default index. The fixed `HEAD:file` probe drops from 1 CLI launch to 0. Regression assertions compare root/directory/file, executable, symlink and index-stage results with Git in both object formats, prohibit native CLI launches, and exercise an alternate index. The earlier full inventory had 170 mode-query launches. Validation: revision, backend and positional-argument tests passed with Gix first (101 tests, 15.60s), then CLI (54 tests, one skipped, 12.65s). Ruff lint/format, mypy, basedpyright and `git diff --check` passed.
Submodule enumeration and cache refresh each launched `git config list` over `.gitmodules`. Use the dedicated `ModulesFile` binding for raw path, URL and branch fields, sharing the existing selection of worktree versus historical blob sources and the existing path-containment checks. Retain Git's parser for other/duplicate sections, ambiguous bracket syntax, missing or implicit fields, and native parse errors. The bindings cannot yet enumerate arbitrary sections; conservative guards preserve declaration order and the existing errors without implementing a configuration grammar in Python. Public and writable config parsers retain their full contracts. The fixed probe enumerating one submodule and reloading its URL drops from 4 CLI launches to 0, including the former standalone version checks. The regression compares declaration order, repeated raw URL values and refreshed fields with Git, asserts zero native launches, and checks duplicate-section and include behavior. The earlier full inventory recorded 976 relevant configuration reads before secondary version probes. Validation: all affected submodule and backend tests passed with Gix first (497 tests, three skipped, one expected failure, 117.80s), then CLI (448 tests, four skipped, one expected failure, 431.08s). Ruff lint/format, mypy, basedpyright and `git diff --check` passed.
Audit the remaining configuration consumers after converting supported `.gitmodules` reads. Record concrete differences in remote declaration order and scope, last/raw versus first URL selection, arbitrary actor-reader sources, tracking-ref mapping, cached fetch-refspec presence checks and include handling. Official GixPython 0.1.0 exposes scalar generic config getters, a setter and serialization, but lacks the enumeration, multivalue and source-selection operations needed to preserve these contracts. `OpenOptions.isolated()` also disables includes. Keep these callers on Git pending binding additions rather than implementing another config grammar or changing their public behavior. The fixed general-config probe remains 2 CLI launches before and after this investigation: reduction 0. This commit records the blockers and makes no claim that these reads have been converted. Local probes produced the same GitPython results with Gix first and CLI second while demonstrating each native API mismatch. Configuration tests passed with Gix first (19 tests, 1.07s), then CLI (19 tests). `git diff --check` passed.
Reuse successful minimum-version checks for equivalent `Git` wrappers, keyed by the resolved executable and its metadata, working directory, effective environment and `Git.refresh()` generation. Keep public `version_info` caching per instance and reject unsupported versions before repository creation. Executable replacement and newly preferred PATH candidates trigger new probes; ambiguous Windows search and global options retain the existing behavior. This is shared CLI housekeeping for both backends: it caches actual Git answers and records no native success. It does not implement a missing Gix capability. The cache is bounded to 128 contexts; external launcher state still requires an explicit `Git.refresh()` when executable/environment metadata is unchanged. The fixed pair of extra warm version checks drops from two CLI launches to zero. With the Gix-only fallbacks restored, warm opening/discovery launch counts fall from 2/4 to 1/3 for Gix and 11/13 to 9/11 for CLI. Update the benchmark ceilings accordingly while retaining real reference-format and negative-path queries. Cold contexts can still add a version probe. Validation: 68 affected version, refresh, guard, counter and ceiling tests passed with Gix first (1.50s), then CLI (1.74s). All 11 benchmark result digests match and the adjusted ceilings pass; `git diff --check` passed.
GixPython 0.1.0 exposes no hook lookup or execution API. The previous fast path read configuration through Gix but used Python `os.stat()` to declare an absent hook successful, so it was not a Gix implementation. Retain the existing CLI path for every requested hook and document the required native binding as `GIX-23`. The missing-hook probe now deliberately retains one CLI launch per invocation (zero reduction), replacing the former zero-call Python shortcut. Tests assert six actual calls for three absent hooks across ordinary and linked worktrees, retain pre-dispatch NUL rejection, and check existing/configured hook fallback. The native handler and its special `GIT_EDITOR` config permission are removed. Validation: Gix index, backend and safety tests passed first (236 passed, 2 skipped in 34.38s), then CLI (166 passed, 3 skipped in 28.61s). Ruff lint/format and `git diff --check` passed. Native mutation failures continue to be protected from CLI retries by the existing dispatch contract.
Extend the existing `pyperf` journey with submodule inventory, tree/index revision paths and modes, and raw commit/tree readback. Fresh public wrappers on the retained `Repo` join the separate benchmark CI job, result-parity checks and per-measurement reporting. Read existing signed objects without rewriting them, and keep opening/discovery separate from the already-open journey. Each new read has a zero-CLI ceiling backed by Gix APIs. The expanded warm journey records CLI/Gix launch counts of 43/1; submodule inventory is 11/0, revision paths 8/0 and object readback 1/0. Opening/discovery retain their real fallback requirements: 9/1 and 11/3. A Python substitute must not be used to lower those ceilings. Update the coverage overview for the expanded workload. This measurement-only commit reduces backend CLI launches by zero. Its purpose is to make genuine conversions and remaining fallbacks visible and enforceable as GixPython gains capabilities. Validation: 13 counter/comparison tests passed with Gix first and CLI second (0.05s each). All 14 untimed result digests match and corrected CLI ceilings pass. Repository-wide Ruff lint/format, `mypy` (47 files), `basedpyright` (zero errors/warnings) and `git diff --check` passed. Fresh full-suite and timed measurements are recorded separately in the following journal commit.
Treat observed differences from equivalent Git operations as compatibility bugs, even when intentional or covered by an adapter workaround. Require an upstream fix or a verified API/mode as strict as Git. Define `native` as Gitoxide execution through GixPython, name the underlying Gix calls, and forbid Python implementations of missing Git behavior. Keep reproduced bugs, missing APIs and unverified coverage distinguishable in the follow-up ledger. The ledger now records 25 items. Add object/branch type validation (`GIX-24`) and dangling-reference enumeration (`GIX-25`), expand identity cleanup (`GIX-21`), and spell out the missing HEAD reflog updates and incorrect old OID on symbolic detachment (`GIX-10`). Preserve bare-worktree classification and canonical gitfile behavior as upstream compatibility bugs (`GIX-14`) even though the adapter obtains compatible results from Gix metadata and reopening. Include Git/Gix expectations, API names, reproduction instructions, versions and the existing guard regressions. Keep current CLI fallbacks and budgets. Verify 12 direct Gix/Git comparisons for each of SHA-1 and SHA-256 on released GixPython 0.1.0 and existing CPython 3.12.14: six object/branch type checks, identity cleanup, dangling enumeration and four reflog transactions. Gix ran before Git for each case. All 24 comparisons confirmed the documented outcomes; ledger IDs, status labels, cross-references and local links are consistent. Preserve the earlier Gix-only audit measurements at `5ef5c62f`: the 1,710-test full suite, 22,694 CLI launches, and 14 matching `pyperf` result digests. Keep the subsequent 120-Gix/34-CLI-test capability validation tied to its runtime snapshot `ee947c98`. The historical zero-launch lifecycle results are not current coverage claims. This documentation follow-up reruns neither the full suite nor timed benchmarks and does not claim those measurements for a new runtime revision. This commit reduces CLI launches by zero. Runtime code, tests and CLI budgets are unchanged by the strict-compatibility documentation update. All 74 original Tix changes remain represented, and `git diff --check` passes. No additional interpreter was installed.
The downstream runner always installed plain `GitPython` into a fresh
virtual environment. Starting it from a Gix-enabled environment therefore
still exercised the CLI backend and left downstream Gix compatibility
untested.
Add `--backend {cli,gix}`, retaining CLI as the default. Gix runs install
this editable checkout with `.[gix]` and verify the selected backend before
running downstream tests. Include the backend in `result.json`, status
output and retained directory names. Expand the five-project CI matrix to
ten independent jobs and document local backend selection.
Validated all five selected suites with official `GixPython` 0.1.0 first,
then CLI, using the existing Python 3.12.14 interpreter and Git 2.54.0:
72 passed and one credential-dependent DataHub skip per backend. Tested
LangChain Community 0.4.2, MLflow 3.17.0, Bandit 1.9.4, SWE-bench 5.0.2,
and DataHub 1.7.0.14 with matching release sources on both backends.
Verified the default CLI installation and backend mismatch rejection in
both directions, frozen dependencies, result metadata, and workflow matrix
wiring. `ruff check`, `ruff format --check`, runner `--help`, and
`git diff --check` passed. The ten local runs took 69.75 seconds for Gix
and 80.80 seconds for CLI including source and environment setup; these
are downstream validation timings, not controlled performance benchmarks.
PR #2274's Python 3.12 CI jobs spent 57m 25s in Windows `pytest` versus 5m 2s on Ubuntu, with 85,777 and 82,683 `Git.execute` launches respectively. A local profile of the file-handle and recursive submodule regressions spent about 65 of 77 seconds executing Git, including 44 seconds in repository construction. Repeated process startup makes the separate metadata probes expensive. Query reference format, object format, bare status and the absolute common directory in one `git rev-parse` call on the CLI construction path. Keep the path last and split only the three fixed scalar fields so embedded newlines remain intact. Git still computes every value. Supported Gix discovery keeps its native metadata and existing CLI format-validation query, with unchanged native launch ceilings and fallbacks. The focused selection drops from 2,447 to 2,009 launches: three saved for each of 146 repository opens. Three paired Windows measurements of 20 opens each reduce the median from 300.74 ms to 221.18 ms per open, with matching metadata and 11 versus eight launches. Add `--durations=30` to CI so future slow tests are visible, and record the profiling evidence and reproduction requirements in `doc/gix-backend.md`. Cover both object formats, both reference formats, bare and non-bare repos, spaces and embedded newlines. The eight applicable Windows cases fail before the fix and pass afterward; all 16 cases pass on Ubuntu WSL. The optional Windows Gix selection reproduces the same 17 failures on the original and patched sources. Document the demonstrated path-formatting and undecodable `commondir` differences under `GIX-14`, distinguishing them from Windows test assumptions. Keep these bugs open pending compatible upstream behavior. Validation: the full Windows CLI suite passes with 1,662 passed, 60 skipped, nine expected failures, two unexpected passes and 40 passing subtests. It records 77,732 launches in 3,715.16s. The initial baseline records 86,032 in 3,990.28s; its only two failures pass on unchanged code after restoring CI's `core.autocrlf=true` and placing `pytest`'s temporary directory outside a Git checkout. The overlapping full runs are not a controlled elapsed-time comparison. Repository-wide Ruff lint/format, all pinned pre-commit hooks, `mypy` (47 files), `basedpyright` (zero errors or warnings), the Sphinx HTML build and `git diff --check` pass.
The Windows suite with `GixPython 0.1.0` had 17 failures that the CLI-only
package matrix did not exercise. Format paths returned by `Repository.common_dir()`,
`workdir()` and `git_dir()` with the existing Windows separator adapter,
including repository properties and worktree listings. Gix still resolves
the locations, and POSIX filenames retain their literal backslashes.
Recover from `RuntimeError("native worker panicked")` only during repository
discovery, letting Git validate malformed metadata and provide its normal
error. Propagate unrelated errors and errors after a native mutation starts.
Keep the upstream discovery bug open as `GIX-14` in `doc/gix-backend.md`.
Correct the gitfile and symlink controls to use the platform's Git behavior.
Exercise status with portable filenames on Windows while retaining POSIX
newline coverage. Compare direct-branch and symbolic-alias blob updates with
Git, which rejects the former and accepts the latter in the tested versions.
Add a Windows/Python 3.12 Gix job while preserving all 28 existing CLI jobs
and check names. Prepare Rust for the released `GixPython` source build,
cache `pip` wheels, verify backend selection and retain JUnit results plus
backend operation counts for every job.
Validation:
- Full Windows Gix suite: 1,760 passed, 40 subtests passed, 58 skipped,
nine expected failures and three unexpected passes in 2,242.66 seconds.
The baseline reproduced all 17 failures; the unexpected passes and the
path-deprecation warning also occurred there. Overlapping runs do not
establish a performance improvement for these compatibility fixes.
- Windows CLI regressions: 138 passed and 14 subtests passed.
- Ubuntu WSL Gix regressions: 43 passed and 14 subtests passed, including
both hash formats and POSIX newline filenames.
- `ruff`, `mypy`, `basedpyright` and pinned `pre-commit` checks passed. Matrix
expansion and existing GitHub check names were verified.
The full-suite matrix only installed `.[test,gix]` on Windows, so Ubuntu and macOS exercised the CLI backend even where `GixPython` was available. Add one Python 3.12 Gix job on each platform, covering all three runner OSes while preserving the 28 existing CLI jobs and their check names. Use the official PyPI release: macOS can install wheels, while Linux and Windows build the source distribution with stable Rust. Keep `CARGO_TARGET_DIR` outside `pip`'s temporary source directory and cache it alongside Cargo registry and Git dependencies. Cache keys include OS, architecture, Python, Rust and `gix-requirements.txt`, allowing dependency reuse across releases on the same platform and toolchain. Retain the existing `pip` wheel cache and document the job names and build requirements. Validation: - `actionlint` 1.7.12 and the repository's pinned `pre-commit` checks passed. Matrix expansion verifies 31 jobs and unchanged CLI combinations/names. - Built the SHA-256-verified `GixPython` 0.1.0 source distribution using the existing Python 3.12.14 and Rust 1.99.0. The first build, including test environment setup, took 117.51 seconds. A rebuild from a different source path reused all 228 Cargo crates and completed in 1.57 seconds. - Ran the workflow's installation/backend assertion with that source-built wheel and verified the installed extension matches it byte-for-byte. - Focused repository/backend tests passed with Gix first: 16 tests and 14 subtests. CLI then passed 12 tests and 14 subtests, with the Gix-only module skipped. These are local checks; the new CI jobs have not run yet.
Byron
force-pushed
the
gix-backend
branch
3 times, most recently
from
October 8, 2026 06:22
f3193ff to
af1de6a
Compare
On Windows, `git init --separate-git-dir` can try to rename a submodule's metadata directory onto itself. An open file inside that directory makes the rename fail with `Directory not empty`, as seen in the CI tutorial. Close the cloned `Repo` before the shared reconnect helper validates and repairs its gitfile, releasing retained object handles before Git might need to initialize or move repository storage. Add `test.cleanup.cleanup_directory` and `TemporaryDirectory` for disposal of isolated test directories. Remove whatever is possible, log filesystem errors, and leave locked files behind without changing the test result. Retry read-only Windows files and directories without changing symlink or junction targets. Support both cleanup callback APIs and Python 3.8+. Migrate test contexts and fixture teardown to these helpers while keeping `git.util.rmtree`, operations under test, and fixture reuse strict. Release repository handles before deletion, run the missing performance test teardown, and wait for the killed Git daemon to exit. Invalidate cached fixture layouts when disposal fails so subsequent tests rebuild at fresh paths. Give `test/run-local.py` a private pytest temporary root to avoid shared-root ownership failures. Remove the diff test's cleanup xfail and document the test cleanup convention. Regression coverage includes real Windows file locks, metadata reconnects through ordinary and 8.3 paths, read-only directories, unchanged symlink targets, cleanup retries, preserved test-body exceptions, decorator teardown, and recovery from a locked cached fixture. Alpine and Ubuntu CI exposed a POSIX daemon leak in the new cleanup test: `git ls-remote daemon_origin` contacted the previous fixture's server and failed before the cleanup assertions. Killing the `git daemon` wrapper left its `git-daemon` child listening with the old base directory. Launch the daemon executable directly on every platform, extending the existing Windows approach, so the existing kill-and-wait tears down the server itself. Once the daemon actually stopped, Linux CI's `TestRemote.test_fetch_unsafe_branch_name` exposed a restart failure: earlier server connections could leave the shared port in `TIME_WAIT`, so the next daemon could not bind it and `ls-remote` reported `Connection refused`. Enable Git's `--reuseaddr` option to allow these restarts. The socket regression makes the server close first, verifies that shutdown refuses new connections, and restarts on the same port. It fails without the option and passes with it. Windows 3.14t CI reported `ConnectionResetError: [WinError 10054]` in the socket regression: Git for Windows resets the rejected initial request instead of returning EOF. Accept that specific reset during `recv`; both outcomes establish that the server closed the connection. Keep the real shutdown, connection-refusal and same-port restart assertions. Validation: - Windows with Git 2.55.0.windows.5: 164 passed and 1 skipped in the CLI regression selection; final focused checks passed with 25 passed and 1 skipped for each of the CLI and Gix backends. - Linux under WSL: 27 passed and 4 skipped in the portable regression selection. - Pinned pre-commit hooks, `mypy`, and `basedpyright` passed. - All base, cleanup and remote tests passed on macOS with Gix first (53 passed, one Windows-only skip in 16.44 seconds), then CLI (53 passed, one skip in 21.44 seconds), including the daemon restart regression. Pinned `pre-commit` hooks, `mypy` and `git diff --check` passed for the daemon corrections. - The Windows reset correction passed all cleanup tests with Gix first (15 passed, one Windows-only skip in 2.62 seconds), then CLI (15 passed, one skip in 2.91 seconds). The reset path also passed with two simulated Windows resets and real daemon restarts on macOS. Pinned `pre-commit`, `mypy` and `git diff --check` passed; native Windows validation of this final test correction is still pending.
Windows process startup makes repeated Git version probes and missing-hook checks costly in the submodule tests. A relative `git` executable also prevents the existing minimum-version cache from being shared on Windows. Resolve `git.exe` from `PATH` to an absolute `GIT_PYTHON_GIT_EXECUTABLE` for Windows CI and the local runner, preserving explicit overrides. Use the selected executable for the runner's direct Git commands. Pass `skip_hooks=True` at 70 submodule fixture commit sites, preserving normal `IndexFile.commit()` defaults and dedicated hook coverage. Match CI's private configuration by appending the alias fixture directly and retaining `core.autocrlf=true` on Windows. The previous `include.path` forced Gix repository reloads during local measurements. Document the Git Bash invocation and distinguish these local setup corrections from CI gains. Validation also exposed a daemon-test deadline shorter than Windows' closed loopback-port refusal delay: fresh closed ports returned `TimeoutError` at two seconds and `ConnectionRefusedError` just after 2.02 seconds. Allow five seconds for the post-shutdown check while retaining its strict refusal assertion and restart coverage. A controlled, serial 37-case Windows Gix sample with identical flat config, profiling enabled and coverage disabled improved from 87.10s to 77.20s. Git launches fell from 1,437 to 1,128: 132 fewer version probes and 177 fewer hook checks. This is about 11% faster in the sample, not a full-suite estimate. Validation used CPython 3.12.13, Git 2.55.0.windows.5 and `GixPython` 0.1.0: - Both complete submodule modules and related checks ran with CI's coverage options. CLI finished with 542 passed, 1 skipped, 1 xfailed and 3 xpassed. - The initial Gix selection had 546 passed, 1 failed, 1 xfailed and 3 xpassed. Its sole failure was the daemon deadline corrected here; that regression passed on a targeted rerun and in the subsequent CLI run. - Replaying the original submodule source from `c3a2832b` reproduced the CLI `test_base_rw` file-lock skip, confirming it predates these changes. - Pre-commit checks passed for all changed files. Git Bash checks covered executable resolution, explicit overrides and the non-Windows branch; changed Python files also parse with Python 3.8 syntax.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tasks
Created by Codex on behalf of Byron. Byron will review before this is ready to merge.
Summary
Important
This PR will become GitPython 3.3 some weeks before Git 3 is officially released in March.
Users can already test and run their software against this branch. Please report compatibility problems and provide feedback here in this PR.
To try it with Git 2.52 or newer installed:
python -m pip install --upgrade "git+https://github.com/gitpython-developers/GitPython.git@gix-backend"GitPython now delegates repository discovery, configuration, references, reflogs, object storage, tree construction, index operations, and revision parsing to guarded Git commands. The default backend supports SHA-1 and SHA-256 objects with either files or reftable reference storage, without interpreting Git's binary storage formats in Python.
The deprecated
GitDBremains explicitly selectable, and rawrepo.gitcommand access remains available. The migration notes inchanges.rstdescribe the API changes and differences in Git CLI behavior.Changes
1. Git CLI implementation of the GitPython API surface
The ordinary installation implements the supported GitPython API through guarded
git.cmd.Gitcalls. Git owns repository storage and format handling, including SHA-1/SHA-256 objects and files/reftable references. This removes Python storage-format parsing from the default backend while retaining common documented workflows, rawrepo.gitaccess, and the explicitly selectable deprecatedGitDB. The existing context and migration notes below describe safety controls and low-level API changes.2. GixPython implementation for better performance
The optional
GitPython[gix]installation uses the publishedGixPython==0.1.0release on CPython 3.11+. GitPython selects it automatically when thegixmodule is importable; the ordinary installation continues to support Python 3.8+. To try the native backend:python -m pip install --upgrade "GitPython[gix] @ git+https://github.com/gitpython-developers/GitPython.git@gix-backend"Supported library-managed operations use Rust-backed object reads/writes, revision and graph queries, tree enumeration and construction, index reads/edits, references and reflogs, commit creation/statistics, tree diffs, status, untracked files, and ignore matching. Unsupported formats and options retain the guarded CLI implementation. In particular, reftable and
extensions.compatObjectFormatrepositories currently use Git. Native mutation failures are reported without attempting a second CLI mutation. Publicrepo.gitcommands retain their CLI behavior.Recorded local measurements with official GixPython 0.1.0 on CPython 3.12.14, macOS arm64:
These are individual local validation runs, not statistical benchmarks or a guarantee for every application. The full suites collect different backend-specific tests; the focused selections overlap. See the backend guide and performance journal for conversion coverage, remaining CLI cases, reproduction details, and the 19-item GixPython/Gitoxide follow-up ledger. Native repository-handle reuse remains future work and is not included in these gains.
3. Significantly faster CI test execution
Repeated repository construction now happens once where safe: historical dependency sources are prepared lazily, and submodule/revision fixtures copy prepared baselines into independent writable repositories. Actor/exception tests avoid unnecessary repository setup, and historical tree reads avoid cloning and checking out a worktree. Tests retain fresh wrappers, independent mutable Git metadata, existing security snapshots, and explicit isolation regressions.
On the same local coverage-enabled GixPython suite, fixture optimizations reduced wall time from 757.85 s (12m38s) to 404.61 s (6m45s): 353.24 s saved, 46.6% less time, about 1.87× faster. CLI fallback decisions decreased from 39,002 to 21,402; these counters do not count every Git subprocess. This is the before/after fixture improvement, separate from the backend comparison above. The changes reduce repeated work in the CI suites; the quoted timings are local measurements, not a hosted GitHub Actions before/after benchmark. No matching pre-optimization full CLI run was recorded.
The branch also adds released downstream compatibility checks for LangChain, MLflow, Bandit, SWE-bench, and DataHub, plus reproducible local test execution and backend-operation reports.
Context
The goal is to support Git's object and reference formats while reducing the attack surface of Python implementations and limiting argument injection and unintended CLI side effects.
git.cmd.Git, existing unsafe option/protocol checks, validated operands, and framed stdin records. Shell execution and option reordering past protective flags are rejected. Persistentcat-filerequests use NUL framing.git hook run. Signing/editor options and custom archive format commands require an unsafe-options opt-in.GitCommandErrorwith Git's status and diagnostics.oldhexsha, precompressed object streams and custom object-output writers,Submodule.rename(), and directRepo.alternatesmutation. Configuration follows Git syntax and no longer subclassesRawConfigParser. Detailed replacements and limitations are in the changelog.The change also updates documentation, format/backend and injection regressions, the minimum-Git CI job, and fuzz harnesses. Existing performance benchmarks use bounded samples to keep subprocess-based runs practical. The release announcement above describes the planned release; this PR does not change
VERSIONyet.CI follow-up handles platform path conventions and rejects unsupported index filenames before changing the original index. Submodule updates now close their internally opened repositories, preventing retained log records from keeping Windows checkouts open through Git processes. Cygwin builds the supported Git version in its normal installation layout and verifies detection before testing.
User Prompts