Add a setting to share a doctest namespace without merging blocks into one test

[tony]

Summary

#87 delivers a shared doctest namespace by merging a namespace's blocks into one doctest.DocTest, and therefore one pytest item. That works everywhere, including under distribution, but it spends per-block node ids for the blocks that share. #88 asks for the opposite trade and is right that Sphinx's mechanism keeps both — its objection is not answerable by argument, because the two designs are optimal under different run configurations.

This issue proposes shipping both, as orthogonal settings, so neither is imposed. It is strictly additive: no existing run changes, no stdlib or pytest class is subclassed, and the new path is refused loudly rather than silently when the run cannot support it.

The insight that makes this cheap: #87 conflated two independent choices.

• Scope — what shares a namespace: a block, a declared group, or the document.
• Items — whether sharing merges those blocks into a single test, or leaves each block as its own test.

doctest_docutils_namespace_scope already exists and answers the first. This issue adds a second setting answering the second, defaulting to today's behaviour.

Background

The reason this needs a mechanism at all is a two-line contract in the stdlib. DocTest.__init__ copies the namespace it is handed, and DocTestRunner.run clears it afterwards by default at Lib/doctest.py#L1577-L1578. Between the copy and the clear, N DocTest objects cannot share state by default.

There are exactly two ways around it, and both are sanctioned:

• Merge — one DocTest holding every block's examples. This is what pytest's own text-file collector does, calling parser.get_doctest once for a whole file.
• Share the mapping — assign the same dict to each block's test and disable the clear. This is what sphinx.ext.doctest does, with the two obstacles annotated in-line: # DocTest.__init__ copies the globs namespace, which we don't want and # also don't clear the globs namespace after running the doctest. The stdlib itself uses the override — DebugRunner.run passes clear_globs=False.

Neither is a hack. They differ only in what unit becomes a pytest item.

What per-block node ids buy, and what merging costs

Carried over from #88, which this issue supersedes.

A page collected with one item per block yields one id per block, and those ids are what make a block reachable:

• pytest page.md::page.md[6] to re-run one block while iterating on it
• --lf re-running only the block that failed, rather than the whole page
• -k and --deselect reaching a block
• a JUnit report naming the block that failed

The measurement that motivated #88, on a 16-block page — note this is not an argument from speed:

Layout	Wall clock
16 items, -n auto	6.73 s, 6.38 s
serial (what 1 item costs)	4.65 s, 5.10 s

Per-block items are in fact slower at that size, because xdist worker startup outweighs the parallelism. The case for keeping them is debuggability and tooling reach, not throughput.

The cost merging does not advertise

One item means one function-scoped fixture setup for the whole namespace, not one per block. That is not a detail — it is a written contract in the largest consuming project. libtmux's own contributor documentation states it verbatim:

Every code block on a page is an independent doctest. Blocks do not share a session: each one gets a fresh copy of the namespace and a fresh tmux server. A later block cannot use a pane an earlier block created — the name is simply not defined there.

and its README advertises "Fresh tmux server/session/window/pane fixtures per test".

Merging keeps that promise literally — a merged page is one test, so it still gets one fresh server per test — while inverting it in practice, because the number of tests on the page collapsed from sixteen to one. Measured on gp-libs' own suite: a page with two gated blocks went from 1 fixture setup to 3 when blocks were lifted into their own items, and back to 1 when they were marked before setup. The same arithmetic runs the other way for merging.

This is the strongest single argument for the setting proposed here, and it is the reason a project like libtmux would want per-block even at the cost of pinning a scheduler.

The constraint that shapes the design

A shared mapping is a Python object, so it does not cross processes. pytest-xdist gives every worker its own collection and its own memory (docs/how-it-works.rst), and its documentation is explicit that state is not shared across workers (docs/how-to.rst).

Measured against a shared-mapping build, on the two-block page from #83:

• serial — 2 passed
• -n 2, default scheduler — 1 failed, 1 passed
• -n 2 --dist loadfile — 2 passed
• -n 2 --dist loadgroup, no xdist_group marker — 1 failed, 1 passed

--dist defaults to no, and -n without --dist promotes it to load at src/xdist/plugin.py#L304-L334. loadfile keeps a file's items on one worker by splitting the scope on the nodeid's path (scheduler/loadfile.py#L35-L60). loadgroup reads the xdist_group marker on the worker and suffixes the nodeid with @group (src/xdist/remote.py#L236-L254); an item with no marker distributes as plain load.

So a plugin can emit the marker but cannot choose the scheduler. Sharing the mapping is therefore conditionally correct — and, critically, the condition is detectable.

Detection is available and public

On the controller, config.option.dist holds the real value at pytest_configure time. Verified empirically:

• serial → no
• -n 2 → load
• -n 2 --dist loadfile → loadfile
• -n 2 --dist loadgroup → loadgroup
• -n auto → load

On a worker it is forced to "no" at src/xdist/remote.py#L392-L400, so the check must be gated on the public is_xdist_controller helper. That is the difference between a footgun and a supported option: the plugin can refuse to run silently-wrong.

Profiling

Neither shape is a throughput win, in either direction. A 16-block Markdown page, three runs averaged:

• block scope, serial — 16 items, 635 ms
• document scope (merged), serial — 1 item, 662 ms
• block scope, -n auto — 16 items, 3351 ms
• document scope (merged), -n auto — 1 item, 3057 ms

Serially the difference is noise. Under -n auto, worker startup dominates both by roughly 5x. This confirms the measurement in #88 and extends it: item granularity is an ergonomic choice, not a performance one, so the decision should be made on debuggability and safety alone.

Proposal

Add one setting, defaulting to current behaviour.

doctest_docutils_namespace_items, ini plus a matching command-line flag:

• merged (default) — a namespace becomes one DocTest and one test. Today's behaviour, unchanged.
• per-block — every block keeps its own DocTest and its own node id; each is handed the same namespace mapping, and the runner is told not to clear it between blocks.

Under per-block, the plugin additionally:

1. Emits an xdist_group marker per namespace during collection, so --dist loadgroup groups a document's blocks onto one worker. The marker is consumed at remote.py#L236-L254.
2. Registers that marker unconditionally at pytest_configure, so --strict-markers users who never opt in are unaffected — the idiom pytest-asyncio uses for its own marker.
3. Raises pytest.UsageError on the controller when the setting is per-block, sharing is on, and the scheduler is load — naming --dist loadfile as the fix. Erroring on an unsupported configuration is established pytest-asyncio practice.
4. Reports the resolved setting in pytest_report_header, so the mode is discoverable without reading source — again pytest-asyncio's pattern.

Why this is additive

• stdlib doctest — nothing subclassed, nothing monkeypatched. clear_globs is a documented parameter and test.globs a documented attribute; DebugRunner already sets the precedent. One consequence to document rather than prevent: compileflags is derived from the namespace at run time via _extract_future_flags(test.globs), so a shared mapping shares __future__ flags between blocks.
• _pytest.doctest — untouched. DoctestItem keeps taking one dtest per item (#L251); this changes only what doctest_docutils emits. Following syrupy's example, no core item class is subclassed — the plugin works through collection hooks and its own state.
• doctest_namespace — pytest merges fixtures with self.dtest.globs.update(globs) at #L288-L293. Under per-block, seeding the mapping before the first block preserves that for every block.
• existing suites — the default is merged, and the default scope is block, so nothing changes until a project opts in twice.
• pytest-xdist — the incompatible combination is refused at configure time rather than producing a wrong result.
• Sphinx builds — unaffected; the directive nodes are unchanged, and a ```{doctest} fence still resolves through the same docutils directive registry that reST uses.

The discipline here is the one tmux applies to its own option surface: add the new spelling, keep the old one working, and say so. See CHANGES#L1164 — "refresh-client -F has become -f (-F stays for backwards compatibility)" — and #L1300 on terminal-overrides.

Acceptance criteria

1. With no configuration, collection, node ids, item counts and reported lines are byte-identical to Share doctest namespaces across a page, and honour directive options #87's behaviour.
2. Under namespace_items = per-block with sharing enabled, a document's blocks share state and each keeps its own node id (page.md::page.md[1]).
3. Under namespace_items = per-block, pytest <nodeid> for a block that depends on an earlier block fails informatively rather than mysteriously — the limitation is inherent, so it must be documented, not hidden.
4. Under namespace_items = per-block with --dist load (including bare -n auto), the session stops with a pytest.UsageError naming --dist loadfile.
5. Under namespace_items = per-block with --dist loadfile, a state-building page passes.
6. Under namespace_items = per-block with --dist loadgroup, a state-building page passes, driven by an emitted xdist_group marker.
7. --strict-markers passes for projects that never opt in.
8. doctest_namespace fixtures reach every block under both settings.
9. testsetup / testcleanup ordering and :skipif: / :options: semantics are identical under both settings.
10. python -m doctest_docutils and testdocutils() behave sanely under both settings, since neither has a scheduler.
11. The resolved setting appears in pytest_report_header.

How this relates to the open issues

• Share doctest namespaces without collapsing per-block node ids #88 — this is that issue's criterion 1, made shippable by scoping it to the schedulers where it can hold. Share doctest namespaces without collapsing per-block node ids #88 stays the record of why per-block ids are worth keeping.
• Share doctest namespaces across a page, and honour directive options #87 — the PR delivering the merge setting. This issue is a follow-up to it, not a competitor; it adds the second setting on top.
• Doctest blocks in one document do not share a namespace, unlike pytest text files #83 — the original namespace request. Satisfied by either setting; this one satisfies it without spending ids.
• pytest node IDs embed an absolute path #85 — node ids. Its Expected now keeps the [N] index, which is exactly what the shared setting preserves for every block.
• Doctest directive options are parsed and then discarded #84 and Doctest :pyversion: raises InvalidVersion at collection #86 — directive options and :pyversion:. Same construction site (_find), so they should land before this to avoid a second rewrite of that function. Both are fixed in Share doctest namespaces across a page, and honour directive options #87.
• doctest: Upstream updates #26 — upstream doctest updates. This issue adds a dependency on the clear_globs / test.globs contract, which is the surface doctest: Upstream updates #26 would need to re-check against a new CPython. Worth cross-linking permanently.
• pytest-asyncio incompatibility #43 — pytest-asyncio incompatibility. Adjacent: both concern how gp-libs coexists with another collection-time plugin, and pytest-asyncio is also the design precedent cited above.
• doctests in python open source: Position paper #10 — the position paper on doctests in open source. The scope/setting split is the kind of distinction that paper argues for; worth folding in if it is ever written up.
• [Agentic DX] gp-libs: low-to-high lift plugin/API clarity roadmap #62 — the API-clarity roadmap. A second public setting raises exactly the stable-API-versus-internals question that issue raises; the setting and scope names should be declared stable if this ships.
• docs(pytest-plugin): Render fixture cards on /doctest/pytest/ via sphinx_autodoc_pytest_fixtures #71 — fixture-card rendering. Unaffected, but the new option should appear in the same generated reference.

Open questions

1. Is doctest_docutils_namespace_items the clearest name, or should the two settings be collapsed into a single enumeration?
2. Should per-block auto-upgrade the scheduler when it detects load and a single worker, or always refuse? Refusing is simpler and never surprising.
3. pytest_xdist_make_scheduler (newhooks.py#L95-L99) is the documented extension point for custom distribution. Supplying a scheduler is more invasive than refusing, but would remove the user-facing requirement entirely. Worth evaluating before committing to the UsageError.
4. Should per-block be marked experimental for one release, with the deprecation-warning discipline pytest-asyncio uses when a default is going to move?