Skip to content

Python 3: language-layer cleanup, portable CLR loading and out-params, plus compat tests - #3484

Open
ChrisCrosley wants to merge 19 commits into
pyrevitlabs:developfrom
ChrisCrosley:fix/improve-python3-support
Open

Python 3: language-layer cleanup, portable CLR loading and out-params, plus compat tests#3484
ChrisCrosley wants to merge 19 commits into
pyrevitlabs:developfrom
ChrisCrosley:fix/improve-python3-support

Conversation

@ChrisCrosley

Copy link
Copy Markdown
Contributor

TLDR: This is a long description but Claude did a pretty good job reasoning through this and I thought it would be worth sharing in full. This PR attempts to pick up all of the Python3 specific language changes to make IronPython3.4 stable, and is a first step towards CPython + pythonnet migration. The PR comes with a command line script to search for Python2 issues that extension developers can also run on their repos and 2 test buttons. There are some know issues at the bottom that may be worth discussion.

Description

One small step along the migration path from IronPython 2.7 -> CPython + pythonnet. That path splits into (at least...) two layers: Python-3 language idioms, which break on any Python 3 engine (IronPython 3 included), and pythonnet-bridge marshaling idioms, which break only on CPython+pythonnet (IronPython has native .NET integration and never hits them).

This branch attempts to clear the language layer in full and the first two of roughly eight bridge classes - IronPython-only CLR loading (clr.AddReferenceToFileAndPath) and clr.Reference out-params - plus the test infrastructure to verify them per engine. It deliberately does not complete the bridge: the remaining marshaling classes (generic-collection construction, interface __namespace__, enum/indexer/IDisposable/overload idioms) are surveyed and tracked as a measured backlog, not fixed here - see Scope. It does not attempt the pyrevit/forms WPF/CPython port or change engine selection; language-layer residuals inside forms/_ipy.py are fixed where found (e.g. the __bool__ aliases and the Python-3 view/iterator bugs below).

Net effect: this makes pyrevitlib and the shipped extensions correct under IronPython 3 (the language layer is exactly what IPY3 exposes), and lands the first layer of CPython support - not CPython-correctness, which the bridge backlog gates.

Every change here keeps working on the default IronPython 2 engine; nothing in this branch changes engine selection or public API signatures.

The changes are covered by a static checker (pipenv run check-py3) and an in-Revit suite run by two DevTools buttons (Stage 0). The suite is green on all three engines, with documented skips for the known issues below.

Stages

  • Stage 0 - test infrastructure (the safety net; landed first)
  • Stage 1 - Python-2 syntax residuals (iteritems, bare unicode, __nonzero__/__bool__ aliases)
  • Stage 2 - framework CLR shim (engine-agnostic assembly loading; removes clr.AddReferenceToFileAndPath from all call sites outside pyrevit.framework)
  • Stage 3 - out-param marshaling (the two clr.Reference sites in revit/db/create.py and revit/db/query.py)
  • Stage 4 - heterogeneous-sort hardening (the Fix/logging bug fixes #3477 fix pattern applied to the remaining risky keyless sorts; checker at zero; CI gate on)
  • Stage 5 - extended checker coverage ( PY3-VIEW view/iterator class + the 2to3/pylint --py3k rules PY2-MODULE/PY2-BUILTIN/PY2-NEXT, and the residuals they found)

Stage 0 - test infrastructure

Static checker: dev/scripts/check_py3_compat.py

AST-based scanner for the Python-2-only and IronPython-only patterns this branch targets. Stdlib-only and path-agnostic: third-party extension authors can run it against their own .extension folders with a stock Python 3.

pipenv run check-py3              # scan pyrevitlib + shipped extensions
python dev/scripts/check_py3_compat.py MyExt.extension   # any folder
Code Flags
SYNTAX file does not parse as Python 3
SYNTAX-WARN parse-time SyntaxWarning (e.g. '\d' escapes in non-raw strings - a hard error in future Pythons)
PY2-ITER .iteritems() / .iterkeys() / .itervalues(); Py2-only itertools members (ifilterfalse, izip, ...) in any usage form
PY2-HASKEY .has_key()
PY2-NAME xrange / basestring / unicode / unichr / long / StandardError
PY2-BUILTIN a removed builtin called: raw_input / execfile / apply / cmp / coerce / intern / buffer / reduce / reload / file (skipped when the name is imported)
PY2-MODULE import of a stdlib module renamed/removed in Python 3 (StringIO, Queue, cPickle, ConfigParser, urllib2, ...); guard-aware
PY2-NEXT iterator class defines next() (with __iter__) but no __next__
PY2-BOOL class defines __nonzero__ without __bool__
IPY-CLR clr.AddReferenceToFileAndPath outside pyrevit.framework
CLR-REF clr.Reference outside the sanctioned dispatch sites (query.py, create.py)
SORT-KEY keyless sorted() over dict .items()/.values()
PY3-VIEW a bare view/iterator (keys/values/items, map/filter/zip) that escapes local scope - indexed, returned, yielded, or stored to an attribute (was a list in Python 2)

Engine-guarded shims are recognized and skipped (if PY2: branches, @skipUnless(PY2, ...) decorators, try/except NameError fallbacks, local redefinitions). pyrevitlib/rpw and pyrevit.coreutils.markdown are excluded (see Vendored-library policy).

Baseline at Stage 0 was over 30 findings.

In-Revit test suite: pyrevit.unittests.test_py3_compat

Revit-hosted suite exercising the compatibility hotspots. Two DevTools buttons under pyRevitDev → Debug → Unit Tests run the identical suite on the attached IronPython engine and on CPython (#! python3), making it a per-engine parity dashboard: a failing test is a coverage signal marking exactly what a later stage fixes.

Test procedure: run both buttons under the default (IPY2) attach; switch the attached engine to IPY342 in Settings, reload, and run again. The out-param tests build fixtures inside rolled-back transactions (no trace left in the document) and reuse the existing DevTools A.rfa.

Results on a Revit 2025+ (.NET 8) host - all cells verified in Revit through Stage 2, with the CPython column re-verified fully green after Stages 3 and 4:

Test Exercises IPY2 IPY3 CPython
test_core_module_imports core pyrevit.* modules import pass pass pass
test_interop_module_imports managed interop modules' CLR loading (dxf, ifc) skip† skip† skip†
test_interop_native_module_imports rhino/adc imports - native/external binaries; opt-in via TEST_NATIVE_INTEROP skip (opt-in) skip skip
test_framework_asm_file_reference the Stage 2 shim, via an extensionless path pass pass pass
test_basewrapper_repr ElementWrapper.__repr__ (iteritems) pass pass pass
test_forms_listitem_truthiness TemplateListItem __nonzero__ pass pass skip
test_forms_paramdef_truthiness ParamDef __nonzero__ pass pass skip
test_get_param_by_name / test_get_category_by_name query string-identifier lookups pass pass pass
test_load_family_out_param clr.Reference in create.load_family pass pass pass
test_curve_intersect_out_param clr.Reference out-param of query.get_gridpoints, on pure geometry pass pass pass
test_envvars_sortable_by_name the #3477 sort pattern pass pass pass

† On .NET 8 hosts the assemblies are not shipped (see known issue below). On netfx hosts (Revit ≤2024) the managed interop test runs: expected pass under IronPython, and it is the test that proves the Stage 2 shim for interop under CPython - i.e. that verification requires a Revit ≤2024 host.

The rhino/adc import tests are opt-in because importing them loads unmanaged and version-uncontrolled binaries into the Revit process: rhino3dmio_native.dll (rhino3dm's native C++ core; also netfx-only, so loading it into a .NET 8 host is untested territory) and whatever assemblies the machine's Desktop Connector install carries. Managed load failures are catchable and just turn a test red; a bad native load is an access violation nothing can catch. A test suite's worst failure mode must be a red row, not a dead host. This gating is precautionary - these imports were the initial suspect for the IPY342 hard-crashes but were exonerated (the actual cause was the markdown stack overflow below); the risk class is real regardless. Certifying these modules means flipping TEST_NATIVE_INTEROP deliberately, one module at a time.

Stage 1 - Python-2 syntax residuals

All PY2-ITER, PY2-NAME, and PY2-BOOL findings cleared. Flips the test_basewrapper_repr matrix cells green on IPY342/CPython and test_forms_listitem_truthiness on IPY342.

  • revit/db/__init__.py - pdata.iteritems().items(); this was the crash inside BaseWrapper.__repr__ on any Python 3 engine.
  • forms/_ipy.py - __bool__ = __nonzero__ aliases on TemplateListItem and ParamDef. Python 3 ignores __nonzero__, so under IPY342 those objects were silently always-truthy, breaking checked-state logic.
  • Vendored coreutils/markdown/ - restored the text_type definition in util.py's Python-2 branch (it was commented out) and routed all bare unicode uses through it, including class AtomicString(unicode)(text_type); __version__.py uses plain str (ASCII by construction). The package no longer depends on compat.py's unicode builtins injection.
  • revit/db/query.py - isinstance(x, (str, unicode))isinstance(x, str). Sound on every engine pyRevit ships: IronPython 2 has a single string type (str is unicode), and CPython 2 was never a pyRevit engine.
  • Override Text.pushbutton - unicode(el.Text)str(el.Text), same reasoning.

Two additional vendored-markdown bugs, found by running the new suite in Revit (the CPython cell showed 2 errors beyond the expected Stage 2/3 reds):

  • extensions/footnotes.py - Element.getchildren() was removed in CPython 3.9 (still present in IronPython 3.4, which is why only the CPython run errored); replaced with len(li) / list(parent).
  • Extension loading by name was broken on every engine, IPY2 included: both extra.py's sub-extension list and build_extension's fallback hardcoded a top-level markdown.extensions. path that does not exist in the vendored copy. Both now resolve against the vendored package, so markdown.markdown(text, extensions=["tables"]) works for the first time. Verified under local CPython by importing the package standalone.

The markdown fixes are kept (they are correct and help third-party callers on every engine - the extension-loading bug was broken under IPY2 too), but the four markdown tests were removed and coreutils/markdown is now excluded from the checker: the package has no first-party runtime consumers and is a deprecated unbundling candidate (see the vendored-library policy), so it is no longer part of the maintained surface. The two known-issue sections below document why.

Stage 2 - framework CLR shim

  • New public API: framework.add_reference_to_file(asm_file), the engine-portable replacement for IronPython's clr.AddReferenceToFileAndPath. It dispatches to that API under IronPython (byte-for-byte legacy behavior); under CPython/pythonnet it loads the file via Assembly.LoadFrom, appending .dll when missing (detected by suffix, not splitext - assembly names contain dots: IxMilia.Dxf). Documented as public surface: "how do I load a DLL portably" is the first question a #! python3 extension author hits.
  • Converted all 13 active call sites: interop/adc.py (×6), dxf.py, ifc.py, rhino.py, coreutils/git.py (which also loses its inline engine branch), coreutils/mathnet.py, and loader/asmmaker.py's Lokad.ILPack load. Not converted: framework.py internals (the shim site, already engine-gated), rpw (frozen legacy), and the DevTools IronPython-compile test (IronPython-only by design).
  • Distinct from coreutils.assmutils.load_asm_file, which bare-loads via Assembly.LoadFrom and does not register the assembly for import under IronPython - a same-sounding name with a different contract, which is why the new shim lives in framework (the CLR-loading hub) under a distinct name.
  • New suite test test_framework_asm_file_reference loads a shipped managed assembly (pyRevitLabs.Json) through the shim via an extensionless path, on every engine.

With the shim in place, pyrevit.interop becomes importable under #! python3 - verifiable only on a netfx host (Revit ≤2024), since .NET 8 builds do not ship the interop assemblies (see known issue).

Stage 3 - out-param marshaling

Ports the two clr.Reference out-param sites in pyrevitlib to engine-dispatched code: IronPython keeps the explicit clr.Reference/StrongBox path byte-for-byte; CPython/pythonnet uses the tuple-return convention with a None filler for the out argument (plain omission would resolve to the wrong overload where a no-out sibling exists, e.g. Curve.Intersect(Curve)).

  • query.py - new public helper query.intersect_curves(c1, c2) wrapping Curve.Intersect's out-param; get_gridpoints now uses it, and the suite tests it directly on pure geometry.
  • create.py - load_family branches the Document.LoadFamily call, and FamilyLoaderOptionsHandler gets the two things pythonnet requires of Python-implemented .NET interfaces: a __namespace__ attribute, and ref-param handling by convention - IronPython mutates the passed StrongBox, pythonnet returns the updated values in a tuple.

Verified in Revit: the CPython suite run came back fully green (12 pass / 4 skip / 0 errors), proving on pyRevit's pythonnet fork the None-filler out-arg convention, the tuple return, and a Python-implemented .NET interface (IFamilyLoadOptions with __namespace__ and tuple-returned ref-params) receiving live callbacks from Revit.

Extension follow-through. A full-tree sweep confirmed pyrevitlib's two sites were the only library sites, but three shipped extension scripts carried the same idiom; all three now use the new wrappers:

  • Section Box Navigatorquery.intersect_curves (its code carried a "weird ironpython stuff - change when cpython works" comment; that day came)
  • Load More Types and Load Familiesrevit.create.load_family, which also replaces crash-on-failed-load with a graceful empty list

Also verified: no other Python-implemented .NET interface in first-party code has ref/out params in its callbacks - IFamilyLoadOptions was the only one. A new checker rule (CLR-REF) flags clr.Reference anywhere outside the two sanctioned dispatch sites (query.py, create.py) so the idiom cannot creep back.

Manual testing of the converted Load Families tool under IPY342 surfaced one more pre-existing Py2 residual in a sibling lib: file_utils.py used itertools.ifilterfalse (renamed filterfalse in Python 3). It was replaced with a set comprehension, which also restores the documented set type that the iterator had silently replaced. This exposed a checker blind spot, now closed: Py2-only itertools members (ifilter, ifilterfalse, imap, izip, izip_longest) are flagged in all three usage forms (attribute access, from itertools import ..., bare names), guard-aware as usual. A tree-wide re-scan found no other instances.

Stage 4 - heterogeneous-sort hardening

Applies the #3477 fix pattern (key=lambda kv: str(kv[0])) to the three remaining keyless dict-view sorts: Get RVT Info (over a .NET dictionary - the closest analog of the original Settings crash), ParametersValuesToParameter (category name → BuiltInCategory pairs), and the ExportIFC dev example. The survey's other suspects (multiline sorted() calls in forms/_ipy.py) turned out to be false alarms from line-based grep - each carries a key= on its continuation line, which the AST checker had correctly not flagged.

This cleared the checker's original baseline to 0, wired into CI as the py3-compat job - any reintroduced Py2 idiom, IronPython-only CLR call, stray clr.Reference, deprecated escape sequence, or risky keyless sort fails the build. Manual test for the three fixed tools: run each button; only the sort order of displayed entries could differ.

Stage 5 - extended checker coverage

Stages 1-4 drained the checker's original rule set. Two things then extended it: an IronPython-3 Search-button bug report, and a review of whether the checker was comprehensive against the canonical Py2→3 catalog. Both added detection and drained the residuals they surfaced, keeping the tree at zero.

Python-3 view/iterator residuals

Chasing an IronPython-3 Search-button report ("not subscriptable") surfaced a language-layer class the original sweep missed - dict.keys()/.values()/.items() and map/filter/zip return lists in Python 2 but non-indexable lazy views in Python 3 (breaks on any Py3 engine, IronPython 3 included; it is not the bridge "collections are views" item, which is pythonnet-only .NET collections). Fixes:

  • forms/_ipy.py - search_matches returned a bare KeysView that the search loop then indexed (results[i]); the grouped SelectFromList similarly did .index() on a bare view. Both wrapped in list(). (These are language residuals in _ipy.py, in the same category as Stage 1's __bool__ fix - not the deferred WPF/CPython port.)
  • five public functions that returned a bare view/iterator (pyutils.pairwisezip, query.get_schedulesfilter, two extension load_configsfilter) and two that stored one to a WPF ItemsSource (later .index()-ed) - all wrapped in list().

New checker rule PY3-VIEW, designed around escape: a bare view is flagged only when it leaves local scope where its use is unknowable - indexed, returned, yielded, or stored to an attribute. It deliberately does not flag a plain-local x = d.keys() (usually just iterated - too noisy) nor foo(d.keys()) call-arguments. The call-argument case was measured (15 sites, mostly SelectFromList.show/output.print_table) and left out for a reason worth recording: all the core consumers already iterate their inputs rather than index them (for item in ctx_items), so passing a view is already safe - and a call-arg rule couldn't be satisfied by the correct fix (hardening the consumer) anyway. That is the "stricter on the core library" answer in practice: the library is defensive in both directions - it does not return bare views, and it iterates (never indexes) collection inputs, which protects every caller without touching call sites. Rule limitation: it catches syntactic escape shapes, not data-flow (a view assigned to a local and then indexed elsewhere, like the grouped-select bug, still needs code review - that one was found by hand).

Rounding out the rule set (2to3 / pylint --py3k)

A completeness review against the canonical catalog: all pure-syntax removals were already caught free by the SYNTAX parse step, but three statically-detectable classes were still missing and are now covered:

  • PY2-MODULE - stdlib module renames (StringIO, Queue, cPickle, ConfigParser, urllib2, ~35 total), guard-aware so compat.py's if PY2: imports are skipped.
  • PY2-BUILTIN - removed builtins when called (raw_input, execfile, apply, cmp, reduce, reload, file, ...), import-aware so from functools import reduce; reduce(...) is not flagged.
  • PY2-NEXT - iterator classes with next() + __iter__ but no __next__ (the PY2-BOOL analog). Plus StandardError added to PY2-NAME, and local-name detection made import-aware.

These found and fixed 4 residuals, all PY2-MODULE, all in extensions: three unused import urllib2 (removed) and one import StringIO in Wipe Site Designer Line Styles → a plain-list accumulator (sidesteps io.StringIO str/unicode ambiguity on the default IPY2 engine). PY2-BUILTIN and PY2-NEXT found nothing - clean, so they are pure regression insurance. (The semantic-only changes the checker deliberately leaves to the in-Revit suite - division, str/bytes, dict-ordering - are listed under Stage 0.)

Vendored-libraries

Two vendored libraries came up repeatedly. They are different cases and get different treatment; stating both explicitly so it survives into the migration docs.

pyrevitlib/rpw (revitpythonwrapper 1.7.4, unmaintained upstream) - engine-locked legacy. It subclasses WPF types and loads IronPython.Wpf, so it genuinely cannot run on CPython. Supported only under the opt-in legacy IronPython engine; excluded from the Python-3-supported surface, this branch's checker, and the shim conversions. pyrevit.forms is the documented replacement for its UI layer. Its real constituency is third-party IronPython scripts, which keep the legacy engine regardless.

pyrevit.coreutils.markdown (python-markdown 2.6.8) - deprecated, portable orphan (unbundling candidate). Unlike rpw it is pure Python (regex + ElementTree, no clr/WPF), so it runs on any engine by nature - it was never engine-locked. It was vendored in December 2016 as a genuine core dependency (the output window rendered print_md through it), then orphaned in June 2026 when the output refactor moved rendering to C# (ScriptOutput.cs), leaving zero first-party runtime consumers. Because it has no internal consumer, this branch stops investing in it: the four markdown compat tests were removed and coreutils/markdown is excluded from the checker. The Stage 1 syntax and vendoring fixes are kept - they are correct, cheap, and help third-party callers on every engine (the extension-loading bug was broken under IPY2 too) - but reverting them would be pointless churn, not a reason to keep maintaining the package. It stays at its current path rather than moving to site-packages/, because a top-level markdown there would both break every pyrevit.coreutils.markdown importer and collide with pip-installed markdown 3.x under the shared CPython interpreter. #! python3 scripts should use the real markdown from pip - PyPI access being the point of the migration - and the vendored copy is a candidate to unbundle and delete.

Both libraries are deletion candidates when the legacy IronPython engine sunsets.

Known issue: markdown conversion hard-crashes Revit under IPY342

Running the suite under IPY342 hard-crashed Revit (no error dialog). Reproduced and root-caused outside Revit by hosting the repo's bin/netfx/engines/IPY342 DLLs in a disposable process: markdown conversion dies with a .NET StackOverflowException (exit 0xC00000FD), the one exception .NET cannot catch. IronPython 3.4's DLR stack frames are much fatter than IPY2's or CPython's, and markdown's parser is recursion-heavy; measured standalone, the conversions need ~768KB of fresh stack. On Revit's already-deep UI thread that headroom does not exist.

This was originally handled with skipIf(IRONPY3) markdown tests, but those tests were later removed entirely (see the vendored-library policy): the crash is not test-specific - any script calling pyrevit.coreutils.markdown under IPY342 inside Revit risks it - and the package has no first-party runtime consumers (verified: output.print_md renders via C# in ScriptOutput.cs). This crash is itself part of why markdown is treated as a deprecated, unbundling candidate rather than a maintained surface. The same investigation surfaced a related managed (non-crash) issue in the extra extension, below.

Known issue: markdown extra/attr_list broken under IronPython (2 and 3)

attr_list.py requires re.Scanner, which IronPython's .NET-regex-based re does not provide on either engine version; its from sre import Scanner fallback is dead in Python 3 and unavailable in the shipped IPY2 stdlib. The Stage 1 vendoring fix exposed this: previously, name-based extension loading failed so early that extra never worked on any engine - which is also evidence it has no users, and another data point for deprecating the vendored copy. No fix is planned; scripts needing extra under a Python-3 engine should use pip markdown.

Known issue: interop assemblies missing on .NET 8 hosts (Revit 2025+)

Found by running the suite on a .NET 8 Revit: IxMilia.Dxf, Ifc.NET, and Rhino3dmIO ship only in dev/libs/netfx/bin/netfx/; the netcore lib set omits them. pyrevit.interop.dxf/ifc/rhino therefore cannot import on Revit 2025+ on any engine - a packaging gap in shipped pyRevit that predates this branch (the docs site advertises these modules). The managed interop test is skipIf(NETCORE) with this reason. Closing the gap needs either Stage 2 verification on a netfx host (Revit ≤2024) or a decision to ship netcore copies (IxMilia.Dxf targets netstandard and should port; Ifc.NET and Rhino3dmIO need evaluation).

Third-party impact

None breaking. Details were reviewed change by change: public signatures and return types are preserved; the Stage 2 shim is additive (and makes pyrevit.interop importable under #! python3 where it previously hard-failed); the Stage 1 __bool__ fix restores documented IPY2 truthiness behavior under IPY3. The compat.py unicode builtins injection stays - its removal would be a separate, announced deprecation. The checker doubles as a "check your extension" tool for the eventual community migration guide.

Links:

a static AST checker (pipenv run check-py3), a Revit-hosted test suite run by twin DevTools buttons on IronPython and CPython, and the checker's 30-finding baseline.
iteritems in ElementWrapper.__repr__, bare unicode uses in the vendored markdown package, missing __bool__ aliases in forms, and (str, unicode) isinstance tuples. Checker baseline 30 -> 15.
- markdown: resolve extension short names against the vendored package
  (the hardcoded top-level `markdown.extensions.*` paths never worked on
  any engine); replace Element.getchildren(), removed in CPython 3.9;
  skip conversions under IronPython 3, where fat DLR frames overflow the
  host thread's stack (~768KB needed, measured standalone) — an
  uncatchable StackOverflowException that hard-crashes Revit.
- suite: split native-binary interop imports (rhino/adc) into an opt-in
  test so a bad native load cannot kill the host; skip managed interop
  imports on .NET 8 hosts (IxMilia.Dxf/Ifc.Net are not shipped for
  netcore); replace the grid-based out-param test with Curve.Intersect
  on pure geometry (Grid.Curve is view-dependent on this host); skip
  markdown extra under IronPython (re.Scanner unavailable in both 2 and
  3).
- runner: print failing tests' tracebacks to the output window
  (printErrors was a no-op).
framework.add_reference_to_file() wraps clr.AddReferenceToFileAndPath (IronPython) / Assembly.LoadFrom (pythonnet); all 13 active call sites converted. Checker IPY-CLR findings 12 -> 0.
query.intersect_curves() wraps Curve.Intersect; load_family branches LoadFamily; FamilyLoaderOptionsHandler gains __namespace__ + dual ref-param conventions for pythonnet interface callbacks. IronPython paths unchanged; CPython suite run fully green (12 pass / 0 errors).

Three extension scripts converted to the new wrappers (Section Box Navigator, Load More Types, Load Families). Testing those surfaced two more fixes: itertools.ifilterfalse in file_utils.py (gone in Py3) and deprecated '\d' escapes in non-raw strings (rpws, Logs button).

Checker: new CLR-REF rule (clr.Reference outside the sanctioned dispatch sites), Py2-only itertools coverage in all usage forms, and parse-time SyntaxWarnings reported as SYNTAX-WARN findings. Baseline is now 3 (SORT-KEY only).
Apply the pyrevitlabs#3477 sort-key pattern to Get RVT Info, ParametersValuesToParameter, and ExportIFC. Checker is now at 0 findings across the tree
From a review of the branch:
- check_py3_compat.py now exits non-zero on a missing or zero-file
  path (a typo'd path was passing green, neutering the gate)
- markdown convert() decodes bytes explicitly instead of stringifying
  them to "b'...'"
- markdown extra.py resolves its package via __name__, not the
  IronPython-unreliable __package__
- TemplateListItem.__bool__ returns a real bool (Py3 requires it)
- opt-in reload test (TEST_HANDLER_REBAKE) for the __namespace__
  re-bake question on FamilyLoaderOptionsHandler

Wires check-py3 as a CI gate (py3-compat job) now that the tree is at
zero findings. The generic-collection List[T](pylist) class surfaced
in review (~55 sites, mostly extensions) is deferred to its own PR.
pyrevit.coreutils.markdown has no first-party runtime consumers
(output.print_md renders via C#), so stop treating it as a maintained
surface: remove its four compat tests and the IPY3 skip helper, drop
it from the import test, and exclude coreutils/markdown from
check_py3_compat.

Unlike rpw (engine-locked WPF legacy) it is pure Python and portable -
a deprecated unbundling candidate, not frozen legacy. The Stage 1
syntax/vendoring fixes are kept: they are correct and help third-party
callers on every engine; #! python3 scripts should use pip markdown.
dict.keys()/values()/items() and map/filter/zip are non-indexable
views in Python 3 (lists in Python 2), breaking on any Py3 engine.
Fixes the IPY3 Search button (search_matches indexed a bare KeysView),
the grouped SelectFromList (.index() on a view), five functions
returning bare iterators, and two ItemsSource stores.

New PY3-VIEW checker rule flags bare views that escape local scope
(indexed/returned/yielded/attr-stored). Call-arg is intentionally not
flagged: the core consumers (SelectFromList.show, print_table) already
iterate inputs, so passing a view is safe.
Add three statically-detectable Py2->3 rules: PY2-MODULE (stdlib
renames - StringIO/Queue/urllib2/..., guard-aware), PY2-BUILTIN
(removed builtins when called - raw_input/execfile/reduce/..., skipped
when the name is imported), and PY2-NEXT (iterator class with next()
but no __next__). StandardError added to PY2-NAME, and local-name
detection is now import-aware.

Fixes the 4 residuals found, all in extensions: three unused
`import urllib2` (removed) and one `import StringIO` -> plain-list
accumulator (sidesteps io.StringIO str/unicode ambiguity on the
default IPY2 engine)
From a review of the checker:
- recognize try/except ImportError as a guard, so the canonical
  portable module shim (try: from StringIO import StringIO / except
  ImportError: from io import StringIO) is no longer flagged PY2-MODULE
- match the immediate receiver name for CLR-REF and PY2-ITER instead of
  walking the whole receiver subtree, so foo[clr].Reference and
  obj[itertools].imap() no longer false-match (framework.clr.Reference
  and real itertools.imap still do)

Receiver-blindness of PY3-VIEW / SORT-KEY (.items() on a non-dict) is
left as-is: the suggested list()/key= fix is always safe and the
first-party false-positive rate is zero. No real detection lost; the
tree stays at 0.

@devloai devloai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary:

  • Cleans up Python-2-only language idioms (iteritems, bare unicode, __nonzero__/__bool__, keyless heterogeneous sorts, bare views/iterators) across pyrevitlib and shipped extensions so the code runs correctly under IronPython 3.
  • Adds an engine-portable CLR loading shim (framework.add_reference_to_file) replacing clr.AddReferenceToFileAndPath at all first-party call sites outside framework.py.
  • Ports the two clr.Reference out-param sites (query.intersect_curves, create.load_family/FamilyLoaderOptionsHandler) to dispatch between IronPython's StrongBox idiom and pythonnet's tuple-return convention.
  • Ships a stdlib-only AST checker (dev/scripts/check_py3_compat.py) and an in-Revit unittest suite (test_py3_compat.py, two DevTools buttons) to verify parity across IronPython2/IronPython3/CPython engines.

Review Summary:

Traced the core marshaling changes against the actual Revit API interface signatures (IFamilyLoadOptions.OnFamilyFound/OnSharedFamilyFound ref-param ordering, Curve.Intersect out-param) and confirmed the IronPython vs. pythonnet dispatch branches are consistent with each other and with prior behavior. Verified pyrevit.compat.IRONPY correctly spans both IronPython 2 and 3 so the clr.Reference branches remain exercised on IPY3 as intended. Spot-checked all converted CLR-loading call sites (adc.py, dxf.py, ifc.py, rhino.py, git.py, mathnet.py, asmmaker.py) for leftover unused clr imports and extension-handling edge cases in the new framework.add_reference_to_file helper — none found. Reviewed the vendored markdown fixes, the extension script call-site conversions (sectionbox_navigation.py, family_utils.py, file_utils.py, etc.), and the new checker/test infrastructure for logic correctness. The only issue found is a low-severity dangling documentation reference to a file (IRONPYTHON_TO_PYTHON3_ANALYSIS.md) that isn't shipped in the repo or this PR. Overall this is a well-scoped, thoroughly self-tested change with no functional regressions identified.

Suggestions

  • Add the missing IRONPYTHON_TO_PYTHON3_ANALYSIS.md doc referenced by the new checker script. Apply
  • Fix the pre-existing report('|', end='') call in Wipe Site Designer Line Styles (report() doesn't accept end=). Apply

Comment thread dev/scripts/check_py3_compat.py
@sanzoghenzo

Copy link
Copy Markdown
Contributor

Hi @ChrisCrosley , can you do me a favor? don't write essays for PR descriptions. Get straight to the point. It took me less time to read the actual code edits than the description itself 🤣

@ChrisCrosley

Copy link
Copy Markdown
Contributor Author

🙃🙃 point taken! 🙃🙃

@sanzoghenzo sanzoghenzo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few minor comments, overall it looks good!

Comment thread pyrevitlib/pyrevit/interop/adc.py Outdated
Comment thread pyrevitlib/pyrevit/revit/db/query.py Outdated
ChrisCrosley and others added 3 commits July 12, 2026 17:05
…Examples.panel/ExportIFC.pushbutton/script.py

Co-authored-by: Andrea Ghensi <andrea.ghensi@gmail.com>
Co-authored-by: Andrea Ghensi <andrea.ghensi@gmail.com>
@ChrisCrosley
ChrisCrosley marked this pull request as ready for review July 12, 2026 21:30
Comment thread pyrevitlib/pyrevit/interop/adc.py Outdated
@sanzoghenzo

Copy link
Copy Markdown
Contributor

There are some conflicts to solve, then we should be ready to go!

How can we make sure that future PRs for the python side have run pipenv run check-py3 ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Search.pushbutton ipython 3 No prompt

3 participants