Decouple extension UI layout from folder structure - #3386
Conversation
When using custom layouts stored in AppData, adding a tool that wasn't in the original layout caused "Revit cannot run available command" errors. The assembly DLL only contained command types for layout-referenced tools, and changes to custom layout files outside the extension directory didn't invalidate the hash-based cache. Both C# and Python parsers now track which tools from the tool index are referenced by the layout and compile unreferenced tools as assembly-only components. The layout controls ribbon UI placement only — all discovered tools have compiled command types available in the cached assembly.
Visual Studio's WPF designer generates *_wpftmp.csproj files during build that should never be committed. Remove the accidentally-tracked file and add a gitignore rule.
Reformats the 7 fully-new files introduced by this feature so they comply with the project's recommended black style. Existing files modified by this branch are intentionally left untouched: upstream develop is not black-formatted, and reformatting them here would churn hundreds of unrelated lines.
Replace hardcoded '.extension', '.tab', '.panel', '.stack', and 'tools' string literals with the existing constants in pyrevit.extensions (UI_EXTENSION_POSTFIX, TAB_POSTFIX, PANEL_POSTFIX, STACK_BUTTON_POSTFIX, TOOLS_DIR_NAME).
Replace the ~80-line hand-rolled YAML serializer in layout_cli.py (serialize_layout_yaml, _yaml_dump, _yaml_scalar, _write_panel_yaml) with the existing pyrevit.coreutils.yaml.dump_dict wrapper around YamlDotNet. The Layout Builder pushbutton script is updated to call dump_dict directly instead of importing the removed helper. Also normalizes the 3 existing extension_layout.yaml files (Core, Tools, Templates) to a canonical unquoted-scalar block style, since they were previously hand-authored with always-quoted strings. Final output style is verified at runtime against YamlDotNet; minor touch-ups may be needed after first in-Revit regeneration.
- layout_parser.py: drop unused `coreutils`, `GenericUIContainer`, `LayoutItem` imports. - layout_cli.py: drop unused `os` import; remove the `stack_buffer` list in `_build_panel_layout` along with its flush blocks (the buffer was initialized and cleared but never appended to). - Layout Builder/script.py: drop unused function-local `Windows` import; drop top-level `codecs` import (no longer needed after the yaml refactor). - Test Layout Parsing/script.py: drop unused `Tab`, `Panel`, `GenericStack` imports.
Replace the cross-module write to the private _assembly_only_commands attribute from layout_parser with a public setter on Extension. Keeps the underscore-prefixed storage internal to the class while giving the layout parser a clean entry point.
- parser.get_parsed_extension: remove the try/except that hid ImportError from the layout_parser/toolindex imports. The imports are unconditional and any failure was masquerading as "use legacy mode", which would silently disable the new feature for everyone. - layout_parser.get_layout_file: log the swallowed exception at debug level instead of pass. - Layout Builder _get_custom_layout_path: same; add a module-level mlogger via script.get_logger().
Move the function-local imports of Tab, Panel, and GenericStack from inside _create_tab, _create_panel, and _create_stack to the module header. components.py does not import from layout_parser/toolindex/ layout_cli, so there is no circular-import risk. The user_config import inside get_layout_file is intentionally left function-local since user_config performs config-file I/O on import.
- toolindex.py: collapse the Args/Returns docstrings on the two trivial helpers (_get_extension_name, _get_dir_extension) into a single line. - layout_parser.py: collapse the Args/Returns block on _create_tab into a single line. - Layout Builder/script.py: remove the four inner banner comments (# -- properties exposed to WPF binding --, etc.) that only restate what the method group does. The outer section banners are kept for navigation in this 700-line file. - userconfig.py: add docstring on disable_custom_layouts setter. - PyRevitConfig.cs: drop the noisy extra blank line introduced by the branch (matches upstream spacing). Project convention per CLAUDE.md is to favor WHY-comments over WHAT-comments; the C# XML doc comments on private helpers are kept because they match the surrounding C# style.
The method's accessibility is `internal`, not `public`, so its name was misleading. Updates both call sites in LayoutParser.cs.
Black cleanup after removing the hand-rolled YAML serializer left two trailing blank lines at end of file.
…re-0x5KJ Claude/pr checklist new feature 0x5 kj
…r an extension. plus cleanup and documentation for PR
…sley/pyRevit-fork into feature/decouple-layout
![devloai[bot]](https://avatars.githubusercontent.com/in/886113?s=60&v=4){kind=small}
There was a problem hiding this comment.
PR Summary:
Introduces an optional YAML-based layout system that decouples ribbon UI structure from the on-disk folder hierarchy. Extensions can declare tabs/panels/tools in extension_layout.yaml while tools live in a flat tools/ directory. Fully backward-compatible — legacy extensions are untouched. Includes a WPF Layout Builder tool, per-extension custom layout import/export in Settings, C# loader parity via LayoutParser.cs, and migrations of core bundled extensions.
Review Summary:
The overall architecture is well-thought-out and the backward-compatibility story is solid. The Python-side layout parser and tool indexer are clean and well-documented. The C# LayoutParser correctly mirrors the Python logic with appropriate logging.
Two higher-severity issues were found: (1) the Layout Builder script runs its entire entry-point (extension discovery, SelectFromList dialog, build_tool_index, window creation) at module level — a known anti-pattern in pyRevit's IronPython engine where module-level code persists between executions; (2) the layout cache directory is computed differently in the Layout Builder (%APPDATA%\pyRevit\Layouts\) vs the Settings window (PYREVIT_APP_DIR\Layouts\), which will silently diverge on non-standard installations, causing the builder to write layouts to a location Settings never reads. A latent circular import between toolindex.py and parser.py was also flagged — currently safe due to lazy import ordering but fragile. On the C# side, HasLayoutFile and GetLayoutFilePath duplicate their config-lookup logic independently, and ParseLayout has a null-coalescing fallback to a hard-coded filename after the tool index is already built. Exception handling in the Settings import callback re-raises without a user-visible alert.
Suggestions
- Consolidate the layout cache directory path into a single shared utility function (e.g. in
layout_parser.py) imported by bothLayout BuilderandSettingsscripts to eliminate the divergent path computation. Apply - Move the module-level entry-point logic in
Layout Builder.pushbutton/script.py(lines 687–738) into amain()function guarded byif __name__ == '__main__':to follow the pyRevit IronPython anti-stale-global pattern. Apply
- Consolidate layout cache dir into get_layout_cache_dir() in layout_parser.py (canonical PYREVIT_APP_DIR source); Layout Builder and Settings now share it instead of computing divergent paths - Move Layout Builder module-level entry point into main() behind __main__ guard per IronPython anti-stale-global pattern - Break latent circular import: toolindex.py now lazy-imports parser.py helpers inside functions; drop unused _create_subcomponents - C# HasLayoutFile delegates to GetLayoutFilePath, removing duplicated config-lookup logic - C# ParseLayout guard-fails early when no layout file resolves, before building the tool index - Settings import callback now logs and alerts the user on failure instead of re-raising silently out of the WPF event handler - Fix Black violation (double blank line) in _resolve_string_entry - Update built loader DLLs Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
@ChrisCrosley Have you tested creating a single layout that includes the full list of default pyRevit tools? If so, how did it perform? This is great work and aligns with discussions @sanzoghenzo and I had a while back.
|
jmcouffin
added
Tools
- Adjusted layout file path resolution to prioritize the layout directory over the extension directory, ensuring correct file access. - Enhanced the unique name generation for stacks by incorporating the panel name, preventing potential naming collisions. - Updated command assembly logic to include child commands from container tools, improving command retrieval accuracy.
|
Hi @ALL |
It would have been better if that proposal (#2648) was taken in consideration and discussed further, before creating a PR... But I'm becoming the old man ranting about the old times, that nobody should listen to anymore 🤣 Just one question: if one wants to set tooltips, background colors, media files, contents, supported revit versions, and so on, does it still have to be inside a bundle.yaml? If yes, and if everything is moved to the tools/ directory, doesn't it create conflicts? I tried to take a look at the code to answer this myself, but it may be my firefox not working well with github anymore, or just GitHub enshittification that can't handle these big PRs... |
|
@sanzoghenzo, I didn’t see your original RFC or I would have restarted the discussion there! I’ve refactored the core extension with the proposed structure if you want to see the folder structure and proposed yaml. The main difference from your original proposal is that I’m keeping the tool bundle.yaml for tooltips, colors, etc. I’m just consolidating all nesting directories and tab/panel/stack yaml files. [extensions/pyRevitCore.extension/extension_layout.yaml] This way the UI is cleanly separated from the tool config properties the developer will want to maintain control over. And buttons within /tools/ will still have subfolders to contain related files. The crawler even allows multiple levels of nesting if the developer wishes to group tools. Input is welcome! @jmcouffin I did some testing early on and it was either a wash or faster than the original method for the same layout, but if you simplify the layout to just a handful of buttons then the UI build goes faster - especially if you’re removing smart buttons. I’m planning to try and do some more testing to confirm. |
Oh, that makes sense, I read "flat tools/ directory" and I most likely was too tired to think clearly 😅 This is definitely a simpler, easier alternative to my proposal since it has a lower friction on the future adoption. Still not much time and willpower to scan through the code, but what about two extensions declaring the same tab/panel name? (I don't even remember if and how are handled now, it is just to check if it is something that has been thought of) |
@sanzoghenzo last time I checked, a couple of months back, it worked fine; you could declare the same tab for both extensions, and they would sit under the same tab. |
The separator/slideout/stack/tool wire format inside a panel's "layout" list was encoded and decoded independently in the loader (layout_parser), the CLI generator (layout_cli), and the Layout Builder UI script. A change to the format had to be mirrored across all three or they would drift. Introduce classify_layout_entry plus encode_* helpers in layout_parser as the single source of truth, and route all three consumers through them. Also tighten layout_cli stack detection from a 'stack' substring match to an exact STACK_BUTTON_POSTFIX comparison.
A tool is a single compiled command whose id is independent of its UI location. Referencing the same tool in two panels/stacks aliased one component into two parents, corrupting its control/unique id. Both the Python and C# layout parsers now detect an already-placed tool, warn, and skip the duplicate rather than supporting an impossible case. Separately, a malformed or empty layout file used to leave the extension with zero components and cache that empty ribbon. Layout parsing now reports success/failure and falls back in order: custom override -> bundled extension_layout.yaml -> legacy directory walk, so a broken layout never wipes the ribbon.
generate_layout's split-panel mode named files after the panel only, so two panels with the same name in different tabs overwrote each other. Qualify the filename with the tab name (and a numeric suffix on any remaining clash) so each panel gets its own file. Layout import/export only copies files matching the .panel.yaml naming convention, so a panel referencing a file by another name silently rendered empty after reload. Parse the layout's layout_file references and warn the user about any that are misnamed or missing, instead of dropping them quietly.
…dits Tools present on disk but not in the active layout are compiled into the assembly so layout edits don't force a rebuild. The ASCII cacher wrote them out (via __dict__) but never restored them, so a cached reload with bin_cache=False produced an assembly missing those command types. Restore them in get_cached_extension, reconstructing command types recursively so nested types (LinkButton/InvokeButton under NoScriptButton) resolve. Separately, the extension cache hash only walked the extension directory, so a custom layout file stored outside it (e.g. in the Layouts cache dir) never invalidated the cache when edited or switched. Fold the active custom layout's mtime into the hash in both the Python and C# hashers.
Layout-defined tabs/panels are virtual (no bundle.yaml), so highlight/ collapsed/background/is_beta and localized titles were dropped. - Parser reads these from the layout entry or external .panel.yaml. - generate_layout emits them (and the real panel title) so migrating a legacy extension preserves appearance. - Layout Builder: resolve localized titles for display, carry through metadata it doesn't edit, and add a Properties dialog (highlight, collapsed, is_beta, color-picker backgrounds, and a per-locale title grid) reachable from a right-justified toolbar button enabled for the selected tab/panel. - Document the new keys in the layout guide.
pyRevitCore now loads via extension_layout.yaml, so the Reload command's class is named pyrevitcore_reload (extensionname_toolname), not the legacy path-based pyrevitcore_pyrevit_pyrevit_tools_reload. Update the hardcoded PYREVIT_CORE_RELOAD_COMMAND_NAME (and the sessionmgr docstring example) so Reload / Save & Reload can find the command again. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The new-loader LayoutParser ignored background/highlight/collapsed/is_beta, so layout-defined panels rendered with no background on the C# path (the Python parser already read them). Parse these keys onto ParsedComponent in CreateTab/CreatePanel, reading from the inline entry or external .panel.yaml, mirroring the Python layout parser and the legacy BundleParser.
- Added extension_layout.yaml for DevTools to demonstrate full metadata compatibility. - Updated Layout.panel with bundle.yaml - Renamed ButtonA in Test Panel Colors to avoid tool name conflict (there were 2 ButtonA's
|
I dug more into bundle.yaml documentation trying to find which features can be applied to tabs and panels - not just buttons. I came up with:
None of these were supported in the initial PR. I've updated the parser and layout generator to handle these, and updated the layout builder to have an additional dialog to set metadata for tabs and panels. (all button metadata still lives in bundle.yaml) 
I still need to do a lot of manual testing to make sure this is all working properly, but I wanted to get a first pass out there since this is getting attention. The panel coloring is working in DevTools. 
A note from Claude worth maybe discussing as a seperate issue: in the C# new loader, |
|
Cool stuff again @ChrisCrosley |
|
We can definitely make metadata a 3rd column if that's the preference. It would just be empty unless a panel is selected because I don't think we want to edit tool metadata here - which is why I landed on making it a dialog. Feel free to experiment. We can remove tab and panel highlighting if there is not a plan to support that. That would remove the tab metadata completely. |
|
Allow me to mark as draft since I don't want this to be part of the next release but rather the one after |
|
Sorry for the unexpected closing of the issue. I was doing some cleanup on branches and hit the wrong one. @ChrisCrosley @romangolev @sanzoghenzo @MohamedAsli the link below exposes the future of the PR I had in mind |
5 participants
Description
This PR introduces an optional YAML-based layout system that separates an extension's UI layout (tabs, panels, button arrangement) from its on-disk folder structure. Extensions can now declare their ribbon layout in
extension_layout.yamlinstead of encoding it in nested.tab/.panel/.stack/directories. Tool bundles can live in a flattools/directory, and admins or end users can override the bundled layout with a custom one — without forking the extension.The system is fully backward-compatible: extensions with no
extension_layout.yamlcontinue to load via the existing folder-walking parser, unchanged. Extensions that opt in can migrate incrementally —tools/and legacy.tab/.panel/directories are both scanned, so tools can be moved over piecemeal.Example layout
Included as optional presets in .extension/tools/
Layout Builder
What's in this PR
Python-side parser (
pyrevitlib/pyrevit/extensions/)layout_parser.py— readsextension_layout.yaml/*.panel.yaml, resolves tool references against a tool index, and builds the same component treeuimaker.pyalready consumes.toolindex.py— recursive scanner fortools/(and the legacy hierarchy) that produces a name → component dictionary. Plain subfolders intools/are treated as organizational only.layout_cli.py— programmatic API forgenerate_layout()(emit YAML mirroring an existing legacy structure) andlist_tools()(diagnostic listing).parser.py— dispatches to layout-based or legacy parsing based on whether a layout file is present.components.py—Extensionnow carries an_assembly_only_commandslist so unreferenced tools still get compiled into the DLL (see "assembly-only commands" below).userconfig.py— adds a globaldisable_custom_layoutstoggle.C# loader parser (
dev/pyRevitLoader/pyRevitExtensionParser/)LayoutParser.cs— mirrors the Python parser for the new-loader code path. Handles tool indexing, custom-layout-path resolution from the INI config, panel layout files, stacks, separators, and slideouts.ExtensionParser.cs— dispatches toLayoutParser.ParseLayoutwhen a layout file is detected; exposes a newParseSingleComponententry point so the layout path can reuse all existing bundle/script/icon parsing.ParsedExtension.cs— carriesAssemblyOnlyComponents; adds.extensionandtoolsto the directory-hash inputs so cache invalidation is correct under the new structure.PyRevitConfig.cs— readsdisable_custom_layouts(global) andcustom_layout_path(per extension, in[<name>.extension]).PyRevitConsts.cs— adds the matching config key constant.Layout YAML format
Tools are referenced by their folder name without the postfix (
MyTool.pushbutton→"MyTool"). Lookups are case-insensitive. Panel layouts can be inline or split out into<Name>.panel.yamlfiles alongsideextension_layout.yaml.Custom user layouts
%APPDATA%\pyRevit\Layouts\<ExtensionName>\.custom_layout_pathin the user INI; fall back to the bundled layout if absent or if the globaldisable_custom_layoutsis set.Custom/Defaultstatus column, and a global "Disable all custom layouts" checkbox.Layout Builder (new Core tool)
extensions/pyRevitCore.extension/tools/Layout Builder.pushbutton/— WPF visual editor. Two-pane UI: left side lists available tools with search and a "Show All / Hide Placed" toggle; right side is the tab/panel/stack/tool tree with add / move-up/down / remove buttons. Saves to the user's custom-layout cache directory and reloads pyRevit. Optional developer-shipped presets (<ext>/layouts/*.layout.yaml) appear via a "Load Preset" button when present.Dev tools (
extensions/pyRevitDevTools.extension/)Generate Layout.pushbutton— generates anextension_layout.yamlfrom any existing legacy extension as a starting point for migration. Shift-click splits panels into separate files.List Tools.pushbutton— diagnostic listing of all tools discoverable in an extension.Test Layout Parsing.pushbutton— exercises the parser for sanity checks during development.Migrated bundled extensions
pyRevitCore.extension— fully migrated: tools moved frompyRevit.tab/pyRevit.panel/*.stack/...to a flattools/directory, structural folders removed, layout declared inextension_layout.yaml. Adds the new "Layout Builder" button. The Settings smartbutton picks up new XAML for the Extension Layout section.pyRevitTools.extension— opts in withextension_layout.yamlonly, demonstrating the hybrid case where tools remain in the legacy folder structure but the ribbon is now layout-driven.pyRevitTemplates.extension— minimal example layout plusextension-layout-guide.mdandmigration-instructions.mdfor extension authors.Assembly-only commands
When layout mode is active, any tool found on disk but not referenced by the layout is still compiled into the extension assembly — it just doesn't appear in the ribbon. This was a deliberate design choice so that user-facing layout edits (via the Layout Builder or by hand) don't trigger a full assembly rebuild on the next reload: the command type is already available, the layout just decides whether to surface it. Python side:
Extension.set_assembly_only_commands()registers the unreferenced tools, andExtension.get_all_commands()merges them with UI commands for assembly compilation. C# side:ParsedExtension.AssemblyOnlyComponents, also folded intoCollectCommandComponents().Backward compatibility & safety
extension_layout.yamluse the existing parser unchanged — no behavioral change..tab/.panel/tree; both locations are scanned and merged into the tool index (tools/wins on duplicate name).<extensionname>_<toolname>, flat) vs. legacy path-based IDs. Cache directory hashing inputs were updated (.extensionandtoolsadded) so stale caches don't mask the new structure. Users with leftover compiled DLLs from older builds may need to clear%APPDATA%\pyRevit\<RevitYear>\*.dllonce — this is documented inmigration-instructions.md.Notes for reviewers
namefield onbundle.yaml, but the implementation instead derives the name from the folder name in all cases. I could go either way here. Open to feedback.extensions/pyRevitTemplates.extension/extension-layout-guide.mdandmigration-instructions.mdare the authoritative end-user docs.Layout Builderbutton is placed in the corepyRevittab rather than under Dev Tools because it's user-facing, not developer-only.Checklist
pipenv run black {source_file_or_directory}Related Issues
Additional Notes
bin/were rebuilt against the newLayoutParser.cs/ExtensionParser.csso the new loader can parse layout-mode extensions out of the box.ParsedExtension.cs(adding.extensionandtools) is required for layout-mode extensions but is also a strict improvement for legacy extensions — the previous hash inputs missed changes inside the extension root folder itself.