Fix D3D12 swapchain hook for Wine/D3DMetal (CrossOver on macOS)#1786
Closed
AlexGRDev wants to merge 8 commits into
Closed
Fix D3D12 swapchain hook for Wine/D3DMetal (CrossOver on macOS)#1786AlexGRDev wants to merge 8 commits into
AlexGRDev wants to merge 8 commits into
Conversation
- Add adapter enumeration fallback when D3D12CreateDevice(nullptr) fails (D3DMetal may reject the nullptr adapter used for dummy device creation) - Hook all three IDXGIFactory2 swapchain creation methods [15/16/24]: CreateSwapChainForHwnd, CreateSwapChainForCoreWindow, CreateSwapChainForComposition so the hook fires regardless of which method D3DMetal uses internally - When any factory hook fires, bootstrap s_swapchain_vtable and s_command_queue_offset directly from the real game swapchain instead of requiring a successful dummy swapchain creation - Fallback to factory hooks (instead of returning false) when dummy swapchain creation fails, allowing Present to hook on next recreation Fixes REFramework overlay not appearing in RE4R under CrossOver 26.2 with CX_GRAPHICS_BACKEND=d3dmetal on macOS.
MSVC requires explicit cast — void* cannot implicitly convert to void**. All three bootstrap locations (create_swapchain, for_corewindow, for_composition) now use *(void***)(*swap_chain) consistent with the rest of the codebase.
DXMT hooks D3D12CreateDevice to translate calls to Metal. When REFramework temporarily restores original bytes and calls the raw function, there is no native D3D12 on macOS — the call crashes/hangs. Detect Wine via wine_get_version in ntdll.dll. When Wine is present, keep DXMT's hook intact and call D3D12CreateDevice through it. DXMT handles the translation and device creation can succeed normally.
…n fails DXMT wraps D3D12CommandQueue internally and does not store the passed COM pointer directly in the swapchain object, causing the memory scan to find nothing. Detect this case (Wine + offset == 0) and instead of returning false, keep a ref to the dummy queue we created and use it in present() — both are backed by the same Metal GPU so rendering commands reach the same hardware. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The dummy queue was created from the dummy device used for vtable scanning, not from the game's actual D3D12 device. DXMT routes each D3D12 device to its own Metal command queue, so executing REFramework's command lists (recorded on the game's device) via the dummy device's queue causes them to submit to a different Metal command buffer — appearing behind or not composited with the game frame. Fix: on the first present() call we have the game's real ID3D12Device4 from GetDevice(). Create a new queue from that device and replace the dummy queue. All subsequent overlay rendering now submits to the correct Metal command queue and is composited in the right order. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Three improvements to fix overlay z-ordering in fullscreen/borderless: 1. CPU fence wait in present(): after m_on_present() executes ImGui commands on our separate Metal queue, Signal+WaitForSingleObjectEx forces CPU-side synchronization before DXMT's Present blit. Metal hazard tracking should then see our completed writes to the backbuffer texture. 2. Vtable-matching scan (3rd pass): if direct pointer scan fails (DXMT stores queue via internal pointer not COM pointer), scan for an object in the swapchain whose vtable matches the command queue vtable. Extends search to 1024 pointers. 3. Fence+event creation when queue is upgraded to game-device queue. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… queue The previous scans all targeted the DUMMY swapchain created in hook(). DXMT may store the command queue differently in real vs dummy swapc hains. On the first present() call we have the REAL game swapchain (swap_chain param). Scan it immediately after the queue upgrade using vtable matching — look for any object whose vtable matches our DXMT ID3D12CommandQueue vtable. If found, replace s_wine_fallback_queue with the game's actual original queue. DXMT uses this queue for Present synchronization, so rendering on it guarantees our ImGui commands are included in the Metal presentation ordering. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes REFramework overlay not appearing in RE4R under CrossOver 26.2 with CX_GRAPHICS_BACKEND=d3dmetal on macOS.