Fix HIP host-visible memory and managed globals - #166
Draft
AWoloszyn wants to merge 1155 commits into
Draft
Conversation
zjgarvey
reviewed
Jul 9, 2026
zjgarvey
left a comment
zjgarvey
left a comment
Collaborator
There was a problem hiding this comment.
Some extra tests might be good.
E.g., managed-global roundtrip, or malformed ELF parsing, or heap-header-population test, or a test checking that a dispatch with a non-zero heap pointer ends up in implicit_args would be useful. If these are covered by the hip tests, feel free to skip them (but I'd like to know which tests this PR fixes so I can look at them).
Comment on lines
+1004
to
+1006
| for (iree_host_size_t i = 0; i < registration->symbol_count; ++i) { | ||
| iree_hal_streaming_symbol_registration_t* symbol_registration = | ||
| ®istration->symbols[i]; |
Collaborator
There was a problem hiding this comment.
Might want to hold the registry lock here.
| "group segment max size default must be non-zero"); | ||
|
|
||
| // ROCm device-libs initialize hidden_heap_v1 with a 128KiB heap header and | ||
| // 2MiB initial slabs. These constants mirror __ockl_dm_init_v1's heap layout. |
Collaborator
There was a problem hiding this comment.
Is this stable? E.g., if __ockl_dm_init_v1 changes or adds a new code, I'd like to be able to at least run a test to make sure these magic numbers here will be compatible.
| return iree_ok_status(); | ||
| } | ||
|
|
||
| static iree_status_t iree_hal_streaming_managed_device_name( |
Collaborator
There was a problem hiding this comment.
This is identical to module_managed_device_name in module.c:1005.
| IREE_RETURN_IF_ERROR(iree_hal_streaming_module_copy_cstring( | ||
| module, pointer_name, &pointer_name_string)); | ||
| iree_status_t status = | ||
| iree_hal_streaming_module_initialize_managed_pointer( |
Collaborator
There was a problem hiding this comment.
This init is also called in prepare_module around registry.c:861
|
|
||
| if (managed_found && managed_symbol) { | ||
| if (found && symbol) { | ||
| IREE_RETURN_IF_ERROR(iree_hal_streaming_module_initialize_managed_pointer( |
Collaborator
There was a problem hiding this comment.
And here (calling init, which does a blocking D2H), on every lookup. This also allocates and frees a ".managed" string on every call.
Might be worth considering whether to hoist a single pointer-init routine, initialize once at load, and cache + lookup. I'm not sure how important this optimization is in practice, but it the blocking transfers seem bad to throw around if not strictly necessary.
| while (remaining > 0 && iree_status_is_ok(status)) { | ||
| iree_device_size_t this_chunk = | ||
| remaining < d2h_chunk_size ? remaining : d2h_chunk_size; | ||
| status = iree_hal_device_transfer_d2h( |
Collaborator
There was a problem hiding this comment.
Does this get routed through queue_execute? If so, I'm concerned that if the d2h queue is the same as the one we put the parent function-call on, we might hit a deadlock.
|
|
||
| // HIP reports a successful zero-size malloc by returning NULL. Keep just enough | ||
| // thread-local state for hipMemPtrGetInfo(NULL, &size) to report that result. | ||
| static iree_thread_local bool iree_hip_zero_size_allocation_pending = false; |
Collaborator
There was a problem hiding this comment.
I looked at o.g. hipMemPtrGetInfo and it just unconditionally returns {0, hipSuccess} for NULL. Is there like a stricter version of this we need where we reject NULL?
| } | ||
| iree_slim_mutex_unlock(&transfer->context->direct_transfer_mutex); | ||
| if (iree_status_is_ok(status)) { | ||
| memcpy(transfer->dst, transfer->staging, transfer->size); |
Collaborator
There was a problem hiding this comment.
I'm not sure I understand why the transfer needs staging. The staging memory is just normal unpinned host memory, right? Why not just do the d2h directly into transfer->dst?
| (uint64_t)symbol->size_bytes); | ||
| } | ||
|
|
||
| IREE_RETURN_IF_ERROR(iree_hal_device_transfer_d2h( |
Collaborator
There was a problem hiding this comment.
Do we need to hold the direct_transfer_mutex here?
Several test factories initialized an iree_hal_resource_t inside a local aggregate and returned that aggregate by value. The resource header owns an atomic reference count, so copying or relocating it after initialization is not a valid C++ object operation and MSVC rejects the implicit copy. Give each fixture final stack storage first and initialize its resource header at that address. Apply the same lifetime rule to fake devices and executable caches across the C binding, AMDGPU, and SPIR-V test suites.
An AQL packet becomes visible to the command processor when its header is made valid. The publication width is the packet's first dword, containing both the 16-bit header and the following 16-bit setup field; publishing only the header does not satisfy that ABI contract. Keep setup out of the packet until all body and kernarg writes are complete, then publish header and setup together with release ordering. Use the Windows interlocked exchange under MSVC and the compiler atomic store elsewhere so the raw-HSA example preserves the same ordering on every supported host.
The low-asm inference harness assigned empty aggregates over its context and block-pool members after deinitialization. The block pool contains an atomic free-list head, so aggregate assignment attempts to assign a non-copyable std::atomic when the C API is included from MSVC C++. Let the lifecycle flags describe whether each member is live. Context deinitialization already clears its storage, and block-pool initialization fully initializes the pool before its next use, so the extra aggregate assignments provided no lifecycle guarantee.
Registry tests passed bare function designators to EXPECT_EQ while the other operand was a stored function pointer. MSVC preserves the designator as a function reference during GoogleTest template deduction, which leads the printer to qualify a function type and emits C4180 under the warnings-as-errors baseline. Take each callback address explicitly so both assertion operands have the same function-pointer type. This also states the behavior under test: registry composition must preserve callback identity rather than invoke the callback.
The scalar conversion hook used GNU typeof to construct entries of an anonymous local structure. MSVC C treats typeof as an undeclared function, so the scalar hook library could not compile with the baseline Windows toolchain. Give conversion candidates a named representation and construct each plain C value through a small typed helper. Keep the local array and its count-based selection intact. This removes the compiler extension without introducing a separate container abstraction or changing generation weights.
LoomOpt preserves an explicit executable path exactly, but resolves a bin-directory override to the host's executable name. The tests used POSIX-only absolute paths and always expected a suffixless tool, which encoded the Linux result instead of the resolver contract. Build both fixtures below the test runner's native temporary directory. Keep the exact-path assertion for explicit resolution and select the expected .exe suffix only for directory-based discovery on Windows.
A bare -2147483648 expression does not retain the intended signed 64-bit type under MSVC C. Unary negation wraps at the literal type before assignment, turning lower bounds positive and inverting generated target-contract ranges. Centralize signed 64-bit literal spelling around INT64_C and INT64_MIN, and route lower-rule bounds, memory strides and offsets, attribute literals, and diagnostic values through it. Low descriptor emission now shares the same checked helper so out-of-range values are rejected consistently.
Repository-wide Bazel wildcard builds name private implementation targets directly, bypassing the consumer alias platform selection. The Linux libbacktrace implementation therefore attempted to compile on Windows and failed in its configuration header before the broad test suite could run. Declare the implementation target Linux-compatible. Normal runtime alias behavior is unchanged, while //... now treats the backend as incompatible on Windows instead of compiling unsupported third-party source there.
Launch timing called clock_gettime directly even though the common binding is portable and already depends on IREE base. This prevented the shared stream implementation from compiling with MSVC because CLOCK_MONOTONIC is POSIX-only. Use iree_time_now, whose monotonic nanosecond contract maps to CLOCK_MONOTONIC on POSIX and QueryPerformanceCounter on Windows. The clock remains queried only when HRX_LAUNCH_TIMING is enabled, so the default launch path gains no new timing calls or state.
Keep runtime paths in UTF-8 and convert them only at the Win32 API boundary. The shared converter rejects embedded NULs and invalid UTF-8, preserves explicit namespace paths, and resolves filesystem paths to absolute extended-length names. Route file handles and stdio streams through CreateFileW and _wfopen. This removes their dependence on ANSI code pages, executable manifests, and the process long-path opt-in. Exercise relative, UNC, and device-path conversion plus real file-content and stdio opens at a greater-than-MAX_PATH UTF-8 filename.
IOCP converted file-open paths to UTF-16 but passed relative names directly to CreateFileW. Those names still inherit MAX_PATH unless the embedding executable and host have both opted in to long-path support. Reuse the shared Win32 path boundary so async opens reject invalid UTF-8, preserve explicit namespace paths, and use absolute extended-length filesystem paths. Free the transient conversion before the operation enters its completion path.
Bazel build events encode output paths as file URIs. Decode their path components with the host platform's URI rules so Windows drive roots do not retain the URI's leading slash. Apply the same conversion to compile-command collection and paranoid presubmit output discovery so both developer workflows consume build events consistently.
The AMDGPU source-build option carries a Linux path into CMake. Use POSIX serialization so the host path flavor cannot alter CI plans.
Model compiler query results at the subprocess boundary instead of relying on fake POSIX shell executables. Keep the subprocess wrapper covered with the active Python interpreter so process invocation is still exercised on every host.
External tools can produce host-specific line endings, including doubled carriage returns when preformatted CRLF passes through a Windows text-mode stdout. Normalize only outputs that tool wrappers explicitly expose as text while keeping the general process capture API byte-preserving. Force non-colored SPIR-V disassembly so Windows terminal assumptions cannot inject presentation escapes into golden output. Apply the same newline contract to SPIR-V and Wasm version and disassembly text.
Target bundle storage contains pointers into its own embedded snapshot, export plan, and configuration. Returning rebound storage by value lets ABI-dependent copies leave those pointers aimed at the callee's dead stack frame. Initialize the two test-owned copies at their final caller addresses before rebinding their internal views. This makes the ownership and address-stability requirement explicit and removes the redundant rebind that had masked one instance.
Loom's tool runner accepted UTF-8 paths and arguments but passed them to ANSI Win32 APIs. Toolchains, checkouts, or temporary directories outside the active code page could therefore fail or launch with altered arguments. Keep command construction and public temporary paths in UTF-8, convert the mutable command line to UTF-16 immediately before CreateProcessW, and use wide APIs for capture and temporary files. Check the process wait result so capture begins only after confirmed termination. Exercise the boundary by copying the test executable into a Unicode path, passing and capturing a Unicode argument, and forcing capture and public temporary files through a Unicode directory under both Bazel and CMake.
The LLVM adapter carried a complete private copy of argument allocation, Windows and POSIX process launch, output capture, waiting, and temporary file handling. That copy had already diverged from the shared SPIR-V and Wasm runner, including continued use of ANSI Win32 APIs. Expose process outputs and results through the shared types directly, and remove adapter-specific type aliases and forwarding ownership helpers from LLVM, SPIR-V, and Wasm. Route all LLVM invocation and temporary-file work through the shared implementation while retaining only tool resolution, LLVM-specific diagnostics, and file flows. This removes parallel launch and ownership contracts and ensures every external tool inherits the same UTF-8, quoting, capture, and error-handling behavior.
Loom and IREE used distinct enum parameter types and opaque status tags for allocator callbacks. The layouts happened to match, but the callback function types did not and boundary adapters required function-pointer casts. Give allocator commands the same fixed-width scalar type and deliberately share the opaque status carrier. This establishes an exact callback type without a trampoline, allocation, or additional state. The allocation parameter object carried through `const void*` has its own language identity and is not covered by the signature alignment.
Use a release barrier followed by the naturally aligned 32-bit packet header store on Windows. This preserves the AQL publication ordering without reinterpreting unsigned packet storage as a signed LONG object.
Keep argument ownership, output ownership, and public API validation in the shared process source. Move Win32 process and temporary-file mechanics into process_win32.c, POSIX mechanics into process_posix.c, and unsupported-host behavior into a small stub. The platform boundary is now one private dispatch contract instead of two large conditional regions containing most of the implementation. Bazel and CMake compile the same guarded source set so platform selection is not duplicated in build metadata.
External compiler tools should receive only their redirected standard streams. Unrelated inheritable handles can otherwise keep resources live or cross process boundaries unexpectedly. Configure a null stdin and an explicit descriptor policy for each host OS. Win32 uses an extended startup handle allowlist, Darwin uses its close-on-exec spawn policy, and Linux uses close-from actions where the libc provides them with descriptor enumeration for older libc implementations. Keep the Linux enumeration directory live until spawn so every queued close action refers to a valid descriptor. Mark process-owned POSIX descriptors close-on-exec as a second line of defense. Self-launching regression probes verify that an intentionally inheritable event or pipe does not cross the child process boundary.
Several in-tree HRX integrations link private runtime symbols and cannot substitute the public shared library for the Bazel hrx_static target. Generate the corresponding CMake library so consumers resolve the same target in both build systems. Apply the nested-project header filtering and object-library compile dependencies consistently to the static and shared variants.
The libhrx core and public ABI can run on local HAL drivers and should not select AMDGPU as a global build side effect. Leave driver selection to the product configuration instead. The current conformance suite embeds AMDGPU kernels, so default it on only when tests and the driver are enabled and reject explicit incompatible configurations with a direct diagnostic.
The ELF ABI bridge is required by the Windows x86-64 target ABI for both MSVC and clang-cl. Select it from WIN32 and the repository's normalized IREE_ARCH value instead of a compiler-mode architecture regex that also matched ARM64. Remove the unused ASM_MASM source property because the custom command owns assembly and contributes only its generated object to the target.
The passthrough libraries currently depend on ELF export controls and a POSIX dynamic loader. Default the option on only where those targets exist and diagnose an explicit incompatible request instead of failing later on missing CMake targets. Guard target-specific properties by target existence so a future Windows implementation can enable the option once its native loader and exports are ready. This is an implementation readiness gate, not a product restriction.
Capture Win32 error codes before closing failed file and mapping handles so cleanup cannot overwrite the diagnostic. Close POSIX files when initial truncation fails, preserving errno across that cleanup. Treat zero as a valid duplicated descriptor in the stdio failure path. The only invalid descriptor sentinel is -1.
Exact pools own backing allocated for one normalized parameter set. Reject zero, non-power-of-two, and stronger reservation alignments instead of silently returning storage with a weaker contract. This keeps the ordinary queue-allocation path unchanged while making direct and generic pool use obey the public reservation API.
RegisterTypeEquals had no callers and converted rich lookup failures into a boolean without propagating or consuming their status objects. Remove it instead of retaining an unused status-as-control-flow policy in test support.
Add pass.if_changed as a structured pipeline operation whose body observes the mutation result of its immediately preceding sibling. Carry that result through parsing, printing, verification, program compilation, builders, and interpreter execution so cleanup work can be skipped when a transform is a no-op. Represent static pipeline calls as nested program instructions instead of flattening their bodies into the caller. This preserves the callee's aggregate mutation result at the call boundary and gives subsequent conditional cleanup the same semantics for direct passes and called pipelines.
Teach vector insertion and extraction canonicalization about exact static bank coordinates. Forward reads from the slot just written, bypass disjoint writes, remove unchanged reinsertion, and collapse overwritten writes without guessing about dynamic or out-of-bounds coordinates. Also fold exact dynamic insert coordinates to static indices and recover the trailing source vector from a leading-axis broadcast. These rules make aggregate bank cleanup symmetric and reusable by structural transforms instead of coupling it to one scalarization pass.
Represent target residency as direct resource cliff chains plus optional derived resources assembled from rounded direct-resource contributions. The model is target-neutral and keeps stable resource identities suitable for scheduling, placement, final occupancy, diagnostics, and reports. Provide one validated query that evaluates every resource, identifies the limiting set, and reports the pressure required to recover or lose a tier. Unavailable, empty, saturated, and malformed models retain explicit semantics instead of leaking target-specific occupancy arithmetic.
Generate AMDGPU occupancy data as the target-neutral residency model and pass that same immutable policy through scheduling, allocation repair, occupancy reporting, native emission, and loom-check analysis. Central cliff evaluation now defines both greedy pressure penalties and final occupancy boundaries. Direct and derived resource identities stay stable across those phases, eliminating the duplicated low-codegen model and preventing planning from disagreeing with final target accounting.
Apply a source-derived memory summary to the unique dependency effect whose concrete memory space matches it. Single-memory-effect packets retain their existing unambiguous behavior, while generic or ambiguous multi-effect descriptors remain conservative. This lets cluster async packets refine their LDS destination without misclassifying the same packet's global read. Scheduling witnesses cover disjoint roots and residues as well as overlapping, unknown, consumer, and workgroup-barrier ordering.
Exercise the async-legality rejection for two outstanding cluster gathers that target the same strided workgroup interval. This is the negative counterpart to the disjoint-slot lowering witnesses and keeps same-root overlap conservative.
Allow func.apply to produce a kernel async group at verification time. Template selection and callable inlining erase that boundary before the whole-function async-lifetime pass, which then diagnoses duplicate waits, dropped groups, and escaping groups on the inlined operations. Runtime func.call results remain unsupported ownership boundaries and continue to fail verification.
Replace statically addressed leading dimensions of loop-carried vector aggregates with one scalar or trailing-vector recurrence per slot. Preserve ordinary unroll configuration while rebuilding the loop and replace exact result extracts without carrying an aggregate into target lowering. Validate the complete recurrence before rewriting. Dynamic prefixes, nested aggregate uses, escaping results, incompatible tails, and unrepresentable expansion fail through structured diagnostics instead of leaving partially scalarized IR.
Run vector-bank scalar replacement after SCF unrolling exposes static bank slots. This removes aggregate loop-carried recurrences before single-use promotion and register allocation consume them. Only run canonicalization and common-subexpression elimination when the scalar-replacement pass mutates the IR. This keeps cleanup coupled to the transformation that creates its opportunities.
The target residency model now owns every policy and query formerly exposed through the low-codegen pressure header. Remove the dead compatibility target and make the frame's residency dependency explicit in both build graphs.
Rename the shared target-neutral model and AMDGPU accessor so APIs describe the residency tiers they expose. Keep scheduler pressure names for local value and resource accounting, separating the target policy from its scoring mechanism.
Represent one repeated non-wrapping byte interval per dynamic stride and use comparable alias roots to prove disjoint packet slots. Source memory planning derives the residue interval from exact single-term address plans, while incomplete roots, strides, and bounds retain conservative may-alias behavior.
Make the checked-in pass operation declarations match the Python dialect source of truth. This retains the append-only operation ordinal while keeping the generated header reproducible by the normal C table generator.
Attach a versioned memory_access payload to low.op so source-derived alias roots, byte intervals, and strided residues survive lowering, text round-trips, and packet rewrites. Frame construction rebuilds the scheduling table from this payload when no ephemeral lowering table is available. Reject malformed or unsupported precision at the IR boundary, including sentinel alias IDs and fields that disagree with their precision flags. Canonical low.op fallback keeps compiler-owned metadata lossless inside target assembly regions.
Make source-memory recording explicitly opt into durable low-IR attachment and use it for compiler-owned workgroup allocations in AMDGPU memory, matrix-fragment, and cluster-async lowering. Cluster gather records its LDS destination summary because its packet combines a global read with a workgroup write. Source-lowering coverage captures distinct allocation roots and strided packet slots, while operand-form rewrites prove the metadata survives packet replacement.
Add XCNT as a gfx125x progress class with an encodable s_wait_xcnt packet. Memory descriptors retain every packet source through XCNT so allocation and wait planning can distinguish completion counters from source-translation lifetime. Keep the descriptor overlay target-specific: earlier RDNA4 sets do not expose the counter or wait packet. The gfx1250 WMMA oracle also records the resulting allocation contract: live address VGPRs displace conversion temporaries into dead payload registers without serializing on XCNT.
Build dependency links from materialized branch-edge copies to the producers of their copied values. Coalesced copies remain transparent, while register-repair moves now receive waits before the move instead of deferring them to a later SSA consumer. Missing allocation copy plans for branch payloads fail the planner contract instead of silently omitting a hardware dependency.
Propagate assignment-backed storage leases through the wait frontier so pending target source and result lifetimes survive control-flow edges. Check materialized results, branch-edge repairs, and their temporary register writes before processing a packet's logical effects. Incoming leases conservatively require a full counter drain when their dynamic position is path-dependent, while ordered same-class VMEM writeback can retire the exact older instance without an extra wait.
Model the source-register lifetime counter introduced by gfx125x as a first-class wait-planning progress domain. Translation packets now retain their scalar inputs until XCNT proves completion, and the control-flow frontier carries both retained leases and the active VMEM or SMEM translation group across block boundaries. Use ordered VMEM completion to emit minimum waits or rely on the implicit progress caused by a subsequent VMEM source overwrite. SMEM reuse remains a full-drain operation because scalar translations may complete out of order. Group transitions, system packets, branches, and program termination record the architectural drains they provide, while EXEC writes explicitly protect outstanding VMEM translations. Publish implicit-drain instruction families as generated descriptor traits instead of coupling the planner to mnemonic spelling. This keeps specialized forms, including cluster workgroup-id reads, under the same target semantic contract.
Split branch-edge repair writes from packet-local result placement in the wait planner. Edge copies execute before the structural branch packet and therefore cannot consume the branch's implicit XCNT drain; ordinary descriptor results and allocation reuse execute after it. Keep the native order explicit so retained source leases are discharged before an edge copy overwrites their physical registers without introducing redundant waits before implicitly draining packet results.
Make compact, parameterized kernels compile to the same scalar-recurrence shape as manually expanded schedules. Statically indexed loop-carried vector banks can remain structured in authored Loom source, be exposed by configured loop unrolling, and disappear before target lowering and register allocation. ## Design `sroa-vector-banks` recognizes `scf.for` operands whose leading vector dimensions form a finite, statically addressed bank. It validates the complete recurrence before changing the IR, rebuilds the loop with one scalar or trailing-vector value per bank slot, preserves unrelated loop-carried state, and rewrites exact result extracts. Dynamic prefixes, nested-region uses, escaping aggregate results, and incompatible updates produce structured diagnostics instead of a partial transform. Vector canonicalization exposes the bank structure by collapsing overwritten insert chains, folding exact dynamic coordinates, and recovering trailing vectors from leading-axis broadcasts. These are general aggregate canonicalizations and contain no target-specific schedule knowledge. The production source pipeline runs bank scalar replacement after configured unrolling and before single-use sinking, private-fragment promotion, and CFG lowering. The new `pass.if_changed` statement makes pass mutation an explicit pipeline condition, so canonicalization and CSE run only when the preceding transform changed the IR. ## Compiler contract Vector banks are an authoring representation, not an allocation unit. A selected bank must either decompose completely into ordinary SSA recurrences or fail with a diagnostic; it cannot silently survive as a giant target vector, aggregate copy, private allocation, or spill. The transform is target-independent. It turns compact schedule intent into the scalar and trailing-vector values already understood by target planning and register allocation, allowing schedule dimensions to vary without coupling allocator policy to authoring aggregates.
…182) Close the gfx1250 path from Loom source semantics to native code objects and AMDGPU HAL submission. This adds the target's matrix, tensor-memory, extended-register, workgroup-cluster, residency, and XCNT contracts as one executable system while preserving the established path for ordinary AMDGPU programs. ## Target and scheduling model Gfx125x now has an explicit 320 KiB LDS limit, its native matrix feature set, tensor-memory packet descriptors and wait counter, and post-allocation extended-VGPR state planning. Target-family provider selection lets reusable gfx125x implementations refine to an exact invocation target without weakening exact target identity. The old scalar pressure approximation is replaced by a target-neutral residency model. Direct and derived resources describe allocation granularities and occupancy cliffs once; scheduling penalties, allocation repair choices, final occupancy, diagnostics, and reports query the same model. This preserves the architectural question—whether a change crosses a residency tier—without encoding AMDGPU-specific register thresholds throughout the compiler. The resulting schedules also require general compiler repairs: tied storage survives alias coalescing, compact wide concatenations reserve contiguous spans, loop headers relocate to recurrent edge storage, bounded remainder and address-suffix facts canonicalize without losing dynamic expressions, and target-inserted native packets contribute to final reports. ## Workgroup clusters end to end Static cluster shape is part of the source and low-level launch contract. It is validated and preserved through AMDHSA V6 metadata, decoded by the HAL, checked against runtime-reported device limits, and submitted with AMD's extended 64-byte AQL dispatch packet. Ordinary dispatch remains on the standard packet; malformed, unsupported, indirect, and PM4 clustered launches fail explicitly. Loom models cluster and within-cluster identities, reconstructs gfx1250 launch state from the clustered ABI, proves cluster-convergent all-participant collectives and corresponding source and destination address agreement, and lowers cluster gathers to native async-to-LDS packets. The checked-in B128 multicast witness uses a nonuniform payload, two recipient workgroups, an explicit async wait, split gfx12 barrier packets, and independent LDS reads so identity, recipient remapping, transfer width, wait domain, and barrier placement are observable. ## Memory and wait correctness Source memory effects survive low IR as concrete roots, address spaces, byte intervals, and strided residues. Packet replacement refines only the matching effect, retains workgroup alias precision, and stays conservative for overlapping cluster destinations. Gfx125x tensor and cluster transfers are modeled from issue through group commit, wait, barrier, CFG edges, physical result writes, and exit. XCNT source addresses, payload registers, scalar bases, EXEC state, and translation groups remain leased until the required progress is proven. Branch-edge repair copies are emitted before a packet's implicit drain is credited, matching native instruction order and preventing a false proof at CFG joins. ## Behavioral contract Cluster metadata, executable validation, device capabilities, packet encoding, and queue submission describe one launch contract. A clustered executable is either supported consistently through that path or rejected before submission; metadata-only and runtime-only partial states are not accepted. Residency and XCNT reasoning are shared compiler mechanisms with gfx125x target data, not target-local bypasses. Non-clustered programs continue to use standard dispatch packets, and targets without gfx125x counters or extended-register state retain their existing behavior.
Use the topology-assigned logical device index and flattened logical queue ordinal as the canonical queue axis. Decode that representation for physical queue consumers and key each logical device epoch table by the same identity. This prevents independent HAL devices from aliasing in a device group while preserving distinct queues within composite devices and stable physical queue locations in profiling output.
AMDGPU queue frontiers now use the device-group topology index and flattened logical queue ordinal as their canonical identity. Independent logical devices previously restarted physical device and queue ordinals at zero and could therefore alias when placed in one group, allowing a cross-device axis to resolve against the wrong local queue. The epoch signal table is scoped to one logical device and indexed by flattened queue ordinal. Consumers that require physical coordinates decode through the queue-affinity domain, while profiling continues to expose physical queue ordinals. Fence-scope selection performs an inline identity and range check on the submission path. The regression constructs two AMDGPU logical devices through the production device-group builder and proves that local epoch resolution succeeds, cross-device resolution fails, and fence scopes distinguish local from cross-device queues. Composite logical devices retain distinct axes for every physical queue.
Exercise configuration binding, schedule-loop unrolling, and fragment-bank scalar replacement through final gfx1250 code generation from one production Loom source. The largest schedule carries an 8x4 bank whose aggregate representation exceeds the target vector contract, so successful lowering proves that structural scalar replacement occurs before target lowering. The same source binds launch geometry and workgroup storage so invalid configurations exercise cross-knob bounds, workgroup limits, and LDS capacity diagnostics.
Keep host-visible and managed allocations on shared host/device backing, preserve queue-compatible staging, and synchronize managed globals through stream timelines. Propagate kernel resource and native runtime parameter metadata from HSACO. Bind device heap, hostcall, and printf services per dispatch through generic HAL host notifications, with immutable patch lists validated and prepared outside the dispatch hot path. Retain runtime resources with queued work and preserve runtime parameter metadata in replay streams. Reject buffered-printf graph instantiation until reusable graph command buffers can provide race-free per-launch FIFO bindings.
AWoloszyn
force-pushed
the
users/awoloszyn/dev-03-04-01-host-visible-memory
branch
from
July 20, 2026 14:24
4685205 to
a3cbb3a
Compare
Serialize context-wide direct transfers and consolidate managed-global publication so host-visible allocations and module registrations remain coherent under concurrency. Make module registration transactional and remove redundant transfer staging. Centralize printf formatting and hostcall packet traversal with strict bounds, cycle limits, and dispatch-lifetime resource retention. Validate runtime patches against the selected kernel argument layout, reject unsupported graph runtime services, and prevent missed host-notification wakeups. Keep replay streams free of process-local runtime values and resources. Require complete executable vtables with backend-owned unsupported implementations, and cover the new parsing, packet, patching, graph, and notification invariants.
Retain runtime argument resources until deferred dispatches complete and avoid clearing newly allocated host-visible storage. Move executable-global discovery into the HAL metadata contract so the AMDGPU loader owns HSACO parsing and streaming module setup consumes backend-neutral declarations. Split runtime-service code into scoped components, keep cache probes inline, remove dead physical-device state, and document notification wake semantics.
Initialize cloned dispatch runtime parameter storage unconditionally so retained commands cannot observe stale arena state. Add coverage for both absent and present patch lists. Consume hostcall allocation failures, complete the Metal executable runtime metadata vtable, and snapshot managed buffers while holding the registry lock before performing blocking transfers.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This allows us to run some of the hip specific runtime bindings in hrx. For example printf and kernel-side memory allocations.