V80 SLASH shell silently drops host `s_axi_control` writes to offsets `0x40–0xBF`

[dkumaramd]

V80 SLASH shell silently drops host s_axi_control writes to offsets 0x40–0xBF (i.e. when addr[7]≠addr[6])

1. Environment

1. Git dev branch commit 7a52870
2. Tools: Vivado/Vitis 2025.1 · SLASH VRT

2. Bug description

Host control-register accesses (VRT kernel.read()/write() over QDMA/PCIe) to a kernel's
s_axi_control interface are not delivered to the kernel for register offsets in the first 256
bytes of the control aperture whose address bit [7] ≠ bit [6] — i.e. 0x40–0x7F and 0x80–0xBF.
Observed behavior on these offsets: a host write does not update the kernel register (confirmed by
JTAG accesses and ILA probes on s_axi_control), yet a host read of the same offset returns the value
the host last wrote there — so the write-then-readback self-validates and the failure is invisible/masked.
Kernel argument pointers written to affected offsets stay 0, producing wrong results with no error
reported.

Addresses checked (V80 compute shell, 4 KB control aperture at 0x20200000000):

aperture offset	A[7:6]	result
0x00–0x3F	00	reaches kernel ✅
0x40–0x7F	01	does not reach kernel ❌
0x80–0xBF	10	does not reach kernel ❌
0xC0–0xFF	11	reaches kernel ✅
0x100–0xFFF	any	reaches kernel ✅

3. Temporary Workaround (kernel-side) which works for TAPA design

The failing 0x40–0xBF registers were re-mapped into 0x100–0xFFF (a region that does reach the kernel), and C_S_AXI_ADDR_WIDTH was widened 8 → 12 to make those offsets addressable. (As the good 0x00–0x3F and 0xC0–0xFF sub-bands were not large enough).

[hpc-aulmamei]

Thanks for the detailed writeup, @dkumaramd — I've been through the control path on dev and the reproducer in 20260911_tapa_bm25_ila.zip. Three findings, and then two captures I'd like to ask you for, because I think the current model of the bug may not hold.

1. The failing set looks sparse, not a 0x40–0xBF band

In the zip, ip_repo/indexer_top_1_0/src/indexer_top_control_s_axi.v has a hand-edited localparam ADDR_* block that diverges from the HLS-generated map in the same file's header comment. Only three arguments were relocated:

arg	HLS offset	relocated to
inst_mem	0x50	0xd4
doc_mem_1	0x68	0xe0
doc_mem_5	0x98	0xec

doc_mem_0 (0x5c), doc_mem_2 (0x74), doc_mem_3 (0x80), doc_mem_4 (0x8c), doc_mem_6 (0xa4) and doc_mem_7 (0xb0) all sit inside the reported dead band and were left in place — so at least at that point the failure was three specific registers, not every offset with A[7] != A[6]. Was the whole-band result measured later, by sweeping every dword? If it is really three registers, no address decoder produces that pattern and the A[7:6] framing is a coincidence of where 64-byte boundaries fall.

2. Write-combining on the control BAR is ruled out

This was my first suspicion, since the runtime does register access through an mmap of the BAR with no fences anywhere in the path, and the symptom — a write that never lands but reads back the value you wrote — is what a pending CPU-side write buffer looks like.

It isn't that. The driver picks the page protection from the prefetchable bit (driver/slash_dmabuf.c:187), and on a V80 the control aperture is PF2 BAR0 (128 MB → 0x202_0000_0000), which is not prefetchable: top.tcl sets PREFETCHABLE {1} only for PF2 BAR4, and on a live board Linux places BAR0 and BAR2 in the 32-bit MMIO window (0xa8000000, 0xb0000000) while BAR4 goes above 4 GB (0x380070000000) — the allocator's usual split for prefetchable vs not. So wc = 0, the mapping is pgprot_device, and every register write leaves the CPU as its own uncached dword. There is no host-side buffer that could absorb a write and satisfy the readback.

3. So the readback is probably telling the truth

With an uncached mapping, a readback that returns your value means something really is storing it. The simplest remaining explanation is that the write does reach s_axi_control and the HLS register file does hold it — the readback is correct — but those arguments never propagate into the kernel's datapath, which would leave the pointer at 0 inside the kernel exactly as you describe.

What makes me suspect that rather than the shell: the same zip carries hdl_patches.patch and hdl_patches_2.patch, which add power-up initial values to FIFO state and extend the kernel reset by 16 cycles (ap_rst_shift), with a comment about the region's synchronous reset being missed while the user clock is already running. Something in this design was already coming out of reset wrong. An argument register sampled by a sub-module before it is properly reset would read back correctly over AXI and still be 0 in the datapath.

What would settle it

Two captures from the unpatched register map, please:

1. JTAG read of 0x50, 0x68, 0x98 after the host has written them and before ap_start. Does the register file hold the written value, or 0?
2. ILA capture on s_axi_control showing AWADDR/WDATA/WVALID/BVALID/BRESP for those three writes, with the kernel-internal inst_mem, doc_mem_1, doc_mem_5 signals in the same window.

If the register file holds the value and the internal signal is 0, this is kernel/TAPA-side and the shell is exonerated. If the register file reads 0, the write genuinely is not landing and the search moves to the NoC → SmartConnect leg — with write-combining off the table.

Unrelated, but worth checking in your workaround

In the zip, component.xml declares reg0 range = 4096 while indexer_top_control_s_axi.v still has C_S_AXI_ADDR_WIDTH = 8 and ADDR_BITS = 8. The shell trusts component.xml (linker/slashkit/emit/hw/user_region/addr_ctx.py) and opens a 4 KB window onto a slave that decodes 256 bytes, so offsets 0x100+ alias back down into 0x00–0xFF. That is probably why 0x100–0xFFF appeared to reach the kernel. If the final fix relies on registers living above 0x100, please confirm the widening to 12 bits went into the RTL and the indexer_top.v instantiation as well as component.xml — an 8-bit slave behind a 4 KB window will alias silently.

[quetric]

@hpc-aulmamei i've been working since yesterday on a theory that this is caused by MSI-X registers for the respective BAR aliasing over the kernel aperture. See the associated development branch. Disabling MSI-X does not work by itself, i've now moved the MSI-X register base up to the end of the BAR and doing another build to check if that fixes it.

[dkumaramd]

@hpc-aulmamei - not sure which Zip file you are referring to ... please syn-up with @quetric as I had provided the passing & failing builds to him.