The link is up and RX sees the packet, but the received packet is not counted as a good packet.

[think-gaoxing]

DCMAC Integrated in Static Service Shell: Clarify Network App Flow and Fix RX Good-Packet Validation

Summary

In the newer service-shell architecture, DCMAC appears to be integrated directly into the static service shell rather than being packaged and hot-loaded through a service partial PDI.

After flashing the latest service static shell to V80 flash and syncing the matching userspace/kernel software stack, the DCMAC MMIO registers are accessible directly through PF2 BAR. The installed DCMAC validation script can verify link state and generate traffic without loading examples/06_dcmac as a partial design.

Environment

• Board: AMD Alveo V80
• BDF: 0000:38:00
• Static shell type: service
• SLASH source revision: e6a014bfd5414f72ea83be7658a0d5a2dc77299d
• Static shell PDI:
  /home/nic/slash-dev2/build/hdpr130_work/artifacts/static_route_fix_post_noc_fresh_flash/amd_v80_gen5x8_25.1.pdi
• Static shell PDI SHA256:
  10a01c4baa5d97fd5e981fb763cae7aac1ee97cbdddcb414e5f05b36ca5f203d
• Tool version:
  v80-smi version -> SMI v1.0.0
• Kernel module:
  slash/1.0.0, DKMS installed
• DCMAC validation script:
  /usr/lib/python3.12/dist-packages/slashkit/resources/dcmac/driver/network_end2end_test.py

What Was Done

1. Flashed the newer service static shell to V80 flash.
2. Reset the board into the flashed service shell.
3. Rebuilt and installed the matching software stack from the same newer source tree:
   • v80-smi
   • vrtd
   • libvrt
   • libvrtd
   • libvrtdpp
   • libslash
   • slashkit
   • slash DKMS kernel module
4. Verified board enumeration:
   • PF0: ami
   • PF1: slash_qdma
   • PF2: slash_ctl
   • VRTD: OK
5. Ran DCMAC link validation directly against the static shell.

Validation Command

sudo python3 /usr/lib/python3.12/dist-packages/slashkit/resources/dcmac/driver/network_end2end_test.py -d 38 -v 1

Note: for this script, -d 38 maps to /sys/bus/pci/devices/0000:38:00.2/resource2.

Observed Result

The DCMAC links are up:

dcmac0.link_up=True
dcmac1.link_up=True

Traffic generation also changes counters:

dcmac0 TX:
  TOTAL_BYTES=1412
  TOTAL_PACKETS=1
  TOTAL_GOOD_PACKETS=1
  BAD_FCS=0
  FRAME_ERROR=0

dcmac1 RX:
  TOTAL_BYTES=1412
  TOTAL_PACKETS=1
  TOTAL_GOOD_PACKETS=0
  INRANGEERR=1
  BAD_FCS=0
  BAD_CODE_COUNT=0

Current Issue

The link is up and RX sees the packet, but the received packet is not counted as a good packet.

The suspicious part is:

dcmac1 RX TOTAL_PACKETS=1
dcmac1 RX TOTAL_GOOD_PACKETS=0
dcmac1 RX INRANGEERR=1

This suggests the static-shell DCMAC path is alive, but the generated frame may not fully satisfy the RX MAC packet validity checks, or the traffic generator/test script is producing a frame length/type/FCS/statistics condition that DCMAC reports as INRANGEERR.

Architecture Question

Given this newer architecture, should network applications that use DCMAC avoid generating or hot-loading a service partial PDI?

Current understanding:

• DCMAC, GT, and the base network/trafficgen MMIO blocks are now part of the static service shell.
• network_end2end_test.py can validate link and traffic directly through BAR access.
• examples/06_dcmac no longer needs to be packaged into or loaded as a service partial PDI for basic DCMAC link/traffic validation.
• Future network applications should use the static service shell's resident DCMAC/network infrastructure unless they need additional custom programmable logic outside the static shell.

Please confirm whether this is the intended architecture.

Expected Behavior

For the built-in static-shell DCMAC traffic test:

1. dcmac0.link_up and dcmac1.link_up should both be True.
2. A frame transmitted from one DCMAC should be received by the peer DCMAC.
3. RX statistics should count the received frame as a good packet, assuming the traffic generator emits a valid Ethernet frame.

Actual Behavior

Link is up and RX packet counters increment, but RX good-packet counters do not increment and INRANGEERR is observed.

Questions

1. Is examples/06_dcmac still expected to produce a service partial PDI in the new architecture?
2. Or is DCMAC intended to be permanently resident in the static service shell?
3. For future network applications, should we avoid overwriting/loading service partial PDI unless adding custom non-DCMAC logic?
4. Is INRANGEERR=1 expected for the current network_end2end_test.py trafficgen packet, or does it indicate a frame formatting/datapath bug?
5. If service partial PDI hot-load is no longer part of the DCMAC flow, should the documentation for examples/06_dcmac be updated to distinguish the old partial-PDI flow from the new static-shell DCMAC flow?

[mariodruiz]

Hi @think-gaoxing,

I think large part of your questions is answered in the documentation. https://slash-fpga.readthedocs.io/en/latest/explanation/dcmac.html#dcmac-integration-in-slash

The DCMAC lives in the service region. This region only gets loaded when the board boots up.
The kernels talking with the network go into the user region.

Can you please elaborate on how the network interfaces are connected? it would be good if you share the full script log.
We are working on the example 06 as there are other issues as #218

[think-gaoxing]

Hi @mariodruiz

The physical setup was a direct QSFP56 cable between QSFP0 and QSFP2 on the V80. No Ethernet switch was used.

The interface mapping is:

V80 QSFP0
  -> service-layer hierarchy qsfp_0_n_1
  -> DCMAC0, port 0
  -> eth_0

V80 QSFP2
  -> service-layer hierarchy qsfp_2_n_3
  -> DCMAC1, port 0
  -> eth_2

The intended AXI-Stream connections are:

traffic_producer_0.axis_out -> eth_0.tx0
eth_0.rx0                  -> traffic_consumer_0.axis_in

traffic_producer_1.axis_out -> eth_2.tx0
eth_2.rx0                  -> traffic_consumer_1.axis_in

Therefore, across the external cable, the traffic path is:

producer_0 -> DCMAC0/QSFP0 TX
           -> external QSFP0-to-QSFP2 cable
           -> DCMAC1/QSFP2 RX -> consumer_1

producer_1 -> DCMAC1/QSFP2 TX
           -> external QSFP2-to-QSFP0 cable
           -> DCMAC0/QSFP0 RX -> consumer_0

The DCMAC control registers are accessed through PF2 BAR2:

PF2 BAR2 device AXI base: 0x0203_0000_0000

DCMAC0 / QSFP0:
  device address: 0x0203_0200_0000
  BAR2 offset:    0x02000000
  aperture:       64 KiB

DCMAC1 / QSFP2:
  device address: 0x0203_0300_0000
  BAR2 offset:    0x03000000
  aperture:       64 KiB

The static-shell address audit confirmed that both DCMAC address segments are present in the service address space and in both CPM PCIe-
NoC master spaces. 

 The relevant result was:

DCMAC +0x2000000 revision=0x1
DCMAC +0x3000000 revision=0x1

INITIAL:
  DCMAC0 link_up=True
  DCMAC1 link_up=True

LINK:
  DCMAC0 link_up=True
  DCMAC1 link_up=True

DCMAC0 TX:
  TOTAL_BYTES=1412
  TOTAL_PACKETS=1
  TOTAL_GOOD_PACKETS=1
  BAD_FCS=0
  FRAME_ERROR=0

DCMAC1 RX:
  TOTAL_BYTES=1412
  TOTAL_PACKETS=1
  TOTAL_GOOD_PACKETS=0
  INRANGEERR=1
  BAD_FCS=0
  BAD_CODE_COUNT=0
  TRUNCATED=0

FAIL: Direction 0->1: expected one good packet

Both physical links were therefore up, and the packet reached the opposite MAC. 

[mariodruiz]

From what you said, you ran network_end2end_test.py. Can you please share the full output of the script?

[think-gaoxing]

After update code to the latest dev code 1ae9b77, the ports don't bring up.
sudo python3 network_end2end_test.py --dev 38
dcmac0.link_up=False
dcmac1.link_up=False
Link not detected in at least one of the DCMACs