fix(dataset): make mmap conversation reads concurrency-safe

[cquil11]

Summary

• read conversation records through cursor-free mmap slices
• prevent parallel worker threads from racing a shared mmap cursor
• add deterministic concurrent-read regression coverage

Root cause

MemoryMapDatasetClient.get_conversation() used seek() followed by read() on one mmap object shared by executor threads. Parallel AgentX warmup reads could move the shared cursor between those calls, returning the wrong or truncated JSON record.

Validation

• uv run pytest tests/unit/dataset/test_payload_mmap.py tests/unit/dataset/test_memory_map_compress.py tests/unit/dataset/test_mmap_cache.py -q (94 passed)
• full unit suite completed locally
• pre-commit run --files src/aiperf/dataset/memory_map_utils.py tests/unit/dataset/test_payload_mmap.py

[github-actions]

Try out this PR

Quick install:

pip install --upgrade --force-reinstall git+https://github.com/ai-dynamo/aiperf.git@67a42758b366589951cdecc06087e0a6cf88603d

Recommended with virtual environment (using uv):

uv venv --python 3.12 && source .venv/bin/activate
uv pip install --upgrade --force-reinstall git+https://github.com/ai-dynamo/aiperf.git@67a42758b366589951cdecc06087e0a6cf88603d

---

Last updated for commit: 67a4275 • Browse code