Skip to content

Fix add_to_suite losing the testcase under junitparser>=5 - #179

Merged
jhutar merged 3 commits into
redhat-performance:mainfrom
NewtonChutney:fix-junitcli-deepcopy
Jul 16, 2026
Merged

jhutar merged 3 commits into
redhat-performance:mainfrom
NewtonChutney:fix-junitcli-deepcopy

Conversation

@NewtonChutney

Copy link
Copy Markdown
Contributor

Summary

  • TestSuite.add_testsuite() in junitparser 5.x deep-copies the suite before appending it to the tree (a behavior change from 4.x). add_to_suite() created a new TestSuite, passed it to add_testsuite() immediately, then mutated the same (now orphaned) suite object by adding the testcase afterward — the tree kept the pre-testcase deep copy, so the case silently vanished with no exception anywhere in the call chain.
  • Since a caller's per-run-unique junit filename always hits the "new suite" path, this affects every single such run consistently (not intermittent) once an environment resolves junitparser>=5, while environments still pinned to junitparser<5 are unaffected — a pip install -e git+... without an upper bound means different agents can silently drift onto different junitparser majors.
  • In our case this produced ReportPortal launches with a bare item and no attached log, despite the pipeline reporting success at every step (add, upload, and Ibutsu import all exit 0/succeed since nothing raises).

Fix

Attach the testcase to the suite before it is ever passed to add_testsuite(), so newer junitparser's deep copy carries it along. Both the new-suite and existing-suite code paths are otherwise unchanged.

Test plan

  • Reproduced the bug against a real production status-data-*.json/log files with junitparser==5.0.1: add for PASS/FAIL/ERROR all produced an empty <testsuite/> (no exception, exit code 0).
  • Applied the fix and reproduced both the new-suite and existing-suite code paths locally — both now correctly populate the <testcase>.
  • Existing tests/test_junit_cli.py::test_add_to_suite still passes.
  • Verified end-to-end against a real Jenkins-triggered pipeline run (patched the fix into the agent's venv mid-run): the resulting junit-*.xml went from 111 bytes (empty) to ~15KB with a populated <system-out>, and the resulting ReportPortal launch showed a proper 3-level hierarchy (launch → suite → item) with the log attached.

🤖 Generated with Claude Code

TestSuite.add_testsuite() started deepcopying the suite before
appending it to the tree in junitparser 5.x. add_to_suite() created a
new TestSuite, passed it to add_testsuite() immediately, then mutated
the same (now orphaned) suite object by adding the testcase afterward
- the tree kept the pre-testcase deepcopy, so the case silently
vanished with no exception anywhere in the call chain.

Reproduced against a real production status-data file: under
junitparser==5.0.1, every add for a scenario that hadn't already
created its suite in that run's junit.xml (i.e. every single kessel-k6
run, since each uses a unique per-run filename) produced a
testcase-less <testsuite/>, which explains ReportPortal launches
showing a bare item with no attached log despite the pipeline
reporting success at every step.

Fix: attach the testcase to the suite before it is ever passed to
add_testsuite(), so newer junitparser's deepcopy carries it along.
Verified against both the new-suite and existing-suite code paths on
junitparser 5.0.1, and the existing test_add_to_suite unit test still
passes.
@coderabbitai

coderabbitai Bot commented Jul 15, 2026 •

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Enterprise

Run ID: 37d8b894-9c1c-42fc-ac1e-172422bcd797

📥 Commits

Reviewing files that changed from the base of the PR and between 8262193 and bd157df.

📒 Files selected for processing (2)
  • core/opl/junit_cli.py
  • tests/test_junit_cli.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • core/opl/junit_cli.py

📝 Walkthrough

Summary by CodeRabbit

  • Bug Fixes
    • Fixed an issue where test cases could be missing from generated JUnit XML reports.
    • Ensured test cases are immediately added when targeting an existing suite.
    • When the target suite doesn’t exist, it is now created and the testcase is correctly preserved, including results, output, timing, and start/end details.
  • Tests
    • Added regression coverage for creating new suites and verifying both in-memory and persisted JUnit XML output.

Walkthrough

JUnitXmlPlus.add_to_suite now fully populates each testcase before selecting a suite, then explicitly inserts it into existing suites or newly created suites before registration. Regression tests cover in-memory and persisted results.

Changes

JUnit suite insertion

Layer / File(s) Summary
Populate and insert testcases
core/opl/junit_cli.py
add_to_suite populates testcase fields before suite selection, adds the testcase to matching existing suites, and inserts it into new suites before registration.
Regression coverage
tests/test_junit_cli.py
Tests verify that adding to a new suite preserves the testcase in memory and after reloading the saved XML.

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly states the main fix and matches the changeset.
Description check ✅ Passed The description directly explains the junitparser 5 issue and the fix applied.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
core/opl/junit_cli.py (1)

111-117: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Stop iteration early and use for...else to simplify suite lookup.

Adding a break prevents redundant iterations and avoids adding the testcase multiple times if duplicate suite names exist. Furthermore, using Python's for...else construct eliminates the need for the suite_found flag, streamlining the logic.

♻️ Proposed refactor
-        suite_found = False
         for suite in self:
             if suite.name == suite_name:
                 logging.debug(f"Suite {suite_name} found, going to add into it")
                 suite.add_testcase(case)
-                suite_found = True
-        if not suite_found:
+                break
+        else:
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@core/opl/junit_cli.py` around lines 111 - 117, Refactor the suite lookup
around the loop in the JUnit CLI code to use Python’s for...else construct
instead of the suite_found flag. In the suite.name == suite_name branch, add the
testcase once and immediately break to stop iteration; move the current
not-found handling into the loop’s else block, preserving existing behavior when
no matching suite exists.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@core/opl/junit_cli.py`:
- Around line 111-117: Refactor the suite lookup around the loop in the JUnit
CLI code to use Python’s for...else construct instead of the suite_found flag.
In the suite.name == suite_name branch, add the testcase once and immediately
break to stop iteration; move the current not-found handling into the loop’s
else block, preserving existing behavior when no matching suite exists.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Enterprise

Run ID: 5323c281-b6b6-4c0d-b39d-956bfb681dc2

📥 Commits

Reviewing files that changed from the base of the PR and between af14f1e and 8262193.

📒 Files selected for processing (1)
  • core/opl/junit_cli.py

Addresses review nitpick. No behavior change: break on match instead
of continuing to iterate, and move the not-found handling into the
loop's else clause. Re-verified against junitparser==5.0.1 (existing
test_add_to_suite unit test, plus a direct two-call accumulation test
against the same suite) - both new-suite and existing-suite paths
still work correctly.
Both new tests fail against the pre-fix code under junitparser==5.0.1
(confirmed on a real environment with that version installed):
add_to_suite() for a suite that doesn't exist yet silently drops the
testcase, both in-memory and once the file is reloaded from disk (the
latter matching how junit_cli.py is actually used - "add" and "upload"
are separate CLI invocations reading back a previously written file).

Also confirmed the existing test_add_to_suite fails the same way under
junitparser==5.0.1, it just hadn't been exercised against that version.
@jhutar
jhutar merged commit d75925e into redhat-performance:main Jul 16, 2026
2 checks passed
@NewtonChutney
NewtonChutney deleted the fix-junitcli-deepcopy branch July 18, 2026 07:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants