Skip to content

Make coverity happy for 4.0#623

Merged
bosilca merged 13 commits into
ICLDisco:masterfrom
bosilca:fix/coverity-happy-for-4.0
May 11, 2026
Merged

Make coverity happy for 4.0#623
bosilca merged 13 commits into
ICLDisco:masterfrom
bosilca:fix/coverity-happy-for-4.0

Conversation

@bosilca
Copy link
Copy Markdown
Contributor

@bosilca bosilca commented Jan 28, 2024

Coverity identified 692 legitimate issues. Luckily for us most of them are in the generated code and have relatively simple solutions. This PR is a placeholder to address all of the issues that can be addressed in a reasonable time frame.

@bosilca bosilca requested a review from a team as a code owner January 28, 2024 04:34
@bosilca bosilca force-pushed the fix/coverity-happy-for-4.0 branch from b6c8edf to a1e80e5 Compare January 28, 2024 05:05
@bosilca bosilca changed the title Fix/coverity happy for 4.0 Make coverity happy for 4.0 Jan 28, 2024
@bosilca bosilca force-pushed the fix/coverity-happy-for-4.0 branch from a1e80e5 to 635cdc5 Compare January 28, 2024 05:49
@abouteiller abouteiller added this to the v4.0 milestone Jan 31, 2024
Comment thread parsec/interfaces/ptg/ptg-compiler/jdf2c.c Outdated
Comment thread parsec/utils/output.c
Comment thread tests/dsl/dtd/dtd_test_simple_gemm.c
Comment thread parsec/interfaces/ptg/ptg-compiler/jdf2c.c
@bosilca bosilca force-pushed the fix/coverity-happy-for-4.0 branch from 635cdc5 to 538dea3 Compare February 2, 2024 04:14
@bosilca bosilca marked this pull request as draft February 5, 2024 16:47
@bosilca
Copy link
Copy Markdown
Contributor Author

bosilca commented Feb 5, 2024

let's keep it as a draft for now, there are some tests failing and I can't yet figure out why.

@abouteiller
Copy link
Copy Markdown
Contributor

abouteiller commented Feb 7, 2024

Snapshot of investigation:

During parsec_redistribute_dtd_New we create an adt that we cleanup later in that same function.

376         adt = parsec_dtd_create_arena_datatype(parsec, &TARGET);
...
(gdb) n
426         adt = parsec_dtd_get_arena_datatype(parsec, TARGET);
(gdb) p *adt
null

we later dereference null+8 as an input to parsec_datatype_free, which may also explain why we would see these weird MPI_TYPE_FREE errors in some cases recently.

The same code for SOURCE works, and I suspect that this code in add2arena causes the index to be overwritten with 0 when it should be 1.

260     adt->ht_item.next_item = NULL;  /* keep Coverity happy */
261     adt->ht_item.hash64    = 0;  /* keep Coverity happy */
262     adt->ht_item.key       = 0;  /* keep Coverity happy */

@abouteiller
Copy link
Copy Markdown
Contributor

abouteiller commented Feb 7, 2024

For posterity, a piece of discussion from Slack about the dtd_arena_create and our plan to address the problem

I see what is happening. In general we create the arena with a datatype and we keep it for as long as we need it.
13:56 well … not in DTD. Here we create an empty arena, I guess just to get an ID that will then be attached to the arena we create with the normal function
13:57 so the “create_arena” cannot reset the ht_key (which is used as the ID) because in DTD this holds important data
13:57 so create is now not really a create, but more like “initialize some fields”
13:58 well, coverity was not complaining about the use of ht_item in DTD, because there it is used. IT was complaining about it in PTG where it is not used at all
14:00we need to go back to a sane use of the arena. We create and that function will set all fields, and then in DTD we will register it with the taskpool, and then it will get its ID (edited)
14:01 with this approach we will keep most of the infrastructure we have today in place. Which means we will still use the hash-table for the DTD arenas for something with an id that monotonically increases …

@abouteiller abouteiller self-requested a review February 22, 2024 14:04
abouteiller added a commit to abouteiller/dplasma that referenced this pull request Apr 22, 2024
@abouteiller abouteiller force-pushed the fix/coverity-happy-for-4.0 branch from 193050d to 45d6c62 Compare April 24, 2024 20:53
@abouteiller abouteiller linked an issue Apr 24, 2024 that may be closed by this pull request
@abouteiller
Copy link
Copy Markdown
Contributor

CI failure is due to #641 this is ready

Comment thread parsec/data.c Outdated
Comment thread parsec/data_dist/matrix/apply_wrapper.c Outdated
Comment thread parsec/interfaces/ptg/ptg-compiler/jdf_unparse.c
Comment thread parsec/utils/argv.c
Comment thread tests/collections/reshape/testing_reshape.c
abouteiller added a commit to abouteiller/dplasma that referenced this pull request Oct 24, 2024
@abouteiller abouteiller modified the milestones: v4.0, v5.0 Nov 6, 2024
@bosilca bosilca force-pushed the fix/coverity-happy-for-4.0 branch from 12fdcd4 to 358aad4 Compare May 10, 2026 04:56
@bosilca bosilca marked this pull request as ready for review May 10, 2026 04:57
bosilca and others added 4 commits May 10, 2026 00:57
Don't generate the args code if there are no successors
compute the key using 64 bits arithmetic
ensure vpid is within expected range

Signed-off-by: George Bosilca <bosilca@icl.utk.edu>
This is only used by the DTD interface, but it needs to be completely
initialized in all cases.

Signed-off-by: George Bosilca <bosilca@icl.utk.edu>
Signed-off-by: George Bosilca <bosilca@icl.utk.edu>
Signed-off-by: Aurelien Bouteiller <bouteill@icl.utk.edu>
abouteiller and others added 8 commits May 10, 2026 00:57
Fix the arena datatype lifecycle in PTG examples, wrappers, and tests.

Generated PTG taskpools embed their arena datatype objects, but user code is
responsible for initializing the opaque datatype assigned to those arenas and
for releasing it before freeing the taskpool. Update examples and helper
wrappers to initialize the embedded taskpool arena directly with
parsec_arena_datatype_set_type() or the matrix ADT helpers, instead of copying
temporary arena datatype objects into the taskpool.

Add matching taskpool-specific free helpers where needed so tests can destroy
the arena datatype and then free the generated taskpool. Also free generated
taskpools in parser/runtime tests that previously only created them, and remove
a redundant DTD arena datatype reinitialization after parsec_matrix_adt_new_rect()
already created the requested type.

While touching the generated PTG release path, scope the dynamic termination
ready-list loop variable locally to the emitted loop.

Signed-off-by: George Bosilca <gbosilca@nvidia.com>
Pair several internal allocations with their matching teardown paths exposed by
the debug leak sweep.

Release temporary MCA component-selection strings, free selected device component
arrays during device shutdown, and make MCA parameter teardown release source
file metadata regardless of where the active value came from.

Clean up scheduler-owned queues consistently by removing the duplicate LTQ
system-queue allocation and releasing LHQ hierarchical queue metadata and shared
system queue state during scheduler removal.

Also release the map-operator task-class array from the taskpool destructor.

Signed-off-by: George Bosilca <gbosilca@nvidia.com>
Fix test-side ownership leaks reported during the debug leak sweep.

Destroy matrix data and descriptors with the matching PaRSEC teardown APIs, free
temporary result buffers and helper allocations, release DTD tile handles and
task classes, and clean up custom test distribution state before process exit.

Update class and runtime tests to release barriers, hwloc state, list/lifo
objects, timing buffers, and thread arrays. Keep the remaining DTD
data_flush_all lifetime findings documented in the leak-sweep TODO for later
investigation.

Signed-off-by: George Bosilca <gbosilca@nvidia.com>
@bosilca bosilca force-pushed the fix/coverity-happy-for-4.0 branch from 358aad4 to c07129a Compare May 10, 2026 05:11
Correct spelling mistakes across comments, documentation strings, debug
messages, and test diagnostics. This keeps generated help text and
developer-facing messages consistent without changing behavior.

Signed-off-by: George Bosilca <gbosilca@nvidia.com>
Copy link
Copy Markdown
Contributor

@devreal devreal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but there are some open points of discussion.

Comment thread parsec/utils/argv.c
Comment thread tests/collections/reshape/testing_reshape.c
@bosilca bosilca merged commit 36d655a into ICLDisco:master May 11, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Rename parsec_add2arena

3 participants