Skip to content

fix(ignore): suppress components ignored by a related purl (#120) - #125

Merged
mscasso-scanoss merged 4 commits into
crc64from
fix/ignore-related-purls
Oct 27, 2026
Merged

mscasso-scanoss merged 4 commits into
crc64from
fix/ignore-related-purls

Conversation

@mscasso-scanoss

@mscasso-scanoss mscasso-scanoss commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #120

Problem

The ignore list (-b SBOM.json) was only ever compared against the single primary purl stored in the url record (field 6 → purls[0]). ignored_asset_match() is applied per url record at fetch time, so it never sees a component's related purls.

A project is frequently published under several purls — e.g. a GitHub repo that is also indexed as a golang / nuget / conda package. Those sibling identities are materialised only later, at report time, by fetch_related_purls(). Consequently, ignoring a purl that happened to be a related purl of the matched component had no effect: the match survived under its primary identity and the ignored purl resurfaced verbatim in the reported "purl": [...] array.

This is exactly the reporter's scenario in #120: ignoring the first purl worked (it was the primary one), but ignoring a second purl of the same upstream project did nothing.

Fix

After load_matches() builds the candidate component list, and before best-match selection, drop every component whose primary OR related purl is in the ignore list. A legitimate non-ignored candidate for the same snippet region then wins, or the finding is suppressed entirely when no candidate survives.

  • ignored_purl() — checks a single purl string against the ignore list.
  • component_ignored_by_purl() — resolves the related purls (fetch_related_purls) and tests every purl of the component.
  • component_list_remove_ignored() — filters the component list in place.
  • load_matches() — invokes the filter, guarded by ignore_components so there is zero overhead when -b is not used.

Verification (ossv2 KB)

Reproduced the exact bug with a real multi-purl component. pkg:github/curl/curl carries pkg:nuget/libcurl-v143-shared as a related purl:

Scenario Before (crc64 tip) After
ignore the related purl pkg:nuget/libcurl-v143-shared pkg:github/curl/curl still reported, nuget purl still in the array component suppressed, region falls through to the next candidate
scan without -b — byte-identical output (only elapsed differs)
ignore a primary purl works still works
  • valgrind: the new code paths are clean.
  • Existing regression suite (tests/snippet_multicomponent) still passes.

Test

tests/ignore_related_purl/ — a data-driven regression test: it peels region candidates until a component exposing ≥2 purls surfaces, asks the engine to ignore one of that component's related purls, and asserts the whole component disappears. It skips gracefully when no KB / no multi-purl component is reachable, so it never breaks CI images built without a KB. Confirmed to fail on the pre-fix binary and pass after the fix.

Also fixed in this PR: normalize_license() buffer overflow

While testing with -b and valgrind, a pre-existing heap buffer overflow surfaced in the license-printing path (unrelated to #120 but easy to hit from the same scans):

normalize_license() rewrites a license string in place with its canonical SPDX id (e.g. Apache → Apache-2.0, GPLv2 → GPL-2.0-only), which can be longer than the matched text. license_add_to_list() sized the buffer to the original string length, so normalizing a short alias into a longer canonical id wrote past the end (valgrind: Invalid write of size 1 in strcpy, then an Invalid read in string_clean() running off the unterminated buffer).

Fix: reserve at least MAX_NORMALIZED_LICENSE_LN (32) bytes for license buffers (longest canonical id is LGPL-3.0-or-later, 17 chars) and defensively skip the in-place copy if a future table entry would not fit. After the fix, valgrind shows no Invalid read/write in the license path, normalization still emits the correct SPDX ids, and output without -b is unchanged. (Remaining valgrind "definitely lost" entries are pre-existing leaks in libldb init and health.c, untouched here.)

Also in this PR: snippet fallback when a full-file match is fully ignored

Related behaviour for the full-file case. When a file produces a full-file match (MATCH_FILE) but every candidate component is removed by the ignore list, the result used to collapse to "none" — the winnowing/snippet pass is skipped for file matches, so nothing was left to report.

The engine now detects that case and automatically retries once as a fresh snippet scan over the same file, honoring the same ignore/include lists, so a legitimate snippet match can still surface instead of reporting nothing.

  • scan_data_reset() discards the scan results (matches, matchmap, best match) while keeping the scan identity, config and the reusable hash/line buffers — the "throw the results away and start fresh" step, mirroring the result-freeing half of scan_data_free().
  • ldb_scan()'s file/snippet/compile body now runs in a loop; it resets and re-runs forcing snippets (once) when a full-file match is fully filtered under an active ignore list.

Scope & safety: zero behaviour change unless -b is used and a full-file match is fully filtered (verified byte-identical output otherwise). Verified on ossv2: a full-file curl match whose purls are all ignored now falls back to a snippet match instead of "none"; valgrind shows no new leaks or invalid accesses from the reset/second pass. Adds tests/snippet_fallback_on_ignore/ (data-driven, skips without a KB; fails pre-change, passes after).

Notes

  • Base branch is crc64; the ignore and scan logic is identical on main, so this forward-ports cleanly if desired.

🤖 Generated with Claude Code

mscasso-scanoss and others added 2 commits October 26, 2026 15:48
The ignore list (-b SBOM.json) was only compared against the single
primary purl stored in the url record (field 6, purls[0]). A project is
often published under several purls (e.g. a github repo also indexed as a
golang / nuget / conda package); those sibling identities are materialised
only later by fetch_related_purls() at report time. As a result, ignoring
a purl that was a *related* purl of the matched component had no effect:
the match survived under its primary identity and the ignored purl
resurfaced verbatim in the reported "purl" array.

Fix: after load_matches() builds the candidate component list, drop every
component whose primary OR related purl is in the ignore list, before
best-match selection. A legitimate non-ignored candidate for the same
region then wins, or the finding is suppressed entirely when none remain.

- ignored_purl(): check a single purl string against the ignore list.
- component_ignored_by_purl(): resolve related purls and test them all.
- component_list_remove_ignored(): filter the component list in place.
- load_matches(): invoke the filter, guarded by `ignore_components` so
  there is zero overhead when -b is not used.

Verified against the ossv2 KB: ignoring pkg:nuget/libcurl-v143-shared (a
related purl of pkg:github/curl/curl) now suppresses that component, while
a scan without -b is byte-identical to before (only elapsed time differs).
valgrind is clean for the new code paths. Adds a data-driven regression
test that auto-discovers a multi-purl component and asserts suppression.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
normalize_license() replaces a license string in place with its canonical
SPDX id (e.g. "Apache" -> "Apache-2.0", "GPLv2" -> "GPL-2.0-only"), which
can be LONGER than the matched text. License buffers were allocated by
license_add_to_list() sized to the ORIGINAL string length (rounded up to an
8-byte boundary), so normalizing a short alias into a longer canonical id
wrote past the end of the buffer.

valgrind flagged this as an "Invalid write of size 1" in strcpy
(license.c) followed by "Invalid read" in string_clean() walking off the
end of the corrupted, non-terminated buffer. It reproduces whenever a
reported component declares a license whose canonical SPDX id is longer
than its stored alias.

Fix:
- license_add_to_list(): reserve at least MAX_NORMALIZED_LICENSE_LN (32)
  bytes so any canonical id in license_normalization[] (longest is
  "LGPL-3.0-or-later", 17 chars) fits even when the original text is shorter.
- normalize_license(): defensively skip the in-place copy if the canonical
  id would not fit, guarding against future table entries.

Verified with valgrind on the same scan: the Invalid write/read in the
license path are gone, normalization still emits the correct SPDX ids
(e.g. GPL-2.0-only, Apache-2.0), and a scan without -b is unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
mscasso-scanoss and others added 2 commits October 26, 2026 16:42
… ignored

When a file produces a full-file match (MATCH_FILE) but every candidate
component is removed by the ignore list (-b), the result used to collapse
to "none": the winnowing/snippet pass is skipped for file matches, so there
was nothing left to report once the components were filtered.

The engine now detects that case and automatically retries once as a fresh
snippet scan over the same file, honoring the same ignore/include lists, so
a legitimate snippet match can still surface instead of reporting nothing.

Implementation (src/scan.c):
- scan_data_reset(): discards the scan RESULTS (matches, matchmap, best
  match) while keeping the scan identity, config and the reusable hash/line
  buffers, mirroring the result-freeing half of scan_data_free(). This is
  the "throw the results away and start fresh" step.
- scan_has_reportable_match(): true if any match still has components.
- ldb_scan(): the file/snippet/compile body now runs in a loop. If the first
  pass was a MATCH_FILE, an ignore list is active, and nothing is reportable,
  it resets and re-runs forcing snippets (once, guarded against re-entry).

The trigger is captured before compile_matches(), which resets match_type to
MATCH_NONE once the collapsed match has no components.

Scope and safety:
- Zero behavior change unless -b is used AND a full-file match is fully
  filtered: verified byte-identical output otherwise.
- Verified against the ossv2 KB: a full-file curl match whose purls are all
  ignored now falls back to a snippet match instead of "none"; valgrind shows
  no new leaks or invalid accesses from the reset/second pass.
- Adds tests/snippet_fallback_on_ignore/ (data-driven, skips without a KB;
  confirmed to fail pre-change and pass after).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…pet fallback

Bumps SCANOSS_VERSION for the changes in this PR (#120):
- suppress components ignored by a related purl
- fix normalize_license() buffer overflow
- fall back to a snippet scan when a full-file match is fully ignored

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@mscasso-scanoss mscasso-scanoss self-assigned this Oct 27, 2026
@mscasso-scanoss
mscasso-scanoss merged commit 52ac81e into crc64 Oct 27, 2026
1 check passed
@mscasso-scanoss
mscasso-scanoss deleted the fix/ignore-related-purls branch October 27, 2026 18:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant