Skip to content

skillspector baseline leaves duplicate occurrences of a finding unsuppressed #633

Closed

Description

@Skeptomenos

Summary

skillspector baseline writes one fingerprint per deduplicated finding. scan --baseline checks suppression per occurrence, before deduplication. When the same match appears in two places, the generated baseline covers one occurrence, and the other one is still reported on the next scan.

Reproduction (v2.12.0; also v2.11.2)

demo/SKILL.md             (frontmatter: name, description)
  line 8:  The upstream service deletes unused files, and the link dies with no warning.
demo/references/notes.md
  line 3:  The mirror drops stale entries with no warning.
$ skillspector baseline demo/ --no-llm -o bl.yaml
Wrote baseline with 1 suppressed finding(s) to: bl.yaml

$ skillspector scan demo/ --no-llm --format json -o r.json --baseline bl.yaml
# r.json: issues = [AR2 @ references/notes.md:3], suppressed_count = 1

On v2.11.2 the same steps leave AR2 @ SKILL.md:8 open instead, so which occurrence survives also varies.

Expected

A baseline generated from a scan suppresses every finding that scan reported. The next scan of unchanged content reports nothing new.

Cause, as far as we can tell

  • baseline builds entries from effective_findings(result), the deduplicated list, via build_baseline_dict.
  • The report node calls partition_findings(selected_findings, baseline, …) on the pre-deduplication list. It deduplicates only the kept findings afterwards (deduplicate(active_findings) in nodes/report.py).
  • So an occurrence folded into another finding never gets its own fingerprint, but it is checked on its own.

Workaround

Iterate. Scan with the generated baseline, fingerprint whatever is still open, and repeat until nothing is open. Alternatively, keep a glob rules: entry, which matches per occurrence.

Environment

Python 3.12 on macOS, --no-llm.

Activity

  1. rng1995 commented on Sep 28, 2026

    @rng1995
    Collaborator

    Implementation is in progress in PR #637, which fingerprints the active findings before deduplication so each occurrence retains its own context and snippet. The PR is open and has not merged, so this issue remains open.

  2. rng1995 commented on Oct 4, 2026

    @rng1995
    Collaborator

    Resolution update: PR #637 has merged. Baseline generation fingerprints the original active findings before report deduplication, retaining each occurrence's own file, line, context, and snippet.

    Current-main baseline generation/rescan checks suppress all repeated same-file and cross-file occurrences, with no active findings left in the tested reproductions. This supersedes the earlier pending status; this issue was automatically closed by the merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions