Skip to content

feat: parallelize container monitor dependency requests#6748

Closed
bgardiner wants to merge 3 commits intomainfrom
cursor/container-monitor-parallel-61fd
Closed

feat: parallelize container monitor dependency requests#6748
bgardiner wants to merge 3 commits intomainfrom
cursor/container-monitor-parallel-61fd

Conversation

@bgardiner
Copy link
Copy Markdown
Contributor

Pull Request Submission Checklist

  • Follows CONTRIBUTING guidelines
  • Commit messages
    are release-note ready, emphasizing
    what was changed, not how.
  • Includes detailed description of changes
  • Contains risk assessment (Low | Medium | High)
  • Highlights breaking API changes (if applicable)
  • Links to automated tests covering new functionality
  • Includes manual testing instructions (if necessary)
  • Updates relevant GitBook documentation (PR link: ___)
  • Includes product update to be announced in the next stable release notes

What does this PR do?

This PR improves performance for ecosystem/container monitoring by changing the /monitor-dependencies request flow from fully sequential to hybrid execution:

  • For each path, it always executes the first ScanResult request first (base OS project ordering retained)
  • It then executes the remaining ScanResult requests in parallel
  • It preserves output ordering by collecting parallel responses and appending them in scan order
  • It preserves existing error semantics:
    • 401 still throws AuthFailedError
    • other 4xx errors still throw MonitorError
    • non-4xx failures are collected as monitor errors and execution continues

Where should the reviewer start?

  • src/lib/ecosystems/monitor.ts
  • test/jest/unit/ecosystems-monitor-docker.spec.ts

How should this be manually tested?

  1. Run monitor unit tests:
    • npm run test:unit -- --runTestsByPath test/jest/unit/ecosystems-monitor-docker.spec.ts
  2. Optionally run a representative multi-project container monitor command and compare runtime:
    • snyk container monitor <image>

What's the product update that needs to be communicated to CLI users?

Container monitor execution is now faster for images that produce multiple scan results. The CLI now keeps base OS processing first and parallelizes the remaining dependency API requests.

Risk assessment (Low | Medium | High)?

Low. This is a scoped orchestration change with focused tests that verify ordering and existing error semantics.

Any background context you want to provide?

Large images that produce many scan results can experience long monitor durations due to sequential API calls; this change addresses that bottleneck.

What are the relevant tickets?

N/A

Open in Web Open in Cursor 

Co-authored-by: bgardiner <bgardiner@users.noreply.github.com>
@snyk-io
Copy link
Copy Markdown

snyk-io Bot commented Apr 24, 2026

Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
Open Source Security 0 0 0 0 0 issues
Licenses 0 0 0 0 0 issues
Code Security 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 27, 2026

Warnings
⚠️ There are multiple commits on your branch, please squash them locally before merging!

Generated by 🚫 dangerJS against 26562d4

@bgardiner
Copy link
Copy Markdown
Contributor Author

Superseded by #6757, which uses bounded concurrency via the SNYK_REQUEST_CONCURRENCY knob (introduced in #6756) instead of unbounded Promise.all.

Two reasons for the re-implementation:

  1. Unbounded Promise.all is a footgun on large images. Wildfly (quay.io/wildfly/wildfly:34.0.1.Final-jdk21) produces ~512 ScanResults — this PR would fire 500+ concurrent PUT /monitor-dependencies requests, risking 429s, socket exhaustion, and thundering-herd backend load. The new PR caps it at 10 by default with an env override.

  2. The "base OS first, then parallelize the rest" hybrid pattern isn't necessary. pMap returns results in input order regardless of completion order, so output ordering is preserved without special-casing the first ScanResult. The simpler uniform-parallelization shape is easier to reason about.

Will close this PR once #6756 and #6757 are merged.

@bgardiner
Copy link
Copy Markdown
Contributor Author

Closing — superseded by #6757 (bounded concurrency via SNYK_REQUEST_CONCURRENCY, simplified ordering). See comment above for rationale.

@bgardiner bgardiner closed this Apr 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants