Skip to content

fix(export-dynamic): verify registry artifacts before skipping workspace (RHDHBUGS-3635) - #160

Open
gashcrumb wants to merge 5 commits into
redhat-developer:mainfrom
gashcrumb:fix/RHDHBUGS-3635-verify-registry-artifacts
Open

gashcrumb wants to merge 5 commits into
redhat-developer:mainfrom
gashcrumb:fix/RHDHBUGS-3635-verify-registry-artifacts

Conversation

@gashcrumb

Copy link
Copy Markdown
Member

Summary

Fixes RHDHBUGS-3635

When determining whether to skip a workspace because its last commit is an ancestor of the last published commit, verify that all expected container images for the workspace actually exist in the registry and contain valid, non-empty dynamic package metadata (io.backstage.dynamic-packages).

If any expected artifact is missing from the registry or has invalid/empty dynamic package metadata (such as from a cancelled publish or export failure), do not skip the workspace — proceed with exporting and publishing to self-heal the missing/broken artifact.

@gashcrumb
gashcrumb requested a review from davidfestal August 26, 2026 12:57
Comment thread export-dynamic/export-dynamic.sh Outdated
…ace (RHDHBUGS-3635)

Assisted-By: opencode
Signed-off-by: Stan Lewis <gashcrumb@gmail.com>

rh-pre-commit.version: 2.4.0
rh-pre-commit.check-secrets: ENABLED
@gashcrumb
gashcrumb force-pushed the fix/RHDHBUGS-3635-verify-registry-artifacts branch 2 times, most recently from ca3e7a5 to 74b9cbc Compare September 22, 2026 15:17
Assisted-By: opencode
Signed-off-by: Stan Lewis <gashcrumb@gmail.com>

rh-pre-commit.version: 2.4.0
rh-pre-commit.check-secrets: ENABLED
@gashcrumb
gashcrumb force-pushed the fix/RHDHBUGS-3635-verify-registry-artifacts branch from 74b9cbc to 164415b Compare September 22, 2026 15:28
Comment thread export-dynamic/action.yaml Outdated
Comment thread export-dynamic/export-dynamic.sh
Comment thread export-dynamic/export-dynamic.sh
…ification

- Write WORKSPACE_SKIPPED_UNCHANGED_SINCE=false to GITHUB_OUTPUT before
  exiting when skopeo is missing, so the action step captures the output
  even though the shell step runs with `|| true`; downstream steps then
  correctly treat the workspace as non-skipped instead of green-lighting
  a broken artifact.
- Remove the stray reference to a `push-container-image` input in the
  action.yaml description for `last-publish-commit`; that flag is
  hardcoded in the step env, not an action input.
- Correct the README claim that the script 'fails with an error' (true
  when invoked directly, not when called through the action) to describe
  the actual observable outcome via the action.
- Add a README note explaining that a `versions.json` bump on a release
  branch causes all workspaces to be re-exported on the next run because
  the tag prefix changes; skopeo then finds no images under the new
  prefix and the skip is bypassed for all of them.

Assisted-By: opencode

rh-pre-commit.version: 2.4.0
rh-pre-commit.check-secrets: ENABLED
@jonkoops
jonkoops requested a review from nickboldt October 6, 2026 15:10
nickboldt
nickboldt previously approved these changes Oct 6, 2026
@nickboldt

Copy link
Copy Markdown
Member

to validate this is working we should run it against the broken scorecard images?

…ion test suite

- Support multi-platform OCI image indexes and manifest lists in verify_registry_artifact by dereferencing child manifests when top-level annotations are not present.
- Extract verification logic into export-dynamic/verify-registry-artifacts.sh following the pack-dist-dynamic pattern.
- Add comprehensive test suite in export-dynamic/verify-registry-artifacts.test.sh and wire into test-pack-dist-dynamic workflow.

Assisted-By: opencode
Signed-off-by: Stan Lewis <gashcrumb@gmail.com>

rh-pre-commit.version: 2.4.0
rh-pre-commit.check-secrets: ENABLED
@gashcrumb

Copy link
Copy Markdown
Member Author

Great suggestion! We investigated this and tested against those images:

Background & Quay vs GHCR

  1. Where the broken images were: The broken scorecard images with empty io.backstage.dynamic-packages annotations were downstream Konflux builds in Quay (quay.io/rhdh/..., from RHDHBUGS-3834), caused by an empty pipeline parameter. The upstream scorecard artifacts that export-dynamic publishes to GHCR were never broken and have always had valid annotations.
  2. Image Indexes vs Manifests: Quay/Konflux builds are multi-architecture OCI image indexes (application/vnd.oci.image.index.v1+json). On an image index, the io.backstage.dynamic-packages annotation is stamped on the child platform manifests rather than the root index.

Updates in this PR

We updated the verification check (export-dynamic/verify-registry-artifacts.sh) so that when skopeo inspect --raw returns an image index / manifest list without root annotations, it resolves .manifests[0].digest and inspects the child platform manifest.

Test Results

Tested against both Quay multi-arch images and GHCR single manifests:

  1. Broken Quay scorecard image (:2.0.0--1.1.1 from RHDHBUGS-3834):
    • Inspects index $\rightarrow$ resolves child manifest $\rightarrow$ catches empty annotation "" $\rightarrow$ Correctly rejected (skipWorkspace=false, workspace cannot be skipped).
  2. Fixed Quay scorecard image (:2.0.0--1.1.2):
    • Inspects index $\rightarrow$ resolves child manifest $\rightarrow$ verifies valid base64 annotation $\rightarrow$ Correctly accepted.
  3. Valid GHCR image (bs_1.54.9__1.4.0):
    • Single manifest with annotation $\rightarrow$ Correctly accepted.
  4. Non-existent tag / missing annotation:
    • Correctly rejected.

We have also added an automated test suite (export-dynamic/verify-registry-artifacts.test.sh) covering offline mock fixtures (valid manifests, empty annotations, missing annotations, empty arrays, index dereferencing) and live registry checks, wired directly into .github/workflows/test-pack-dist-dynamic.yaml so CI validates this on every PR.

…fact

Merge collapsible if statements in verify-registry-artifacts.sh to resolve SonarCloud S1066.

Assisted-By: opencode
Signed-off-by: Stan Lewis <gashcrumb@gmail.com>

rh-pre-commit.version: 2.4.0
rh-pre-commit.check-secrets: ENABLED
@sonarqubecloud

sonarqubecloud Bot commented Oct 6, 2026

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants