Skip to content

test: wait for uploaded report discovery - #11

Merged
neilime merged 1 commit into
mainfrom
docs/release-1.0.0
Nov 4, 2026
Merged

neilime merged 1 commit into
mainfrom
docs/release-1.0.0

Conversation

@neilime

@neilime neilime commented Nov 4, 2026 •

Copy link
Copy Markdown
Member

Hosted report assertions can run before GitHub's artifact list exposes a successfully uploaded report. Wait up to one minute for discovery, then keep the existing checks for missing, empty, expired or unsafe reports. This fixes the observed race without changing the published actions.

Also regenerate the five action READMEs through make docs after publishing 1.0.0 so their usage examples pin 89e3bf93adc94b592188e714f61d4abc391aab46. No generated files were hand-edited.

Validation: all 43 action unit tests, full repository lint and documentation generation pass. A focused local probe verifies immediate visibility, delayed visibility and bounded failure when reports stay missing. Hosted CI passes, including real artifact discovery, native reports and VM lifecycle tests on both backends.

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown

Hi, thank you for creating your PR, we will check it out very soon

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown

Ansible tests - playbook - failure

Ansible tests - playbook - failure

Test Results

Status Count
✅ Passed 1
❌ Failed 1
⏭️ Skipped 0
Total 2

❌ Pass Rate: 50.0%

Failed Tests
  • [fixture] Exercise a synthetic Ansible test assertion: Assert the selected fixture succeeds (suite: valid)
    • File: tests/fixture/ansible/valid.yml:8
    • Error: Task failed: Action failed: Assertion failed

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown

Ansible tests - playbook - success

Ansible tests - playbook - success

Test Results

Status Count
✅ Passed 2
❌ Failed 0
⏭️ Skipped 0
Total 2

✅ Pass Rate: 100.0%

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown

Molecule local scenario - success

Molecule local scenario - success

Test Results

Status Count
✅ Passed 7
❌ Failed 0
⏭️ Skipped 1
Total 8

✅ Pass Rate: 87.5%

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown

Molecule local scenario - failure

Molecule local scenario - failure

Test Results

Status Count
✅ Passed 4
❌ Failed 2
⏭️ Skipped 1
Total 7

❌ Pass Rate: 57.1%

Failed Tests
  • [local-fixture] Verify the local test resource: Assert configuration and selected outcome (suite: assertions)
    • File: tests/fixture/molecule/local/assertions.yml:14
    • Error: Task failed: Action failed: Assertion failed
  • scenario (suite: molecule)
    • Error: Molecule failed, timed out or produced no native assertions

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown

Ansible tests - sanity - failure

Ansible tests - sanity - failure

Test Results

Status Count
✅ Passed 6
❌ Failed 1
⏭️ Skipped 27
Total 34

❌ Pass Rate: 17.6%

Failed Tests
  • yamllint (suite: ansible-test)
    • File: sanity
    • Error: The test `ansible-test sanity --test yamllint` failed with 2 errors:

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown

Ansible tests - sanity - success

Ansible tests - sanity - success

Test Results

Status Count
✅ Passed 7
❌ Failed 0
⏭️ Skipped 27
Total 34

✅ Pass Rate: 20.6%

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown

Molecule local scenario - timeout

Molecule local scenario - timeout

Test Results

Status Count
✅ Passed 3
❌ Failed 1
⏭️ Skipped 0
Total 4

❌ Pass Rate: 75.0%

Failed Tests
  • scenario (suite: molecule)
    • Error: Molecule failed, timed out or produced no native assertions

@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Super-linter summary

Language Validation result
BIOME_FORMAT Pass ✅
BIOME_LINT Pass ✅
CHECKOV Pass ✅
GITLEAKS Pass ✅
GIT_COMMITLINT Pass ✅
GIT_MERGE_CONFLICT_MARKERS Pass ✅
JSCPD Pass ✅
MARKDOWN Pass ✅
MARKDOWN_PRETTIER Pass ✅
NATURAL_LANGUAGE Pass ✅
PRE_COMMIT Pass ✅
SPELL_CODESPELL Pass ✅
TRIVY Pass ✅

All files and directories linted successfully

For more information, see the GitHub Actions workflow run

Powered by Super-linter
Super-linter revision: 2da136927bd4a73596db63044b504547c62cb854
Super-linter version: 9.0.0

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown

Goss - qemu - valid

Goss - qemu - valid

Test Results

Status Count
✅ Passed 7
❌ Failed 0
⏭️ Skipped 0
Total 7

✅ Pass Rate: 100.0%

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown

Goss - qemu - invalid

Goss - qemu - invalid

Test Results

Status Count
✅ Passed 0
❌ Failed 2
⏭️ Skipped 0
Total 2

❌ Pass Rate: 0.0%

Failed Tests
  • goss-execution (suite: disposable-vm)
    • Error: Guest assertions or report collection failed; inspect test outputs
  • File /iac-missing-synthetic-file exists (suite: goss)
    • Error: Test failed

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown

Goss - qemu - empty

Goss - qemu - empty

Test Results

Status Count
✅ Passed 0
❌ Failed 1
⏭️ Skipped 0
Total 1

❌ Pass Rate: 0.0%

Failed Tests
  • goss-execution (suite: disposable-vm)
    • Error: Guest assertions or report collection failed; inspect test outputs

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown

Molecule - success

Molecule - success

Test Results

Status Count
✅ Passed 10
❌ Failed 0
⏭️ Skipped 0
Total 10

✅ Pass Rate: 100.0%

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown

Molecule - assertion-failure

Molecule - assertion-failure

Test Results

Status Count
✅ Passed 3
❌ Failed 2
⏭️ Skipped 0
Total 5

❌ Pass Rate: 60.0%

Failed Tests
  • scenario (suite: molecule)
    • Error: Molecule failed, timed out or produced no native assertions
  • [smoke] Verify the disposable test configuration: Assert the expected content (suite: verify)
    • File: tests/fixture/molecule/smoke/verify.yml:10
    • Error: Task failed: Action failed: Assertion failed

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown

Molecule - idempotence-failure

Molecule - idempotence-failure

Test Results

Status Count
✅ Passed 5
❌ Failed 1
⏭️ Skipped 0
Total 6

❌ Pass Rate: 83.3%

Failed Tests
  • scenario (suite: molecule)
    • Error: Molecule failed, timed out or produced no native assertions

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown

Goss - lima - valid

Goss - lima - valid

Test Results

Status Count
✅ Passed 7
❌ Failed 0
⏭️ Skipped 0
Total 7

✅ Pass Rate: 100.0%

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown

Goss - qemu - timeout

Goss - qemu - timeout

Test Results

Status Count
✅ Passed 0
❌ Failed 1
⏭️ Skipped 0
Total 1

❌ Pass Rate: 0.0%

Failed Tests
  • goss-execution (suite: disposable-vm)
    • Error: Guest assertions or report collection failed; inspect test outputs

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown

Goss - lima - invalid

Goss - lima - invalid

Test Results

Status Count
✅ Passed 0
❌ Failed 2
⏭️ Skipped 0
Total 2

❌ Pass Rate: 0.0%

Failed Tests
  • goss-execution (suite: disposable-vm)
    • Error: Guest assertions or report collection failed; inspect test outputs
  • File /iac-missing-synthetic-file exists (suite: goss)
    • Error: Test failed

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown

Goss - lima - empty

Goss - lima - empty

Test Results

Status Count
✅ Passed 0
❌ Failed 1
⏭️ Skipped 0
Total 1

❌ Pass Rate: 0.0%

Failed Tests
  • goss-execution (suite: disposable-vm)
    • Error: Guest assertions or report collection failed; inspect test outputs

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown

Goss - lima - timeout

Goss - lima - timeout

Test Results

Status Count
✅ Passed 0
❌ Failed 1
⏭️ Skipped 0
Total 1

❌ Pass Rate: 0.0%

Failed Tests
  • goss-execution (suite: disposable-vm)
    • Error: Guest assertions or report collection failed; inspect test outputs

@neilime
neilime force-pushed the docs/release-1.0.0 branch from 0a39023 to 97ef8ee Compare November 4, 2026 18:26
@neilime neilime changed the title docs: generate usage examples for release 1.0.0 test: wait for uploaded report discovery Nov 4, 2026
@neilime
neilime merged commit 5387e63 into main Nov 4, 2026
39 of 40 checks passed
@neilime
neilime deleted the docs/release-1.0.0 branch November 4, 2026 20:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant