Skip to content

Commit d316296

Browse files
KSemenenkoKSemenenko
committed
Preserve PR test qualification and record workflow delivery
1 parent 79b7cf2 commit d316296

8 files changed

Lines changed: 292 additions & 11 deletions

File tree

‎.github/workflows/tests.yml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,12 +2,13 @@ name: Tests
22
on:
33
push:
44
branches: [main]
5+
pull_request:
56
workflow_dispatch:
67
permissions:
78
contents: read
89
concurrency:
910
group: keyload-tests-${{ github.ref }}
10-
cancel-in-progress: false
11+
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
1112
jobs:
1213
repository-checks:
1314
name: Repository checks

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -347,7 +347,7 @@ The later `5bbf30f70` [Linux source run37104211481](docs/implementation/isolated
347347

348348
- **CI** (`ci.yml`): PR build, formatting, analyzers and repository rules.
349349
- **Tests** (`tests.yml`): project unit/scalar tests, process recovery and
350-
Docker/Aspire RF3 SDK/MCP checks on main or manual runs.
350+
Docker/Aspire RF3 SDK/MCP checks on main, PR or manual runs.
351351
- **Benchmarks** (`benchmarks.yml`): all performance comparisons and TimeSeries
352352
image checks, with isolated Linux runners and authenticated result artifacts.
353353
- **Release** (`release.yml`): builds NuGet packages from a version tag or the

‎docs/ADR/ADR-062-workflow-separation.md‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ Requirements: REQ-WF-001..006. Acceptance: AC-WF-001..006.
77

88
Use five plainly named pipelines with distinct responsibilities:
99
CI handles pull requests/manual build, format, analyzers and repository rules;
10-
Tests handles main/manual project unit/scalar, process recovery and Docker/Aspire
10+
Tests handles main/PR/manual project unit/scalar, process recovery and Docker/Aspire
1111
RF3 qualification; Benchmarks handles every comparative performance test,
1212
including TimeSeries image checks; Release builds NuGet package artifacts from
1313
version tags/manual source; Website separately qualifies and publishes the site.
@@ -16,6 +16,7 @@ Remove standalone governance and per-feature TimeSeries workflows.
1616
```mermaid
1717
flowchart LR
1818
PR[Pull request] --> CI[CI rules build format analyzers]
19+
PR --> Tests[Tests ordinary unit recovery RF3]
1920
Main[Exact main source] --> Tests[Tests ordinary unit recovery RF3]
2021
Main --> Benchmarks[Benchmarks isolated native cells and image checks]
2122
Tag[Version tag or manual source] --> Release[Release NuGet package artifacts]
@@ -32,7 +33,7 @@ flowchart LR
3233
reviews every diff and owns combined evidence. Workers cannot commit/push.
3334
2. CI retains an independent always-triggered repository-checks job, full Release/
3435
formatter checks and real analyzer regressions. It has PR/manual triggers and
35-
no push trigger. Tests retains all original ordinary main qualification gates,
36+
no push trigger. Tests retains all original ordinary main/PR qualification gates,
3637
including full Release, formatter, rules, analyzers, normal/scalar units,
3738
real-process recovery and genuine RF3 through real .NET/official MCP SDKs.
3839
3. Move every existing native comparison job to benchmarks.yml unchanged except
@@ -42,7 +43,7 @@ flowchart LR
4243
image test/job/artifact there under the same build prerequisite.
4344
4. New isolated contracts bind to exact Benchmarks/benchmarks.yml identity; C#
4445
native/fault/image admission and unit fixtures match. TimeSeries also validates
45-
the exact workflow ref and retains it in its context. Positive/negative
46+
the exact workflow ref without changing the retained receipt shape. Positive/negative
4647
provenance regressions remain; no producer-name fallback is introduced.
4748
5. Historical legacy report producer runs retain KeyLoad CI/ci.yml identity.
4849
Current metadata is explicitly named CI via a separate workflowDisplayName

‎docs/Architecture.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -785,7 +785,7 @@ classDiagram
785785

786786
[ADR-062](ADR/ADR-062-workflow-separation.md) separates PR CI, project Tests, all Benchmarks, Release
787787
package builds and Website publication. `ci.yml` (`CI`) always checks repository rules on PR/manual runs.
788-
`tests.yml` (`Tests`) retains main/manual full build/format/unit/scalar/recovery/
788+
`tests.yml` (`Tests`) retains main/PR/manual full build/format/unit/scalar/recovery/
789789
Docker RF3 gates. `release.yml` (`Release`) builds actual NuGet package artifacts.
790790
`benchmarks.yml` (`Benchmarks`) owns every native comparison and TimeSeries image
791791
check, with its own full build/format/rules prerequisite and isolated Linux cells.

‎docs/implementation/workflow-layout-acceptance.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ public NuGet publication is presumed from the request to build packages.
1414
- AC-WF-001 / REQ-WF-001: exactly ci.yml, tests.yml, benchmarks.yml, release.yml
1515
and pages.yml exist at top level with names CI, Tests, Benchmarks, Release,
1616
Website. CI is PR/manual checks: repository rules, full Release/formatter and
17-
analyzer regression checks. Tests is main/manual project qualification:
17+
analyzer regression checks. Tests is main/PR/manual project qualification:
1818
full Release/format/rules/analyzer, unit/scalar, process recovery and genuine
1919
Docker/Aspire RF3. Repository rules are inside those pipelines, never standalone.
2020
- AC-WF-002 / REQ-WF-002: Benchmarks is main/manual performance qualification;

‎docs/implementation/workflow-layout-execution.md‎

Lines changed: 47 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -31,11 +31,11 @@ Ordered steps and done conditions:
3131
exact native job/step/artifact names. Workers cannot commit or push.
3232
- [x] Run static governance, syntax/whitespace and integrated source review. Preserve
3333
all pre-existing dirty work; stage only task-owned hunks/files.
34-
- [ ] Commit and push the stable layout milestone on current main. Inspect actual
34+
- [x] Commit and push the stable layout milestone on current main. Inspect actual
3535
push-triggered Tests/Benchmarks, manual CI/Release and Website exact-SHA jobs. No duplicate full dispatch.
36-
- [ ] Download changed-job artifacts, record actual SHA/run/job URLs and gate status;
36+
- [x] Download changed-job artifacts, record actual SHA/run/job URLs and gate status;
3737
fix layout/provenance failures and repeat relevant GitHub checks if required.
38-
- [ ] Close all AC or state exact pre-existing/external blockers without claiming
38+
- [x] Close all AC or state exact pre-existing/external blockers without claiming
3939
failed/skipped/incomplete measurement, readiness or publication success.
4040

4141
Baseline failures: main9b3bd8ed64b42e7e3bd37d454face7984dfc6b5f CI37111280400 and
@@ -56,7 +56,7 @@ Existing incomplete product qualifications are not silently included as new repa
5656

5757
## Final owner clarification and task graph extension
5858

59-
- [x] TASK-WF-ROOT separates CI PR checks from Tests main/manual project suites,
59+
- [x] TASK-WF-ROOT separates CI PR checks from Tests main/PR/manual project suites,
6060
names Benchmarks/Website plainly, and adds Release tag/manual package builds.
6161
- [x] TASK-WF-CS updates the bounded C# fixtures and minimal source-contract
6262
regressions to all five final workflow roles/names; no runtime behavior changes.
@@ -84,3 +84,46 @@ HEAD archive plus only this task's source changes passes the existing validator:
8484
The sandboxed development build returned exit1 with0 warnings/0 errors after
8585
5min and is not a passing build result. Required compilation/runtime evidence
8686
comes from the exact delivered GitHub workflows. No local tests were run.
87+
88+
## Delivered GitHub milestone
89+
90+
Commit `9e0532fdeed7c55a17f9c857f8d5a264215341e7` is pushed to main.
91+
The authenticated workflow inventory contains exactly the five active names
92+
CI, Tests, Benchmarks, Release and Website. CI run
93+
[37113420787](https://github.com/managedcode/KeyLoad/actions/runs/37113420787)
94+
passes repository rules, the complete Release/format build and analyzer tests.
95+
Release run
96+
[37113424709](https://github.com/managedcode/KeyLoad/actions/runs/37113424709)
97+
passes and retains nine actual `0.1.0-dev` NuGet packages. Downloaded package
98+
versions, sizes and SHA256 values agree with its source/run-bound `packages.json`.
99+
Artifact `packages-0.1.0-dev` has authenticated id `11271266373`.
100+
The [qualification receipt](workflow-layout-qualification.json) retains the exact
101+
run/job/artifact identities, verified package manifest and real image test summary.
102+
103+
Benchmarks run
104+
[37113337516](https://github.com/managedcode/KeyLoad/actions/runs/37113337516)
105+
passes its complete build/format/rules prerequisite and the moved real pinned
106+
TimeSeries image job `111176089078`. The unchanged native resource-model suite
107+
then fails four Timescale image-tag assertions (expected `2.30.2-pg18`, actual
108+
empty tag). The incomplete cohort cannot refresh website metrics.
109+
110+
Tests run
111+
[37113337508](https://github.com/managedcode/KeyLoad/actions/runs/37113337508)
112+
passes complete build/format/rules and 118 analyzer cases. All four new
113+
workflow-layout regression cases pass, as do all 183 process-recovery cases.
114+
The ordinary unit suite passes 1856/1857; its sole failure is the same pre-task
115+
SDK-status assertion listed in the baseline above. RF3 passes 62/63 and reports
116+
one existing SQL admission assertion
117+
`AcAisql007InsufficientDataBytesRejectSdkAndOfficialSqlBeforeIdClaimWhileDirectControlKeepsRf3Healthy`.
118+
The moved RF3 job definition is unchanged. Downloaded genuine TUnit reports bind
119+
these counts and focused assertions to the exact milestone source SHA.
120+
121+
Website run
122+
[37113337499](https://github.com/managedcode/KeyLoad/actions/runs/37113337499)
123+
fails authenticated comparison selection and missing coverage inputs, matching
124+
the pre-task publication blocker. The complete isolated archive needed by the
125+
SiteTests before-session setup remains unavailable, so a new source/provenance
126+
test is compiled but cannot be counted as runtime-qualified. Publication gates
127+
remain intact. These failures do not establish successful database, benchmark
128+
or website qualification. The follow-up preserves the original PR trigger and
129+
PR cancellation in the separate Tests workflow.

‎docs/implementation/workflow-layout-qualification.json‎

Lines changed: 235 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,235 @@
1+
{
2+
"schemaVersion": 1,
3+
"scope": "Workflow roles and exact producer identities; not complete database or performance qualification",
4+
"sourceRevision": "9e0532fdeed7c55a17f9c857f8d5a264215341e7",
5+
"delivery": "pushed-to-main",
6+
"activeWorkflows": [
7+
{
8+
"name": "CI",
9+
"path": ".github/workflows/ci.yml",
10+
"id": 372183043
11+
},
12+
{
13+
"name": "Tests",
14+
"path": ".github/workflows/tests.yml",
15+
"id": 373808966
16+
},
17+
{
18+
"name": "Benchmarks",
19+
"path": ".github/workflows/benchmarks.yml",
20+
"id": 373808964
21+
},
22+
{
23+
"name": "Release",
24+
"path": ".github/workflows/release.yml",
25+
"id": 373808965
26+
},
27+
{
28+
"name": "Website",
29+
"path": ".github/workflows/pages.yml",
30+
"id": 372547499
31+
}
32+
],
33+
"ci": {
34+
"runId": 37113420787,
35+
"attempt": 1,
36+
"url": "https://github.com/managedcode/KeyLoad/actions/runs/37113420787",
37+
"conclusion": "success",
38+
"jobIds": {
39+
"analyzers": 111175550784,
40+
"repositoryChecks": 111175550969,
41+
"completeBuildAndFormat": 111175579013
42+
}
43+
},
44+
"release": {
45+
"runId": 37113424709,
46+
"attempt": 1,
47+
"url": "https://github.com/managedcode/KeyLoad/actions/runs/37113424709",
48+
"conclusion": "success",
49+
"jobId": 111175559567,
50+
"artifact": {
51+
"id": 11271266373,
52+
"name": "packages-0.1.0-dev",
53+
"digest": "sha256:c095c1a378ebabfc88c7419795d969983315cb9e9a3bb027b8da7517ee0cdbcf"
54+
},
55+
"downloadVerification": "All nine files match manifest SHA256/size and embedded nuspec version",
56+
"manifest": {
57+
"sourceRevision": "9e0532fdeed7c55a17f9c857f8d5a264215341e7",
58+
"runId": "37113424709",
59+
"runAttempt": "1",
60+
"version": "0.1.0-dev",
61+
"packages": [
62+
{
63+
"file": "KeyLoad.Abstractions.0.1.0-dev.nupkg",
64+
"bytes": 299678,
65+
"sha256": "acd0988eb239f8e9749f4709b5fafcef6c9fe1431a88de2851281bfbebf8cdc1"
66+
},
67+
{
68+
"file": "KeyLoad.Artifacts.0.1.0-dev.nupkg",
69+
"bytes": 11113,
70+
"sha256": "20e09d3cc364c7451dd6100273820f193a9a768ca0210d6946e982ed260f3f4c"
71+
},
72+
{
73+
"file": "KeyLoad.Client.0.1.0-dev.nupkg",
74+
"bytes": 23808,
75+
"sha256": "48ae3f7e1d2d664c52825bcbe35277e64e01091f9c1ca398b45e335f158e65d9"
76+
},
77+
{
78+
"file": "KeyLoad.Core.0.1.0-dev.nupkg",
79+
"bytes": 164947,
80+
"sha256": "9e288123a2c2429848d2635535685c20b1bb1b79aa19202d829d7858d827a7e5"
81+
},
82+
{
83+
"file": "KeyLoad.Orleans.0.1.0-dev.nupkg",
84+
"bytes": 112358,
85+
"sha256": "5aec3e75fc2609d00deb68548bfdb4a549e4f97e3481a7c49867f6348e83d632"
86+
},
87+
{
88+
"file": "KeyLoad.Query.0.1.0-dev.nupkg",
89+
"bytes": 42808,
90+
"sha256": "dfbfbc759eb35403fa057ea37cbfdfcf2d4c3402c63ef14a5b03475bac32f9cf"
91+
},
92+
{
93+
"file": "KeyLoad.Replication.0.1.0-dev.nupkg",
94+
"bytes": 80706,
95+
"sha256": "8bf7d003ec629551a3ad2c080d12dcf1e4b3d11b4cf53def2dfc4cc0b90e66c5"
96+
},
97+
{
98+
"file": "KeyLoad.Security.0.1.0-dev.nupkg",
99+
"bytes": 8058,
100+
"sha256": "52b1ae571c15400ec69810690fdfe1d7e78fa49f997e9c2f764aafac58531563"
101+
},
102+
{
103+
"file": "KeyLoad.Storage.ZoneTree.0.1.0-dev.nupkg",
104+
"bytes": 59517,
105+
"sha256": "91d09b17d98f659daa0b966579d75b9f1b0f7b871761e6c8ec212b26d5a20470"
106+
}
107+
]
108+
},
109+
"feedPublication": "not-requested-or-performed"
110+
},
111+
"benchmarks": {
112+
"runId": 37113337516,
113+
"attempt": 1,
114+
"url": "https://github.com/managedcode/KeyLoad/actions/runs/37113337516",
115+
"conclusion": "failure",
116+
"completeBuildAndFormatJob": {
117+
"id": 111175309107,
118+
"conclusion": "success"
119+
},
120+
"pinnedImage": {
121+
"jobId": 111176089078,
122+
"conclusion": "success",
123+
"artifactId": 11270701273,
124+
"artifactDigest": "sha256:266bb2f735969a70e02a1f60a2d020c9422ccef32fd480dd65c4a2899be6e20e",
125+
"factsSha256": "8e12eb3c53baa95c36f3188e3a52c254730256f1cf463b4f254867c7883e38b3",
126+
"context": {
127+
"sourceRevision": "9e0532fdeed7c55a17f9c857f8d5a264215341e7",
128+
"runId": 37113337516,
129+
"attempt": 1,
130+
"repository": "managedcode/KeyLoad",
131+
"ref": "refs/heads/main",
132+
"event": "push",
133+
"workflow": "Benchmarks",
134+
"job": "timeseries-image-facts",
135+
"runnerOs": "Linux",
136+
"hostOs": "Ubuntu 24.04.5 LTS",
137+
"hostArchitecture": "X64",
138+
"observedAt": "2026-10-03T09:38:22.252593+00:00",
139+
"runUrl": "https://github.com/managedcode/KeyLoad/actions/runs/37113337516"
140+
},
141+
"imageReference": "docker.io/timescale/timescaledb:2.30.2-pg18@sha256:e72689191e1c977892c53d6f2c344dbc4a9657a867dc8cc1899229f9d3672b2e",
142+
"cleanupCompleted": true,
143+
"tunitSummary": {
144+
"total": 1,
145+
"passed": 1,
146+
"failed": 0,
147+
"skipped": 0,
148+
"cancelled": 0,
149+
"timedOut": 0,
150+
"flaky": 0
151+
}
152+
},
153+
"nativeModels": {
154+
"jobId": 111176089115,
155+
"failedCases": 4,
156+
"reason": "Unchanged Timescale resource-model assertions expect tag 2.30.2-pg18; actual image annotation has empty tag."
157+
},
158+
"completeCohort": "unavailable; measured target jobs and aggregate skipped"
159+
},
160+
"tests": {
161+
"runId": 37113337508,
162+
"attempt": 1,
163+
"url": "https://github.com/managedcode/KeyLoad/actions/runs/37113337508",
164+
"statusAtInspection": "completed",
165+
"ordinaryUnitStep": "failure",
166+
"scalarStep": "skipped",
167+
"recoveryStep": "success",
168+
"rf3": {
169+
"jobId": 111175308931,
170+
"conclusion": "failure",
171+
"failedCases": 1,
172+
"case": "AcAisql007InsufficientDataBytesRejectSdkAndOfficialSqlBeforeIdClaimWhileDirectControlKeepsRf3Healthy",
173+
"totalCases": 63,
174+
"passedCases": 62
175+
},
176+
"fullQualification": "not-passing",
177+
"conclusion": "failure",
178+
"unitSummary": {
179+
"total": 1857,
180+
"passed": 1856,
181+
"failed": 1,
182+
"skipped": 0,
183+
"cancelled": 0,
184+
"timedOut": 0,
185+
"flaky": 0
186+
},
187+
"unitFailure": "AcTsi006UnknownProblemNamesAndUnavailableStatusStayUnknown; identical pre-task baseline assertion",
188+
"workflowLayoutCases": [
189+
{
190+
"class": "WorkflowLayoutCiSourceTests",
191+
"case": "AcWf001OnlyFiveWorkflowsRemainWithSeparateCiAndTestGates",
192+
"status": "passed"
193+
},
194+
{
195+
"class": "WorkflowLayoutPipelineSourceTests",
196+
"case": "AcWf005ReleaseBuildsAndRetainsNuGetPackagesWithoutPublishing",
197+
"status": "passed"
198+
},
199+
{
200+
"class": "WorkflowLayoutPipelineSourceTests",
201+
"case": "AcWf002BenchmarksRetainsTheIsolatedAndPinnedImageGraph",
202+
"status": "passed"
203+
},
204+
{
205+
"class": "WorkflowLayoutPipelineSourceTests",
206+
"case": "AcWf004WebsiteFollowsBenchmarks",
207+
"status": "passed"
208+
}
209+
],
210+
"recoverySummary": {
211+
"total": 183,
212+
"passed": 183,
213+
"failed": 0,
214+
"skipped": 0,
215+
"cancelled": 0,
216+
"timedOut": 0,
217+
"flaky": 0
218+
},
219+
"artifactName": "test-results-ubuntu-latest",
220+
"tunitReportSha256": {
221+
"KeyLoad.UnitTests-linux-net10.0.tunit-report.json": "06c3f05698f86c2dc944351eb89a014a92331a757e4b662ee84668c31c7c42e4",
222+
"KeyLoad.RecoveryTests-linux-net10.0.tunit-report.json": "27f808bd6ad4acc59974f587ed7a08f5f5637f674f53674d97829b3983eaf221"
223+
}
224+
},
225+
"website": {
226+
"runId": 37113337499,
227+
"attempt": 1,
228+
"url": "https://github.com/managedcode/KeyLoad/actions/runs/37113337499",
229+
"conclusion": "failure",
230+
"qualificationJobId": 111175308778,
231+
"reason": "No complete authenticated isolated cohort and native coverage inputs; publication skipped. Same publication blocker before task.",
232+
"newSiteRegression": "compiled by full solution build; runtime blocked by mandatory before-session authentic complete-cohort inputs"
233+
},
234+
"followUp": "Preserve original PR test trigger and cancellation in separate Tests; evidence above is bound only to its stated milestone SHA."
235+
}

‎tests/KeyLoad.UnitTests/Features/RepositoryGovernance/WorkflowLayoutCiSourceTests.cs‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,7 @@ await Assert.That(WorkflowLayoutSource.JobBlock(ci, "analyzer-rules")
4040
var tests = WorkflowLayoutSource.Read(TestsFile);
4141
await Assert.That(tests.StartsWith("name: Tests\n", StringComparison.Ordinal)).IsTrue();
4242
var testEvents = WorkflowLayoutSource.EventBlock(tests);
43+
await Assert.That(testEvents.Contains("pull_request:", StringComparison.Ordinal)).IsTrue();
4344
await Assert.That(testEvents.Contains("workflow_dispatch:", StringComparison.Ordinal)).IsTrue();
4445
await Assert.That(testEvents.Contains("branches: [main]", StringComparison.Ordinal)).IsTrue();
4546
var testJobs = WorkflowLayoutSource.JobIds(tests);

0 commit comments

Comments
 (0)