Skip to content

Commit 7e218a1

Browse files
KSemenenkoKSemenenko
committed
Checkpoint TimeProvider deadline ownership and verification
1 parent 51ce19c commit 7e218a1

6 files changed

Lines changed: 183 additions & 26 deletions

File tree

‎README.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -376,6 +376,11 @@ of this migration remain pending.
376376

377377
For detailed status, see the [implementation tracker](docs/implementation/status.json) and the [qualification records](docs/implementation/). We publish performance numbers only from real GitHub Actions runs, on the [website](https://www.keyload.cloud/).
378378

379+
The [TimeProvider migration](docs/Features/ResourceExecution/TimeProvider.md)
380+
supplies explicit clocks for timestamps, elapsed budgets and managed timers.
381+
Its local Aspire analyzer suite passes 388 tests; controlled real-operation
382+
test sources are joined, while full build and runtime qualification remain open.
383+
379384
Website publication runs independently in CI when source changes. It uses the newest completed benchmark run with a verified aggregate when available; otherwise it publishes the product site without performance figures. A completed benchmark run triggers a fresh website build. [ADR-112](docs/ADR/ADR-112-independent-website-publication.md) records the source, artifact and publication checks; the revised route still needs delivered-source Linux CI and Pages verification.
380385

381386
## FAQ

‎docs/Features/ResourceExecution/TimeProvider.md‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,11 @@
11
# ResourceExecution: controlled time
22

3-
Status: implementation in progress; local development verification is distinct from Linux RF3 qualification.
3+
Status: source joined; full build and runtime qualification remain incomplete. Local development verification is distinct from Linux RF3 qualification.
44
Decision: [ADR-115](../../ADR/ADR-115-time-provider.md).
55

66
## Goal and scope
77

8-
All KeyLoad-owned C# current-time reads and elapsed/deadline/timer work use native .NET `TimeProvider`. Existing DateTime/DateTimeOffset static clocks are already rejected by KLD0022; hidden System-provider calls and Stopwatch still bypass controlled clocks. Use injected clocks in execution owners and select System only in constructor defaults or composition roots. Preserve UTC persisted timestamps, signed-request validity, RF3 ordering, native Orleans scheduling, monotonic budgets and actual benchmark provenance. No new packages or replacement database/transport dependencies. Provider control is explicitly authorized for tests. Runtime-host and business clocks may remain separately composed where their ownership differs.
8+
All KeyLoad-owned C# current-time reads and elapsed/deadline/timer work use native .NET `TimeProvider`. KLD0022 rejects DateTime/DateTimeOffset static clocks, native Stopwatch properties, fields, calls and creation, and Environment tick access. Use injected clocks in execution owners and select System only in constructor defaults or composition roots. Preserve UTC persisted timestamps, signed-request validity, RF3 ordering, native Orleans scheduling, monotonic budgets and actual benchmark provenance. No new packages or replacement database/transport dependencies. Provider control is explicitly authorized for tests. Runtime-host and business clocks may remain separately composed where their ownership differs.
99

1010
## Requirements and acceptance
1111

@@ -28,6 +28,12 @@ All KeyLoad-owned C# current-time reads and elapsed/deadline/timer work use nati
2828

2929
Validation skills in order: quality-ci (check native gates), analyzer-config (existing severity), format (canonical dotnet format), csharpier (N/A: dotnet format owner), code-analysis (Release analyzers), roslynator/meziantou/stylecop (review current package ownership; no new packages), complexity (existing source-owned limits plus changed-code review), crap-score (actual source-bound coverage only). Baseline and final outcomes will be recorded here after checks. A failed build is not completion. No worker commits; lead checkpoints only coherent scope without unrelated work.
3030

31+
## Development evidence, 2026-10-06
32+
33+
The [development receipt](../../implementation/time-provider-development-2026-10-06.json) records actual commands, native report hash and unresolved gates. The complete Aspire-owned analyzer suite passed 388/388 tests, including readonly Stopwatch fields and the controlled migration deadline. Product clock modules, ComparisonHost and the final integration test sources compile with analyzers. The complete solution and canonical formatter remain blocked by parallel BackupRestore/cancellation-flow diagnostics; process recovery stops during prior-source preparation. The local RF3 suite failed (16 passed, 142 failed) on cluster-profile reparse paths and unavailable prior-server proof. These are local macOS development observations, with no Linux qualification, coverage, CRAP or performance claim.
34+
35+
Controlled real-operation sources: `TimeProviderOperationTests` (JSON/native command timestamp and retry, query/due-work expiry, UTC versus monotonic budget), `NativeReadCutClockTests` (real snapshot expiry, released slot and preserved bytes), and `DueWaitControlledClockTests` (real apply waiter fallback/cancellation and joined timers). Fresh UnitTests compilation and execution remain required. The controlled timer supports these single-threaded one-shot workflows; it is not qualified as a general concurrent or periodic timer implementation. Framework-native synchronous process/thread waits without a provider overload remain bounded OS observations; managed delays and timed task waits use provider overloads.
36+
3137
```mermaid
3238
flowchart LR
3339
Composition[System provider or controlled test clock] --> Owner[Execution owner]

‎docs/implementation/status.json‎

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -4,16 +4,18 @@
44
"status": "in_progress",
55
"feature": "docs/Features/ResourceExecution/TimeProvider.md",
66
"decision": "docs/ADR/ADR-115-time-provider.md",
7-
"sourceStage": "clock_migration_and_deterministic_real_operation_regressions_in_progress",
7+
"sourceStage": "clock_migration_and_deterministic_real_operation_regressions_joined",
88
"canonicalBuildPassed": false,
99
"canonicalFormatterPassed": false,
1010
"runtimeQualified": false,
1111
"fullTaskComplete": false,
12-
"verificationStage": "owned_analyzer_release_build_passed_other_gates_pending",
12+
"verificationStage": "Aspire_analyzers_388_passed_shared_build_and_runtime_gates_blocked",
13+
"analyzerTestsPassed": 388,
14+
"localDevelopmentReceipt": "docs/implementation/time-provider-development-2026-10-06.json",
1315
"remaining": [
14-
"Final canonical solution build and format",
15-
"Aspire analyzer, unit, process recovery and Docker RF3 client suites",
16-
"Source-bound functional coverage and Linux qualification"
16+
"Fresh passing complete solution build and canonical formatting",
17+
"Execute controlled engine/read-cut/replica tests through freshly compiled Aspire unit runner",
18+
"Complete unit/scalar/recovery/RF3 and exact-source Linux gates"
1719
]
1820
},
1921
"ORLEANS-PRIMITIVES-001": {

‎docs/implementation/time-provider-development-2026-10-06.json‎

Lines changed: 135 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,135 @@
1+
{
2+
"date": "2026-10-06",
3+
"scope": "TimeProvider local development; not Linux qualification",
4+
"feature": "docs/Features/ResourceExecution/TimeProvider.md",
5+
"decision": "docs/ADR/ADR-115-time-provider.md",
6+
"sourceRevisionObserved": "51ce19cfbb086b316d41cd959e98946d78cc26e3",
7+
"concurrentSourceChangesExcluded": true,
8+
"canonicalBuildPassed": false,
9+
"canonicalFormatterPassed": false,
10+
"runtimeQualified": false,
11+
"fullTaskComplete": false,
12+
"analyzers": {
13+
"command": "dotnet run --project src/KeyLoad.AppHost --no-build --no-restore --configuration Release -- --KeyLoadTests:Suite=analyzers",
14+
"exitCode": 0,
15+
"summary": {
16+
"total": 388,
17+
"passed": 388,
18+
"failed": 0,
19+
"skipped": 0,
20+
"cancelled": 0,
21+
"timedOut": 0,
22+
"flaky": 0
23+
},
24+
"nativeReport": "TestResults/analyzers/KeyLoad.Analyzers.Tests-macos-net10.0.tunit-report.json",
25+
"nativeReportSha256": "b5f02f448bc940fbfb50be86db8c1d26e7ef307c0f0ac70d7ae5114ba539189e",
26+
"publicationGeneration": "15a5a712955e4dd48ba50337feeccc5f",
27+
"operatingSystem": "macOS 27.0.1",
28+
"runtimeVersion": ".NET 10.0.12",
29+
"log": "/Users/ksemenenko/.aspire/logs/cli_20261006T082534_8247a03f.log",
30+
"clockCases": [
31+
{
32+
"className": "AggregateTypeClockTests",
33+
"summary": {
34+
"total": 1,
35+
"passed": 1,
36+
"failed": 0,
37+
"skipped": 0,
38+
"cancelled": 0,
39+
"timedOut": 0,
40+
"flaky": 0
41+
}
42+
},
43+
{
44+
"className": "SystemClockAccessTests",
45+
"summary": {
46+
"total": 3,
47+
"passed": 3,
48+
"failed": 0,
49+
"skipped": 0,
50+
"cancelled": 0,
51+
"timedOut": 0,
52+
"flaky": 0
53+
}
54+
},
55+
{
56+
"className": "SystemClockElapsedTests",
57+
"summary": {
58+
"total": 10,
59+
"passed": 10,
60+
"failed": 0,
61+
"skipped": 0,
62+
"cancelled": 0,
63+
"timedOut": 0,
64+
"flaky": 0
65+
}
66+
}
67+
]
68+
},
69+
"builds": {
70+
"canonical": "failed: 33 parallel BackupRestore artifact diagnostics and one AppHost signal join; join subsequently repaired and scoped AppHost build passed",
71+
"canonicalLog": "/tmp/keyload-time-solution-build-final.log",
72+
"compiledModules": [
73+
"Core",
74+
"Query",
75+
"Security",
76+
"Diagnostics",
77+
"Storage.IO",
78+
"Storage.ZoneTree",
79+
"BenchmarkScenarios",
80+
"Benchmarks",
81+
"Client",
82+
"Comparisons",
83+
"ComparisonHost",
84+
"Replication",
85+
"Orleans",
86+
"ServiceDefaults",
87+
"Server",
88+
"Analyzers.Tests",
89+
"CrashHost",
90+
"RecoveryTests"
91+
],
92+
"scopedAppHost": "passed 0 warnings/errors, current product references; no replacement for solution gate",
93+
"unitSource": "blocked by 8 parallel cancellation-flow and BackupRestore diagnostics; controlled operation cases not executed",
94+
"unitLog": "/tmp/keyload-time-unit-source-final-r2.log",
95+
"integrationSource": "passed 0 warnings/errors after async deadline ownership and disposal join repair",
96+
"integrationLog": "/private/tmp/keyload-time-runtime-integration-source-build.log"
97+
},
98+
"recovery": {
99+
"command": "dotnet run --project src/KeyLoad.AppHost --no-build --no-restore --configuration Release -- --KeyLoadTests:Suite=recovery",
100+
"exitCode": 134,
101+
"status": "blocked before test execution: prepare-native5-probe original exit1, prior-source CA1822 compilation diagnostics",
102+
"log": "/Users/ksemenenko/.aspire/logs/cli_20261006T082313_208b3de5.log"
103+
},
104+
"format": {
105+
"command": "dotnet format KeyLoad.slnx --verify-no-changes --no-restore",
106+
"exitCode": 2,
107+
"status": "parallel BackupRestore and cancellation-flow diagnostics; scoped clock formatting passed",
108+
"log": "/tmp/keyload-time-format-final.log"
109+
},
110+
"governance": "passed 27 projects and 4 modules",
111+
"performance": "unmeasured; no acceleration claim",
112+
"coverageAndCrap": "unmeasured",
113+
"remaining": [
114+
"Fresh passing complete solution build and canonical formatting",
115+
"Execute controlled engine/read-cut/replica tests through freshly compiled Aspire unit runner",
116+
"Complete unit/scalar/recovery/RF3 and exact-source Linux gates"
117+
],
118+
"rf3": {
119+
"command": "dotnet run --project src/KeyLoad.AppHost --no-build --no-restore --configuration Release -- --KeyLoadTests:Suite=rf3",
120+
"exitCode": 2,
121+
"summary": {
122+
"total": 158,
123+
"passed": 16,
124+
"failed": 142,
125+
"skipped": 0,
126+
"cancelled": 0,
127+
"timedOut": 0,
128+
"flaky": 0
129+
},
130+
"nativeReport": "TestResults/rf3/KeyLoad.IntegrationTests-macos-net10.0.tunit-report.json",
131+
"nativeReportSha256": "98fa6f4065dab9fa6fd13af1dd0232305ffcdc5cf1d204f9d16876bd40b9e54d",
132+
"status": "failed local development: cluster profile rejects macOS reparse temp paths; genuine prior-server proof unavailable; no passing Docker RF3 claim",
133+
"log": "/Users/ksemenenko/.aspire/logs/cli_20261006T083253_a37d4221.log"
134+
}
135+
}

‎tests/KeyLoad.IntegrationTests/Features/ClusterRouting/Helpers/RequestCqrsAuthorityFaultLifecycleRunner.cs‎

Lines changed: 16 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -11,8 +11,8 @@ internal static async Task RunAsync(bool officialMcp, CancellationToken cancella
1111
using var parentTimeout = new CancellationTokenSource(RequestCqrsRf3Protocol.ParentDeadline, TimeProvider.System);
1212
using var parent = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken, parentTimeout.Token);
1313
var lifecycle = new RequestCqrsLifecycleEvidence();
14-
var scenario = new RequestCqrsAuthorityFaultScenario(officialMcp, lifecycle, cancellationToken);
15-
await ServerFailureObserver.ObserveAsync(() => scenario.ExecuteObservedAsync(parent.Token),
14+
var scenario = new RequestCqrsAuthorityFaultScenario(officialMcp, lifecycle, TimeProvider.System, cancellationToken);
15+
await ServerFailureObserver.ObserveAsync(() => ExecuteObservedAsync(scenario, lifecycle, parent.Token),
1616
scenario.Failures).ConfigureAwait(false);
1717
if (scenario.Failures.Count == 0)
1818
{
@@ -21,8 +21,21 @@ await ServerFailureObserver.ObserveAsync(
2121
scenario.Failures).ConfigureAwait(false);
2222
}
2323
lifecycle.RecordFirstFailureIfAny(scenario.Failures);
24-
await ServerFailureObserver.ObserveAsync(scenario.CleanupAsync, scenario.Failures).ConfigureAwait(false);
24+
var cleanup = scenario.DisposeAsync().AsTask();
25+
await ServerFailureObserver.ObserveAsync(() => cleanup, scenario.Failures).ConfigureAwait(false);
2526
lifecycle.RecordTerminal();
2627
lifecycle.ThrowWithContext(scenario.Failures);
2728
}
29+
30+
private static async Task ExecuteObservedAsync(RequestCqrsAuthorityFaultScenario scenario,
31+
RequestCqrsLifecycleEvidence lifecycle, CancellationToken parentToken)
32+
{
33+
try
34+
{ await scenario.ExecuteAsync(parentToken).ConfigureAwait(false); }
35+
catch (Exception)
36+
{
37+
lifecycle.RecordFirstFailure();
38+
throw;
39+
}
40+
}
2841
}

‎tests/KeyLoad.IntegrationTests/Features/ClusterRouting/Helpers/RequestCqrsAuthorityFaultScenario.cs‎

Lines changed: 12 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@
99
namespace KeyLoad.IntegrationTests.Features.ClusterRouting;
1010

1111
internal sealed class RequestCqrsAuthorityFaultScenario(bool useMcp, RequestCqrsLifecycleEvidence lifecycle,
12-
CancellationToken callerToken)
12+
TimeProvider clock, CancellationToken callerToken) : IAsyncDisposable
1313
{
1414
private const string MissingOwner = "The authority-fault wave did not retain its owner.";
1515
private const string AdminJson = RequestCqrsAuthorityFaultAssertions.AdministratorJson;
@@ -39,18 +39,7 @@ internal static Task RunAsync(bool officialMcp, CancellationToken cancellationTo
3939

4040
internal List<Exception> Failures { get; } = [];
4141

42-
internal async Task ExecuteObservedAsync(CancellationToken parentToken)
43-
{
44-
try
45-
{ await ExecuteAsync(parentToken).ConfigureAwait(false); }
46-
catch (Exception)
47-
{
48-
lifecycle.RecordFirstFailure();
49-
throw;
50-
}
51-
}
52-
53-
internal async Task CleanupAsync()
42+
public async ValueTask DisposeAsync()
5443
{
5544
try
5645
{
@@ -60,16 +49,23 @@ await RequestCqrsAuthorityFaultCleanup.RunAsync(root, rootCreated, waveStartupAt
6049
}
6150
finally
6251
{
63-
operationTimeout?.Dispose();
52+
try
53+
{
54+
operationDeadline?.Dispose();
55+
}
56+
finally
57+
{
58+
operationTimeout?.Dispose();
59+
}
6460
}
6561
}
6662

67-
private async Task ExecuteAsync(CancellationToken parentToken)
63+
internal async Task ExecuteAsync(CancellationToken parentToken)
6864
{
6965
lifecycle.SetStage(RequestCqrsLifecycleStage.WaveStartup);
7066
root = RequestCqrsAuthorityFaultProvisioning.NewPrivateRootPath();
7167
RequestCqrsAuthorityFaultProvisioning.CreatePrivateRoot(root, () => rootCreated = true);
72-
operationTimeout = new CancellationTokenSource(RequestCqrsRf3Protocol.WaveDeadline, TimeProvider.System);
68+
operationTimeout = new CancellationTokenSource(RequestCqrsRf3Protocol.WaveDeadline, clock);
7369
operationDeadline = CancellationTokenSource.CreateLinkedTokenSource(parentToken, operationTimeout.Token);
7470
lifecycle.SetTokens(callerToken, parentToken, operationDeadline.Token);
7571
await PrepareAsync(operationDeadline.Token).ConfigureAwait(false);

0 commit comments

Comments
 (0)