Skip to content

Commit 3985008

Browse files
committed
Implement resource policy CAS and integrate search and messaging acceptance fixtures
1 parent 58567d3 commit 3985008

160 files changed

Lines changed: 6714 additions & 340 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/ci.yml‎

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -105,6 +105,8 @@ jobs:
105105
run: dotnet run --project src/KeyLoad.AppHost --no-build --no-restore --configuration Release -- --KeyLoadTests:Suite=analyzers
106106
- name: Test prompt termination on Aspire resource failures
107107
run: dotnet run --project src/KeyLoad.AppHost --no-build --no-restore --configuration Release -- --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/AspireFailure*/*' --KeyLoadTests:ResultsDirectory=TestResults/aspire-failure --KeyLoadTests:ReportTrx=true
108+
- name: Test Aspire recovery prerequisite ownership
109+
run: dotnet run --project src/KeyLoad.AppHost --no-build --no-restore --configuration Release -- --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/PriorProbePrerequisiteTests/*' --KeyLoadTests:ResultsDirectory=TestResults/recovery-prerequisites --KeyLoadTests:ReportTrx=true
108110
- name: Test live benchmark progress capture
109111
run: dotnet run --project src/KeyLoad.AppHost --no-build --no-restore --configuration Release -- --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/ComparisonProgress*/*' --KeyLoadTests:ResultsDirectory=TestResults/live-benchmark-progress --KeyLoadTests:ReportTrx=true
110112
- name: Test live benchmark process entry
@@ -117,11 +119,8 @@ jobs:
117119
run: dotnet run --project src/KeyLoad.AppHost --no-build --no-restore --configuration Release -- --KeyLoadTests:Suite=unit
118120
- name: Run unit tests without CPU intrinsics
119121
run: dotnet run --project src/KeyLoad.AppHost --no-build --no-restore --configuration Release -- --KeyLoadTests:Suite=unit-scalar
120-
- name: Build the previous storage executable
121-
id: native5
122-
run: bash scripts/Features/StorageRecovery/build-native5-probe.sh
123122
- name: Test recovery after process crashes
124-
if: ${{ !cancelled() && steps.build.outcome == 'success' && steps.native5.outcome == 'success' }}
123+
if: ${{ !cancelled() && steps.build.outcome == 'success' }}
125124
run: dotnet run --project src/KeyLoad.AppHost --no-build --no-restore --configuration Release -- --KeyLoadTests:Suite=recovery
126125
- name: Save test results
127126
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
@@ -131,7 +130,6 @@ jobs:
131130
path: |
132131
**/TestResults/**
133132
artifacts/qualification/**
134-
artifacts/native5-probe/**
135133
if-no-files-found: ignore
136134
- name: Save build diagnostics
137135
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1

‎docs/ADR/ADR-089-event-projection-lineage.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,9 @@ REQ/AC-LINEAGE-001–005, original KL-097 and later KL-039; ADR-015/019/022/025/
1818
030/078/081/082. Ordered stages, exact worker/root scope, tests, baseline,
1919
migration/rollback, native data and public/RF3 integration are frozen there.
2020
This ADR becomes Implemented only after all required artifacts and evidence.
21+
The new private effect identity uses the stable integer `DistanceMetric` value
22+
as a key-codec-v1 scalar. Apply, eligibility and deletion use the identical key;
23+
the existing key codec and prior persisted records do not change.
2124

2225
```mermaid
2326
flowchart LR

‎docs/ADR/ADR-090-graph-search.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,8 @@ intermediate JSON or edge payloads, use an independent read gate, cross partitio
1717
silently truncate a graph, or substitute ANN for an exact branch. A shared budget
1818
charges all roots/operators; shortest-hop identity deduplication prevents cycles
1919
and multiple paths from multiplying contributions.
20+
Label filtering requires the persisted graph field policy at canonical `/label`,
21+
including empty label filters; a plain `label` name cannot bypass that policy.
2022

2123
Implementation contract:
2224

@@ -43,6 +45,13 @@ Implementation contract:
4345
lifecycle_wave owns the bounded QueryExecution parser, manifest profile and
4446
mapped real-store lowering/parity tests. Scalar QueryPage stays unchanged.
4547

48+
6. G3 strengthens the existing same-partition KL-056 path using deliberately
49+
conflicting text/vector/graph branch ranks, a separately computed weighted
50+
RRF oracle, current authorization, and actual direct/SQL .NET/MCP RF3 parity.
51+
The exact frozen corpus/test ownership and private patch join are in
52+
GraphRetrieval's G3 section. This adds qualification for existing contracts,
53+
not a second executor or global distributed rank claim.
54+
4655
No persisted data converter, automatic store migration, dependency change or
4756
physical placement change is involved. Nodes lacking the application capability
4857
cannot accept this request. Rollback removes endpoint admission and fails
Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
# ADR-093: Compare-and-set resource field policies
2+
3+
Status: Accepted for implementation; qualification pending.
4+
5+
Vector projection lineage and graph label authorization require current resource
6+
classifications, but ConfigureResource previously rejected every changed
7+
definition. A raw-store rewrite in a test would conceal this missing public flow.
8+
9+
Decision: extend the existing authorized canonical ConfigureResource request with
10+
an optional expected schema version and admit only field/header policy changes,
11+
with an exact version increment. Physical shape, indexes, authority, quotas,
12+
paused state and typed schema remain immutable through this request. Full
13+
resource migrations remain explicit separate work. Blind replacement and a
14+
second policy store are rejected.
15+
16+
Implementation contract: [ResourcePolicyUpdates](../Features/Authorization/ResourcePolicyUpdates.md),
17+
REQ/AC-RPOL-001–004 and AC-LINEAGE-004. Its ordered stages, exact root/Luna file
18+
ownership, native ID3 compatibility, current-version fences, tests, rollback,
19+
baseline and join conditions are mandatory. Persist through the existing RF3
20+
metadata apply gate and deduplicated outcome; no new dependency or data epoch.
21+
This ADR becomes Implemented only with all mapped implementation and evidence.
22+
23+
```mermaid
24+
flowchart LR
25+
Request[Existing signed request] --> Authority[Persisted administrator]
26+
Authority --> CAS[Expected resource version]
27+
CAS --> Shape[Only policy arrays may change]
28+
Shape --> Commit[Canonical RF3 metadata and outcome]
29+
Commit --> Reads[Current policies and version fences]
30+
```
Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
# ADR-094: Orleans coordination of canonical recurring and saga due work
2+
3+
Status: Accepted for S2 implementation; qualification pending.
4+
Date:2026-10-04. Integration owner:root. Coding owner:Luna cluster_wave.
5+
6+
KL-100's S1 commands persist schedules and Waiting sagas but have no autonomous
7+
runtime. Adopt the exact bounded discovery, native service/coordinator/request
8+
chain, retry and lifecycle contract in
9+
[DueCoordination](../Features/Messaging/DueCoordination.md), REQ/AC-DUE-001..004
10+
and REQ/AC-JOBS-005. Existing canonical records and RF3 atomic transitions remain
11+
the authority; a disposable fixed-tail sweep avoids a new durable due-index
12+
format and prevents continuously arriving keys from extending a sweep forever.
13+
14+
Implementation contract:
15+
16+
1. Root freezes this specification and narrowly rehomes the existing native
17+
identity scope/admission to Orleans ClusterRouting/Identity, preserving exact
18+
claims, limits, RequestContext restoration and stable serializer identities.
19+
2. cluster_wave owns NEW Core Features/Messaging Queries/Models/Validation and
20+
Orleans Features/Messaging GrainServices/Grains/Contracts/Execution, plus new
21+
Messaging UnitTests Cases/Helpers/Assertions/Models. Native records retain
22+
unchanged aliases/IDs. New transferred hint DTOs have explicit v1 aliases and
23+
stable generated IDs; no persisted or public S1 discriminator changes.
24+
3. Root alone adds Core internal visibility, Server silo DI, AddGrainService and
25+
native Graph transitions. All writes use a separate signed IRequestGrain and
26+
the existing bounded ManagedCode Communication CQRS consumer. No direct Apply,
27+
synthetic privileged scheduler principal, copied dispatcher or hosted timer
28+
outside Orleans is allowed.
29+
4. Tests cover finite sweep/cursor/byte/deadline/corruption/cancellation bounds,
30+
current creator authority and stable unknown-response retry. Actual Aspire
31+
RF3 SDK/official MCP tests must observe autonomous emission/expiry, duplicates,
32+
leader loss/restart, revocation, cancellation and healthy following calls.
33+
Existing S1 process cut tests remain mandatory and prove only process durability.
34+
5. Root integrates, reviews, builds/formats/governance-checks, runs Aspire suites,
35+
commits each completed stage and retains exact-source Linux original artifacts
36+
before full KL-100 closure. Scoped tests and code presence do not close S2.
37+
38+
Dependencies:ADR-092, existing queue admission, native request/identity boundaries,
39+
epoch7 reader admission, original quorum/leader/read-cut contracts. Rollout uses
40+
homogeneous capable readers; rollback stops due coordination while retaining all
41+
watermarks, outcomes, queues and saga state. No automatic data migration occurs.
42+
Failed pages/jobs remain observable with safe structured errors; no raw retained
43+
templates, credentials or subject/entity identities are logged.
44+
45+
```mermaid
46+
sequenceDiagram
47+
participant Service as Leader silo grain service
48+
participant Coordinator as Partition coordinator grain
49+
participant Request as Fresh request grain
50+
participant Host as Node-local RF3 apply owner
51+
Service->>Coordinator: Finite canonical due hint
52+
Coordinator->>Request: Subject-only scoped identity and signed Batch
53+
Request->>Host: Reload policy, ordered logged-time commit
54+
Host-->>Coordinator: Native CQRS terminal outcome
55+
Coordinator-->>Service: Joined settlement or explicit failure
56+
```

‎docs/Architecture.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -117,6 +117,12 @@ Task replies; the stream RPC/transport and durable-operation contracts remain
117117
pending. Native enumeration does not transfer node-owned storage or supply
118118
persisted authority, retry receipts or durability.
119119

120+
[ResourcePolicyUpdates](Features/Authorization/ResourcePolicyUpdates.md) and
121+
[ADR-093](ADR/ADR-093-resource-policy-updates.md) freeze policy-only CAS through
122+
the existing RF3 metadata ConfigureResource path. Advancing its resource version
123+
invalidates existing read/index fences; physical schema and placement remain
124+
separate migrations. Implementation and public qualification are in progress.
125+
120126
[ADR-052](ADR/ADR-052-timeseries-bounded-aggregates.md) accepts the additive
121127
[TimeSeries](Features/TimeSeries.md) latest/aggregate/window slice. New typed
122128
Abstractions DTOs mirror Client/Server/Core TimeSeries owners and real matching
Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
# ResourcePolicyUpdates within Authorization
2+
3+
Accepted source contract: [ADR-093](../../ADR/ADR-093-resource-policy-updates.md).
4+
This closes the public metadata update gap exposed by AC-LINEAGE-004 without
5+
performing a physical resource, index, authority, quota or schema migration.
6+
Canonical feature ownership is Authorization. Existing ConfigureResource
7+
admission, persisted administrative authority, metadata RF3 and native outcomes
8+
remain the sole write path.
9+
10+
`ConfigureResourceRequest` gains nullable `ExpectedSchemaVersion` at generated
11+
Orleans field ID3, omitted from public JSON when null. Null preserves create and
12+
identical-definition behavior and still rejects a changed existing definition.
13+
A non-null value is a compare-and-set on an existing resource. The replacement
14+
must increment SchemaVersion exactly once and may change only FieldPolicies and
15+
HeaderPolicies. All other definition properties must be canonically identical.
16+
No policy change may bypass the existing complete resource/path/budget validation.
17+
No-op version increments are invalid. Existing native aliases and IDs0–2 stay
18+
unchanged; older executors reject changed definitions rather than ignore policy.
19+
20+
| Requirement | Acceptance and automated evidence |
21+
|---|---|
22+
| REQ-RPOL-001: authorized policy replacement is one canonical metadata commit | AC-RPOL-001: real ConfigureResource apply changes only policy arrays and advances the resource version; exact native persisted definition and original outcome survive reopen; owner data/vector/event bytes are unchanged. `ResourcePolicyUpdateTests` |
23+
| REQ-RPOL-002: concurrent/retried updates cannot restore stale authority | AC-RPOL-002: stale/missing expected version, missing resource, overflow, unchanged policy/version-only update, changed kind/domain/index/authority/quota/schema/paused state and malformed pointers fail with typed errors and no partial state; same original command retry reuses its receipt. `ResourcePolicyUpdateRejectionTests` |
24+
| REQ-RPOL-003: current reads reauthorize after policy ACK | AC-RPOL-003: projection reclassification, raw-field search denial, graph `/label` denial, and existing change-feed/live-query schema fences use the new persisted definition; old cursors cannot return cached sensitive payload. `EventProjectionPolicyTests`, `ResourcePolicyUpdateVisibilityTests` |
25+
| REQ-RPOL-004: public native clients and RF3 retain the trust boundary | AC-RPOL-004: SDK and official MCP ConfigureResource carry the CAS field through one request grain; current-policy queries on all replicas agree after ACK and fail closed after revocation/restart. `ResourcePolicyUpdateRf3Tests` |
26+
27+
Ordered execution and task graph:
28+
29+
1. Root approves this contract/ADR, owns request field, existing dispatcher join,
30+
public transport/schema inventory and architectural/status documentation.
31+
2. Luna query_wave owns new Core/Features/Authorization/Commands/
32+
ResourcePolicyUpdates.cs plus new UnitTests/Features/Authorization/{Cases,Helpers}
33+
files and the existing EventProjection fixture's policy-update calls. Its
34+
helper validates the nullable prior/new definition and expected version; it
35+
owns no storage handles, routing, alternate authorizer or external side effect.
36+
The root invokes it after reading previous metadata and before any write.
37+
3. The helper distinguishes Validation (invalid/no-op increment), RevisionConflict
38+
(missing or stale CAS), UnsupportedCapability (physical/schema migration or
39+
legacy changed-definition request) and ResourceExhausted (version overflow).
40+
It cannot weaken the existing admin/tenant check or resource validation.
41+
4. Root reviews the complete diff and joins the helper, then builds the solution
42+
and runs the actual Aspire unit/scalar/recovery and SDK/MCP RF3 entries. Worker
43+
gates/Git are forbidden; no test run can race a shared binary replacement.
44+
In parallel, Luna lifecycle_wave owns only new IntegrationTests/Features/
45+
Authorization/{Cases,Helpers,Assertions}/ResourcePolicyUpdate* files. It uses
46+
the actual SDK and official MCP client to qualify CAS, current read/use policy,
47+
old cursor rejection and healthy following requests against Aspire RF3. No
48+
shared fixture, transport, routing or production edits are delegated to it.
49+
5. Delivered-source Linux artifacts and every AC are required for closure. A
50+
local focused pass is development evidence only.
51+
52+
Baseline: full local Aspire Unit36 on unchanged build36 source has3019/3165
53+
passing and146 failures; policy reclassification is a diagnosed real product
54+
gap. No passing policy-update evidence exists yet. No packages or storage format
55+
change is required. SchemaVersion is already in text-generation and change-feed
56+
fences; store position invalidates committed RAM/cache cuts. Rollback disables
57+
new policy admission and retains the committed definition/version; it must never
58+
restore older policies or rewrite user data. Frontend N/A: existing admin API;
59+
SQL N/A for this stage: it does not advertise a new SQL management statement.
60+
61+
```mermaid
62+
sequenceDiagram
63+
participant Caller as SDK or official MCP
64+
participant Request as Orleans request grain
65+
participant Meta as Canonical RF3 metadata owner
66+
Caller->>Request: ConfigureResource with expected version
67+
Request->>Meta: Persisted admin authorization and CAS
68+
Meta->>Meta: Validate policy-only replacement and commit
69+
Meta-->>Caller: New persisted definition and version
70+
Caller->>Request: Search or current page
71+
Request->>Meta: Current resource policy and version fence
72+
```

0 commit comments

Comments
 (0)