Skip to content

conformance: the capability check catches overreach but not under-delivery #39

Description

@JumpTechCode

zeroOutsideCapabilities in internal/adapter/conformance checks one direction:
that no field is non-zero outside a declared capability. A field that is declared
and never filled is zero, and zero always passes.

The Claude Code adapter passes the kit cleanly while over-declaring four
capabilities (filed separately). Every unfilled governed field —
Workspace.Revision, Workspace.Origin, ToolResult.Truncated,
Delegation.ID, ToolCall.DelegatedID, Delegation.Usage — is invisible to the
check.

The kit's own comment calls the table "what makes a capability declaration
binding rather than advisory". Against under-delivery it is still advisory, and
should not be described otherwise until it isn't.

Separately, Verify runs against a synthetic four-line store in the adapter's
test, so the kit has never been exercised against a real transcript.

Possible fix

An adapter declaring capability C, and not marking it partial, must fill at
least one field governed by C in a session where the evidence for C is present.
That is harder than it sounds — it needs a fixture per capability that is known
to carry the evidence — and it may not be worth the machinery. Correcting the
comment is the minimum.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions