feat: D1 Durable Dana Conversation — restart-resumable ACP agent - #27
Open
ngoclam9415 wants to merge 44 commits into
Open
feat: D1 Durable Dana Conversation — restart-resumable ACP agent#27ngoclam9415 wants to merge 44 commits into
ngoclam9415 wants to merge 44 commits into
Conversation
Add DanaACPAgent implementing the ACP Agent protocol over stdio JSON-RPC, enabling dana-console to connect to Dana as a Custom ACP Agent. - dana/apps/acp/agent.py: DanaACPAgent translating ACP calls (initialize, session/new, session/load, session/resume, session/prompt, session/cancel) to AgentSession operations, streaming HostEvents back as session_update notifications - dana/apps/acp/translation.py: HostEvent → ACP update chunk translation (ACP types never enter STAR core) - dana/apps/acp/__main__.py: entry point with stderr-only logging - dana/__init__/init_environment.py: redirect structlog to stderr so stdout stays clean for JSON-RPC frames - pyproject.toml: dana-acp console script entry point - tests/integration/test_acp_agent.py: 14 in-process + subprocess tests covering load capability, replay-before-return, chunk streaming, burst ordering, busy, cancel, malformed content, stderr/stdout discipline
Untrack /sprint/, /v2/, CLAUDE.md (local-only working docs, consistent with existing AGENTS.md/.claude/.opencode ignores). CLAUDE.md removed from repo; local copy retained via gitignore.
Per-agent intercept-capable EventBus: first-wins aggregation, sync+async handlers, raise isolation, emit_sync via Misc.safe_asyncio_run. Lazy mount on BaseSTARAgent. Non-dict results warned+skipped. 17 tests.
.agents/, .codegraph/, .codex/, .superpowers/, memories/, tests/unit/core/guard/ — local tool artifacts (consistent with .claude/.opencode).
_build_native_tools_if_supported now returns early if schemas already built. Rebuilding every build_prompt re-ran inspect.signature on every resource method under the tracing chain, exhausting the recursion budget on long sessions (librarian console crash). Structural deps don't change per turn -> build once. Adds 2 tests.
Milestone M3 (longest pole of v2.0 extensibility backbone). Wires the S1
EventBus into both tool-execution paths and adds a deny-only PermissionPolicy.
- ext/operation.py: Operation + ToolIdentity (thin, read-only via MappingProxyType)
- ext/permission.py: PermissionPolicy deny-only (a tool_call subscriber)
- ext/guard.py: reference rm-rf + protected-path policy (S4 discovery target)
- tool_executor.py: emit tool_call/tool_result around dispatch in both single-
call paths; split out _dispatch_single_call[_async] (dispatch NOT merged)
- runtime/{protocols,__init__}.py: remove dead ToolHookProtocol/ApprovalProtocol
scaffold + constructor params hooks/approval (never wired)
Adversarial review fixes: non-dict tool_result modify no longer crashes the
batch (isinstance guard + warn); guard substring rules str()-coerce (defeats
list-arg bypass); strict-bool block (is True); Operation.arguments immutable.
Tests: 27 new (S3.1-S3.15 + 7 adversarial fix-regressions). Regression green:
tests/unit + tests/integration 1614 passed, 37 skipped, 1 xfailed.
Milestone M2. Emits see_end/think_end/act_end/reflect_end around the STAR phases in query()/aquery() so handlers can observe, modify, or block each phase. STAR contract (_see/_think/_act/_reflect) unchanged. - base_star_agent.py: _emit_phase[_async] helpers + per-phase block/modify wiring in _do_query/_do_aquery; reflect_end emit in the reflect wrappers. - Orchestrator-based wiring (not scatter-site): STARAgent._think/_act_async broadcast inline without super(), so base-site wiring would miss them. - Fix latent S1 bug: event_bus property used getattr(self,_event_bus,None) but STARAgent.__getattr__ returns a magic-method stub for any unknown attr, so the bus was never created on the real agent. Now reads self.__dict__. - Adversarial fixes: per-phase exit uses EXIT_FLAG is True (not _do_exit_star_loop, avoiding the empty-dict false-exit quirk); act_end block sets phase_blocked (skips reflect, prevents repeat); non-dict modify ignored + warned. Trade-off: broadcast fires before emit, so legacy broadcast observers see the pre-modify result (accepted for minimal blast radius; modify still changes the result for later phases). Tests: 10 new (T2.1-T2.8 + 2 adversarial). Regression: tests/unit + tests/integration 1624 passed, 37 skipped, 1 xfailed (the 9 done-flag-autonomy tests caught the event_bus bug pre-fix).
Milestone M4 — completes the v2.0 extensibility backbone (M1-M4 all shipped).
Drop-in Python extensions discovered from ~/.dana/extensions/ (global, always)
and .dana/extensions/ (project, trust-gated via DANA_TRUST_PROJECT_EXTENSIONS)
and bound to the agent's EventBus via a setup(agent) factory using agent.on().
- ext/extensions.py: ExtensionManager — discover/load/reload + LoadReport.
* Per-agent, lazy via agent.extensions (__dict__ storage, same __getattr__
lesson as S1/S2).
* Loader bypasses the pyc cache (read_text+compile+exec): SourceFileLoader
keys .pyc on (mtime,size) so a same-byte-size edit within 1s would exec
stale code — fatal for hot-reload correctness.
* Reload: unsub tracked handlers, pop stale sys.modules, re-exec, emit
SESSION_RELOAD. Must run at idle (S1 Finding A).
* Sub tracking via wrapping bus.subscribe during setup (try/finally).
- base_star_agent.py: agent.on alias + extensions property + load_extensions/
reload_extensions delegates. NOT auto-loaded at construction (host calls it;
zero regression risk to agent init).
- Trust gate: global = user's home (trusted); project = explicit flag.
Adversarial fixes: failing setup rolls back partial handler registrations
(transactional; was a reload leak); reload pops stale sys.modules entries.
Tests: 10 new (T4.1-T4.8 + 2 adversarial). Regression: tests/unit +
tests/integration 1634 passed, 37 skipped, 1 xfailed.
Concise showcase of the intercept-capable EventBus: drop-in extensions (~/.dana/extensions/*.py), the setup(agent)/agent.on contract, the block/modify handler shapes, and the 3 concrete examples (rm-rf guard, arg rewrite, STAR observer).
feat: v2.0 extensibility backbone (M1-M4)
…ce in _get_or_create_worker, remove dead _InFlight.wait()
- Add CancellationTree with cascade/detach/keep ownership semantics - Add CancellationNode with acknowledged/timeout/effect-unknown outcomes - Add kill escalation (force-kill all descendants regardless of ownership) - Extend FactType enum with D2 tool lifecycle facts (non-terminal + terminal) - Add comprehensive tests: cancellation matrix (6 contexts), outcome distinction, terminal fact enforcement, serialization, edge cases
- Add DurableJobManager with handoff request/confirm/fail lifecycle - Add DurableJobRecord with status tracking (handoff_requested, running, completed, failed, cancelled, handoff_failed) - Add cascade/detach integration: RUNNING jobs survive parent cancellation, HANDOFF_REQUESTED jobs fail on parent cancel - Add serialization round-trip for records and manager state - Add comprehensive tests: lifecycle, cascade/detach, edge cases, crash-before-handoff, serialization
…ion matching, add evaluator tests
- Add tool lifecycle HostEventTypes (TOOL_REQUESTED, TOOL_STARTED, TOOL_PROGRESS, TOOL_RESULT, TOOL_FAILURE, TOOL_ACKNOWLEDGED, TOOL_TIMED_OUT, TOOL_EFFECT_UNKNOWN, TOOL_CANCELLATION_REQUESTED, TOOL_AUTHORIZED_OR_DENIED, THOUGHT) - Extend host_events.py fact-to-event mapping for all D2 tool facts - Extend ACP translation with thought, tool-call, tool-update, result, and cancellation state mappings - Add AgentSession tool lifecycle wiring: emit_thought, journal_tool_*, execute_tool_call with full lifecycle - Add rollback flag (use_legacy_executor) for non-ACP host fallback - Add comprehensive unit tests for all five ACP states and tool lifecycle
…e wiring Also fixes D4 Model Catalog: add __init__.py, input validation, tighten types
…r into sessions, add rollback flag
- perform_handshake: official mcp package initialize handshake - discover_tools: tools/list integration with capability gating - call_tool: tool invocation via ClientSession - mcp_tool_to_catalog_entry: MCP Tool -> ToolCatalogEntry with namespaced identity (server_name:tool_name) and original name as alias - MCPHandshakeResult frozen dataclass for handshake results - In-process fake-server tests using mcp.shared.memory utilities - Schema conversion tests verify ToolIdentity mapping fidelity - ADR-008: uses official mcp>=1.28,<2 package - ADR-004: namespaced Tool Identity, alias for original MCP name
- MCPStdioTransport: wraps stdio_client + ClientSession with managed subprocess lifecycle (connect, handshake, list_tools, call_tool) - MCPHttpTransport: wraps streamable_http_client + ClientSession with optional connection pooling via pre-configured httpx.AsyncClient - Both transports raise RuntimeError if methods called before connect - close() is idempotent on both transports - Integration tests verify full protocol flow (handshake -> list_tools -> call_tool) through in-process fake server - ADR-008: stdio servers default to dedicated managed processes; HTTP connections may be pooled when descriptors + credentials match
Migration gate (spec §Dependencies and Gates): official-protocol parity achieved + cleanup tests pass. Removed: - dana/lib/resources/mcp/ (MCPClientResource, BrightQueryResource, GitHubMCPResource, SlackMCPResource) - dana/lib/resources/mcp_client.py (duplicate of mcp/mcp_client.py) - Updated dana/lib/resources/__init__.py to remove MCP re-exports All 678 unit tests pass (8 new MCP tests + 670 existing).
…otected state compatibility gating
- MCPCatalogAdapter: discover MCP tools, convert to namespaced ToolCatalogEntry, deterministic collision detection, invalidation, and re-discovery - MCPLease/MCPLeaseManager: session lease lifecycle (pending/active/failed/degraded/released), required-lease failure stops preflight, optional-lease failure degrades, restore from persisted state (session load) - MCPConfig/load_mcp_config: JSON config loading with env allowlist, rollback via mcp_enabled flag, filter_env_for_server - 57 new tests covering all ACs and edge cases - Updated mcp/__init__.py exports AC #1: Deterministic collisions AC #2: Dynamic catalog invalidation AC #3: Allowlisted environment enforced AC #4: Configuration loading + rollback
- ContentNormalizer normalizes text/image/embedded_resource/file_resource blocks - Validation module enforces MIME type, size limits, and path-safety checks - ArtifactStore provides hash-based dedup with OwnerScope isolation - ArtifactRetentionManager tracks independent retention policies - New FactTypes: ARTIFACT_REFERENCE, ARTIFACT_DELETED - NewJournalFact extended with artifact_refs field - SQLite and Postgres adapters pass through artifact_refs on append - Full test coverage: 63 new tests (validation, normalizer, store, retention)
…store - Empty payload, zero-byte file, unsupported MIME, symlink traversal - Concurrent duplicate uploads, large content - Missing artifact during sweep, retention expiry race
…e redundant traversal check
…ndler, and cleanup
…tent blocks, and capability advertisement
…ync, add SIGKILL fallback and waitpid fix(D6): return content_blocks_payload from _normalized_blocks_to_text_blocks, pass to session.prompt, fix dead D6 multimodal handlers in translation
…rt in session.prompt
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Dana now streams multi-turn text conversations through
dana-acpand automatically resumes the same session after the ACP process restarts. This is the first delivery (D1) of the ACP AgentSession kernel — a Session Journal backed by SQLite/PostgreSQL is the sole durable authority for every turn.What shipped
dana/core/session/{models,protected_state}.py—OwnerScope,JournalFact,FactType,ProtectedStateCodec(AES-GCM + HKDF + AAD)dana/core/session/journal/{sqlite,postgres,protocol,schema,models}.py— one contract, two real databases, optimistic concurrencydana/core/session/projections/{conversation,host_events}.py— ConversationView (committed-turn gating) + HostEvent streamdana/core/session/agent_session.py— serialized text turns, streaming seam, bounded flush, crash-safe journalingdana/core/session/legacy_timeline_migration.py— interrupted-turn detection + idempotent legacy Timeline importdana/apps/acp/—initialize,session/new,session/load,session/resume,session/prompt,session/canceldana/core/session/health.py— redacted operational reportKey invariants
TURN_STARTED+USER_CONTENT_FINALappended before the model callASSISTANT_CONTENT_FINAL+ terminal in one atomic batchbusyacp.*imports confined todana/apps/acp/; STAR core untouchedTest results
Dependencies added
agent-client-protocol>=0.10,<0.11aiosqlite>=0.20.0asyncpg>=0.30.0What's NOT changed
dana-code,adana,dana-replCLIs remain on legacy Timeline (incremental migration)STARAgent.query()/aquery()source-compatibleConsole companion PR
Branch
feat/acp-agent-session-restartin dana-console addssessionStoragesession-ID persistence +session/loadresume on reconnect.Design spec
Approved at
docs/superpowers/specs/2026-07-07-acp-star-adapter-design.md.