feat: use YAML for harness project configuration

[aidandaly24]

Summary

• Replace project harness.json files with commented harness.yaml configuration.
• Generate the file from a readable Handlebars asset template using the existing renderer, with yaml.stringify for dynamic values. A complete default-output snapshot replaces fragment-level presentation assertions.
• Keep agent instructions in the conventional system-prompt.md file. Inline systemPrompt text overrides that file for deployment and CLI export; summary prompts are inline text.
• Newly scaffolded harnesses use managed memory instead of the prior omitted/disabled default because create_harness api creates memory by default. Explicit memory choices are preserved, existing configurations with omitted memory stay omitted, and tools remain opt-in.
• Preserve the thin generated CDK app introduced by feat(project): vend a thin CDK app built on transformAgentCoreJson #2249. Harness loading now lives in the library's existing project transform, with YAML support supplied by CDK #383; CLI export continues reading locally.

Generated harness.yaml

# Optional settings are shown with example values.

name: assistant

# Instructions come from system-prompt.md unless systemPrompt is set inline.
# systemPrompt: You are a helpful assistant.

# Model
model:
  provider: bedrock
  modelId: global.anthropic.claude-sonnet-4-6
  # Output tokens per model call, rather than across the whole invocation.
  # maxTokens: 4096

# Tools
# Code Interpreter and Browser use built-in resources when no ARN is specified.
# awsIam uses the harness execution role, which must allow Gateway invocation.
# https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/gateway.html
# tools:
#   - name: code_interpreter
#     type: agentcore_code_interpreter
#   - name: browser
#     type: agentcore_browser
#   - name: research
#     type: remote_mcp
#     config:
#       remoteMcp:
#         url: https://mcp.example.com/mcp
#   - name: company_tools
#     type: agentcore_gateway
#     config:
#       agentCoreGateway:
#         gatewayArn: arn:aws:bedrock-agentcore:us-west-2:123456789012:gateway/example-1234567890
#         outboundAuth:
#           awsIam: {}

# Tool patterns: @<server-name>/<tool-name> or @builtin.
# This controls agent tool selection, not IAM permissions.
# allowedTools:
#   - "@builtin"
#   - "@research/search"

# Skill path sources refer to files already present in the runtime container.
# skills:
#   - s3Uri: s3://your-skills-bucket/skills/research/
#   - gitUrl: https://github.com/your-org/agent-skills.git
#     path: skills/research
#   - path: /opt/skills/research

# Memory
# https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/memory.html
# Alternatives: existing (name or ARN), disabled.
memory:
  mode: managed
  # Managed-memory settings. Absent values use service defaults.
  # Event retention is in days.
  # strategies:
  #   - SEMANTIC
  #   - SUMMARIZATION
  # eventExpiryDuration: 30
  # Existing-memory reference: project resource name or external ARN.
  # name: ConversationMemory
  # arn: arn:aws:bedrock-agentcore:us-west-2:123456789012:memory/example-1234567890

# Execution limits apply per invocation, across all model calls.
# maxIterations: 15
# maxTokens: 20000
# timeoutSeconds: 300

# Truncation changes the context sent to the model, not the saved memory.
# truncation:
#   strategy: sliding_window
#   config:
#     slidingWindow:
#       messagesCount: 40

# dockerfile and containerUri are mutually exclusive. With neither set, the
# service-provided environment is used. Dockerfile paths are relative to this directory.
# dockerfile: Dockerfile
# containerUri: 123456789012.dkr.ecr.us-west-2.amazonaws.com/my-harness:latest

# Environment values are stored in plaintext.
# environmentVariables:
#   LOG_LEVEL: info

# Deployment creates a role when executionRoleArn is absent.
# executionRoleArn: arn:aws:iam::123456789012:role/MyHarnessRole

# Tags
# tags:
#   team: support
#   environment: development

Breaking change: Project harness configuration now uses harness.yaml; harness.json is no longer supported.

Release dependency: The thin app requires the CDK refactor API plus CDK #383. Before shipping, publish that CDK build and sync the generated app's package pin to the release containing both; verification below uses a local refactor build, not the older CDK main.

Testing

• Merged CLI refactor through de54554996bbf7f7c0be948d162cb248d753049f; 3,386 CLI tests passed, with six snapshots. Typecheck, lint, formatting, secrets checks, and build passed.
• 960 CDK tests passed on the matching refactor build with ci: add Slack notification for new issues #383, including the existing inline/fixed-file tests and focused YAML syntax and prompt-content coverage.
• The built Node CLI scaffolded a fresh project, installed the locally built CDK package with normal peer dependencies, and successfully ran agentcore project build. Separate synthesis checks confirmed fixed-file loading and inline override.
• Installed CDK bytes match the local build. The generated app now uses aws-cdk-lib ~2.269.0, satisfying the CDK refactor's existing peer requirement.
• Upstream removed the generated Jest scaffolding; this merge preserves that removal and keeps the harness loading tests in the CDK library. No new AWS deployment was needed.

Observed service limitation: AWS runtime creation rejected a summary prompt with a trailing newline. The successful smoke run used a single-line summary; the CLI does not trim prompt contents.

[agentcore-devx-automation]

Claude Security Review: no high-confidence findings. (run)

[agentcore-devx-automation]

AgentCore Harness Review

Verdict: Looks good

Nice migration from harness.json to harness.yaml. The design is careful and the test coverage is thorough — I especially appreciated:

• The FIFO / symlink-race hardening in readPrompt (nonblocking O_RDONLY + post-open stat), and the subprocess-bounded test.
• The YAML alias / shared-node test in harnessConfig.test.ts ("does not read misplaced prompt keys or overwrite shared YAML aliases") — the shallow-clone-on-descent in the reader threads that needle correctly.
• The obsolete-JSON diagnostic in both readers, and the migration guidance in README.md covering the copied agentcore/cdk app.
• Running the same reader test suite against both the CLI copy (src/io/harnessConfig.ts) and the vended CDK copy (src/assets/cdk/io/harnessConfig.ts), which mitigates drift between the near-duplicate files.
• The HarnessAuthoringSchema split that keeps file:// references out of HarnessSpecSchema while still surfacing empty-file:// errors before any files are written.
• test("reports schema errors with the YAML path before creating export output", ...) asserts export output isn't created on failure.

Nothing worth blocking on. A couple of small non-blocking notes for future consideration:

• The CLI and CDK reader files are nearly line-for-line duplicates (only the error-wrapping differs). The shared cross-runner test loop makes drift detectable, but a longer-term refactor to share source (e.g., via @aws/agentcore-cdk) would be nicer.
• readFile(configPath, "utf8") has no size guard on the YAML itself (only the referenced prompt files are capped at 1 MiB). Low risk for author-controlled input, but a cheap stat guard would be consistent with the prompt-file policy.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.21%. Comparing base (de54554) to head (f8fd917).

Additional details and impacted files

@@             Coverage Diff              @@
##           refactor    #2297      +/-   ##
============================================
+ Coverage     97.20%   97.21%   +0.01%     
============================================
  Files           607      607              
  Lines         40904    40988      +84     
============================================
+ Hits          39760    39848      +88     
+ Misses         1144     1140       -4     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[agentcore-devx-automation]

Claude Security Review: no high-confidence findings. (run)

[agentcore-devx-automation]

Claude Security Review: no high-confidence findings. (run)

[agentcore-devx-automation]

Claude Security Review: no high-confidence findings. (run)

[AlexanderRichey]

Any reason why we're not using handlebars like the other templates? I also don't see a test asserting YAML output anywhere. Without either of these, I can't tell what the generated YAML would look like.

[jariy17]

Pretty good, see if we can use yaml and jinja to simplify the code here.

[jariy17]

Can't we use jinja to generate the yaml file?

[jariy17]

Can't we use a library like 'yaml' to convert the yaml into a typescript object

[jariy17]

nevermind, I see we need to process another file for the system prompt.

[jariy17]

Same as the comments above.

[agentcore-devx-automation]

Claude Security Review: no high-confidence findings. (run)

[agentcore-devx-automation]

Claude Security Review: no high-confidence findings. (run)

[agentcore-devx-automation]

Claude Security Review: no high-confidence findings. (run)

[jariy17]

why you are bumping the aws-cdk-lib?

[AlexanderRichey]

This looks solid. One thing I noticed is that we removed the systemPrompt: file://system-prompt.md syntax, which I liked since it makes it explicit where the system prompt content is coming from. Can we restore that in a follow up?

Also wanted so show some examples of this kind of commented file pattern:

• https://github.com/ohmyzsh/ohmyzsh/blob/master/templates/zshrc.zsh-template
• https://gist.github.com/nishantmodak/d08aae033775cb1a0f8a