PR4: Grok Build install harness

.ddx/beads.jsonl

@@ -77,7 +77,7 @@
 {"acceptance": "Queue hygiene assessed; test issues and stale artifacts identified", "created_at": "2026-04-04T19:56:05.798325039Z", "id": "helix-25836a2d", "issue_type": "task", "labels": ["helix", "kind:review", "kind:review", "area:tracker"], "parent": "helix-71894dd3", "priority": 0, "schema_version": 1, "spec-id": "helix.prd", "status": "closed", "title": "Review: Tracker queue hygiene", "updated_at": "2026-04-04T20:04:48.434428903Z"}
 {"acceptance": "All review passes complete; findings filed as beads with scope-appropriate area labels; AGENTS.md updated if needed", "claimed-at": "2026-04-11T10:53:39Z", "claimed-machine": "eitri", "claimed-pid": "1978252", "created_at": "2026-04-11T10:53:39.087804569Z", "description": "<context-digest></context-digest>\nFresh-eyes review of last-commit resolved to implementation commit a671506734e3ad50fc9825398b8e57879656b4f1.\nReview target: last-commit (tracker-only close commit skipped per automated run-loop rule).\nReviewed execution bead: helix-e28dc2fa.\nReviewed files: .ddx/beads.jsonl, scripts/helix, tests/helix-cli.sh.", "id": "helix-25acd775", "issue_type": "task", "labels": ["helix", "kind:review", "kind:planning", "action:review", "area:workflow", "area:testing"], "notes": "<measure-results>\n  <timestamp>2026-04-11T10:57:55Z</timestamp>\n  <status>ISSUES_FOUND</status>\n  <findings total='1' filed='1' critical='0' high='0' medium='1' low='0'/>\n  <agents-md-updated>NO</agents-md-updated>\n  <learnings-filed>0</learnings-filed>\n  <finding-ids>helix-af52f67a</finding-ids>\n</measure-results>\n\nFresh-eyes review of commit a671506734e3ad50fc9825398b8e57879656b4f1 completed.\nEvidence:\n- Re-read AGENTS.md and loaded the active principles, concerns, practices, and reviewed bead context before inspection.\n- Reviewed HEAD tracker-only close commit 57bcd0758fea3408ab8cc2f5dc9b98f57e76c80c and resolved implementation commit a671506734e3ad50fc9825398b8e57879656b4f1 from helix-e28dc2fa.closing_commit_sha per the automated run-loop rule.\n- Pass 1 correctness found one medium-severity defect in scripts/helix:2518-2526: `sync_issue_closing_commit_sha()` mutates the tracker, then runs `git commit --only -qm` without checking that commit's exit status even though both callers execute the helper in an `|| true` context. Bash reproduction: `bash -lc 'set -euo pipefail; f(){ false; echo after; }; f || true; echo survived'` prints `after` and `survived`, showing inner failures are ignored in that call pattern.\n- Pass 2 integration: if the repo-local sync commit fails, HELIX can continue review/loop execution with an uncommitted `.ddx/beads.jsonl` mutation, so the new review-targeting repair is not yet failure-safe.\n- Pass 3 concern-aware quality: active concerns hugo-hextra and demo-asciinema match via areas:all but introduce no additional scope-specific drift for this workflow/testing change.\n- Pass 4 operational learnings: no AGENTS.md updates or separate learnings issues were required.\n- Filed follow-on bead: helix-af52f67a.", "owner": "erik", "priority": 0, "schema_version": 1, "spec-id": "helix-e28dc2fa", "status": "closed", "title": "review: last-commit a671506 helix-e28dc2fa implementation", "updated_at": "2026-04-11T10:57:55.633517304Z"}
 {"acceptance": "Review areas reconciled and durable report updated", "created_at": "2026-04-10T20:16:50.497479046Z", "description": "Repo-wide alignment review for the HELIX repository covering workflow/CLI contract, tracker hygiene, DDx boundary, and public site/demo surfaces.", "id": "helix-25ec03b4", "issue_type": "epic", "labels": ["helix", "kind:review", "kind:review"], "priority": 0, "schema_version": 1, "status": "closed", "title": "HELIX alignment review: repo", "updated_at": "2026-04-10T20:23:02.637459732Z"}
-{"acceptance": "1. test -f tests/install/grok-build/install.sh && test -f tests/install/grok-build/verify.sh\n2. rg -n 'grok-build|grok' tests/install/run-all.sh\n3. bash tests/install/grok-build/verify.sh either exits 0 or exits with documented skip code when grok missing.\n4. Open GitHub PR for Grok install harness.", "created_at": "2026-07-22T22:17:03.810463033Z", "dependencies": [{"issue_id": "helix-26257203", "depends_on_id": "helix-416efaaa", "type": "blocks", "created_at": "2026-07-22T22:17:26Z"}], "description": "Add tests/install/grok-build/{install.sh,verify.sh} and wire into tests/install/run-all.sh. Use Phase 0 install paths. Static verify: grok inspect asserts canonical helix plugin source path (not mere name=helix); verify-skill-layout.sh; adopter no-local-workflows path probe. Explicit skip exit semantics if grok binary unavailable (no false-fail/false-green). Functional mode list gated by TEST_FUNCTIONAL=1. Out of scope: skill content changes; progressive disclosure.", "id": "helix-26257203", "issue_type": "task", "labels": ["helix", "phase:test", "activity:test", "kind:test", "area:testing", "plan-2026-07-22", "runtime:grok", "tranche:pr4"], "parent": "helix-af0bc17b", "priority": 1, "schema_version": 1, "spec-id": "plan-2026-07-22-grok-plugin", "status": "open", "title": "PR4: Grok install harness tests/install/grok-build", "updated_at": "2026-07-22T22:17:26.922303072Z"}
+{"acceptance": "1. test -f tests/install/grok-build/install.sh && test -f tests/install/grok-build/verify.sh\n2. rg -n 'grok-build|grok' tests/install/run-all.sh\n3. bash tests/install/grok-build/verify.sh either exits 0 or exits with documented skip code when grok missing.\n4. Open GitHub PR for Grok install harness.", "created_at": "2026-07-22T22:17:03.810463033Z", "dependencies": [{"issue_id": "helix-26257203", "depends_on_id": "helix-416efaaa", "type": "blocks", "created_at": "2026-07-22T22:17:26Z"}], "description": "Add tests/install/grok-build/{install.sh,verify.sh} and wire into tests/install/run-all.sh. Use Phase 0 install paths. Static verify: grok inspect asserts canonical helix plugin source path (not mere name=helix); verify-skill-layout.sh; adopter no-local-workflows path probe. Explicit skip exit semantics if grok binary unavailable (no false-fail/false-green). Functional mode list gated by TEST_FUNCTIONAL=1. Out of scope: skill content changes; progressive disclosure.", "id": "helix-26257203", "issue_type": "task", "labels": ["helix", "phase:test", "activity:test", "kind:test", "area:testing", "plan-2026-07-22", "runtime:grok", "tranche:pr4"], "owner": "erik", "parent": "helix-af0bc17b", "priority": 1, "schema_version": 1, "spec-id": "plan-2026-07-22-grok-plugin", "status": "closed", "title": "PR4: Grok install harness tests/install/grok-build", "updated_at": "2026-07-22T22:25:51.496990877Z"}
 {"acceptance": "`docker build -t helix-workflow-test:genie tests/workflows/genie/` succeeds (or for Genie: scripts present, no Dockerfile needed). `bash tests/workflows/genie/run-scenarios.sh --list` enumerates the 13 scenarios. Each scenario hooks into shared verifiers.", "closing_commit_sha": "1793ff4aab93799363766e793708544ba648a53c", "created_at": "2026-05-17T20:52:31.559909343Z", "description": "<context-digest>\n<principles>Design for Change · Design for Simplicity · Validate Your Work · Make Intent Explicit · Prefer Reversible Decisions</principles>\n<concerns>workflow-coverage | shared-fixture | runtime-parity | refusal-hardening</concerns>\n<practices>tests/workflows/fixtures/recipe-app/ is the single shared fixture across all 5 runtimes · Pass criteria are runtime-neutral; per-runtime variants vary by invocation/observability only · Static checks free, functional checks gated by TEST_FUNCTIONAL=1 · Genie scenarios may emit `genie:catalog-unreachable` skip until helix-96f7dd34 resolves · Symlink/inherit from tests/install/<runtime>/install.sh where reusable</practices>\n<governing>FEAT-014 RT-* · tests/install/genie/ (reference pattern) · TP-014 §7 layout</governing>\n</context-digest>\n\nReuse install.py + verify.py from tests/install/genie/. Add run-scenarios.py — a Playwright driver iterating each scenario against the workspace. Per-scenario test-procedure-<scenario>.md for manual fallback when Playwright is unavailable. DATABRICKS_HOST + DATABRICKS_TOKEN required. Scenarios 2/4/5/6 may emit genie:catalog-unreachable skip until helix-96f7dd34 resolves.\n\nLayout: `tests/workflows/genie/Dockerfile` (symlink to tests/install/genie/Dockerfile where applicable), `install.sh` (symlink), `run-scenarios.sh` (per-scenario loop), `.tape` files per scenario, optional `recordings/` dir.", "events_attachment": "helix-264e6b8a/events.jsonl", "id": "helix-264e6b8a", "issue_type": "task", "labels": ["helix", "activity:test", "kind:test", "area:testing", "area:runtime"], "owner": "erik", "parent": "helix-88f790e4", "priority": 1, "schema_version": 1, "session_id": "eb-d9f54b6c", "spec-id": "FEAT-014", "status": "closed", "title": "RT-genie: author tests/workflows/genie/ harness", "updated_at": "2026-05-18T04:52:45.625380157Z"}
 {"acceptance": "Sidecar JSON and three screenshots written under tests/install/genie/recordings/; all three new types appear in their activity-specific prompt responses; baseline diff reports zero new failures; go test ./... and lefthook pre-commit pass. Depends on B7b.", "closing_commit_sha": "a74b65e7e21e13cf89228d557eaaf6f29cdadadf", "created_at": "2026-05-18T21:10:23.075059016Z", "description": "PROBLEM: Even with bundle and install confirmed, only a live Playwright drive against the Databricks Genie workspace proves the three new types are reachable via real prompts. Results must be archived as a metadata sidecar plus screenshots and compared to the 2026-05-17-verify-clean baseline to detect regressions.\n\nROOT CAUSE: tests/install/genie/drive-genie.py runs Playwright against the workspace with fresh cookies; tests/install/genie/recordings/ stores per-run metadata sidecars and screenshots. Without a fresh run after B7b there is no evidence the workspace actually surfaces the new types, and without a baseline diff a regression in any other prompt could land silently.\n\nPROPOSED FIX: Run drive-genie.py with fresh cookies; for Prompt 2 ('list the artifact types defined under activity 01-frame') and the analogous prompts for 02-design and 03-test, assert the response text contains data-prd, data-architecture, and data-quality-expectations respectively. Write tests/install/genie/recordings/2026-05-18-verify-data-types.json containing prompts, response excerpts, screenshot paths, and the manifests from B7a/B7b. Save three screenshots. Diff prompt-by-prompt against tests/install/genie/recordings/2026-05-17-verify-clean and report zero new failures.\n\nNON-SCOPE: bundle rebuild (B7a); workspace install (B7b); editing templates or SKILL.md; rotating cookies; extending baseline scope beyond the existing 2026-05-17-verify-clean prompt set.\n\nAC:\n1. TestE2eListsDataPrdIn01Frame — Prompt 2 response text includes data-prd.\n2. TestE2eListsDataArchitectureIn02Design — corresponding prompt response includes data-architecture.\n3. TestE2eListsDataQualityExpectationsIn03Test — corresponding prompt response includes data-quality-expectations.\n4. TestSidecarMetadataWritten — tests/install/genie/recordings/2026-05-18-verify-data-types.json exists and references each prompt, response, and screenshot path.\n5. TestThreeScreenshotsPreserved — exactly three new screenshots exist under tests/install/genie/recordings/ for the three prompts.\n6. TestNoNewFailuresVsBaseline — diff against tests/install/genie/recordings/2026-05-17-verify-clean shows no prompts regressing from pass to fail.\n7. TestGoSuitePasses — go test ./... passes.\n8. TestLefthookGate — lefthook run pre-commit passes.", "events_attachment": "helix-2664deb8/events.jsonl", "id": "helix-2664deb8", "issue_type": "task", "labels": ["helix", "activity:test", "kind:test", "area:artifacts", "area:integration", "area:packaging"], "parent": "helix-72f075e1", "priority": 0, "schema_version": 1, "session_id": "eb-bd5eea57", "status": "closed", "title": "B7c: Playwright e2e drive, metadata sidecar, and baseline diff", "updated_at": "2026-05-18T21:44:54.189721845Z"}
 {"acceptance": "Running `bash scripts/helix help` and comparing it to docs/helix/04-build/implementation-plan.md shows that every command listed under Current Command Surface is present in the help output, including `input` and `commit` if they are claimed as live surfaces.", "claimed-at": "2026-04-11T02:27:44Z", "claimed-machine": "eitri", "claimed-pid": "2204771", "closing_commit_sha": "f62cba864302333b395aa632fad6a3fc8549a085", "created_at": "2026-04-10T11:57:42.721928144Z", "dependencies": [{"issue_id": "helix-268c54ee", "depends_on_id": "helix-ea97cbda", "type": "blocks", "created_at": "2026-04-10T22:33:39Z"}], "description": "<context-digest>\n<principles>Design for Change · Design for Simplicity · Validate Your Work · Make Intent Explicit · Prefer Reversible Decisions</principles>\n<concerns>hugo-hextra | demo-asciinema</concerns>\n<practices>Theme version: Hextra v0.12.1 — pinned in website/go.mod · Hugo version: 0.159.2 extended — pinned in .github/workflows/pages.yml · Current demos: helix-quickstart (full lifecycle), helix-concerns (drift detection), helix-evolve (requirement threading) · Agent harness: ddx agent run --harness claude — not claude -p</practices>\n<governing>docs/helix/04-build/implementation-plan.md — Running bash scripts/helix help and comparing it to docs/helix/04-build/implementation-plan.md shows that every command listed under Current Command Surface is present in the help output, including input and commit if they are claimed as live surfaces.</governing>\n</context-digest>\n\nReview finding from fresh-eyes review.\nFile: docs/helix/04-build/implementation-plan.md:58\nCategory: correctness\nSeverity: medium\nDescription: The refreshed plan says the current command surface was validated against `bash scripts/helix help`, then lists `input` and groups `commit` as part of that validated surface. But `bash scripts/helix help` does not print an `input` command at all, and its Usage block omits `commit` as an invocable surface even though the dispatcher has `commit)` and `input)` handlers in scripts/helix. That leaves the implementation plan internally inconsistent with the verification source it cites, so operators using the plan cannot tell whether the CLI help is stale or the plan is overstating what the documented command surface actually is.\nSuggested fix: Make the implementation plan and `bash scripts/helix help` agree. Either remove commands from the plan until the help text advertises them, or update the help output so every command claimed in the Current Command Surface section is visible in the documented CLI surface.", "id": "helix-268c54ee", "issue_type": "task", "labels": ["helix", "activity:build", "review-finding", "area:cli", "area:docs"], "owner": "erik", "parent": "helix-13cfe23f", "priority": 0, "schema_version": 1, "spec-id": "docs/helix/04-build/implementation-plan.md", "status": "closed", "title": "correctness: reconcile implementation-plan command surface with helix help evidence", "updated_at": "2026-04-11T02:36:20.16752076Z"}

tests/install/grok-build/install.sh

@@ -0,0 +1,53 @@
+#!/usr/bin/env bash
+# Install HELIX into Grok Build from a local checkout (default) or published source.
+#
+# Env:
+#   HELIX_ROOT       path to helix checkout (default: /workspace/helix or repo relative)
+#   TEST_PUBLISHED=1 use DocumentDrivenDX/helix instead of local path
+#
+# Exit:
+#   0  install succeeded
+#   77 grok CLI not available (caller may treat as skip)
+#   1  install failed
+
+set -euo pipefail
+
+if ! command -v grok >/dev/null 2>&1; then
+  echo "SKIP: grok not on PATH"
+  exit 77
+fi
+
+if [[ -n "${HELIX_ROOT:-}" ]]; then
+  ROOT="$HELIX_ROOT"
+elif [[ -d /workspace/helix ]]; then
+  ROOT=/workspace/helix
+else
+  ROOT="$(cd "$(dirname "$0")/../../.." && pwd -P)"
+fi
+
+echo "→ HELIX_ROOT=$ROOT"
+grok plugin validate "$ROOT"
+
+set +e
+if [[ "${TEST_PUBLISHED:-}" == "1" ]]; then
+  echo "→ grok plugin install DocumentDrivenDX/helix --trust"
+  out="$(grok plugin install DocumentDrivenDX/helix --trust 2>&1)"
+  rc=$?
+else
+  echo "→ grok plugin install $ROOT --trust"
+  out="$(grok plugin install "$ROOT" --trust 2>&1)"
+  rc=$?
+fi
+set -e
+printf '%s\n' "$out"
+if [[ $rc -ne 0 ]]; then
+  if printf '%s\n' "$out" | grep -qiE 'already installed|already exists'; then
+    echo "→ plugin already installed; continuing"
+  else
+    echo "FAIL: grok plugin install exit $rc"
+    exit 1
+  fi
+fi
+
+grok plugin list
+echo "install: OK"

tests/install/grok-build/verify.sh

@@ -0,0 +1,113 @@
+#!/usr/bin/env bash
+# Static verify for Grok HELIX plugin install.
+#
+# Exit:
+#   0  pass
+#   77 grok not available (skip — not a failure for CI without grok)
+#   1  fail
+
+set -euo pipefail
+
+if ! command -v grok >/dev/null 2>&1; then
+  echo "SKIP: grok not on PATH"
+  exit 77
+fi
+
+REPO_ROOT="$(cd "$(dirname "$0")/../../.." && pwd -P)"
+SHARED_VERIFY="$REPO_ROOT/tests/install/shared/verify-skill-layout.sh"
+
+echo "→ grok plugin list"
+grok plugin list
+
+echo "→ resolve skill root (prefer installed-plugins helix, else checkout)"
+SKILL_ROOT=""
+# Prefer registry-installed full plugin tree
+if [[ -d "$HOME/.grok/installed-plugins" ]]; then
+  cand="$(find "$HOME/.grok/installed-plugins" -path '*/skills/helix/SKILL.md' 2>/dev/null | head -1 || true)"
+  if [[ -n "$cand" ]]; then
+    SKILL_ROOT="$(dirname "$cand")"
+  fi
+fi
+if [[ -z "$SKILL_ROOT" && -f "$REPO_ROOT/skills/helix/SKILL.md" ]]; then
+  SKILL_ROOT="$REPO_ROOT/skills/helix"
+fi
+if [[ -z "$SKILL_ROOT" ]]; then
+  echo "FAIL: could not resolve skills/helix root"
+  exit 1
+fi
+echo "→ SKILL_ROOT=$SKILL_ROOT"
+
+# Canonical source: path must contain 'helix' and end with skills/helix
+case "$SKILL_ROOT" in
+  */skills/helix) ;;
+  *)
+    echo "FAIL: skill root is not …/skills/helix: $SKILL_ROOT"
+    exit 1
+    ;;
+esac
+
+# Prefer non-legacy sources: reject known DDx legacy skill names when inspecting
+if command -v python3 >/dev/null 2>&1; then
+  python3 - <<'PY' || exit 1
+import json, subprocess, sys
+try:
+    raw = subprocess.check_output(["grok", "inspect", "--json"], text=True)
+    data = json.loads(raw)
+except Exception as e:
+    print(f"WARN: grok inspect --json failed: {e}", file=sys.stderr)
+    sys.exit(0)
+
+def walk(o, out):
+    if isinstance(o, dict):
+        if o.get("name") == "helix" or (
+            isinstance(o.get("source"), dict)
+            and "helix" in str(o.get("source")).lower()
+            and o.get("name")
+        ):
+            out.append(o)
+        for v in o.values():
+            walk(v, out)
+    elif isinstance(o, list):
+        for v in o:
+            walk(v, out)
+
+found = []
+walk(data, found)
+helix = [s for s in found if s.get("name") == "helix"]
+if not helix:
+    print("FAIL: grok inspect has no skill name=helix", file=sys.stderr)
+    sys.exit(1)
+# At least one helix skill should not be the renamed helix-workflow path only
+paths = []
+for s in helix:
+    src = s.get("source") or {}
+    if isinstance(src, dict):
+        paths.append(str(src.get("path") or src))
+    else:
+        paths.append(str(src))
+print("→ helix skill sources:")
+for p in paths:
+    print("  ", p)
+if any("helix-workflow" in p for p in paths) and not any(
+    "/skills/helix/" in p or p.endswith("skills/helix/SKILL.md") for p in paths
+):
+    print("FAIL: only legacy helix-workflow found", file=sys.stderr)
+    sys.exit(1)
+print("ok: canonical helix skill present in inspect")
+PY
+fi
+
+echo "→ verify-skill-layout"
+bash "$SHARED_VERIFY" "$SKILL_ROOT"
+
+# Adopter no-local-workflows path probe against this skill root
+if [[ -f "$SKILL_ROOT/../../workflows/graph.yml" ]]; then
+  echo "ok: full-plugin catalog via ../../workflows/graph.yml"
+elif [[ -f "$SKILL_ROOT/references/graph.yml" ]]; then
+  echo "ok: package floor catalog via references/graph.yml"
+else
+  echo "FAIL: neither ../../workflows/graph.yml nor references/graph.yml from skill root"
+  exit 1
+fi
+
+echo "verify: PASS"