T-79SM-finish-claude-harness-ts-migration

Status: closed/done · Impact: medium · Complexity: large

Goal

The en-masse Python→TypeScript migration converted all of plugin/ and three of the .claude/ project-check checkers (check_pipe_tail, check_skill_prose, check_entities). Four .claude/ harness scripts remain Python — two of them (audit_skill_runtime.py + its eval harness) are still the only .py entries in sdlc.yaml’s quality_checks:, and a fourth checker’s gate (run_base_evals) is parked (commented out) because its subject still shells to a .py validator. Port the remaining four to TypeScript so the whole harness runs on one substrate, the bun test ./.claude verb covers every .claude suite, and the parked gate comes back online.

Today

Location	Role today
.claude/skills/project-check/check_entity_bases.py	Python checker validating each entity type’s base YAML template against its schema (invokes the now-migrated plugin/validators/validate_base.ts). Its sdlc.yaml gate is parked.
.claude/skills/project-check/check_skill_docs.py	Python checker linting SKILL.md doc structure across plugin/skills/*/SKILL.md.
.claude/scripts/audit_skill_runtime.py	Python checker auditing SKILL.md runtime-state drift; a live quality_checks: gate.
.claude/hooks/warn-skill-doc-drift.py	PostToolUse(Edit/Write) hook warning on skill-doc drift; wired in .claude/settings.json.
.claude/skills/project-check/tests/run_base_evals.py	Python eval harness for check_entity_bases (its gate is the parked line in sdlc.yaml).
.claude/skills/project-check/tests/run_skill_doc_evals.py	Python eval harness; after the check_pipe_tail port it now serves only check_skill_docs.
.claude/scripts/tests/audit_skill_runtime/run_evals.py	Python eval harness for audit_skill_runtime; a live quality_checks: gate.
sdlc.yaml	quality_checks: still lists two .py gates (audit_skill_runtime.py, its run_evals.py) plus a commented-out parked run_base_evals.py line.

Proposed

All four remaining .claude/ harness scripts run under bun as .ts, with bun:test suites replacing their Python eval harnesses. The Python files are removed, sdlc.yaml carries zero .py gates, the parked check_entity_bases gate is restored (its .test.ts covered by bun test ./.claude), the warn-skill-doc-drift PostToolUse hook points at the .ts, and remaining .py checker citations in SKILL.md / lefthook.yml are refreshed. The only .py left under .claude/ is intentional test-fixture sample scripts.

Approach

Each checker is an independent port; commit one checker (+ its test + citation refresh + delete) at a time. Mirror the three completed ports (check_pipe_tail.ts, check_skill_prose.ts, check_entities.ts) for bun idioms and bun:test shape.

1. Port check_entity_bases.py → check_entity_bases.ts invoking plugin/validators/validate_base.ts; replace run_base_evals.py with tests/check_entity_bases.test.ts; delete both .py.
2. Port check_skill_docs.py → check_skill_docs.ts; replace run_skill_doc_evals.py (now sole-serving this checker) with tests/check_skill_docs.test.ts; delete both .py.
3. Port audit_skill_runtime.py → .claude/scripts/audit_skill_runtime.ts; replace tests/audit_skill_runtime/run_evals.py with audit_skill_runtime.test.ts; delete both .py. Leave the sample-script fixtures under tests/audit_skill_runtime/fixtures/** as .py (they are test data, not migration targets).
4. Port warn-skill-doc-drift.py → .ts and repoint the PostToolUse hook in .claude/settings.json.
5. Rewire sdlc.yaml: drop the two .py gate lines and the parked run_base_evals.py comment; the three eval suites now ride bun test ./.claude. Add a bare bun run .claude/scripts/audit_skill_runtime.ts live-repo check if a non-test repo-state gate is still wanted (mirror how audit_entities.ts is gated bare).
6. Refresh remaining .py→.ts checker citations in .claude/skills/project-check/SKILL.md and lefthook.yml (the deferred citation sweep from the three dogfood ports).

Files to touch

Location	Kind	Change
.claude/skills/project-check/check_entity_bases.ts	new	TS port invoking validate_base.ts
.claude/skills/project-check/tests/check_entity_bases.test.ts	new	bun:test replacing run_base_evals.py
.claude/skills/project-check/check_entity_bases.py	delete	replaced
.claude/skills/project-check/tests/run_base_evals.py	delete	replaced
.claude/skills/project-check/check_skill_docs.ts	new	TS port
.claude/skills/project-check/tests/check_skill_docs.test.ts	new	bun:test replacing run_skill_doc_evals.py
.claude/skills/project-check/check_skill_docs.py	delete	replaced
.claude/skills/project-check/tests/run_skill_doc_evals.py	delete	replaced (no longer serves any .py checker)
.claude/scripts/audit_skill_runtime.ts	new	TS port
.claude/scripts/tests/audit_skill_runtime/audit_skill_runtime.test.ts	new	bun:test replacing run_evals.py
.claude/scripts/audit_skill_runtime.py	delete	replaced
.claude/scripts/tests/audit_skill_runtime/run_evals.py	delete	replaced
.claude/hooks/warn-skill-doc-drift.ts	new	TS port of the hook
.claude/hooks/warn-skill-doc-drift.py	delete	replaced
.claude/settings.json	modify	repoint the PostToolUse hook to the .ts
sdlc.yaml	modify	drop both .py gates + the parked run_base_evals comment; gates run all-TS
.claude/skills/project-check/SKILL.md	modify	refresh remaining .py→.ts checker citations
lefthook.yml	modify	repoint remaining .py checker refs to .ts

Acceptance criteria

• AC-1: check_entity_bases.ts, check_skill_docs.ts, .claude/scripts/audit_skill_runtime.ts, and .claude/hooks/warn-skill-doc-drift.ts all exist and run under bun; each checker exits 0 on the live repo. The four corresponding .py files (+ the three Python eval harnesses) no longer exist.

• AC-2:

  grep -rn "check_entity_bases.py\|check_skill_docs.py\|audit_skill_runtime.py\|warn-skill-doc-drift.py\|run_base_evals.py\|run_skill_doc_evals.py" .claude lefthook.yml sdlc.yaml

  returns no matches.

• AC-3: bun test ./.claude passes and discovers the new check_entity_bases.test.ts, check_skill_docs.test.ts, and audit_skill_runtime.test.ts suites.

• AC-4: sdlc.yaml’s quality_checks: lists zero .py entries; bun run plugin/scripts/run_quality_checks.ts --config sdlc.yaml --line runs the all-TS gate set; the previously-parked check_entity_bases coverage is back online (via bun test ./.claude).

• AC-5: the warn-skill-doc-drift PostToolUse hook in .claude/settings.json points at the .ts and still fires on Edit/Write.

Out of scope

• The sample-script fixtures under .claude/scripts/tests/audit_skill_runtime/fixtures/**/*.py — they are intentional test data (the audit’s subject), not migration targets.
• plugin/ migration — already complete.

Dependencies

• none

Discovery context

• Completes the .claude/→TypeScript migration begun by the three Tier-1/Tier-2 dogfood ports (check_pipe_tail, check_skill_prose, check_entities). Folds in the SKILL.md/lefthook citation-refresh sweep those tasks deferred to a single follow-up, and restores the parked run_base_evals gate.

Post-mortem

Captured by /sdlc:task-work on 2026-06-02. PR: pending.

Acceptance criteria coverage

• AC-1: agent-manual (partial) — all four .ts checkers exist and run under bun; the four .py + three Python eval harnesses are deleted (verified by ls). Exit-0-on-live-repo holds for check_entity_bases.ts, audit_skill_runtime.ts, and the warn-skill-doc-drift.ts hook. check_skill_docs.ts exits 1 on the live repo, but this is pre-existing drift the faithful port correctly surfaces — 3 skills (pr-respond, spawn-task-pr, standard-new) genuinely lack their docs/skills/<slug>.md companion docs. The TS port produces byte-identical output to the Python original (verified by diff), so the migration introduced nothing; generating the 3 missing docs is /update-skill-doc work, out of scope for a Python→TS port. Surfaced here rather than papered over.
• AC-2: auto — grep -rn "<six .py filenames>" .claude lefthook.yml sdlc.yaml returns no matches (verified).
• AC-3: auto — bun test ./.claude passes (36 tests / 6 files) and discovers the three new suites (check_entity_bases.test.ts, check_skill_docs.test.ts, audit_skill_runtime.test.ts).
• AC-4: auto — sdlc.yaml’s quality_checks: lists zero .py entries (verified by YAML parse); the previously-parked check_entity_bases coverage is back online via bun test ./.claude.
• AC-5: agent-manual — the warn-skill-doc-drift PostToolUse hook in .claude/settings.json points at bun run …/warn-skill-doc-drift.ts; the hook fires on Edit/Write with byte-identical warning output to the Python original (verified by piping synthetic payloads through both).

What worked

• The three already-merged ports (check_pipe_tail.ts, check_skill_prose.ts, check_entities.ts) were an excellent template — bun idioms, pyResolve, splitlines/lineOf, Bun.spawnSync, and the bun:test fixture shape all transferred cleanly.
• Byte-for-byte diff against the still-present Python originals (before deleting them) gave high-confidence fidelity checks on every port.
• The baseline-diff gate correctly classified the bulk of the corpus drift as pre-existing (466 lines), keeping the real signal small.

Friction and automation gaps

• */ inside a JSDoc block comment (plugin/skills/*/SKILL.md, entities/*/base.yaml) silently terminates the /* … */ comment and breaks bun parse — hit it twice. A lint rule (or a port-checklist note) flagging */ inside .ts block comments would catch this before the first run.
• run_quality_checks.ts --diff-against-baseline produced 17 false-positive new-drift lines: 15 from non-deterministic bun test fixture output (ephemeral /tmp/claude-502/… paths + transient commit SHAs) and 2 from audit_entities.ts corpus-count summary lines that shift run-to-run. The baseline diff is exact-line-text based, so non-deterministic stdout defeats it. A normalization pass (mask tmpdir paths / SHAs / count summaries) before diffing would make the gate trustworthy on suites with non-deterministic output. → 2026-06-02-normalize-baseline-diff-nondeterministic-output
• The Step 3a baseline is written to the main repo .sdlc/quality-baselines/, but Step 7’s executor (run from the worktree) defaults --baseline-dir to the worktree’s .sdlc/ and reports baseline not found. Task-work Step 7 should pass --baseline-dir <main-repo>/.sdlc/quality-baselines explicitly, or the executor should resolve the superproject root.
• audit_skill_runtime’s invocation regex matches only .py/.sh, but the plugin/ migration moved all 100 real SKILL.md script citations to .ts — so the auditor now audits zero real flag-bearing scripts (only 2 advisory .sh). The faithful port preserves this, but the auditor is effectively dormant. Extending it to match .ts (introspecting via bun run <s> --help) is a worthwhile follow-up — deliberately deferred here to keep this a pure migration and avoid introducing live-repo gate failures. → 2026-06-02-audit-skill-runtime-match-ts-citations
• check_skill_docs / setup_planning.ts writeBases both glob base.yaml indiscriminately, conflating Obsidian-Bases docs (task/backlog) with S0005 scaffolder default-frontmatter (decision/milestone/principle/standard) that reuses the filename. check_entity_bases.ts works around it by content-sniffing for Bases top-level keys; setup_planning.ts’s writeBases still has the latent bug (would copy the 4 frontmatter files as broken .base docs into a consumer’s vault on /sdlc:setup --obsidian). A follow-up should either rename the scaffolder file or teach writeBases the same discrimination.

Spawned follow-up tasks

• 2026-06-02-normalize-baseline-diff-nondeterministic-output (https://github.com/sksizer/dev/pull/214) — spawned (Upstream-plugin / sdlc-meta): normalize non-deterministic verb output before baseline diff.
• 2026-06-02-audit-skill-runtime-match-ts-citations (https://github.com/sksizer/dev/pull/215) — spawned (Upstream-plugin / sdlc-meta): extend audit_skill_runtime to match .ts script citations.
• Remaining friction bullets (the */-in-block-comment lint, Step-3a baseline-dir resolution — overlaps with #214’s run_quality_checks scope, and the setup_planning.ts writeBases Bases-vs-frontmatter discrimination) are captured above for a reviewer to triage; not spawned as separate PRs to keep this run bounded.

---

← Back to Tasks