T-4ZZX-audit-skill-runtime-match-ts-citations

Status: closed/superseded · Impact: medium · Complexity: medium

Goal

audit_skill_runtime is the surface-area check that catches drift between what a SKILL.md documents and what its cited script actually exposes via --help. After the harness TS migration, every real SKILL.md script citation became a .ts invocation — but the auditor’s invocation regex still matches only .py/.sh. There are no .py citations left in the plugin, so the auditor now audits zero of the ~100 real .ts script citations (only 2 advisory .sh references survive, and .sh is existence-checked, not introspected). This task extends the auditor to match .ts and introspect those scripts’ flags via bun run <script> --help, restoring the check’s intent.

Today

Location	Role today
.claude/scripts/audit_skill_runtime.ts	The shipped TS auditor (the .py original is gone). SCRIPT_INVOCATION_RE matches only `….(?:py
.claude/scripts/tests/audit_skill_runtime/fixtures/	All fixtures are .py-based (widget.py, multiflag.py, detect_quality_runners.py, committer.py) — there is no .ts fixture exercising the new path.
plugin/skills/*/SKILL.md	Carry ~100 ${CLAUDE_PLUGIN_ROOT}…\.ts script citations the auditor currently ignores, plus 2 .sh citations it only existence-checks.

Proposed

The auditor extracts .ts script citations from SKILL.md files and flag-introspects them, so the ~100 real .ts citations are audited for broken-reference and undocumented-flag drift exactly as .py scripts were before the migration.

• SCRIPT_INVOCATION_RE matches .py, .sh, and .ts invocations.
• queryScriptFlags introspects .ts scripts via bun run <script> --help (not bare shebang exec), parses the advertised flags from the help output with the existing FLAG_RE path, and keeps the .sh existence-only and .py shebang-exec behaviors intact.
• The test corpus gains .ts fixture counterparts so the .ts extraction and bun run --help introspection paths are exercised, including a clean case and an undocumented-flag drift case.

Approach

1. Extend SCRIPT_INVOCATION_RE in audit_skill_runtime.ts to include ts in the extension alternation ((?:py|sh|ts)).
2. In queryScriptFlags, add a .ts branch that runs bun run <scriptPath> --help (keeping the existing 30s timeout, timeout-signal handling, and non-zero-exit handling), then feeds stdout through the existing flag parser. Leave the .sh existence-only branch and the .py shebang-exec branch unchanged.
3. Refresh the module docstring’s “Matched invocation extensions” note to say .py, .sh, .ts and to describe the bun run … --help path for .ts.
4. Add .ts fixture counterparts under .claude/scripts/tests/audit_skill_runtime/fixtures/ — at minimum a clean .ts script whose --help advertises the flags its demo SKILL.md cites, and a drift fixture whose SKILL.md cites a flag the .ts script does not advertise — and wire them into audit_skill_runtime.test.ts.

Files to touch

Location	Kind	Change
.claude/scripts/audit_skill_runtime.ts	modify	Add ts to SCRIPT_INVOCATION_RE; add the bun run <script> --help introspection branch for .ts in queryScriptFlags; update the module docstring.
.claude/scripts/tests/audit_skill_runtime/audit_skill_runtime.test.ts	modify	Add cases covering .ts citation extraction and bun run --help flag introspection (clean + undocumented-flag drift).
.claude/scripts/tests/audit_skill_runtime/fixtures/	new	New .ts-based fixture trees (clean + drift) mirroring the existing .py fixture layout.

Acceptance criteria

• AC-1: A .ts script cited in a SKILL.md fixture whose --help does not advertise a cited flag is reported as drift (undocumented flag), proving .ts citations are now extracted and introspected.
• AC-2: A .ts script whose --help advertises every cited flag passes clean (no false-positive drift), proving the bun run … --help path parses real flags.
• AC-3: Running the auditor against the live plugin/skills/ tree now introspects the .ts citations (audited count > 0) rather than auditing zero flag-bearing scripts.
• AC-4: The existing .sh existence-only and .py shebang-exec behaviors are unchanged (their fixtures still pass).

Out of scope

• Behavior assertions about what each cited script does (those belong to eval suites); this stays a documentation-vs-interface surface check.
• Auditing script citations outside plugin/skills/<slug>/SKILL.md (e.g. citations in docs or other skill trees).

Dependencies

• none

Discovery context

Post-mortem follow-up from T-79SM-finish-claude-harness-ts-migration (closed/done). The TS port of the auditor deliberately preserved the .py/.sh-only matching to keep that migration faithful; this follow-up restores the auditor’s intent now that the corpus it audits is entirely .ts. Originally drafted on PR #215, which would have landed a schema-invalid, date-prefixed, id-less task referencing the now-deleted .py auditor; this re-mint reframes it to the shipped .ts auditor and supersedes #215.

---

← Back to Tasks