T-CTTD-skill-md-shell-example-lint

Status: closed/obsoleted · Impact: medium · Complexity: small

Auto-generated from a /sdlc:task-work post-mortem. Review and promote to open/ready before picking up.

Goal

plugin/skills/*/SKILL.md files routinely cite shell invocations of co-located scripts (“bun run new_task.ts <slug> --headline ... --tag ...”) as authoritative usage examples. Today nothing keeps those examples honest against the underlying script’s actual flag set — a flag rename or a newly-required positional silently leaves the SKILL.md example stale, and downstream sub-agents copying the example hit “unknown argument” mid-task. A SKILL.md-example linter that extracts shell-shaped lines and dry-runs them against the actual script’s --help output would catch this class of drift at quality-check time rather than at task-execution time. Surfaced by T-H69K-run-quality-checks-isolates-pre-existing-drift: the --project-root positional was missing from a Step 3a example until audit_skill_runtime.py caught the script-reference drift mid-flight and prompted a fixup commit.

Today

Location	Role today
plugin/skills/*/SKILL.md	Cite shell invocations of co-located scripts as authoritative usage examples; no automated check that those invocations are valid
plugin/scripts/lint_skill_prose.ts	Lints forbidden-phrase invariants from a skill’s invariants.yaml; does NOT extract or validate shell command lines
plugin/scripts/audit_skill_runtime.py	Audits script-reference paths (does the cited script exist?); does NOT validate the flag set on each invocation
plugin/scripts/project_check.py	Composes the existing lint suite; would gain the new linter as another rule

The originating post-mortem captured the friction shape: a sub-agent implementing the originating task wrote a Step 3a example as run_quality_checks.py --baseline-dir ... but the actual script required a --project-root positional. audit_skill_runtime.py flagged the drift, the sub-agent shipped a fixup commit (d7ab434), and the run continued — but only because audit_skill_runtime happened to catch script-reference shape, not because anything actually validated the flag list.

Proposed

A new linter at plugin/scripts/lint_skill_md_examples.py that:

1. Scans a passed SKILL.md for lines that look like a shell invocation of a Python or shell script under plugin/scripts/ or plugin/skills/*/ (regex roughly: (\${CLAUDE_PLUGIN_ROOT}|plugin/)(scripts|skills/[a-z-]+)/[a-z_]+\.(py|sh)).
2. For each extracted invocation, calls the script with --help (Python) or parses the script’s argparse block / inspects flag definitions (shell scripts can be --help too, when conventional).
3. Compares the flag set in the SKILL.md example against the script’s declared flag set: every cited --<flag> must exist in the --help output; every positional in the example must match a declared positional.
4. Reports mismatches with line number + suggested fix (“--project-root is required by new_task.ts but missing from this example”). Exit non-zero on any mismatch.

Wire into project_check.py’s skill-doc lint suite so it runs as part of the standing quality-check gate.

Approach

1. Implement plugin/scripts/lint_skill_md_examples.py:
   • Extract shell-shaped lines via regex (multi-line continuations joined first).
   • For each extracted line, locate the script (resolve ${CLAUDE_PLUGIN_ROOT} to the plugin root).
   • Run <script> --help in a subprocess, parse the argparse-shaped output to extract the flag set.
   • Diff cited flags against declared flags; report each mismatch.
2. Add unit tests at plugin/scripts/test_lint_skill_md_examples.py covering: (a) clean SKILL.md passes; (b) renamed flag is caught; (c) missing required positional is caught; (d) extra flag in example is caught.
3. Wire into plugin/scripts/project_check.py’s skill-doc lint suite.
4. Run against the live plugin/skills/*/SKILL.md corpus and fix any surfaced examples (the originating task’s Step 3a invocation already shipped a fix; other examples may carry similar drift).

Files to touch

Location	Kind	Change
plugin/scripts/lint_skill_md_examples.py	new	The linter described above
plugin/scripts/test_lint_skill_md_examples.py	new	Regression tests
plugin/scripts/project_check.py	modify	Register the new lint rule
plugin/skills/*/SKILL.md	modify	Fix any examples surfaced by an initial run (best-effort; large)

Acceptance criteria

• AC-1: plugin/scripts/lint_skill_md_examples.py <SKILL.md> exits 0 when every cited script invocation’s flag set matches the script’s --help output.
• AC-2: The linter exits non-zero with a one-line-per-mismatch report when a cited example references a flag that doesn’t exist in the script’s --help.
• AC-3: The linter is registered in project_check.py’s skill-doc lint suite and runs as part of the standing quality-check gate.
• AC-4: plugin/scripts/test_lint_skill_md_examples.py covers rename / missing-positional / extra-flag cases; all tests pass.

Out of scope

• Validating prose claims about other skills’ shell behaviour. That class of drift is tracked by T-XSI8-task-define-flags-spec-shell-drift at the task-spec layer; this task is exclusively about SKILL.md-internal examples.
• Validating non-Python-non-shell example syntax (e.g. cited gh ... commands). The first cut targets plugin/-owned scripts whose --help we can call deterministically.

Dependencies

• none

Discovery context

Spawned by /sdlc:task-work post-mortem of T-H69K-run-quality-checks-isolates-pre-existing-drift on 2026-05-23.

Dedup search (spawn-from-post-mortem)

Bullet: The —project-root positional was missing from the SKILL.md Step 3a example in the first wave; the implementing sub-agent’s audit_skill_runtime.py caught it as a script-reference drift and prompted a fixup commit. That gap exists because the SKILL.md example was authored without running the actual script first. Worth adding: a SKILL.md-example linter that extracts shell-shaped lines (script + flags) and dry-runs them against the actual script’s —help to confirm the flag set matches. Keywords searched: audit_skill_runtime, script-reference, project-root, shell-shaped, positional, md-example, sub-agent, prompted Excluded: 2026-05-21-run-quality-checks-isolates-pre-existing-drift Top candidates (score / status / headline):

• 32 / planning/draft / 2026-05-23-subagent-invariant-preflight-lint — Run skill-prose invariant linter inline during sub-agent edits to catch forbidden-phrase violations pre-flight
• 22 / closed/done / 2026-05-20-task-work-sub-agent-verdict-contract-clarity — Tighten task-work sub-agent verdict contract so ensure-ready’s READY marker isn’t mistaken for task-work’s final verdict
• 13 / closed/done / 2026-05-21-skill-md-runtime-drift-audit — Audit /sdlc:task-work SKILL.md against per-project runtime conventions
• 12 / planning/draft / 2026-05-21-orchestrated-sub-agent-design-call-gap — Surface design questions before orchestrated /sdlc:task-work commits to an approach
• 10 / closed/done / 2026-05-19-task-work-uses-per-project-quality-checks — Make /sdlc:task-work quality-check commands per-project configurable Decision: SPAWNED Rationale: Override script’s LINKED-EXISTING → subagent-invariant-preflight-lint. That task lints SKILL.md prose against forbidden-phrase invariants (lint_skill_prose.py rule), not against actual script flag sets. The bullet asks for a structural check between cited shell invocations and the cited script’s —help output — a different mechanism (subprocess + argparse parsing) targeting a different class of drift (script-reference flag drift, not prose-phrase invariants). Adjacent tasks 2026-05-21-task-define-flags-spec-shell-drift (task-spec layer) and 2026-05-21-task-work-relevance-verifies-cli-flag-claims (Today-section claims) cover different scopes; no existing task targets SKILL.md example linting itself.

---

← Back to Tasks