Skip to content
sdlc

T-5CVT-project-check-asserts-skill-docs-match-skill-md

Status: closed/superseded · Impact: high · Complexity: small

Auto-generated from a /sdlc:task-work post-mortem. Review and promote to ready before picking up.

Goal

Section titled “Goal”

The skill-doc drift warning hook is per-edit-event and advisory — it fires when an author edits a SKILL.md without touching the matching doc, but it cannot block, and it doesn’t run on already-merged code. Today, nothing asserts at project-check time that docs/skills/<slug>.md exists for every skill in plugin/skills/ OR that each doc’s Mermaid flowchart actually references every numbered step in the corresponding SKILL.md. A deterministic check would convert the soft hook into a hard gate.

Today

Section titled “Today”
  • .claude/hooks/warn-skill-doc-drift.py (added by T-H9Q4-document-every-skill-with-mermaid-flowchart) only fires post-Edit on the editing author’s machine.
  • .claude/skills/project-check/check_skill_prose.py runs the prose-invariant linter from PR #19 across every SKILL.md, but it doesn’t check the matching doc at all.
  • .claude/skills/project-check/check_entities.py validates entity schemas and frontmatter shape — same pattern as the proposed check (deterministic Python, <path>:<line>: <message> citation, exit codes 0/1/2).
  • 15 skills today have matching docs because they were hand-authored; the next skill added without a doc would only get caught when someone tries to edit its SKILL.md.

Proposed

Section titled “Proposed”

A new .claude/skills/project-check/check_skill_docs.py that:

  1. Walks plugin/skills/*/SKILL.md and confirms a matching docs/skills/<slug>.md exists for each.
  2. Parses each SKILL.md’s ## N. ... numbered steps.
  3. Parses each doc’s ```mermaid block and confirms each step number appears in the flowchart (best-effort — looking for [<N>. or the S<N> node-id convention from plugin/skills/README.md).
  4. Exits 0 / 1 / 2 in the same shape as the other project-check scripts.

The check is wired into .claude/skills/project-check/SKILL.md so running /project-check exercises it alongside the existing check_entities.py and check_skill_prose.py.

Approach

Section titled “Approach”
  1. Steal the structural shape from check_entities.py — same PEP-723 header, same <path>:<line>: <message> citation idiom.
  2. Implement the three checks (doc-exists, step-count-match, step-cited-in-flowchart). Keep the flowchart parser conservative — extract the ```mermaid block as a string and substring-match step numbers; do NOT parse Mermaid semantically.
  3. Add a tests/check_skill_docs/ fixture set under .claude/skills/project-check/tests/ exercising each failure mode (missing doc, step count mismatch, step number absent from flowchart, clean).
  4. Wire the runner into the project-check entry point.

Files to touch

Section titled “Files to touch”
  • .claude/skills/project-check/check_skill_docs.py (new) — the check.
  • .claude/skills/project-check/SKILL.md — add a Step that runs the new check.
  • .claude/skills/project-check/tests/skill-doc-fixtures/... (new) — fixture trees.
  • .claude/skills/project-check/tests/run_skill_doc_evals.py (new) — runner.

Acceptance criteria

Section titled “Acceptance criteria”
  • AC-1: .claude/skills/project-check/check_skill_docs.py exits 0 against the current corpus (every skill has a doc whose flowchart cites every step).
  • AC-2: A fixture with a missing docs/skills/<slug>.md makes the script exit 1 with a path and the missing-doc message.
  • AC-3: A fixture where the doc’s Mermaid block omits step N makes the script exit 1 with a path:line citation naming the missing step.
  • AC-4: /project-check’s SKILL.md runs the new check, and a fresh /project-check invocation surfaces failures from the new check verbatim.

Out of scope

Section titled “Out of scope”

Dependencies

Section titled “Dependencies”
  • none

Discovery context

Section titled “Discovery context”

Spawned by /sdlc:task-work post-mortem of T-AV6J-add-update-skill-doc-skill on 2026-05-19. The warning hook only fires at edit time; this task turns “every skill has a doc that names its steps” into a hard project-check gate.

← Back to Tasks