Skip to content
sdlc

T-S99C-project-check-asserts-shipped-templates-validate

Status: closed/done · Impact: medium · Complexity: small

Auto-generated from a /sdlc:task-work post-mortem. Review and promote to open/ready before picking up.

Goal

Section titled “Goal”

plugin/entities/<type>/template.md files now satisfy the standard-markdown-frontmatter contract (per T-3K8K-templates-html-comment-after-frontmatter) and also validate clean against their own schemas. There is no automated guard preventing that invariant from regressing. Extending plugin/skills/project-check/check_entities.py to assert “every shipped template validates” closes the gap with a CI-visible signal rather than relying on agents remembering to spot-check.

Today

Section titled “Today”

.claude/skills/project-check/check_entities.py (project-local; the project-check skill is NOT under plugin/skills/ — it lives at .claude/skills/project-check/ per its own SKILL.md) walks plugin/entities/<type>/ and confirms artifacts under docs/planning/<plural>/ stay in sync with their schemas. It does not currently validate the templates themselves.

Originating friction bullet:

Approach step 4 (“Add a regression test”) had Files to touch flag the path as “uncertain” — skipped here as out of scope for a small task, but a follow-up task to wire check_entities.py (or a sibling) to also assert “all shipped templates validate clean” would close the AC-2 gap with a CI signal.

Proposed

Section titled “Proposed”

.claude/skills/project-check/check_entities.py iterates every plugin/entities/*/template.md and shells out to plugin/validators/validate_frontmatter.py <template>. A non-zero exit becomes a project-check failure with the same shape as existing drift diagnostics.

Approach

Section titled “Approach”
  1. Add a check_templates_validate() helper inline in .claude/skills/project-check/check_entities.py (no separate file — the check is short enough to live alongside the existing ones; promoting to a dedicated check_templates.py is premature until a second template-shape check appears).
  2. Call it from the script’s main entry point so it runs alongside the existing drift checks.
  3. Surface failures in the same format as today (one line per failing template, plus a tail summary).

Files to touch

Section titled “Files to touch”
  • .claude/skills/project-check/check_entities.py — add inline template-validation helper + main-entry-point call.
  • .claude/skills/project-check/SKILL.md — list the new check in the “What this skill checks today” section.

Acceptance criteria

Section titled “Acceptance criteria”
  • AC-1: Running check_entities.py against the current repo exits 0 and includes a line per template saying it validated.
  • AC-2: Introducing a schema-violating change to any plugin/entities/*/template.md causes the check to exit non-zero and print the offending field.

Out of scope

Section titled “Out of scope”
  • Validating templates that live outside plugin/entities/ (none exist today; can be added when they do).
  • Wiring this check into a separate templates-validate skill — project-check is the right home for it.

Dependencies

Section titled “Dependencies”
  • none

Discovery context

Section titled “Discovery context”

Spawned by /sdlc:task-work post-mortem of T-3K8K-templates-html-comment-after-frontmatter on 2026-05-20.

Post-mortem

Section titled “Post-mortem”

Captured by /sdlc:task-work on 2026-05-21. PR: pending.

Acceptance criteria coverage

Section titled “Acceptance criteria coverage”
  • AC-1: auto — ran .claude/skills/project-check/check_entities.py against the live repo; exit 0 and four template validates: lines (one per shipped template) confirmed.
  • AC-2: auto — verified manually by injecting status: bogus-stage into plugin/entities/task/template.md and re-running the check (exit 1 with the offending field cited). Backed permanently by the new fixture tests/fixtures/bad-template-value/ registered in run_evals.py, which asserts exit 1 plus schema validation failed
    • bogus-stage in the output.

What worked

Section titled “What worked”
  • The existing check_one_entity() already iterated shipped *.md files and accepted --entities-dir, so plumbing the new check through the same signature was a one-line addition at the call site plus a helper function — no architectural change.
  • Shelling out to plugin/validators/validate_frontmatter.py (rather than re-implementing schema validation inline) meant the new check picks up every present and future validator improvement for free, and Check 5’s diagnostics share the exact wording the validator uses everywhere else.
  • The eval-fixture pattern (one dir per case + a CASES table in run_evals.py) made adding two new regression fixtures a four-file mechanical edit.

Friction and automation gaps

Section titled “Friction and automation gaps”
  • sdlc.yaml carries a pr_check: block that plugin/schemas/sdlc-yaml.schema.json does not whitelist — run_quality_checks.py exits 2 (“Additional properties are not allowed”) on every invocation, including Step 4’s worktree_init call (with --allow-empty) and Step 7’s quality-checks call. Worked around manually here by running the relevant per-suite eval scripts directly, but every parallel /sdlc:task-work run hits the same wall. Either pr_check should land in the schema or sdlc.yaml should drop the block until the schema catches up — follow-up task material. → T-DN76-pr-check-schema-promote
  • The fixture for the existing clean case has no template.md, so the new Check 5 silent-skip path (return [] when no template is present) is hit by the existing case but never asserted to be silent. Adding an explicit “no-template” fixture would pin that behaviour; deferred as low-value given the silent-skip is one line.

Spawned follow-up tasks

Section titled “Spawned follow-up tasks”

← Back to Tasks