T-3A6G-implement-entities-migrate

Status: closed/done · Impact: high · Complexity: medium

Goal

Close the audit→migrate loop. /sdlc:entities-audit reports drift in entity instances and tags each entry auto_fixable=true|false; /sdlc:entities-migrate is the destructive counterpart that actually applies the auto_fixable=true items. It’s referenced as planned in three places (setup/SKILL.md Step 4, entities-audit/SKILL.md description, audit_entities.py markdown footer) but doesn’t exist yet — every audit finishes with a recommendation to run a skill that isn’t there.

Today

plugin/scripts/audit_entities.py is the source of truth for what counts as drift. It walks docs/planning/<plural>/ for each entity type, validates frontmatter against plugin/entities/<type>/schema.json, checks schema_version, and reports H2 sections present in the template but missing in the instance. Each drift entry carries auto_fixable: bool. JSON output is supported (--json).
plugin/skills/entities-audit/SKILL.md wraps the audit (read-only). Its description says: “The destructive counterpart is /sdlc:entities-migrate (planned).”
plugin/skills/setup/SKILL.md Step 4 invokes audit_entities.py and says: “wait for /sdlc:entities-migrate to apply mechanical fixes.” No such skill or script exists.
audit_entities.py:291 ends its markdown output with: “Run /sdlc:entities-migrate to apply mechanical fixes (when implemented).”
The only drift the audit currently marks auto_fixable=true is missing schema_version (audit_entities.py:215-221). Everything else — older schema_version (needs real per-version migration), frontmatter validation errors, prose H2 gaps — is auto_fixable=false.
Closed entities are not currently distinguished by the audit. Both task and milestone schemas have closed/* enum values (plus legacy bare-closed forms for tasks). The audit walks every file and reports drift uniformly. There’s an archive/ subdir convention (audit skips it by default — audit_entities.py:181) but no automatic move, and closed-but-not-archived files get treated like active ones.
The pattern for write-then-PR workflow already exists in plugin/skills/task-work/SKILL.md (worktree → commits → PR via gh). The pattern for fixture-based eval suites already exists in .claude/skills/project-check/tests/ (PEP-723 runner + per-case fixture dirs).

Proposed

/sdlc:entities-migrate exists as a plugin skill. Invoking it:

Spins up a worktree (mirrors task-work).
Runs audit_entities.py --json to enumerate drift.
For each drift entry with auto_fixable=true, applies a deterministic fix to the offending file. Today that means: stamp the schema’s current version into schema_version: when missing, preserving frontmatter key order.
Leaves every auto_fixable=false drift untouched — collects them for the PR body.
Commits the changes (one commit per entity type), opens a PR titled chore(entities): apply auto-fixable schema drift, with a body that lists what changed and what’s left for human review.
Reports the PR URL and exits.

The destructive script (migrate_entities.py) is reusable on its own — supports --dry-run, refuses to act on a dirty working tree unless --allow-dirty, idempotent (running twice produces no diff).

Closed-entity policy: migrate skips closed entities by default. A closed entity represents what was true when it shipped; retroactively stamping new fields rewrites history. Override with --include-closed for the rare case it’s actually wanted. audit_entities.py is extended to emit a closed: bool per file in its JSON output so migrate’s filter is mechanical (no re-reading of frontmatter).

audit_entities.py retains sole authority over what’s auto_fixable; migrate doesn’t invent new fix logic. To expand mechanical-fix scope later: flip auto_fixable=true in the audit and add the corresponding fix branch in migrate.

Approach

Extend audit_entities.py to emit closed: bool per file. In FileReport, add a closed field. Compute it in audit_file() from the file’s frontmatter status. Closed-detection helper: status starts with closed/ OR matches the legacy bare-closed enum values (closed, done, superseded, partially-superseded, relocated-upstream, investigated-no-repro). Include closed in the JSON render and in the markdown render’s per-file line (e.g. [closed] suffix). No behavior change to the audit’s exit code or drift list — closed entities still get their drift reported.
Add plugin/scripts/migrate_entities.py (new). PEP-723 self-bootstrapping, same shebang style as the other scripts. Args: --project-root, --entities-dir, --dry-run, --allow-dirty, --include-closed, --type <name> (repeatable). Behavior:
- Refuse if working tree is dirty (unless --allow-dirty). Use git status --porcelain to check.
- Shell out to audit_entities.py --json against the same --project-root/--entities-dir. Parse the JSON.
- Filter out closed entities by default. Skip any file with closed: true unless --include-closed was passed. Closed files are reported in the summary as “skipped (closed)” so the user knows they were considered.
- For each remaining file, for each drift with auto_fixable=true, dispatch on drift.kind:
  - schema_version: load file frontmatter, set schema_version: <entity.schema_version>, re-serialize preserving key order, write back. Body unchanged.
  - (No other kinds today; leave the dispatch table extensible.)
- In --dry-run, print the planned changes but don’t write.
- Print a summary at the end: N files fixed, M files left with manual drift, K files skipped (closed).
- Exit codes: 0 if no fixes needed or all applied cleanly; 1 if a fix attempt failed; 2 on script error or dirty tree.
- Idempotency: re-running on the same files after a successful fix should report 0 fixes (audit returns no auto-fixable drift).
Add plugin/skills/entities-migrate/SKILL.md (new). Allowed-tools: Bash, Read, Edit, Glob, Grep, Agent. Structure (in order):
- Pre-flight: confirm working tree clean (against the main repo, not just current worktree).
- Worktree: git worktree add .claude/worktrees/entities-migrate -b chore/entities-migrate main. Initialize per the same pattern as task-work Step 4.
- Run audit first to surface what’s about to change. If zero auto_fixable=true items, report nothing to migrate and exit (no PR).
- Invoke migrate_entities.py with --project-root <worktree> inside the worktree.
- Stage and commit: chore(entities): apply auto-fixable schema drift. One commit covering all entities (not split by type), one PR.
- Open PR via gh pr create. Body: summary, list of files changed and what was applied, plus the list of auto_fixable=false items left for human review.
- Same close-out and worktree-teardown story as task-work Steps 10a–10c.
Update cross-references to remove “(planned)” / “(when implemented)” hedges:
- plugin/skills/entities-audit/SKILL.md description.
- plugin/skills/setup/SKILL.md Step 4.
- plugin/scripts/audit_entities.py:291 (the markdown footer line).
Eval fixtures at plugin/skills/entities-migrate/tests/ (mirrors .claude/skills/project-check/tests/):
- fixtures/missing-schema-version/ — active entity instance missing the field. Expected: script stamps the field, exit 0, idempotent on second run.
- fixtures/older-schema-version/ — instance has schema_version: 0 while current schema says 1. Expected: script does NOT touch the file (audit marks it auto_fixable=false); migrate’s summary lists it as manual.
- fixtures/closed-with-drift/ — closed entity instance missing schema_version. Expected: script does NOT touch the file by default; summary reports it as “skipped (closed)”. With --include-closed, it IS fixed.
- fixtures/clean/ — instance already at current version. Expected: no fixes, exit 0.
- fixtures/dirty-tree/ — fixture where the working tree is dirty. Expected: script refuses with exit 2 (unless --allow-dirty).
- run_evals.py — invokes migrate_entities.py against each fixture, asserts on exit code, file content delta, and summary substring.
Smoke test end-to-end: invoke /sdlc:entities-migrate on a synthetic dirty repo. Confirm it audits, applies, commits, opens PR. Then re-invoke on the now-clean repo — confirms idempotency.

Files to touch

New:

plugin/scripts/migrate_entities.py — the destructive script (PEP-723).
plugin/skills/entities-migrate/SKILL.md — the orchestrator skill.
plugin/skills/entities-migrate/tests/run_evals.py — eval runner (mirrors .claude/skills/project-check/tests/run_evals.py).
plugin/skills/entities-migrate/tests/fixtures/missing-schema-version/ — fixture: instance missing the field.
plugin/skills/entities-migrate/tests/fixtures/older-schema-version/ — fixture: instance with stale version.
plugin/skills/entities-migrate/tests/fixtures/clean/ — fixture: instance already correct.
plugin/skills/entities-migrate/tests/fixtures/dirty-tree/ — fixture exercising the dirty-tree guard.
plugin/skills/entities-migrate/tests/README.md — how to run and extend.

Modified:

plugin/scripts/audit_entities.py — (a) emit closed: bool per file in both JSON and markdown output; add a _is_closed_status() helper. (b) Drop “(when implemented)” from the markdown footer (audit_entities.py:291).
plugin/skills/entities-audit/SKILL.md — change “(planned)” to “(see also)” or similar.
plugin/skills/setup/SKILL.md — Step 4 text: replace the “wait for /sdlc:entities-migrate” hedge with concrete usage guidance.

Not modified:

plugin/entities/<type>/schema.json — schemas are inputs, not outputs. The schema’s version field already exists.
Any actual docs/planning/<plural>/*.md instance — those get fixed by the script, not hand-edited as part of this task.

Acceptance criteria

Out of scope

Per-version migration logic. Bumping an instance’s schema_version from N to N+1 typically requires field-shape changes (e.g. renaming a key, splitting a string into a list). Those belong in versioned migration scripts (e.g. entities/task/migrations/v1-to-v2.py), invoked on demand. This task only handles the “missing schema_version” case the audit currently marks auto-fixable.
Auto-inserting missing template H2 sections. Body insertions are content decisions — what to write inside the new section is the author’s call. Leave to /sdlc:task-define (for tasks) or to humans (for other entities). The audit will continue to flag these as auto_fixable=false.
Auto-correcting frontmatter validation errors. Wrong enum values, missing required fields with no obvious default, type mismatches — all need judgment. Not mechanical.
A /sdlc:entities-bulk-migrate-style cross-repo runner. This skill operates on the current repo only.
Touching the version field on schema.json itself. Schemas evolve manually; this task is about bringing instances into sync with whatever schema version is current.
Auto-moving closed-with-drift entities into archive/. The audit already skips archive/ by default and manual moves are fine until volume justifies automation. A future task can add --archive-stale-closed to migrate (or a separate /sdlc:entities-archive skill) when the friction is real.
A schema-declared terminal_statuses list. Today closed-detection is a hardcoded pattern check (^closed/ plus the legacy bare-closed enum values). If a future entity type uses a different convention, that’s the moment to add an explicit declaration. Not now.

Dependencies

audit_entities.py exists and exposes stable JSON output with auto_fixable: bool per drift entry — ✓ already in place.
The worktree + PR pattern from /sdlc:task-work — ✓ already in place; this skill will follow that template.
No upstream tasks block this.

Discovery context

Surfaced while reviewing the entity-consistency story after PR #1 (feat/project-check-skill). The audit→migrate split is the natural pair (audit reads, migrate writes), and the audit already produces structured output (auto_fixable: bool) designed to be consumed by a writer. The architecture is in place; this task is the missing implementation.

Minimum-useful scope is the schema_version stamping case (the only auto_fixable=true drift today). Expansion to other auto-fixable cases is a config change in audit_entities.py plus a new dispatch arm in migrate_entities.py — out of scope here but cheap once the skeleton exists.

Post-mortem

Captured by /sdlc:task-work on 2026-05-19. PR: #11.

Acceptance criteria coverage

AC-1: agent-manual — verified plugin/skills/entities-migrate/SKILL.md exists and is picked up by plugin.json’s skills: ./skills glob. A /reload-plugins is needed for the live skill list; left to the merge step.
AC-2: auto — tests/run_evals.py case missing-schema-version; agent-manual smoke against a tmp copy of the live docs/planning/.
AC-3: agent-manual — encoded in SKILL.md Step 2 (“zero auto_fixable=true items → report nothing to migrate and exit”). Script side covered by the clean eval.
AC-4: auto — missing-schema-version-dry-run eval asserts dry-run announces but does not write; the same fixture’s non-dry-run pass confirms equivalence.
AC-5: auto — dirty-tree eval.
AC-6: auto — missing-schema-version eval’s second-run check (commits between runs to mirror real orchestration); agent-manual second-invocation smoke also reported fixed: 0.
AC-7: auto — older-schema-version eval leaves the file untouched; PR-body “Manual review needed” templated in SKILL.md.
AC-8: auto — closed-with-drift and closed-with-drift-included evals; agent-manual smoke for legacy bare-closed (status: superseded).
AC-9: agent-manual — verified JSON has closed: bool peer of path/type; markdown shows [closed] suffix.
AC-10: auto — 7/7 eval cases pass (spec asked for 5; the 2 extras re-use fixtures and exercise AC-4 / --include-closed half of AC-8).
AC-11: agent-manual — grepped all three references; “(planned)” / “(when implemented)” hedges removed.
AC-12: auto — /project-check’s check_entities.py clean against plugin/entities/; project-check eval suite still 7/7.

What worked

Task spec was concrete enough that a single sub-agent implemented end-to-end without follow-up clarifying questions. The “Approach” section’s numbered steps mapped 1:1 to commits.
The audit’s auto_fixable contract held — migrate’s dispatch is a single if drift.kind == "schema_version" arm with room to grow. No new fix logic invented.
The .claude/skills/project-check/tests/ fixture pattern transferred cleanly to plugin/skills/entities-migrate/tests/. Same shape, same shebang style, same runner ergonomics.
Closed-skip default validated against the user’s actual pain point (rust-ontogen’s ~200 closed records) before any code was written.

Friction and automation gaps

task-ensure-ready can’t be re-run from inside task-work after Step 3 — Step 3 sets status: in-progress, but ensure-ready’s contract requires status in {draft, proposed, backlog, ready}. Running it would corrupt the verified state. Orchestrator had to skip Step 5. Fix: either task-ensure-ready should tolerate in-progress when a recent readiness_verified_at is present, or task-work should run ensure-ready BEFORE Step 3.
Sub-agent jumped ahead and opened PR #11 before the orchestrator wrote the post-mortem (Step 8) and ran sync (Step 9). Fix: the sub-agent brief in task-work Step 6 should explicitly say “do not push, do not open a PR — leave the feat branch with a clean committed state; the orchestrator handles Steps 9 and 10.”
/tmp/commit-msg.txt collision — orchestrator used this generic temp path for a git commit -F and a stale file from a previous session was present, so the wrong commit message landed on the start commit (had to --amend). Fix: never use predictable /tmp/<generic>.txt paths; use mktemp or a unique PID/UUID-prefixed name.
git pull --ff-only origin main while on a non-main branch fast-forwarded the wrong branch — orchestrator was on docs/draft-entities-migrate-task (a stale, just-merged branch) and the pull updated that branch to the merge commit while local main stayed behind. Only caught when an Edit failed because the task file wasn’t on local main yet. Fix: task-work Step 3 should git checkout main first and verify branch before any git pull.
Lingering branch from prior task-definition PR — docs/draft-entities-migrate-task was still checked out after the user merged PR #9 on GitHub; nothing in the workflow signaled “your local branch is stale.” Fix: add a pre-flight check in task-work Step 2 that fails fast (or auto-checks-out-main) if HEAD is on a branch already merged into origin/main.
task-work’s Step 4 worktree init (mise trust && just setup-worktree) and Step 7’s quality checks (just full-check, just ci) are templated from a different project that has just; this repo has neither. Orchestrator substituted check_entities.py + the two eval runners. Fix: task-work should declare its quality-check verbs in a per-project config (e.g. .claude/sdlc-config.yaml) or probe for Justfile and substitute.
Compound bash commands chained with && and echo got misparsed under fish — multiple commands had to be re-run as separate Bash calls. Fix: task-work guidance / examples should prefer single-command Bash calls or explicit bash -c '...' wrapping when chaining is needed.

Notes for next run

Sub-agent shipped 7 fixtures vs the 5 named in AC-10 (two re-use existing fixtures with different invocations). The implementation satisfied the AC by passing all five named cases; the extras were additive. AC could read “passes against at least the named five fixtures” rather than implying an exact count.
Sub-agent added a --audit-script flag to migrate_entities.py (not in the task spec). It made the eval runner testable without relative-path assumptions; reasonable scope expansion and called out in the sub-agent’s report.

← Back to Tasks