T-SDB5-audit-entities-baseline-allow

Status: closed/superseded · Impact: medium · Complexity: medium

Auto-generated from a /sdlc:task-work post-mortem. Review and promote to open/ready before picking up.

Goal

plugin/scripts/audit_entities.py runs as a project-level quality check in sdlc.yaml, so every /sdlc:task-work Step 7 invocation includes it. Today it FAILs the whole gate the moment ANY task file has drift — including drift on files unrelated to the current PR. The result is that every PR appears to fail the quality-check gate even when its diff is clean, forcing the operator to inspect the runner output by hand to confirm the drift is pre-existing. Cite T-N2X3-start-task-preserves-readiness-stamp for the live example: 8 unrelated task files showed drift; the gate failed; this PR’s actual diff was clean.

Today

audit_entities.py exits non-zero whenever any audited file under docs/planning/<entity>/ triggers a drift rule. The runner does not distinguish drift that existed before the PR’s first commit from drift the PR introduced. Pre-existing drift includes mechanically unfixable cases like the YAML parse error currently sitting in docs/planning/tasks/2026-05-21-pr-check-schema-promote.md.

plugin/scripts/run_quality_checks.py propagates the audit’s non-zero exit as a FAIL 1/N line, which the task-work skill documents as a hard gate (“do not proceed to the PR until the executor reports OK”). Operators currently work around it by visually verifying the drift is unrelated to their diff and proceeding anyway, which silently erodes the gate.

Proposed

audit_entities.py (or a wrapping invocation) reports two tiers:

Hard fail — drift on files the current PR touches (computed from git diff --name-only origin/main...HEAD or equivalent). Exit non-zero, gate fails, operator must fix.
Warning — drift on files the current PR does NOT touch. Print the list, exit zero. The drift is real but not this PR’s responsibility; an /sdlc:entities-migrate sweep is the right follow-up.

A clean PR against a drifty repo passes the gate. A PR that introduces new drift fails.

Approach

Decide where the “baseline” comes from: git merge-base origin/main HEAD is the natural anchor — drift present in that commit is pre-existing, drift only in the working tree or in PR-only commits is introduced. Other options (a checked-in audit_baseline.json, a CI artifact) trade simplicity for robustness.
Teach audit_entities.py to take a --baseline-rev <rev> flag (or a --pr-base shortcut). When set, the script classifies each drifty file as pre-existing vs introduced and exits non-zero only when there’s at least one introduced.
Update sdlc.yaml’s quality_checks: entry to pass the flag, OR wrap the invocation in a small audit_for_pr.sh shim.
Cover with a test fixture in plugin/scripts/tests/ (or a new plugin/skills/.../tests/run_evals.py style suite) that mocks a repo with drift on a non-touched file and asserts exit 0.

Files to touch

Location	Kind	Change
`plugin/scripts/audit_entities.py`	modify	add baseline classification,
`sdlc.yaml`	new	update the quality_checks entry to pass the new flag.
`plugin/scripts/tests/test_audit_entities.py`	new	(new or extended) —

Acceptance criteria

AC-1: Running audit_entities.py --pr-base origin/main against a repo with drift only on files not touched by the current branch exits 0 and prints the drift as warnings.
AC-2: The same invocation against a branch that introduces drift on a file the branch touches exits non-zero.
AC-3: sdlc.yaml’s quality_checks: invocation is updated so /sdlc:task-work Step 7 stops failing on pre-existing drift.

Out of scope

Auto-fixing the drift. /sdlc:entities-migrate already covers mechanical fixes; this task only changes the gate’s semantics.
Triaging the 8 currently-drifty files. That’s a separate sweep.

Dependencies

none

Discovery context

Spawned by /sdlc:task-work post-mortem of T-N2X3-start-task-preserves-readiness-stamp on 2026-05-21.

Dedup search (spawn-from-post-mortem)

Bullet: audit_entities.py quality check reports pre-existing drift on 8 unrelated task files including a YAML parse error. The runner returns FAIL for any drift so every PR appears to fail the gate even when its diff is clean. Either the audit needs a baseline-allow mechanism that only fails on new drift or the offending files need a triage sweep or the gate needs to drop into a warning tier for pre-existing entries. Keywords searched: audit_entities, baseline-allow, pre-existing, unrelated, including, mechanism, offending, quality Excluded: 2026-05-21-start-task-preserves-readiness-stamp Top candidates (score / status / headline):

33 / closed/done / 2026-05-19-task-work-uses-per-project-quality-checks — Make /sdlc:task-work quality-check commands per-project configurable
21 / closed/done / 2026-05-19-implement-entities-migrate — Implement /sdlc:entities-migrate to apply mechanical schema-drift fixes
15 / closed/done / 2026-05-21-project-local-skill-extension-mechanism — Project-local skill extension / shadowing mechanism
11 / closed/done / 2026-05-19-co-locate-skill-specific-scripts — Co-locate skill-specific scripts under their skill dir; promote when shared
10 / closed/done / 2026-05-19-add-epic-entity-task-depends-on-dependencies — Add epic entity + task depends_on dependencies Decision: SPAWNED

Dedup search (spawn-from-post-mortem)

Bullet: audit_entities.py exits non-zero on pre-existing closed-task drift, blocking the quality_checks runner which short-circuits on first failure. Auto-fix closed-task drift or let audit_entities exit 0 when only closed/* entities drift. Keywords searched: audit_entities, quality_checks, short-circuits, pre-existing, closed-task, non-zero, blocking, auto-fix Excluded: 2026-05-20-task-branches-use-task-prefix Top candidates (score / status / headline):

29 / planning/draft / 2026-05-21-audit-entities-baseline-allow — audit_entities.py: distinguish pre-existing drift from PR-introduced drift
21 / closed/done / 2026-05-19-task-work-uses-per-project-quality-checks — Make /sdlc:task-work quality-check commands per-project configurable
18 / closed/done / 2026-05-19-implement-entities-migrate — Implement /sdlc:entities-migrate to apply mechanical schema-drift fixes
11 / closed/done / 2026-05-20-sdlc-yaml-json-schema-and-validator — Add JSON Schema + validator for sdlc.yaml
10 / planning/draft / 2026-05-21-run-quality-checks-isolates-pre-existing-drift — run_quality_checks.py only fails on drift the current branch introduced Decision: LINKED-EXISTING 2026-05-21-audit-entities-baseline-allow

← Back to Tasks