T-BCNP-quality-gate-ignores-summary-and-corpus-lines

Status: planning/backlog · Impact: medium · Complexity: small

Auto-generated from a /sdlc:task-work post-mortem. Review and promote to open/ready before picking up.

Goal

sdlc quality run --diff-against-baseline flags audit-summary and corpus-growth lines as “new drift” whenever main’s file counts shift between baseline capture and gate invocation, even when the underlying findings haven’t changed. The v1 isolation (T-H69K-run-quality-checks-isolates-pre-existing-drift) named exactly this limitation: line-level diffing false-positives on chatty verbs’ corpus-growth lines (- OK N/M) and summary counts. Make the gate fail only on findings the current branch introduced, not on baseline-shift noise.

Today

Location	Role today
apps/sdlc/lib/services/quality/baseline.ts (diff)	The drift gate’s diff. Per-verb set difference of current-run findings vs the captured baseline, computed after normalizeFinding masks ephemeral tokens (tmpdir paths, SHAs, ports, PIDs, timings) on both sides. Any line present in current-run but absent from baseline counts as “new drift” and fails the gate.
sdlc quality baseline capture (apps/sdlc/lib/services/quality/ops/baseline/capture.ts)	Captures the baseline output blob per quality-check verb at task pickup (Step 3a). Stores raw stdout; no summary/count normalisation.
sdlc quality run --diff-against-baseline (apps/sdlc/lib/services/quality/ops/run.ts, run-checks.ts)	The gate. apps/sdlc/skills/task-work/SKILL.md Step 7 invokes it and treats a non-zero exit as a blocker.
Quality-check verbs (rumdl, entity audits, …)	Emit lines like - OK N/M, audited N files, Found N issues in X/Y files whose N/M are corpus counts. These shift whenever main grows or shrinks even by one file — producing false-positive “new drift” entries on otherwise-unchanged check runs.

Proposed

Extend normalizeFinding in apps/sdlc/lib/services/quality/baseline.ts to mask/drop summary and N/M-tally lines (- OK N/M, audited N files, Found N issues in X/Y files, \d+ findings) before diffing — the same masking pattern already shipped in that function for tmpdir/SHA (T-TWZD-normalize-baseline-diff-nondeterministic-output) and ports/PIDs/timings (T-BQRU-quality-normalize-ports-pids-timings). One recorded constraint: aggregate summary lines should be dropped outright rather than count-masked — a legitimate count reduction false-flags just as a base-shift increase does.

A per-verb finding extractor was the other candidate shape; it stays future scope. The observed false positives are uniform enough that the existing masking pattern covers them at far lower cost, and D-H7FS-op-substrate-surface §4’s --output contract is the natural home for a structured-findings projection if one is ever needed.

Whatever the exact patterns, the AC is “the gate doesn’t fail on a baseline re-capture from current main.”

Approach

1. From the recorded observations (Discovery context) and the quality-check verbs in this repo’s sdlc.yaml, enumerate the summary/tally patterns that actually trip the gate today (rumdl Found N issues in X/Y files / Run rumdl fmt to fix N, audit - OK N/M / audited N files).
2. Extend normalizeFinding (and, where lines are dropped rather than masked, the diff path that applies it) in apps/sdlc/lib/services/quality/baseline.ts with the summary/tally pass.
3. Add a regression fixture under apps/sdlc/lib/services/quality/tests/: a captured baseline + a “current run” that differs only in a summary count. The gate must exit 0 on that pair.
4. Update apps/sdlc/skills/task-work/SKILL.md Step 7 prose if the wording around drift detection changes.

Files to touch

Location	Kind	Change
apps/sdlc/lib/services/quality/baseline.ts	modify	Extend normalizeFinding (and the diff path) to mask/drop summary and corpus-count lines before diffing baseline vs current.
apps/sdlc/lib/services/quality/tests/quality_ops.test.ts	modify	Add regression test covering baseline-shift on summary lines.
apps/sdlc/skills/task-work/SKILL.md	modify	Update Step 7 prose if the drift-detection semantics change.

Acceptance criteria

• AC-1: A regression test exists that captures a baseline at SHA-A, re-runs the same verb at SHA-B (where the only change is a summary count), and verifies the gate exits 0 with no new-drift entries.
• AC-2: Audit-summary lines (whose values are pure aggregates/counts of the underlying findings, e.g. total: N or processed: N) do not register as new drift when their values change between two SHAs on which the underlying findings are unchanged.
• AC-3: The new normalisation logic is documented (either inline in the normalizeFinding docstring in apps/sdlc/lib/services/quality/baseline.ts, which already enumerates the masks, or in a short note in apps/sdlc/skills/task-work/SKILL.md).
• AC-4: Existing tests still pass.

Out of scope

• Re-architecting the baseline format itself — this task layers a normalisation pass over the current shape.
• Per-verb finding extractors / a structured-findings projection (D-H7FS-op-substrate-surface §4 --output) — future scope.
• Per-project tuning knobs (config in sdlc.yaml). Punt to a follow-up if needed; default normalisation should cover the common cases.

Dependencies

• none (T-H69K-run-quality-checks-isolates-pre-existing-drift and T-FNUT-quality-noun-ports have both shipped)

Discovery context

Spawned by the /sdlc:task-work post-mortem of T-FRTD-drop-lease-filesystem-cache on 2026-05-27: 3 audit-summary lines were flagged as new drift because main’s file count moved between baseline capture and the Step 7 gate; resolution cost a baseline re-capture plus one round of human investigation.

Seven later post-mortems linked the same gap here rather than spawning duplicates — independent observations of summary/tally lines false-flagging as new drift:

Linked from	Observation
2026-05-27-add-backlog-capture	Baseline captured in a main checkout carrying untracked files; entity-audit corpus-count/summary lines false-positived. Capture should run in an environment matching the gate (worktree, committed-only) — that nuance folds into this task’s design rather than being triple-tracked.
T-61OI-check-ancestry-flags-stale-base	Uncommitted task-file edits shifted the entity audit’s per-file OK listing; benign reordering flagged as drift.
T-CW4K-scaffold-dashboard-vite-vue	rumdl’s Found N issues in X/Y files summary line gated; a parallel-WIP base move shifted the count (11 → 9) and surfaced spurious new-drift on a branch touching no docs.
T-LX04-activate-s0008-apps-adapter-validator	Same rumdl tally under a now-stale baseline SHA after parallel markdown landed on origin/main.
T-7EJO-extract-corpus-depgraph-module	A legitimate issue-count reduction also false-flags — aggregate summary lines must be dropped, not merely count-normalized.
T-JZL4-generate-dashboard-api-client	task-work’s own start/verify commits advance origin/main between Step 3a capture and Step 7, so the diff runs against a stale corpus; ~5 phantom rumdl/summary findings until a manual re-capture.
T-1YSW-amend-d0001-solutions-tier	Both rumdl SUMMARY lines (Issues: Found 18 issues in 5/972 files and Run rumdl fmt to automatically fix 14 of the 18) counted as findings, so a changed total reported as 2 new findings. Names the fix precisely: the differ should drop non-location-bearing lines.
T-T3QJ-retire-tests-parity-harness	A legitimate file deletion shifted the N/M denominator; two rumdl summary-count lines flagged. (The same bullet’s PID facet is owned by T-BQRU-quality-normalize-ports-pids-timings.)

---

← Back to Tasks