T-K1A8-rebase-markdown-extract-on-markdown-contract

Status: closed/done · Impact: medium · Complexity: large

Goal

plugin/lib/util/markdown_extract.ts is the last module in the repo that parses markdown with unified + remark-* directly, and the whole docs-generation tier rides on it. markdown-contract — already this repo’s markdown engine for every entity contract — exposes parse(), the same projection. Re-basing this module on it leaves markdown-contract the single markdown front-end and lets remark-parse, remark-frontmatter, and unified drop from the direct dependency set.

Today

Location	Role today
plugin/lib/util/markdown_extract.ts	Parses markdown via unified().use(remarkParse).use(remarkFrontmatter); exposes parseMarkdown/parseMarkdownFile, extractFrontmatter, stripFrontmatter, sectionBody (an mdast heading-walk), hasBlockId/extractBlockIdText
package.json	Direct-depends on remark-parse, remark-frontmatter, unified — consumed only by markdown_extract.ts
plugin/lib/services/docs/generate.ts	Calls parseMarkdownFile to read entity frontmatter + bodies
plugin/lib/services/docs/site.ts	Calls parseMarkdownFile for supplemental nav labels
plugin/lib/services/docs/site/projections.ts	Calls parseMarkdownFile to project per-entity child pages

Proposed

markdown_extract.ts keeps its exported API but implements parse / frontmatter / section / block-id over markdown-contract’s parse() projection. No file under plugin/ imports remark-* or unified directly; those three packages are gone from package.json dependencies (they remain transitively under markdown-contract). The three docs-gen consumers are untouched and sdlc docs generate output is byte-identical.

Approach

1. Map each parsing function to the markdown-contract projection: parseMarkdown → parse(text) (the DocTree carrying frontmatter + sections + anchors); extractFrontmatter → the tree’s parsed frontmatter; sectionBody(name) → the matching section node’s subtree text; hasBlockId/extractBlockIdText → the tree’s ^block-id anchors. The fs / heading helpers (entityFiles, entityDirs, dirName, headingPlainText) stay as-is.
2. Reimplement the module body against parse() behind the unchanged exported signatures; delete the remark-parse / remark-frontmatter / unified / yaml imports it no longer needs. If the pinned markdown-contract does not expose ^block-id anchors, keep one local regex for that single helper — it does not reintroduce a remark import.
3. Run bun test for plugin/lib/util/ and plugin/lib/services/docs/; run sdlc docs generate --check and confirm zero drift.
4. Remove remark-parse, remark-frontmatter, unified from package.json dependencies; run bun install; confirm no plugin/ file imports them.

Files to touch

Location	Kind	Change
plugin/lib/util/markdown_extract.ts	modify	Reimplement parse / frontmatter / section / block-id over markdown-contract parse(); drop the remark / unified / yaml imports
package.json	modify	Remove remark-parse, remark-frontmatter, unified from dependencies
plugin/lib/util/tests/	modify	Add/extend tests pinning parseMarkdown / sectionBody / extractFrontmatter / block-id behavior over the markdown-contract backing

Acceptance criteria

• AC-1: grep -rE 'from "(unified|remark-parse|remark-frontmatter)' plugin/ returns no matches.
• AC-2: package.json dependencies no longer lists remark-parse, remark-frontmatter, or unified, and bun install resolves cleanly.
• AC-3: markdown_extract.ts’s exported names and signatures are unchanged, and no file under plugin/lib/services/docs/ is modified.
• AC-4: sdlc docs generate --check exits 0 (regenerated docs are byte-identical to the committed ones).
• AC-5: bun test passes for plugin/lib/util/ and plugin/lib/services/docs/.

Out of scope

• The docs-gen consumers’ own bespoke parsers — site.ts’s hand-rolled YAML frontmatter reader and ## Purpose extraction, the H1-drop / block-id-strip / wikilink transforms in generate.ts and projections.ts. Those are separate cleanups (see [T-Z51X-consolidate-frontmatter-regex-onto-shared-helper](/planning/tasks/consolidate-frontmatter-regex-onto-shared-helper/)).
• Obsidian wikilink / transclusion rewriting in the docs tier — markdown-contract extracts refs, but the site-output rewrite stays here.

Dependencies

• None blocking — markdown-contract’s parse() already ships in this repo’s pinned version. [T-Z51X-consolidate-frontmatter-regex-onto-shared-helper](/planning/tasks/consolidate-frontmatter-regex-onto-shared-helper/) also touches markdown_extract.ts; land one first to avoid a conflict.

Post-mortem

Captured by /sdlc:task-work on 2026-06-30. PR: pending.

Acceptance criteria coverage

• AC-1: auto — command grep -rEn 'from "(unified|remark-parse|remark-frontmatter)' plugin/ returns no matches.
• AC-2: auto — command grep -nE '"(remark-parse|remark-frontmatter|unified)"' package.json returns no matches; bun install resolves cleanly (lockfile saved; the three packages remain transitively under markdown-contract).
• AC-3: auto — all 12 exported names/signatures unchanged; git diff --name-only origin/main -- plugin/lib/services/docs/ is empty.
• AC-4: auto — sdlc docs generate --check exits 0 with drift: [].
• AC-5: auto — bun test plugin/lib/util/ plugin/lib/services/docs/ reports 347 pass, 0 fail.

What worked

• The projection mapped cleanly: markdown-contract’s parse() returns the full mdast Root (with the yaml node and position offsets intact), so the existing mdast-walk helpers and the offset-based sectionBody slice stayed byte-identical — only the processor construction changed.
• The implementer verified parse() output against all 441 real entity files (frontmatter, section bodies, heading plain-text, yaml values) before reimplementing — zero mismatches gave high confidence the docs-gen output would stay byte-identical, which docs generate --check then confirmed.

Friction and automation gaps

• Quality gate raised one false-positive new-drift from plugin/lib/services/dashboard/tests/dashboard.test.ts — a transient running-dashboard listing leaks into bun test stdout with a per-run PID, so the baseline normalizer (which redacts TMPDIR/PORT but not the PID column) can’t subtract it and the line text differs every run. Confirmed unrelated to this change (PID changed across reruns; the branch touches no dashboard code). Gap: either make dashboard.test.ts not emit a live dashboard-list to captured stdout, or extend the baseline normalizer to redact the PID column of dashboard-list lines. → T-BQRU-quality-normalize-ports-pids-timings
• Step 7’s quality run from the worktree could not find the Step 3a baseline: capture wrote it to the main repo’s .sdlc/quality-baselines/, but quality run --diff-against-baseline from the worktree defaults its --baseline-dir to the worktree’s own .sdlc/ and missed it; passing --baseline-dir <main-repo>/.sdlc/quality-baselines explicitly fixed it. Gap: task-work Step 7’s documented invocation omits --baseline-dir, so the worktree-relative default silently misses the main-repo baseline — Step 7 should pass the main-repo baseline dir, or the executor should resolve the baseline dir against the git superproject root. → T-44OO-plugin-scripts-self-discover-project-root

Spawned follow-up tasks

• T-BQRU-quality-normalize-ports-pids-timings — linked (existing open/ready tracker for the dashboard-PID baseline-normalizer false-drift; normalizeFinding PID mask covers this bullet)
• T-44OO-plugin-scripts-self-discover-project-root — linked (existing active tracker for the worktree-vs-main baseline-dir silent-fallback; self-discovery via git rev-parse --git-common-dir covers this bullet)

---

← Back to Tasks