T-6ZNY-headless-bases-syntax-parser-for-eval

Status: closed/done · Impact: low · Complexity: medium

Auto-generated from a /sdlc:task-work post-mortem. Review and promote to open/ready before picking up.

Goal

AC-4 of T-781K-add-obsidian-bases-setup-flag requires opening a generated .base file in Obsidian to confirm it renders as a table without “invalid base” errors. Obsidian is not headless, so this AC is structurally deferred-user: every future change to Bases generation needs a human in the loop to re-verify. A minimal Bases- syntax parser (or shelling out to one) would let the AC run as auto on CI, closing the loop. The post-mortem flagged this explicitly as a “heavier follow-up.”

Today

• Bases is an Obsidian-only renderer; the .base file format is documented at https://help.obsidian.md/bases but there is no first- party CLI parser.
• The originating task’s AC-4 reads: “Opening one generated .base file in Obsidian renders the corresponding planning artifacts as a table without ‘invalid base’ errors (manual verification, noted in PR).”
• The post-mortem observed: “AC-4 is structurally a deferred-user check because Obsidian isn’t headless; an eval pattern that loads a .base through a minimal Bases-syntax parser would close the loop, but that’s a heavier follow-up.”
• No tooling in this repo parses .base files beyond yaml.safe_load, which catches gross YAML errors but not Bases-specific semantic errors (e.g. unknown view type, malformed file.inFolder(...) expression).

Proposed

A small Python module (or shell out to an external tool) that loads a .base file and validates it against the Bases schema enough to catch the same class of errors Obsidian would surface as “invalid base”:

• Top-level keys are recognized (views, filters, formulas, …).
• Each views entry has a recognized type and a valid shape for that type.
• Property references in order, groupBy, sort, and filter parse as either note.<field>, file.<field>, or formula.<name>.
• Filter expressions like file.inFolder("...") and note.status == "..." parse without error.

A new test (or a /project-check extension) loads each generated .base (or each plugin/entities/<type>/base.yaml, since the bytes are copied verbatim) through this parser and fails on any error.

Approach

1. Implement a minimal Python parser at plugin/validators/validate_base.py (mirrors the location of validate_frontmatter.py). Scope: features we actually emit (table view, simple file.inFolder filter, groupBy.property: note.<field>). Reject the shell-out alternative — no stable community CLI exists for the .base format and shelling out adds a runtime dependency we can avoid.
2. CLI: validate_base.py <path-to-.base> (one or more). Exits 0 on pass; non-zero with a per-file, per-key diagnostic on fail.
3. Wire the validator into .claude/skills/project-check/ as a new check script (check_entity_bases.py) that iterates every plugin/entities/<type>/base.yaml and shells out to the validator. project-check is project-local (not in plugin/), which is the right home for SDLC-corpus-specific checks.
4. Update AC-4 of T-781K-add-obsidian-bases-setup-flag from deferred-user to auto, citing the new validator.

Files to touch

• plugin/validators/validate_base.py (new) — the headless validator CLI.
• .claude/skills/project-check/check_entity_bases.py (new, project-local) — iterates plugin/entities/*/base.yaml and shells out to the validator.
• .claude/skills/project-check/SKILL.md — list the new check in the “What this skill checks today” section.
• docs/planning/tasks/2026-05-19-add-obsidian-bases-setup-flag.md — reclassify AC-4 from deferred-user to auto.

Acceptance criteria

• AC-1: A new CLI accepts a .base file path and exits zero on every shipped plugin/entities/<type>/base.yaml.
• AC-2: Hand-corrupting one of the shipped base.yaml files (e.g. changing type: table to type: bogus, or rewriting file.inFolder(".") to file.inFolder() causes the CLI to exit non-zero with a diagnostic that names the offending file and key.
• AC-3: /project-check invokes the validator and fails on any invalid .base template.
• AC-4: AC-4 of T-781K-add-obsidian-bases-setup-flag is reclassified from deferred-user to auto (either by editing that task’s post-mortem coverage line or by superseding it in a follow-up PR).

Out of scope

• A full reimplementation of Bases rendering — only enough syntax validation to catch what Obsidian would flag as “invalid base.”
• Validating community plugins that extend Bases beyond the upstream schema.

Dependencies

• none

Discovery context

Spawned by /sdlc:task-work post-mortem of T-781K-add-obsidian-bases-setup-flag on 2026-05-20.

Post-mortem

Captured by /sdlc:task-work on 2026-05-21. PR: pending.

Acceptance criteria coverage

• AC-1: auto — plugin/validators/validate_base.py exited 0 against all four shipped plugin/entities/<type>/base.yaml files (4/4 file(s) passed).
• AC-2: auto — fixtures bogus-view-type (changes type: table to type: bogus) and broken-filter (rewrites file.inFolder(".") to file.inFolder() both exited 1 with diagnostics naming the offending file and key (views[0].type: 'bogus' not in ['cards', 'table'], filters.and[0]: unrecognized filter expression: 'file.inFolder('). Covered by run_base_evals.py.
• AC-3: auto — .claude/skills/project-check/check_entity_bases.py shells out to the validator, propagates its exit code, and prints FAIL <path> + diagnostics on corruption (verified with a synthetic --entities-dir /tmp/fake-entities run).
• AC-4: auto — AC-4 of T-781K-add-obsidian-bases-setup-flag reclassified from deferred-user to auto in the same commit, citing the new validator. The post-mortem friction bullet that originally flagged this gap was also updated to reference the resolution.

What worked

• Mirroring validate_frontmatter.py / validate_sdlc_yaml.py’s CLI conventions (PEP-723 inline deps, EXIT_OK/EXIT_FAILED/EXIT_CONFIG, --quiet, N/N file(s) passed summary) made the validator drop-in compatible with the existing project-check pattern — check_entity_bases.py is a 60-line subprocess wrapper.
• The “scope = what we actually emit” constraint (table view only, simple file.inFolder filters, note.<field> references) kept the regex set small enough to reason about and matched 4/4 shipped templates on the first run.
• The fixture-based eval pattern from check_entities.py / run_evals.py ported directly — one tuple per drift case, substring assertions on stdout, exit-code check. Seven fixtures cover the disqualifier surface without ballooning.

Friction and automation gaps

• sdlc.yaml at the project root carries a pr_check: block that the bundled schema (plugin/schemas/sdlc-yaml.schema.json) rejects with Additional properties are not allowed ('pr_check' was unexpected). This means plugin/scripts/run_quality_checks.py --config sdlc.yaml fails at the schema gate before running any verb — both for worktree_init (Step 4) and quality_checks (Step 7). Worked around by running each declared verb directly. Root fix: either promote pr_check: into the bundled schema, or document it as a recognized extension the validator should accept. → T-DN76-pr-check-schema-promote
• No sub-agent dispatch tool was available in this harness, so Step 6 ran the implementation directly (no subagent delegation). The skill assumes sub-agent dispatch is always available; degrading gracefully when it isn’t (or detecting and noting the absence) would make the prose more honest about the actual execution mode. (Skipped: harness-level limitation, not a plugin gap.)
• Initial insertion of the new ### 1c. heading in SKILL.md between 1a and 1b (out of order) required a second edit to move it after 1b. A pre-edit “where does this section belong in the existing numbering” check would have caught it once. Minor churn; not worth dedicated tooling. (Skipped: per-bullet judgment, no automation hook.)

Spawned follow-up tasks

• T-DN76-pr-check-schema-promote — created; adds pr_check to the bundled sdlc-yaml.schema.json so the validator stops rejecting this project’s config.

---

← Back to Tasks