T-3MZD-extract-yaml-splice-crate

Status: closed/done · Impact: high · Complexity: large

Goal

Lift determined-vault’s span-preserving YAML field editor into a standalone incubation crate, yaml-splice, implementing the D-7N7I-pure-functional-yaml-editing contract: UTF-8 YAML bytes plus an ordered list of top-level field edits produce new bytes or a typed refusal, with no filesystem, markdown, schema, CAS, or clock dependency. This is the extraction half of M-2VYE-yaml-splice-extraction-and-writer-migrations — the strategy’s best early technical win: real consumers already exist and the active corruption risk (lost YAML comments) is removed, without waiting on the markdown CST, WASM, or vault work.

Today

The seed is real, self-contained (imports only std, saphyr_parser, thiserror, serde_json, and its own siblings — zero determined_* coupling), and heavily tested. It is buried in a crate whose name also owns walks, paths, and fs writes.

Location	Role today
apps/determined/crates/determined-vault/src/yaml/parse.rs	saphyr span parser (parse_with_spans) — being rewritten by PR #1051: alias resolution, explicit-tag honouring, duplicate-key reporting
apps/determined/crates/determined-vault/src/yaml/value.rs	order-preserving Value/Mapping, From<&serde_json::Value> bridge, js_float_to_string (ECMA-262)
apps/determined/crates/determined-vault/src/yaml/splice.rs	splice_key/splice_keys/remove_key/merge_* — no Rename, no semantic no-op, case-sensitive exact-match keys only
apps/determined/crates/determined-vault/src/yaml/canonical.rs	bundles two disciplines: value rendering (used by splice) AND whole-document to_string (js-yaml-compatible)
apps/determined/crates/determined-vault/src/frontmatter.rs	BOM/CRLF framing + splice_note_key — the markdown-container seam (stays behind)
determined-{contracts,engine-client,media,notes,ocr,photos}	6 downstream crates reach in via determined_vault::yaml::* (~26 files)

Proposed

A workspace crate packages/rust/yaml-splice/ exposing the D-7N7I §1/§4 surface — edit_yaml(&[u8], &[FieldEdit]) -> Result<EditOutcome, EditError> with FieldEdit::{Set, Remove, Rename} — dependencies limited to saphyr-parser, thiserror, serde_json. determined-vault keeps a compatibility re-export (pub use yaml_splice as yaml) so all six consumer crates plus notekit and determined-photos — D-7N7I’s first two consumers — compile unchanged. Whole-document canonical to_string stays in determined-vault as the separate canonical write discipline, explicitly out of scope for the pure editor. The adversarial YAML/frontmatter corpus travels with the crate as its conformance suite.

Approach

1. Gate on PR #1051. It rewrites yaml/parse.rs (+701/−63) to add alias/tag handling and duplicate-key reporting — exactly the D-7N7I §2–3 requirements (AmbiguousKey needs duplicate detection). Extract off the post-#1051 parser; extracting first forces a manual reconcile of a ~760-line diff on a moved file. #1051 is stacked on #1053, which also edits determined-media/ocr Cargo.toml.
2. Create the crate. packages/rust/yaml-splice/ (the packages/rust/* workspace glob auto-registers it; dir-aspect is the precedent). Move the span parser and value model verbatim; split canonical.rs — the value-rendering emitter moves, the whole-document to_string stays in determined-vault. Fix super::→crate::.
3. Recast to the D-7N7I contract. Public API becomes edit_yaml + FieldEdit + EditOutcome{bytes, changed}. Add the semantics the seed lacks: case-insensitive-by-default key addressing (Unicode simple case fold, matching Obsidian; case-sensitive an explicit option), with fold-colliding keys (Title/title) → AmbiguousKey; semantic-no-op on Set (§4 — load-bearing: a re-sent list: 'Reminders' must not become list: Reminders); Rename (§4); whole-output reparse (§6); and the stable EditError taxonomy (§6) carrying byte spans.
4. Move the conformance corpus (§8) out of determined-vault’s inline tests into the crate unchanged; keep determined-vault’s own span tests green.
5. Shield consumers. Re-export from determined-vault so the ~26 downstream files stay untouched; flip only the 5 internal determined-vault files (frontmatter, sidecar, path, wikilink, lib) to the new crate.

Files to touch

Location	Kind	Change
packages/rust/yaml-splice/Cargo.toml	new	crate manifest; deps saphyr-parser, thiserror, serde_json only
packages/rust/yaml-splice/src/lib.rs	new	edit_yaml + FieldEdit/EditOutcome/EditError public surface
packages/rust/yaml-splice/src/parse.rs	new	moved span parser (post-#1051)
packages/rust/yaml-splice/src/value.rs	new	moved Value/Mapping + serde_json bridge
packages/rust/yaml-splice/src/splice.rs	new	moved splice ops + new Rename/no-op
packages/rust/yaml-splice/src/render.rs	new	value-rendering emitter split out of canonical.rs
packages/rust/yaml-splice/tests/	new	conformance corpus (moved fixtures)
Cargo.toml	modify	add yaml-splice to [workspace.dependencies]
apps/determined/crates/determined-vault/src/yaml/mod.rs	modify	becomes a pub use yaml_splice re-export shim
apps/determined/crates/determined-vault/src/yaml/parse.rs	delete	moved to the crate
apps/determined/crates/determined-vault/src/yaml/value.rs	delete	moved to the crate
apps/determined/crates/determined-vault/src/yaml/splice.rs	delete	moved to the crate
apps/determined/crates/determined-vault/src/yaml/canonical.rs	modify	keep whole-document to_string; value render moves out
apps/determined/crates/determined-vault/Cargo.toml	modify	depend on yaml-splice
apps/determined/crates/determined-vault/src/frontmatter.rs	modify	repoint internal crate::yaml refs

Acceptance criteria

• AC-1: packages/rust/yaml-splice builds as a workspace member with dependencies limited to saphyr-parser, thiserror, serde_json — no determined_*, no fs/markdown/CAS.
• AC-2: The D-7N7I §8 conformance corpus passes: the prefix and suffix outside every edited span are byte-identical across comments, key order, quoting, BOM, LF/CRLF, block scalars, flow collections, anchors/aliases, empty/comments-only, and unicode/duplicate/case-fold-colliding/non-string keys.
• AC-3: edit_yaml implements Set/Remove/Rename per §4; a semantically-equal Set is a byte-identical no-op (EditOutcome.changed == false, bytes unchanged).
• AC-4: Every refusal returns a typed EditError from the §6 enum with a byte span where known; no panic on any input; no fallback to whole-document serialization.
• AC-5: determined-vault, determined-notes, determined-photos, determined-media, determined-ocr, determined-contracts, determined-engine-client all compile and their existing tests pass against the determined_vault::yaml re-export, unchanged.
• AC-6: whole-document canonical to_string is NOT in yaml-splice’s public API; it remains in determined-vault as the separate canonical discipline.
• AC-7: key addressing is case-insensitive by default (Unicode simple case fold, Obsidian semantics) with case-sensitive as an explicit option; two existing keys that fold to one identity (Title/title) are an AmbiguousKey refusal.

Out of scope

• Whole-document canonical generation — a separate write discipline (canonical-write).
• Markdown fences / frontmatter recognition / body editing — the container layer.
• CAS, atomic write, etag — determined_vault::write, the storage discipline.
• Nested-path, sequence-item, merge-patch, and raw-fragment edits — D-7N7I defers to V2.
• Migrating consumers off the re-export and migrating whole-block writers — that is T-68IB-adopt-yaml-splice-and-reconcile-plan.

Dependencies

• PR #1051 (rust-backends/07-wave4b) must merge first: it rewrites yaml/parse.rs, the file this extraction moves. Its base #1053 edits determined-media/determined-ocr Cargo.toml, which this repoints. (Not task entities, so not in depends_on.)
• D-7N7I-pure-functional-yaml-editing accepted (currently open/proposed).

Discovery context

• Surfaced reviewing the merged polish-experience work (PR #1042): the determined-vault yaml module was the most self-contained extraction candidate. Consumer surface (6 crates / ~26 files) and the #1051 parse.rs collision were measured during that review; D-7N7I charters the boundary.

Post-mortem

Captured by /sdlc:task-work on 2026-07-21. PR: pending.

Acceptance criteria coverage

• AC-1: auto — cargo build -p yaml-splice; cargo tree shows only saphyr-parser/thiserror/serde_json.
• AC-2: auto — cargo test -p yaml-splice (37 conformance tests; assert_single_span proves byte-identical prefix/suffix).
• AC-3: auto — Set/Remove/Rename + semantic-no-op unit tests.
• AC-4: auto — one refusal test per EditError variant + no_panic_on_hostile_input.
• AC-5: auto — cargo build + lib tests for determined-vault and all six consumers (contracts/engine-client/media/notes/ocr/photos), unchanged against determined_vault::yaml.
• AC-6: auto — grep-confirmed to_string/to_frontmatter_block absent from the crate’s public API.
• AC-7: auto — case-insensitive-default / case-sensitive-option / fold-collision→AmbiguousKey tests.

What worked

• The mod.rs re-export shim made the internal determined-vault files (frontmatter/sidecar/path/wikilink/lib) compile with zero repointing — less churn than the plan, same AC-5 result.
• canonical.rs split cleanly: render machinery moved to the crate’s render.rs; whole-document to_string stayed behind as the separate discipline.
• cargo was the real gate and caught everything; all consumers stayed green unchanged.

Friction and automation gaps

• The task worktree had no node_modules (bun-install worktree_init skipped for a Rust task), so determined-contracts/tests/drift.rs — a Rust test that shells out to the TS zod source needing the node package croner — fails environmentally (zero YAML refs, unrelated to this change) — a cross-language Rust test needs node deps armed in the worktree, or should skip when absent.
• fold = to_lowercase rather than a full Unicode simple case-fold table (no unicode crate in the 3-dep budget); documented in-code — a strict-Unicode follow-up must add a fold table/crate or revisit the dep budget.
• The SDLC quality gate (sdlc.yaml) is TypeScript-only and would have passed a broken Rust change — addressed in PR #1062 (adds cargo clippy/cargo test --workspace).

---

← Back to Tasks