Pure functional YAML field editing over bytes

Status: open/proposed

Summary

• yaml-splice is a pure, deterministic library: UTF-8 YAML bytes plus an ordered list of top-level field edits produce new bytes or a structured refusal. It has no filesystem, markdown, schema, clock, locale, or process dependencies.
• A semantic no-op is a textual no-op. Changing a field replaces only that field’s authored span; every non-target byte remains identical. New material is rendered deterministically without reserializing the document.
• V1 deliberately edits one YAML document whose root is a string-keyed mapping. Nested paths ([[#^nested-path]]), raw-fragment injection ([[#^raw-fragment]]), multi-document streams ([[#^yaml-stream]]), formatting, frontmatter fences, validation, CAS, and atomic writes stay outside the library.

Context

Markdown frontmatter is the first consumer, but the mutation problem is YAML’s: change a field without re-emitting comments, key order, quoting, block-scalar style, unrelated whitespace, BOM, or line endings. Keeping this capability under a markdown or vault API would couple a generally useful text transformation to filesystem and document-container concerns it does not need.

The fleet already has a credible implementation seed. determined-vault splits frontmatter without losing BOM/CRLF framing, parses top-level YAML keys with byte spans, replaces values right-to-left, and composes the note before a separate CAS write. determined-notes::notekit uses that path for the settings surface shared by mail, calendar, todo, OCR, media, digest, triage, and scheduler. Several SDLC writers still parse a mapping and reserialize the whole frontmatter block.

The architectural question is therefore not merely “which YAML crate?” It is the observable contract an editor must keep even if its parser or emitter changes.

Decision

1. Pure byte-to-byte boundary

The core operation is observationally equivalent to:

pub fn edit_yaml(
    source: &[u8],
    edits: &[FieldEdit],
) -> Result<EditOutcome, EditError>;

pub struct EditOutcome {
    pub bytes: Vec<u8>,
    pub changed: bool,
}

The same inputs always produce the same result. The function performs no I/O, reads no environment, clock, locale, or global configuration, and has no hidden cache. It returns complete output bytes only after the entire edit batch succeeds. An error exposes no partially edited output.

The implementation may avoid allocation on a no-op, and a binding may expose a borrowed/owned result, but those are optimizations over the same value semantics.

2. Input document profile

V1 accepts:

• UTF-8 bytes, optionally beginning with the UTF-8 BOM EF BB BF;
• one YAML 1.2 document;
• a top-level mapping, or an empty/comments-only document treated as an empty mapping for insertion;
• string scalar keys for fields addressed by the edit API.

Invalid UTF-8, malformed YAML, a non-mapping root ([[#^non-mapping-root]]), and multi-document streams ([[#^yaml-stream]]) are structured refusals. The library never repairs or canonicalizes them implicitly. Untargeted syntax it can preserve safely may remain in the document, including anchors, aliases, tags, flow collections, and block scalars; the conformance suite, not optimistic parsing, defines the supported subset.

The BOM, when present at byte zero, is preserved exactly and is outside every field span. The library never adds a BOM. U+FEFF anywhere else is YAML content, not framing.

This profile describes standalone YAML bytes. A markdown-frontmatter caller passes only the bytes inside its fences; preserving or creating --- fences, the markdown body, and a file-level BOM belongs to the markdown container layer.

3. Field identity and addressing

V1 addresses top-level fields by their decoded string key. Addressing constrains keys, never values — a value may be any YAML value (see §4):

• comparison is case-insensitive by default (Unicode simple case fold), matching Obsidian’s frontmatter property semantics; case-sensitive matching is an explicit option;
• no Unicode normalization is applied beyond that fold;
• key spelling and quoting are preserved when a value changes — matching an existing key never rewrites how the author spelled it;
• two existing keys that fold to the same identity (Title and title) make that key an AmbiguousKey refusal under the default mode;
• non-string keys ([[#^non-string-key]]) are preserved but cannot be targeted;
• duplicate occurrences of a targeted decoded key are an AmbiguousKey refusal.

There is no dotted-path or JSON-Pointer interpretation. The key a.b names the literal top-level key a.b.

4. Edit algebra

V1 has three operations:

pub enum FieldEdit {
    Set { key: String, value: YamlValue },
    Remove { key: String },
    Rename { from: String, to: String },
}

A YamlValue is any YAML value — string, number, boolean, null, sequence, or mapping. Set writes lists and numbers as readily as strings; only the key that addresses a field must be a string.

Edits are interpreted in declared order as transformations of one logical mapping and committed atomically as one output. An implementation may combine non-overlapping span edits and apply them right-to-left, provided the result is identical to ordered evaluation.

Set

• If the field exists and its parsed value is semantically equal to value, the operation is a byte-identical no-op. It does not normalize draft to 'draft', reorder a collection, or rewrite an equivalent scalar spelling.
• If the field exists with a different value, only the assignment/value span is replaced. Its key token, position, leading comments, trailing inline comment, and all unrelated bytes remain unchanged.
• If the field is absent, a new entry is appended after the existing top-level entries and before any explicit document-end marker. Header and trailing comments remain byte-identical. On an empty/comments-only document, existing comments remain before the new entry.

Set replaces the complete field value; it is not a deep merge.

Remove

• If the field is absent, removal is a no-op.
• If present, the key and its value lines are removed.
• Comments lexically inside the removed entry, including its trailing inline comment, are removed with it. Preceding standalone comments are conservatively preserved because V1 does not guess comment ownership.

Rename

• The key token changes in place; value, comments, order, and surrounding bytes are preserved.
• A missing source key is a MissingKey refusal.
• An existing destination key is a KeyConflict refusal. Rename never silently merges, overwrites, or removes either field.

5. Authored bytes and rendering

The library preserves every source byte outside the minimal spans owned by the requested operations. It is not a formatter.

New or changed values are rendered through one deterministic YAML 1.2 emitter. The emitter owns only newly authored value bytes; it never renders the enclosing mapping. Inserted multi-line material uses the document’s first observed line ending; a document with no line ending uses LF. Existing mixed line endings are not normalized.

Raw YAML-fragment replacement is not a V1 operation. A typed YamlValue keeps syntax validity, scalar typing, and escaping under library control. A future raw operation, if justified by consumers, must validate that the fragment is exactly one YAML value and define indentation and no-op semantics before admission.

6. Validity, atomicity, and errors

The source must parse under the supported input profile before any edit is planned. The complete prospective output must parse under the same profile before it is returned. This catches syntax damage and alias/anchor breakage visible to the parser.

The batch is all-or-nothing. Representative errors are:

• InvalidUtf8;
• MalformedYaml;
• UnsupportedDocument (stream, directives, or unsupported syntax);
• RootNotMapping;
• AmbiguousKey;
• MissingKey;
• KeyConflict;
• UnrenderableValue;
• InvalidResult.

Errors carry byte spans when a location is known. They are data, not panics, and never cause a fallback to whole-document serialization.

7. Separation from containers, policy, and storage

yaml-splice does not own:

• markdown frontmatter recognition or --- fences;
• whether malformed frontmatter is treated as body, a finding, or a hard error;
• schema/contract validation or business rules;
• file paths, permissions, watchers, manifests, or locks;
• etags, compare-and-swap, temporary files, fsync, or atomic rename;
• whether a caller is authorized to change a field.

A safe human-document write composes layers explicitly:

markdown split → yaml-splice → compose prospective markdown
  → contract validation → CAS precondition → atomic write

The pure library owns only the middle transformation and its byte-preservation contract. This lets native Rust, WASM, tests, CLIs, and applications use exactly the same editing semantics without granting the library filesystem authority.

8. Conformance is part of the API

The release corpus includes:

• no-op Set over alternative scalar spellings;
• comments, key order, quoted keys and values, blank lines, and inline comments;
• BOM, LF, CRLF, no final newline, and mixed line endings;
• block scalars, nested values, flow collections, anchors, and aliases;
• empty and comments-only documents;
• missing, duplicate, case-fold-colliding, non-string, and Unicode keys;
• ordered multi-edit batches and conflict/refusal cases;
• assertions that the prefix and suffix outside every owned edit span are byte-identical.

Every language binding and replacement implementation runs the same fixtures. Compatibility includes returned bytes, changed/no-op classification, error code, and error span—not implementation details.

Why

Pure transformation is the smallest boundary that contains the hard problem. YAML syntax ownership and byte preservation belong together; filesystem and markdown-container concerns do not. The boundary is deterministic, easily fuzzed, naturally portable to WASM, and safe to compose into different storage policies.

Top-level edits match observed demand: status and schema-version stamps, PR URLs, readiness markers, publish flags, configuration fields, and generated metadata. Starting there converts existing code into a reusable library without committing to a general-purpose YAML DOM editor before a consumer needs one.

Semantic no-op detection is load-bearing. A settings form commonly parses list: 'Reminders', sends the string value back, and would otherwise rewrite it as list: Reminders. The desired data did not change, so the authored bytes must not change either.

Options considered

Keep editing inside determined-vault

Rejected as the permanent boundary. It is the right implementation seed, but its name and package also own walking, paths, wikilinks, and filesystem writes. YAML editing has consumers outside that vault stack and needs none of those dependencies.

Parse to a mapping, mutate, and serialize the whole document

Rejected for human-authored content. It loses comments and scalar spelling, normalizes formatting, and changes unrelated bytes. It remains valid only for an explicit canonical-generation operation over machine-owned documents.

Build a full lossless YAML CST/DOM before shipping

Deferred. It may eventually be useful for nested structural edits and rich editors, but it is not required to solve the observed top-level field-update problem. V1’s public contract does not prevent a CST implementation later.

Accept arbitrary replacement bytes

Rejected for V1. Without a fragment grammar, indentation contract, and output validation, raw insertion pushes syntax correctness onto every caller and makes cross-language parity ill-defined.

Put YAML editing inside the markdown CST

Rejected. YAML is an embedded language with independent consumers and semantics. The markdown layer identifies and preserves the fenced payload; yaml-splice transforms that payload without learning about markdown.

Consequences

• The Markdown ecosystem gains an early, independent deliverable that does not wait for the Markdown CST.
• determined-vault becomes the implementation donor and compatibility consumer, not the permanent owner of the pure editor.
• Existing whole-block writers can migrate incrementally; each migration deletes a serializer-based mutation path.
• A caller cannot use V1 for nested field edits or formatting. It must replace a complete top-level value, retain an existing specialized path, or propose a versioned API extension.
• Minimal textual change is part of compatibility and may constrain parser and emitter upgrades.
• Safe persistence still requires validation, CAS, and atomic writes outside the library; purity does not itself prevent concurrent clobbering.

Migration

1. Move determined-vault’s adversarial YAML/frontmatter fixtures into a shared conformance corpus without changing behavior.
2. Extract its pure span parser, value model, canonical value emitter, and splice operations into an unpublished incubation crate.
3. Leave compatibility re-exports in determined-vault; keep determined-notes::notekit and determined-photos green as initial consumers.
4. Add Rename, output reparse, stable error codes/spans, and the semantics this decision requires where the seed does not yet provide them.
5. Migrate one SDLC writer that changes only top-level fields, then expand by consumer class. Do not classify machine-generated whole-document writers as corruption bugs automatically.
6. Publish or promote the crate only after its API is exercised by at least two independent consumers, following the derived-library placement rule.

This work may proceed alongside contract convergence and before the lossless Markdown CST milestone. The Markdown CST consumes the resulting library; it is not its prerequisite.

Out of scope

• Markdown parsing, frontmatter fences, and body editing.
• Nested-path, sequence-item, merge-patch, and schema-aware edits.
• Multi-document YAML streams and directives.
• Raw fragment insertion and general formatting.
• Canonical whole-document generation; that is a separate API and write discipline.
• Filesystem safety, authorization, validation, and concurrency.
• Choosing the final parser crate.

Open questions

• Build approach Resolved 2026-07-21: build on the existing determined-vault seed. It is functional and the conformance corpus is the real contract, so the implementation choice is not load-bearing — the goal is shipping the dedup and keeping consumers green, not a rewrite. The implementation may still be swapped later behind the same corpus.
• Does demonstrated demand justify nested paths in V2, and if so, what path syntax avoids confusing literal dotted keys with traversal?
• Should custom tags and YAML directives eventually be preserved as opaque syntax, modeled semantically, or remain structured refusals?
• Public name Resolved 2026-07-20: the crate is yaml-splice — yaml-edit is already taken on crates.io, and splice honestly names the top-level V1 scope.

Glossary

• Multi-document YAML stream — one file carrying several YAML documents separated by --- lines. Valid YAML, but frontmatter is a single document, so streams are a structured refusal.
• Non-mapping root — a YAML document whose top level is a bare scalar (hello) or a sequence (- a) instead of key: value pairs. Valid YAML; refused here because frontmatter is conventionally a mapping.
• Nested path — addressing a key below the top level (the b inside a: {b: 1}). Out of V1 scope; a V2 question gated on observed demand.
• Raw-fragment injection — splicing caller-supplied YAML text into the document instead of a typed value. Rejected in V1: it pushes syntax correctness onto every caller and makes cross-language parity ill-defined.
• Non-string key — YAML permits numbers or even collections as mapping keys (123: x). Such entries are preserved untouched but cannot be addressed by the edit API.

References

• D-7VMX-markdown-ecosystem-strategy — parent ecosystem strategy and tiered crate model.
• solutions/determined/crates/determined-vault/src/yaml/splice.rs — current span-preserving implementation seed.
• solutions/determined/crates/determined-vault/src/frontmatter.rs — current byte-framing and composition seam.
• solutions/determined/crates/determined-vault/src/write.rs — storage behavior kept outside this decision’s pure library.
• solutions/determined/crates/determined-notes/src/notekit.rs — multi-product consumer and semantic no-op precedent.
• docs/planning/decisions/D-7VMX-markdown-ecosystem-strategy/review-discussion.md — review context, early-win analysis, and migration candidates.

---

← Back to Decisions