T-MJWQ-oss-skill-fixtures-corpus

State: planning/proposed · Impact: medium · Complexity: medium

Goal

Every fixture the import, publish and spec-conformance work tests against today is hand-authored, so the messy edge cases a real published skill has — odd frontmatter ordering, extra keys, nested resources — never show up in a test run. This task vendors a small corpus of permissively licensed, real-world skills and wires it into the Rust, CLI and e2e suites plus a skills-ref validate CI gate, closing the “no validator oracle” gap the conformance research names.

Today

Location	Role today
solutions/determined/tests/fixtures/agent-pants-recommended/	Synthetic marketplace fixture (one plugin, one marketplace note); hand-authored
solutions/determined/tests/fixtures/agent-pants-freeform/	Synthetic freeform vault fixture (skills, references, two marketplaces); hand-authored
solutions/agent-pants/examples/vault/	Hand-written example vault used in docs, not a test corpus
solutions/agent-pants/crates/agent-adapt-core/tests/fixtures/agent-skills-vault/	Hand-authored Rust integration-test fixture vault
solutions/agent-pants/crates/agent-adapt-core/src/discover.rs	Discover tests run against the hand-authored fixture vault; no real-world skill in the set
solutions/agent-pants/crates/agent-adapt-core/src/adopt.rs	Adopt tests run against synthetic sources; none carry real published frontmatter
solutions/determined/products/agent-pants/tests/e2e/authoring.e2e.ts	Playwright spec against the synthetic fixtures; no real-world import path
solutions/agent-pants/crates/agent-pants/tests/cli.rs	CLI suite against the native binary; no fixture corpus of real skills
.github/workflows/agent-pants-ci.yml	Runs core tests, lint, the codegen drift gate and the CLI suite; no skills-ref validate step

Proposed

solutions/agent-pants/fixtures/oss/<org>-<repo>/ holds a small set of vendored, permissively licensed real-world skills (candidates: anthropics/skills, agentskills.io’s own published examples), each directory carrying the upstream LICENSE and a PROVENANCE.md recording the source URL, the vendored commit sha, and the date. agent-adapt-core’s discover and adopt tests run against the corpus alongside the existing hand-authored fixtures; a new Playwright spec imports one corpus entry into a scratch vault end to end; the CLI suite adopts and publishes one entry against the committed wasm. skills-ref validate runs over every rendered target for the corpus in a new CI step, skipped rather than failed when the tool cannot install.

Approach

1. Pick three to five candidate skills from anthropics/skills and/or agentskills.io’s published examples, checking each one’s license permits vendoring; skip anything without a clear permissive license.
2. Vendor each into solutions/agent-pants/fixtures/oss/<org>-<repo>/ verbatim, with the upstream LICENSE and a PROVENANCE.md (source URL, vendored sha, vendor date).
3. Extend discover.rs’s and adopt.rs’s existing #[cfg(test)] modules to walk the corpus and assert discovery succeeds and adoption round-trips for every entry.
4. Add a Playwright spec that imports one corpus entry through the real import page against a scratch vault, separate from the GitHub-source spec import-from-github-and-source-picker owns.
5. Extend cli.test.ts (or add a sibling test file) to adopt and publish one corpus entry against the committed wasm.
6. Add a skills-ref validate step to .github/workflows/agent-pants-ci.yml: install the validator and run it over the corpus rendered through every target; the step is skipped, not red, when the tool cannot install — decide the install method (pip vs pipx) as an open item.
7. Note in the fixture directory’s own README which fixtures are real-world versus synthetic, so a future contributor does not conflate the two.

Areas

Area	Note
solutions/agent-pants/fixtures/oss/	Vendored real-world skill corpus, one directory per source repo with LICENSE and PROVENANCE.md
solutions/agent-pants/crates/agent-adapt-core/src/discover.rs	Corpus-driven discover test cases alongside the existing fixture tests
solutions/agent-pants/crates/agent-adapt-core/src/adopt.rs	Corpus-driven adopt round-trip test cases
solutions/determined/products/agent-pants/tests/e2e/import-oss-corpus.e2e.ts	Playwright import spec over one corpus entry
solutions/agent-pants/crates/agent-pants/tests/cli.rs	CLI adopt-and-publish assertion over one corpus entry
.github/workflows/agent-pants-ci.yml	skills-ref validate step over the corpus

Acceptance criteria

• AC-1: solutions/agent-pants/fixtures/oss/ holds at least three vendored skill directories, each with an upstream LICENSE and a PROVENANCE.md naming its source URL, vendored sha, and date.
• AC-2: cargo test -p agent-adapt-core discovers and adopts every corpus entry with no change to the vendored content.
• AC-3: The new Playwright spec imports one corpus entry into a scratch vault and passes.
• AC-4: The CLI suite adopts and publishes one corpus entry against the committed wasm and passes.
• AC-5: .github/workflows/agent-pants-ci.yml runs skills-ref validate over every rendered target for the corpus, and the step is skipped rather than red when the tool is absent.

Out of scope

• The CI path-filter and Playwright-in-CI gap for the determined product and experience — T-I7N0-agent-manager-screenshots-docs-ledger owns that.
• Rewriting the vendored skills’ own content; the corpus is copied verbatim.
• The GitHub-source import mechanism itself — T-7V6K-import-from-github-and-source-picker.

Dependencies

• T-7V6K-import-from-github-and-source-picker: the import path this corpus’s Playwright spec exercises end to end.
• T-GLK0-spec-fields-and-skills-ref-oracle: the skills-ref validate oracle this task’s CI step reuses.

Discovery context

From the user’s planning request of 2026-09-07. Closes gap G10 (“Validator oracle: none — skills-ref validate not run anywhere”) in solutions/agent-pants/docs/vault/research/agent-skills-conformance.md.

---

← Back to Tasks