T-HLEW-harness-capability-matrix

State: planning/proposed · Impact: high · Complexity: medium

Goal

Every downstream Agent Manager feature — the field docs, the per-target support warnings, and the harness gap table — needs one answer to “does this harness honor this field or feature”, with a source and a date behind it. agent-adapt already encodes half of that answer as code (which runtime implements which capability, which fields a dialect omits); this task derives one matrix from that code instead of hand-authoring a fourth comparison table next to the three docs/LIBRARIES.md already tracks as agent-config writers.

Today

Location	Role today
`packages/rust/agent-adapt/src/runtime.rs#RuntimeId`	Closed enum of eight runtimes (`ClaudeCode`, `GeminiCli`, `CodexCli`, `OpenCode`, `Amp`, `GithubCopilot`, `NpmPackage`, `Portable`)
`packages/rust/agent-adapt/src/runtime.rs#SkillCapability`	Marker trait a runtime implements to render skills; sibling traits `AgentCapability`, `CommandCapability`, `HookCapability` follow the same shape. Presence/absence of an impl is the only “does this runtime support this asset type” signal in code today
`packages/rust/agent-adapt/src/runtime.rs#FrontmatterDialect`	`omit_fields` lists, per runtime, which canonical fields that dialect drops — the only per-field support signal that already exists in code
`packages/rust/agent-adapt/src/runtime.rs#RuntimePaths`	`Option<ScopedRelative>` fields (`commands_dir`, `hooks_file`, `mcp_config_file`) say whether a runtime has that surface at all
`packages/rust/agent-adapt/src/runtimes/mod.rs#all`	Returns every “full-stack” runtime (skill + agent + command); deliberately excludes `GithubCopilot`, which implements `CommandCapability` alone — a caller needing command-shaped coverage walks `command_renderer_for` instead
`packages/rust/agent-adapt/src/mcp/mod.rs#McpCapability`	The MCP peer capability trait, kept separate from the skill/agent/command/hook family on purpose
`packages/rust/agent-adapt/src/tools.rs#ToolRegistry`	Translates a tool name per runtime (`translate_tool_name`); no allow/ask/deny model anywhere in the crate
`packages/ts/agent-plugin/model.ts#HarnessPermissionsSchema`	A `{allow, deny, ask}` permissions shape agent-pants does not consume; read for contrast, not touched by this task
`solutions/determined/crates/determined-agent-skills/src/lib.rs#get_notes`	Already exposes `known_targets` on the `AgentSkillNotes` wire answer — the precedent this task’s matrix rides beside
`docs/LIBRARIES.md`	”Agent config writers” row already names `agent-adapt`, `@sksizer/agent-plugin` and `agent-mcp-config` as three implementations that write contradictory output; this task reads that duplication, it does not converge it

Proposed

A single capabilities() function in agent-adapt walks runtimes::all() plus the capability-specific lookups (command_renderer_for, agent_renderer_for, skill_renderer_for, McpCapability/HookCapability impls) and returns one matrix: rows are fields and features (frontmatter fields, “renders skills”, “renders hooks”, the un-modelled permission and hook-event row families), columns are RuntimeId, and each cell is native | translated | ignored | rejected. Every cell derivable from runtime.rs/tools.rs today comes from there directly — no hand-duplicated per-runtime fact. Cells with no code signal (permission models: Claude tool lists and permissionMode vs. OpenCode’s permission block vs. Codex’s approval policy; hook event names per harness) carry a hand-seeded evidence record {url, observed_on, version} sourced from harness-comparison.md and the plan’s external facts. The function is exposed on the wire through determined-agent-skills, next to known_targets, so the field docs, the warnings composable, and the gap table all read one source.

Approach

Design the matrix’s row/column shape in agent-adapt (new module src/capabilities.rs): a Capability enum or struct per row family (field-backed rows keyed by canonical field name, feature rows keyed by asset type, evidence-only rows for permissions and hook events), a Support enum (Native, Translated, Ignored, Rejected), and an Evidence struct (url, observed_on, version). Open decision: whether GithubCopilot gets a column derived from command_renderer_for alongside the all()-derived columns for skill/agent/hook rows, or whether command-shaped rows alone carry it — resolve by matching what the gap table (hook-editor-and-harness-gap-table) needs to render per note kind.
Derive field-backed rows from FrontmatterDialect.omit_fields (omitted → Ignored unless a render-time refusal makes it Rejected) and from AGENT_SKILLS_FIELDS/ CLAUDE_CODE_EXTENSION_FIELDS.
Derive feature rows from capability-trait presence (SkillCapability, AgentCapability, CommandCapability, HookCapability, McpCapability) per runtime, walking each capability’s own lookup function rather than only all().
Add the evidence-only row families (permissions, hook events) with hand-seeded Evidence records quoting harness-comparison.md and this milestone’s external specification facts, dated 2026-09-07.
Write a capabilities() function assembling the above into one matrix value; a unit test asserts every code-derivable cell matches what runtime.rs/tools.rs say directly (a regression guard against the matrix silently drifting from its own source).
Expose capabilities() on the wire: a new determined-agent-skills op or a field beside known_targets on AgentSkillNotes, with a matching determined-core::wire type. Confirm moon run :check :test --affected passes.

Areas

Area	Note
`packages/rust/agent-adapt/src/capabilities.rs`	Matrix types, derivation from `runtimes::all()` and the capability traits, evidence records
`packages/rust/agent-adapt/src/lib.rs`	`pub mod capabilities;`
`solutions/determined/crates/determined-agent-skills/src/lib.rs`	Expose the matrix on the wire beside `known_targets`
`solutions/determined/crates/determined-core/src/wire/mod.rs`	Wire types for the matrix, cell support level, and evidence record

Acceptance criteria

AC-1: capabilities() returns a matrix whose columns are produced by walking runtimes::all() and the per-capability lookup functions — no hand-typed RuntimeId list inside capabilities.rs.
AC-2: Every field- and asset-type-backed cell’s support level matches FrontmatterDialect.omit_fields / capability-trait presence for that runtime, checked by a unit test in packages/rust/agent-adapt.
AC-3: The permissions and hook-event row families are present with a non-empty Evidence (url, observed_on, version) on every cell.
AC-4: determined-agent-skills exposes the matrix on the wire, verified by a crate test alongside the existing known_targets coverage.
AC-5: docs/LIBRARIES.md’s “Agent config writers” row is unchanged — no fourth hand-written comparison table is added anywhere in the repo.

Out of scope

Converging agent-adapt, @sksizer/agent-plugin and agent-mcp-config into one writer — docs/LIBRARIES.md’s duplicates row stays open; this task reads the three, it does not merge them.
Rendering the matrix in the UI — the field docs (T-A27K-field-reference-docs-with-evidence), the support warnings (T-8BUZ-marketplace-targets-and-support-warnings) and the gap table (T-0AF7-hook-editor-and-harness-gap-table) each consume it separately.
Modelling HarnessPermissionsSchema consumption in agent-pants — noted for contrast only.

Dependencies

none

Discovery context

From the user’s Agent Manager planning request of 2026-09-07: “Permissions mapping across harnesses” was one of two direct questions asked, answered by auditing packages/rust/agent-adapt/src/tools.rs#ToolRegistry and packages/ts/agent-plugin/model.ts (no allow/ask/deny model exists; a data-driven matrix is the prerequisite for the warnings the user wants). Evidence seeded from the harness survey solutions/agent-pants/docs/vault/reference/harness-comparison.md and the spec-adherence audit solutions/agent-pants/docs/vault/research/agent-skills-conformance.md. Member of M-BMFD.

← Back to Tasks