T-X24I-orchestrate-lease-aware-dispatch

Status: closed/done · Impact: high · Complexity: medium

Goal

Convert /sdlc:orchestrate from a status-frontmatter dispatcher into a lease-aware one. After this task lands, every orchestrate tick (a) fetches the control-plane ref and refuses to dispatch when missing or version-incompatible, and (b) wins a CAS-guarded task lifecycle lease via sdlc lease task claim before spawning a task-work sub-agent. The lease token is injected into the dispatched sub-agent so downstream skills can validate and transition without re-claiming. Closes rollout item 5 of the GitHub Ref Leases ADR.

This task is the first lease-aware skill — it’s where the protocol becomes load-bearing for real work. There is no dual-mode or fallback path: if the control-plane ref is missing or the lease library is unavailable, orchestrate exits the tick with a structured error and waits for the next tick. The intent is a clean cutover; mid-state behavior is intentionally not supported.

Today

/sdlc:orchestrate (introduced by T-3OVF-add-orchestrate-skill) ticks on /loop /sdlc:orchestrate, scans docs/planning/tasks/*.md for status: open/ready (excluding autonomy: human-only), and dispatches /sdlc:task-work <slug> into Agent sub-agents up to the parallelism cap. Claiming is a read-then-write race — between scan and dispatch, two orchestrators on different hosts could each pick the same task because neither holds an atomic claim. No control-plane ref is consulted, so an orchestrator running an older sdlc_version than the authority’s recorded one will happily dispatch incompatible work. PR shepherding (via /sdlc:pr-check) is independent of lease state today.

The lease library + CLI from slice 1 are in place: sdlc lease task claim <id> exists, exits 0 on CAS-CREATE success with CLAIMED task=<id> lease_id=<uuid> lease_token=<uuid> on stdout, and exits 2 with CAS-FAILED ref=... on contention (per T-QC31-add-sdlc-lease-cli-commands). The namespace-conflict guard (T-K3RR-add-lease-namespace-conflict-guard) runs on every CLI invocation. But no skill calls them yet.

Proposed

Two new gates inside /sdlc:orchestrate’s tick, plus a lease-discovery convention and a cache module that every downstream lease-aware skill consumes.

Control-plane compatibility gate. Before dispatch logic runs, orchestrate fetches refs/sdlc/control-plane (via the library’s fetch_ref primitive or the CLI’s sdlc lease inspect). If the ref does not exist, orchestrate exits the tick with CONTROL-PLANE-MISSING on stderr and a one-line log entry — the operator hasn’t run the cutover migration yet, so the protocol is not live. If the ref exists, orchestrate parses the ControlPlane payload, compares its sdlc_version to the orchestrator’s own version (a SDLC_VERSION constant in plugin/lib/lease/version.py, new in this task), and either proceeds (compatible) or exits with CONTROL-PLANE-INCOMPATIBLE expected=<X> actual=<Y> (incompatible). The version-compat predicate follows the ADR’s Control Plane Compatibility section: same major implies compatible; differing major implies refuse.

Lease-claim gate per candidate. For each open/ready candidate that survives the existing autonomy filter, orchestrate subprocesses to sdlc lease task claim <task-id>. The CLI handles all the protocol details (CAS-CREATE on refs/sdlc/tasks/<task-id>, generating lease_id and lease_token, etc.). On exit code 0, orchestrate dispatches the Agent sub-agent for /sdlc:task-work <slug> — no env vars, no argv-passing of lease state. On exit code 2 (CAS-FAILED — someone else won the race), orchestrate logs a one-liner and skips to the next candidate. Other non-zero exit codes propagate as orchestrator-level errors (the tick aborts; next tick retries).

No handoff payload — discovery by branch name. Every lease-aware skill (orchestrate-dispatched or operator-direct) derives its task lease from the current branch: git branch --show-current → strip the task/ prefix → task_id → refs/sdlc/tasks/<task_id> is the lease ref. The library exposes runtime.discover_lease() -> TaskLifecycleLease that performs this derivation, reads the lease from disk cache or FETCH-REF, validates the payload, and returns it. Skills never receive the lease via env vars or arguments — they always discover, so orchestrate-dispatched and operator-direct paths are byte-identical. The lease ref itself is the source of truth; the dispatcher’s role is to CAS-CREATE that ref, not to message a copy of it to the sub-agent.

Per-task on-disk cache. A read-side cache at .sdlc/runtime/lease-cache/<task_id>.json holds the latest observed TaskLifecycleLease payload plus a cached_at timestamp. The library’s cas_create and cas_replace populate the cache as a write-through side effect; cas_delete invalidates it. runtime.discover_lease checks the cache first, returns it if present and fresh, otherwise FETCH-REFs and repopulates. The cache exists because lease-aware skills shell out to sdlc lease heartbeat / sdlc lease task transition repeatedly inside one workflow; each shell-out is a fresh Python process, so an in-memory cache wouldn’t help. The cache is a hint — any CAS operation goes through the live ref. Cache is .gitignored; the .sdlc/runtime/ parent directory is already established by PR #96.

After dispatch, orchestrate does NOT track the lease — that’s the dispatched sub-agent’s responsibility (per T-TZSN-task-work-lease-integration-model-b). The sub-agent heartbeats and transitions the lease through phases; orchestrate’s role ends at “claim won, sub-agent dispatched.”

No fallback path. If the lease library subprocess fails (uv resolution error, missing CLI script, etc.), the tick aborts with a single stderr line; the operator fixes the environment and the next tick proceeds. There is no “skip the lease check” toggle, no degraded mode. Alpha-stage rationale: the protocol is the system; running orchestrate without it is the same shape as running it without git available.

Approach

Identify orchestrate’s tick boundary. Read plugin/skills/orchestrate/SKILL.md end-to-end. Locate the steps that (a) survey ready tasks, (b) decide to dispatch, (c) actually spawn the Agent sub-agent. The new gates wedge between (a) and (c); existing PR shepherding (via /sdlc:pr-check) is untouched.
Add the control-plane gate. Implement a new helper in the lease library — plugin/lib/lease/control_plane.py exposing fetch_and_validate_control_plane(authority) -> ControlPlane and a top-level check_compatibility(local_version, remote_version) -> bool. Both raise structured exceptions on failure (ControlPlaneMissing, ControlPlaneIncompatible). The orchestrate skill calls the helper once per tick, before any candidate iteration.
Add the lease-claim gate. Inside the candidate loop, subprocess to sdlc lease task claim <task-id> and parse the marker line. The CLI handles all error semantics already; orchestrate’s job is just to branch on exit code: 0 → dispatch, 2 → skip + log, other → abort tick. Use the CLI rather than the library directly because the CLI already does authority resolution + namespace-conflict guard + structured stdout. The CLI’s claim path (via cas_create) also writes the resulting lease to the on-disk cache as a write-through side effect (see step 5), so the dispatched sub-agent’s first discover_lease() is a cache hit, not a network round-trip.
Add the cache module. New file plugin/lib/lease/cache.py exposing get(task_id) -> TaskLifecycleLease | None, put(task_id, lease), invalidate(task_id). Cache files live at <project-root>/.sdlc/runtime/lease-cache/<task_id>.json. Each file holds the full TaskLifecycleLease payload plus a cached_at ISO-8601 UTC timestamp. Project root is determined by walking up from cwd until .sdlc/ is found (same convention as other .sdlc/runtime/ consumers introduced in PR #96). Writes are atomic — write to <file>.tmp then os.rename — so concurrent readers never see a partial file. Reads that fail to parse the JSON (corruption, partial write, future schema drift) treat the cache as a miss without raising; the caller falls through to FETCH-REF. The cache is never the source of truth; any inconsistency self-corrects on the next CAS round-trip.
Wire write-through caching into the primitives. Modify plugin/lib/lease/primitives.py so that cas_create and cas_replace against task-lifecycle refs (path matches refs/sdlc/tasks/<task_id>) call cache.put(task_id, lease) on success, and cas_delete calls cache.invalidate(task_id). Non-task refs (control-plane, operation leases) are not cached here — those have different access patterns. Caching is a side effect, not an explicit toggle; callers don’t opt in.
Add the discovery helper. New file plugin/lib/lease/runtime.py exposing discover_lease() -> TaskLifecycleLease. Logic: get current branch via git branch --show-current; if it doesn’t match task/<id>, raise InvalidBranchForLease; derive task_id; check cache.get(task_id) — if the cached payload’s expires_at <= now (stale TTL) OR its phase is in {closed/done, closed/superseded, ...} (terminal phase from a lingering cache after close-out), treat as a miss and fall through; otherwise return the cached payload. On miss, fetch_ref(authority, f"refs/sdlc/tasks/{task_id}"), populate cache, return. Companion helper current_task_id() -> str for skills that only need the ID, and discover_lease(fresh=True) for callers that need authority-fresh state (bypasses cache, still write-throughs the fresh fetch).
Update the orchestrate dispatch. The Agent invocation passes no lease-related environment or arguments — branch name carries everything. The orchestrate prose documents this: “the dispatched sub-agent will be in the worktree at .sdlc/worktrees/<task-id> on branch task/<task-id>; it discovers its lease by calling discover_lease() on startup.”
Update orchestrate’s digest-log format. The per-tick digest line gains two fields: cp=<sdlc_version-or-missing> and a per-candidate claim=won|lost|skipped annotation. The shape stays single-line per tick for grep-ability; this is the operator’s primary observability into the new protocol.
Tests. Add orchestrate-side tests (under plugin/skills/orchestrate/tests/) covering: (a) missing control-plane → tick aborts cleanly, (b) incompatible control-plane → tick aborts with the structured marker, (c) CAS-CREATE contention on one candidate → skipped, next candidate processed, (d) lease library subprocess failure → tick aborts. Library-side tests cover: cache.put/get/invalidate round-trip, discover_lease branch derivation, cache write-through from cas_create / cas_replace, cache invalidation from cas_delete. Use the local-bare-repo authority fixture from slice 1.
Document the lease-aware-skill pattern. Add plugin/conventions/lease-aware-skills.md capturing the branch-derivation discovery rule (skills call discover_lease(), not env vars or argv), the write-through cache semantics (cache populated by CAS, invalidated by CAS-delete; never the source of truth), the “validate before side effect” rule, and the fail-closed rule (skill exits with INVALID-BRANCH or LEASE-MISSING if discovery fails). Consumed by T-TZSN-task-work-lease-integration-model-b and T-SSB8-split-pr-respond-and-close-out-with-lease-reacquire.
Manual smoke run. Against a local bare-repo authority seeded with a control-plane.json and one open/ready task, run a single tick. Confirm: control-plane gate passes, CLI claim succeeds, cache file written at .sdlc/runtime/lease-cache/<task_id>.json, sub-agent dispatched with no env vars, sub-agent’s first discover_lease() hits the cache. Capture the output as a documentation example in the convention doc.

Files to touch

Location	Kind	Change
`plugin/skills/orchestrate/SKILL.md`	modify	Insert control-plane gate + lease-claim gate steps in the tick body; update digest-log format with `cp=...` and `claim=won
`plugin/skills/orchestrate/tests/test_orchestrate.py`	new	Coverage for missing-control-plane, incompatible-control-plane, CAS-claim-lost, lease-library-subprocess-failure paths; assert dispatched `Agent` invocations pass NO lease-state env or argv
`plugin/lib/lease/version.py`	new	Single-source-of-truth `SDLC_VERSION` constant for control-plane compat comparisons
`plugin/lib/lease/control_plane.py`	new	`fetch_and_validate_control_plane(authority)`, `check_compatibility(local, remote)`, `ControlPlaneMissing` / `ControlPlaneIncompatible` exceptions
`plugin/lib/lease/tests/test_control_plane.py`	new	Positive + negative tests against local-bare-repo fixtures (missing ref, incompatible-major, compatible-same-major)
`plugin/lib/lease/cache.py`	new	Per-task on-disk cache at `.sdlc/runtime/lease-cache/<task_id>.json`; atomic write-via-rename; defensive parse; `get` / `put` / `invalidate`
`plugin/lib/lease/tests/test_cache.py`	new	Round-trip put/get; invalidate clears; parse-failure treated as miss; atomic write doesn’t leave partials
`plugin/lib/lease/runtime.py`	new	`discover_lease()` (branch → task_id → cache-or-fetch), `current_task_id()`, `InvalidBranchForLease` exception; `discover_lease(fresh=True)` bypasses cache; honors stale `expires_at` and terminal `phase`
`plugin/lib/lease/tests/test_runtime.py`	new	Discovery happy path; non-task-branch → InvalidBranchForLease; stale `expires_at` triggers refresh; terminal-phase cache treated as miss; cache-hit path doesn’t touch the network
`plugin/lib/lease/primitives.py`	modify	`cas_create` / `cas_replace` against `refs/sdlc/tasks/<id>` write-through to cache on success; `cas_delete` invalidates cache; non-task refs are untouched by cache layer
`plugin/lib/lease/tests/test_primitives.py`	modify	New cases asserting write-through behavior for task refs and that non-task refs (control-plane) skip the cache
`plugin/lib/lease/__init__.py`	modify	Re-export `discover_lease`, `current_task_id`, `InvalidBranchForLease`, `cache` module, and existing control-plane / SDLC_VERSION surface
`plugin/conventions/lease-aware-skills.md`	new	Branch-derivation discovery rule; cache semantics (write-through from CAS, invalidate from CAS-delete, never the source of truth); validate-before-side-effect; fail-closed on `InvalidBranchForLease` or `LEASE-MISSING`. Consumed by T-TZSN-task-work-lease-integration-model-b and T-SSB8-split-pr-respond-and-close-out-with-lease-reacquire

Acceptance criteria

Out of scope

/sdlc:task-work lease integration. That’s T-TZSN-task-work-lease-integration-model-b. This task wires the discovery + cache so task-work can call discover_lease(); task-work’s actual lease-validation / heartbeat / transition wiring lives there.
PR shepherding changes. /sdlc:pr-check is not lease-aware; lease re-acquisition for pr-respond / close-out is T-SSB8-split-pr-respond-and-close-out-with-lease-reacquire.
Reconcile, offline mode, operation leases — all later slices.
The cutover migration itself. Running the migration to plant the initial control-plane.json ref is a one-shot operator action (per ADR Migration of in-flight tasks at cutover) and lives in slice 3.
Heartbeat from inside orchestrate. Orchestrate dispatches and exits; the sub-agent heartbeats during work. Orchestrate does not maintain lease state past dispatch.

Dependencies

T-S0PK-add-lease-protocol-library-and-schemas — primitives, schemas, ControlPlane model
T-QC31-add-sdlc-lease-cli-commands — sdlc lease task claim is the dispatch-time hook
T-K3RR-add-lease-namespace-conflict-guard — already wired through every CLI invocation

Integration target: PRs from this task land on the lease-protocol-integration branch (created off main once this planning PR merges). All three slice-2 tasks merge into that branch; the integration branch is then PR’d to main as the slice-2 cutover. No partial slice-2 state ever reaches main.

Discovery context

This task realizes Rollout Plan item 5 from the ADR. The lease-claim gate is the single highest-leverage change in slice 2 — it’s the point at which the SDLC’s coordination primitive changes from “frontmatter is the lock” to “the lease ref is the lock.” Splitting it from the task-work changes (item 6) is deliberate: orchestrate’s gate is small and contained; task-work’s lease integration is a much larger rewrite. Reviewing them separately keeps each PR’s surface area honest.

← Back to Tasks