T-FRTD-drop-lease-filesystem-cache

Status: closed/done · Impact: high · Complexity: medium

Goal

The lease library currently maintains a write-through filesystem cache at <project-root>/.sdlc/runtime/lease-cache/<task_id>.json. The cache exists as an optimization to skip fetch_ref round-trips between successive ref operations in a single task-work flow. In practice it has been a persistent source of bugs (cache miss → hard fail, worktree-as-project-root resolution, layering violations as skills reach directly into the cache module). The simpler shape is: drop the cache. Each library call (acquire_lease, transition_lease, reacquire_lease, release_to_archive, etc.) fetches the current state from the authority when it needs to, mutates via CAS, and returns the new state to the caller. The caller (the skill) holds whatever it needs across calls as a local value. This collapses three distinct bugs into one fix and unblocks the proposed local-coordination-server model (a later task).

Today

Location	Role today
`plugin/lib/lease/cache.py`	Filesystem cache module: `get`, `put`, `_cache_dir`, `_find_project_root`. Read by start_task.py + heartbeat + library mutators.
`plugin/lib/lease/runtime.py`	Library entrypoints (`acquire_lease`, `transition_lease`, `reacquire_lease`, `release_to_archive`, `read_lease`, etc.) — currently call `cache.put` after every ref op and `cache.get` before reads.
`plugin/lib/lease/primitives.py`	Imports `cache as _cache` for write-through wiring at the primitive layer.
`plugin/lib/lease/__init__.py`	Re-exports `cache` so callers can `from lease import cache`.
`plugin/skills/task-work/start_task.py#main`	Reaches into `lease_cache.get(task_id, project_root=...)` directly to read the cached lease before CAS-REPLACE — the canonical layering violation.
`plugin/skills/task-work/test_start_task.py`	Asserts cache state before/after start_task runs (`lease_cache.get` in test).
`plugin/scripts/lease_heartbeat_loop.py`	Imports `_find_project_root` from runtime; relies on cache to find the lease ref between ticks.
`plugin/scripts/sdlc_lease.py`	CLI subcommands (acquire, transition, reacquire, archive) — operate through library functions that touch the cache as a side effect.
`plugin/lib/lease/reconcile.py`	Has its own state-load path; verify it doesn’t depend on the cache (likely it doesn’t — it fetches from authority).
`plugin/lib/lease/tests/test_cache.py`	Direct cache unit tests — to be removed.
`.sdlc/runtime/lease-cache/`	On-disk cache files. Gitignored. The worktree-as-project-root bug leaves them in the wrong directory.

Proposed

The lease library exposes only high-level operations and each operation is atomic-per-call: it does its own fetch_ref at entry, performs the CAS, and returns. Callers never thread state between library calls — every entrypoint is a self-contained RPC. Cost: one extra git ls-remote per library call (sub-second each, ~5 calls per task lifecycle). Benefit: skills become trivially stateless (“call the library, get the answer”) and the surface area matches a future local-coordination-server model where every call is a self-contained RPC.

Each entrypoint returns a value type (a LeaseHandle or equivalent) bundling current SHA + parsed payload, but only for inspection / logging — successive calls do not require the caller to pass it back.

plugin/lib/lease/cache.py is deleted. No file under plugin/lib/lease/ writes to or reads from <project-root>/.sdlc/runtime/lease-cache/. The _find_project_root walk goes away (callers that need a project root for OTHER reasons — baseline cache, etc. — resolve it themselves; the lease library no longer cares).

start_task.py is rewritten to declare intent at the library level: one call like transition_to_working(authority, task_id) (or transition_lease(authority, task_id, new_phase="working")) wraps the fetch + CAS + return. The script no longer imports lease_cache, cas_replace, validate_dict, or build_lease_commit — those become library internals.

The heartbeat loop fetches the current ref state each tick (instead of reading the cache between ticks). One extra git ls-remote per tick — negligible.

The cache directory is removed from gitignore (no longer needed) and any existing .sdlc/runtime/lease-cache/ directories are cleaned up by the migration of consumers (no special-case removal step is required; the directory just stops being written).

Approach

Audit cache call sites. Compile a complete list of every caller (production + test) that reaches into plugin/lib/lease/cache.py. The grep at task-spec time turned up: primitives.py, runtime.py (4 sites), __init__.py, start_task.py, test_start_task.py, lease_heartbeat_loop.py. Confirm no others before refactoring.
Design the new library surface. Settle on the shape of LeaseHandle (or whatever the return type is named) and the signature of each entrypoint. Pure value type — no methods that hit the authority; explicit library calls to mutate. Document the contract in plugin/lib/lease/__init__.py’s docstring.
Refactor plugin/lib/lease/runtime.py to: (a) accept LeaseHandle inputs where useful, (b) return LeaseHandle outputs, (c) drop every cache.get / cache.put call, (d) use fetch_ref directly where current state is needed. Update existing tests in plugin/lib/lease/tests/test_runtime.py.
Refactor plugin/lib/lease/primitives.py to remove the cache as _cache import. The primitives become pure CAS / FETCH operations with no cache side effects.
Delete plugin/lib/lease/cache.py and its tests (plugin/lib/lease/tests/test_cache.py). Remove the cache export from plugin/lib/lease/__init__.py.
Refactor plugin/skills/task-work/start_task.py to use the new high-level library call. The script becomes: read frontmatter, edit + validate + commit + push (on main), invoke transition_to_working(authority, task_id), git rebase main in worktree. No direct ref / cache / schema / tree imports.
Refactor plugin/scripts/lease_heartbeat_loop.py to fetch each tick instead of read-from-cache. The fetch is on the inner loop; verify the cadence still respects the minimum-frequency floor and we don’t fetch faster than needed.
Refactor plugin/scripts/sdlc_lease.py CLI subcommands to use the new library surface. Each subcommand: parse args → call library → print marker. No cache references.
Update plugin/lib/lease/reconcile.py if it imports from cache; if it has its own state load (fetch_lease_namespace, etc.), it’s already independent.
Update SKILL.md prose in task-work, task-close-out, pr-respond to remove cache-related language (e.g. “the cache is populated via the library’s write-through wiring”, “the cached lease state”, etc.). The prose stops promising cache semantics.
Update .sdlc/.gitignore (or wherever) to remove the lease-cache/ exclusion.
Run end-to-end through the lifecycle without any symlink workaround: /sdlc:task-work on a test task, watch start_task.py succeed without cache, heartbeat tick, /sdlc:task-close-out succeed without cache. AC-6 below pins this.

Files to touch

Location	Kind	Change
`plugin/lib/lease/cache.py`	delete	remove the filesystem cache module entirely
`plugin/lib/lease/tests/test_cache.py`	delete	remove the cache unit tests
`plugin/lib/lease/runtime.py`	modify	drop cache calls; return LeaseHandle values; use fetch_ref directly
`plugin/lib/lease/primitives.py`	modify	remove `cache as _cache` import + write-through side effects
`plugin/lib/lease/__init__.py`	modify	drop `cache` re-export; update docstring
`plugin/lib/lease/tests/`	modify	update tests that asserted cache state
`plugin/skills/task-work/start_task.py`	modify	use high-level library call; drop direct primitive/cache imports
`plugin/skills/task-work/test_start_task.py`	modify	drop cache-state assertions; assert ref state instead
`plugin/scripts/lease_heartbeat_loop.py`	modify	fetch each tick instead of cache-read
`plugin/scripts/sdlc_lease.py`	modify	CLI subcommands use new library surface
`plugin/lib/lease/reconcile.py`	modify	confirm/remove any cache imports
`plugin/skills/task-work/SKILL.md`	modify	remove cache-related prose
`plugin/skills/task-close-out/SKILL.md`	modify	remove cache-related prose
`plugin/skills/pr-respond/SKILL.md`	modify	remove cache-related prose

Acceptance criteria

Out of scope

Local coordination server. The user noted that a local-server model for ref operations may come later. That’s a separate task — the library’s high-level surface is what such a server would expose. This task only removes the filesystem cache.
Performance benchmarking. Assume per-call fetch_ref is fast enough (sub-second). If a future profiling pass reveals real slowness, an in-process memoization shim is a separate task.
Ref operation primitives. cas_create, cas_replace, fetch_ref, fetch_namespace keep their current contracts. This task changes the layer above them, not the layer itself.
Authority schema changes. lease.json, handoff.md, control-plane.json are unchanged.
Backward compatibility. No transition window — alpha-stage protocol, single consumer. The cache is gone in one commit.

Dependencies

Soft: closes (supersedes) T-Y7QU-start-task-fails-loud-on-cache-miss — the original task framing was “make the failure loud”; this task makes the cache go away, which makes the failure mode disappear entirely. Mark the predecessor closed/superseded when this task lands.

Discovery context

Surfaced through slice 4a Task B’s run when start_task.py failed with error: no cached lease for task_id='...'. The workaround was a symlink (<worktree>/.sdlc/runtime/ → <main-repo>/.sdlc/runtime/) so the worktree-based cwd discovery would land on the main repo’s cache. The symlink was repeated for slice 4a Task C’s run. Discussion with the user (2026-05-27) reframed the bug from “fail loud on cache miss” to “the cache shouldn’t exist.” The reframing collapsed three distinct bugs into one fix:

Cache-miss hard fail (the original task).
_find_project_root treating a worktree’s .sdlc/skill-ext/ as a project marker.
start_task.py reaching directly into lease.cache, lease.cas_replace, lease.validate_dict, lease.tree — a layering violation.

The user also flagged that a local coordination server might host the protocol primitives in a future iteration; the library’s high-level surface is what such a server would expose. That’s outside this task’s scope but informs the API shape (clean separation between “what a skill wants” and “how the protocol works”).

Layering-violation survey (2026-05-27)

Before locking the spec, an Explore-agent survey ran across the whole plugin to confirm the violations are concentrated in the lease library and not endemic. Findings:

Confirmed violations (all inside the scope of this task):
- plugin/skills/task-work/start_task.py:308 — from lease.tree import build_lease_commit.
- plugin/skills/task-work/start_task.py:314–330 — inline reimplementation of _find_project_root.
- plugin/scripts/lease_heartbeat_loop.py:107, 261 — from lease.runtime import _resolve_authority, _find_project_root (both private).
No violations in plugin/skills/task-close-out/, plugin/skills/pr-respond/, or the new plugin/lib/prs_field/ consumers — append_pr_url.py and verify_prs_against_pr.py import only from prs_field.__init__’s public surface. That’s the shape we want everywhere.
No cross-skill violations detected (no skill imports a helper from another skill’s directory).
Other libraries (plugin/lib/prs_field/) follow the right shape already. The smell is concentrated in the lease library’s integration layer; this task fixes it there.

The survey’s deeper observation: the lease library exposes primitives + private discovery helpers (_resolve_authority, _find_project_root), but no public helpers for the small number of legitimate things scripts need to know (authority, project root). The atomic-per-call refactor should resolve this by either (a) making the library entrypoints handle authority/project-root internally so scripts never have to ask, or (b) promoting the needed helpers to documented public exports. AC-10 below pins the no-private-imports rule.

Post-mortem

Captured by /sdlc:task-work on 2026-05-27. PR: pending.

Acceptance criteria coverage

AC-1: auto — verified by grep: plugin/lib/lease/cache.py and plugin/lib/lease/tests/test_cache.py are gone (git diff --name-status main..HEAD shows them deleted). No remaining imports of lease.cache, lease_cache, or from .cache under plugin/.
AC-2: auto — plugin/lib/lease/tests/ (175 cases passing) covers each entrypoint with fresh-fetch behaviour. No entrypoint accepts a pre-fetched-SHA argument; every call does find_project_root + fetch_ref at entry.
AC-3: auto — plugin/skills/task-work/start_task.py imports only fetch_lease and transition_lease from lease. No underscore imports, no cas_*, no validate_dict, no build_lease_commit, no TaskLifecycleLease.
AC-4: auto — plugin/scripts/lease_heartbeat_loop.py imports only CASFailed, LeaseError, find_project_root, heartbeat_lease, resolve_authority — all public.
AC-5: agent-manual — runtime AC pinned by the library unit tests. The full end-to-end claim → working → awaiting-review → closing → archive cycle ran on sandbox fixtures across test_lease_transition_claimed_to_working and the existing suite; the property “no symlink workaround needed” is implied by cache.py being gone (there is no cache file to point at). First true end-to-end observation lands on the next /sdlc:task-work invocation post-merge.
AC-6: agent-manual — reconcile detectors are unchanged and fetch from authority directly; they never touched the cache. Confirmed by survey: no lease.cache imports anywhere outside the deleted module.

AC-7: auto —

command grep -n 'cache\|cached\|lease-cache' plugin/skills/task-work/SKILL.md plugin/skills/task-close-out/SKILL.md plugin/skills/pr-respond/SKILL.md

returns only quality-baseline-cache references (a separate system) and matches inside historical post-mortems (acceptable).

AC-8: auto — full lease suite 175 passed; CLI suite 62 passed; test_start_task.py 7 passed. Project quality checks OK 12/12 (baseline-gated) against a re-captured baseline at the current main SHA.
AC-9: auto — command grep -n '_find_project_root' plugin/skills/task-work/start_task.py returns nothing.

AC-10: auto — plugin/lib/lease/__init__.py docstring documents the atomic-per-call public surface.

command grep -rn 'from lease\.\(runtime\|primitives\|cache\|tree\|schemas\) import _' plugin/skills/ plugin/scripts/

is empty.

What worked

The exemplar-driven approach (point at prs_field/__init__.py for the clean shape, point at sdlc_lease.py for the right consumer pattern) gave the sub-agent a concrete target — no design churn.
The pre-spec survey accurately predicted scope: 3 violations in 2 files, all in-scope. Nothing surprising leaked out during implementation.
Baseline-gated quality checks correctly distinguished real new drift from main-branch-movement summary shifts after a baseline re-capture.

Friction and automation gaps

The first sub-agent dispatch died mid-execution with API Error: The socket connection was closed unexpectedly after ~5 minutes. No commits had landed. A fresh dispatch with an abbreviated prompt completed cleanly. The original prompt was long (70+ lines); worth a follow-up — sub-agent prompts above some length threshold may risk this, and the skill’s sub-agent dispatch guidance could note it. → T-JXBQ-task-work-warns-on-long-sub-agent-prompt
The pre-symlink workaround was still needed at worktree creation. The cache fix only takes effect after merge (runtime loads from ${CLAUDE_PLUGIN_ROOT}, not the worktree). Subsequent runs post-merge should not need it.
--baseline-dir explicit override still needed for the quality gate (worktree’s .sdlc/quality-baselines/ doesn’t exist). Separate task: T-5X6Y-task-work-step7-explicit-baseline-dir. → T-5X6Y-task-work-step7-explicit-baseline-dir
Quality-gate’s baseline-shift false positives (3 audit summary lines flagged as “new drift” because main’s file counts moved between baseline capture and gate invocation) cost one round of investigation + a baseline re-capture. Worth surfacing in T-H69K-run-quality-checks-isolates-pre-existing-drift — the diff is too literal; counts/summary lines shouldn’t gate. → T-BCNP-quality-gate-ignores-summary-and-corpus-lines

Spawned follow-up tasks

T-JXBQ-task-work-warns-on-long-sub-agent-prompt — task-work Step 6 grows a prompt-length-budget note flagging long-prompt socket-close risk (created).
T-5X6Y-task-work-step7-explicit-baseline-dir — existing task on quality-gate baseline-dir worktree mismatch; the originating task was added to its related: and the dedup search trail appended (linked).
T-BCNP-quality-gate-ignores-summary-and-corpus-lines — quality-gate strips/normalises baseline-shifting summary and corpus-growth lines before diffing (created).

← Back to Tasks