T-5A82-dedup-search-emits-telemetry

Status: closed/superseded · Impact: low · Complexity: small

Auto-generated from a /sdlc:task-work post-mortem. Review and promote to open/ready before picking up.

Goal

plugin/skills/task-work/dedup_search.py ships --threshold-min-score 6 and --threshold-ratio 0.5 as defaults tuned against a tiny calibration set (the known-overlap pair plus a handful of negative cases). There is no telemetry hook to record how often Step 3 returns LINKED-EXISTING vs SPAWNED in production runs, what the winning candidate’s score and ratio actually were, or whether reviewers later overrode the decision in the search-trail block’s Rationale field. Without that signal the thresholds will drift from optimal silently. Cited by T-A2C6-spawn-from-post-mortem-stronger-dedup.

Today

dedup_search.py search emits the search-trail block on stdout and a one-token decision tag on stderr; nothing is persisted. Each invocation is independent. The thresholds are configuration, not learned.

Proposed

Add an --emit-telemetry-line <path> flag that appends a single JSON line per invocation to the named file. The line carries the decision, top candidate’s score, keyword count, the score/keyword-count ratio, and the originating worktree path (when the caller passes one). A small companion script summarizes the log into a histogram of LINKED-EXISTING vs SPAWNED bucketed by score, so the calibration question (“at what threshold do we flip from too-eager-linking to too-eager-spawning?”) becomes answerable from data.

Approach

1. Add --emit-telemetry-line PATH to the CLI; when set, append-only-write one JSON object per invocation under a schema documented in the script’s docstring.
2. Wire the spawning sub-agent in spawn-from-post-mortem.md Step 3 to pass a default telemetry path (e.g. <worktree>/.claude/dedup-telemetry.jsonl) so every production invocation contributes.
3. Add plugin/skills/task-work/summarize_dedup_telemetry.py that reads the log and prints a histogram + the score/ratio percentiles. Co-located per the plugin/skills/CLAUDE.md script-promotion convention.
4. Document the telemetry schema in the dedup_search.py module docstring so consumers know what to expect.

Files to touch

• plugin/skills/task-work/dedup_search.py — add the flag, the JSON emit, and the schema documentation.
• plugin/skills/task-work/summarize_dedup_telemetry.py (new) — the histogram script.
• plugin/entities/task/spawn-from-post-mortem.md Step 3 — pass --emit-telemetry-line in the example invocation.
• .gitignore — add **/.claude/dedup-telemetry.jsonl if not already covered by .claude/.

Acceptance criteria

• AC-1: Each invocation of dedup_search.py search --emit-telemetry-line <path> ... appends exactly one valid JSON line to <path> with the documented fields.
• AC-2: summarize_dedup_telemetry.py <path> prints a histogram of decisions bucketed by top-candidate score.
• AC-3: After ten production invocations, the log contains ten lines and the summary script runs against it without error.

Out of scope

• Auto-tuning the thresholds from the log. That’s a later decision and probably needs a manual review window first.
• Sending telemetry off-machine. The append-only file in the worktree is enough for a single-developer project; team-scale telemetry is a different conversation.

Dependencies

• T-A2C6-spawn-from-post-mortem-stronger-dedup — the dedup_search script must exist first.

Discovery context

Spawned by /sdlc:task-work post-mortem of T-A2C6-spawn-from-post-mortem-stronger-dedup on 2026-05-21.

Dedup search (spawn-from-post-mortem)

Bullet: dedup script ships threshold defaults tuned against a tiny calibration set; no telemetry hook to record how often Step 3 returns LINKED-EXISTING vs SPAWNED in production runs so thresholds can be revisited empirically. An emit-telemetry-line flag or append-only log would close the calibration loop Keywords searched: emit-telemetry-line, linked-existing, recalibrated, calibration, empirically, threshold, telemetry, revisited Top candidates (score / status / headline):

• 19 / in-progress / 2026-05-21-spawn-from-post-mortem-stronger-dedup — Strengthen spawn-from-post-mortem dedup and record the search trail (self-match; excluded from decision)
• 2 / closed/done / 2026-05-20-orchestrator-categorized-in-flight-limits — Replace orchestrator parallelism cap with categorized in-flight limits configurable in sdlc.yaml
• 2 / planning/draft / 2026-05-20-orchestrator-tick-token-budget-fixture — Token-budget fixture for orchestrator tick parent transcript Decision: SPAWNED Rationale: Top match is the originating task (self-match). After exclusion the next-best score is 2, well below the min-score floor of 6 and unrelated to telemetry/calibration. The self-match pattern is itself tracked in T-HC03-dedup-search-excludes-originating-task.

---

← Back to Tasks