T-94OR-backup-retention-setting
State: planning/draft · Impact: medium · Complexity: medium
A user should set how much history to keep once and stop thinking about it. This task adds a “keep N days” setting, prunes automatically after a scheduled backup, and shows the store’s size and the result of the last prune.
The schedule’s config (enabled, interval_mins, 60 by default) lives in the settings store under
a backup namespace, read by load_config in crates/determined-platform/src/backup_schedule.rs,
with wire types in crates/determined-core/src/wire/backup_schedule.rs. Each scheduled pass calls
BackupPort::run_snapshot, the same method Back Up Now calls. BackupScheduleSettings.vue edits
the config inside BackupSettingsSection.vue. backup_status already reports store_bytes and the
snapshot count. There is no retention setting.
Proposed
Section titled “Proposed”- A retention config beside the schedule’s:
keep_days, absent or off by default, which means keep everything. It is read and written the same way as the schedule config, with defaults merged over a missing or partial entry. The minimum is one day. keep_sinceis now minuskeep_days× 24 hours, as T-A97O defines it. Because the prune keeps the newest snapshot beforekeep_since, a restore’s safety snapshot stays in the store for at leastkeep_daysafter the restore, even when the restore reused an older snapshot. Its Undo works for at least that long.- The scheduled pass prunes, not Back Up Now.
run_snapshotgains an optionalprune_sinceargument. The schedule passes it when retention is on and a prune is due; Back Up Now never does. The job calls T-A97O’sprune::pruneafter the snapshot, while it still holds thevault-backupkey. The prune runs whether or not the snapshot made a commit, so an unchanged vault still gets pruned. - A prune is due at most once a day. The schedule skips it when the last prune ran less than 24
hours ago. It also skips it when the boundary is already the oldest snapshot, which is a cheap
git logcheck, so a pass with nothing to drop never clones. With the default hourly schedule, this means at most one full re-clone of the store a day. - Each prune’s result is written to
vault-backup/last-prune.json, besidestore.git: time, bytes before and after, snapshots dropped, the boundary’s sha and date, or the error. The daily gate and status both read it, so it survives a restart. - A preview: the snapshots a prune at the current setting would drop, the boundary, and the store’s size after. The size comes from a dry run: the same shallow clone into a temp directory, measured, then deleted.
- Status gains the oldest kept snapshot and the last prune’s result.
store_bytesis already there and is shown. - UI in the Backups settings section: the retention control, the status lines, and a “Prune now” button that shows the preview and asks for confirmation before it runs. “Prune now” ignores the daily gate.
- A sha that is not in the store, in a store that has been pruned (
last-prune.jsonrecords a completed prune), reads as “not in the backup store; it may have been removed by retention”. In a store that was never pruned it stays a plain unknown-snapshot error. Git cannot tell a dropped sha from one that never existed, so the message says “may”. - The oldest kept snapshot shows as the start of history, not as a snapshot that added every file.
Approach
Section titled “Approach”- Add the retention config and its wire types, stored next to the schedule’s config.
- Add
prune_sincetorun_snapshot, the daily and nothing-to-drop gates in the schedule, andlast-prune.json. - Add the preview op (dry-run clone) and extend status.
- Build the settings UI. Add stories for its off, on, previewing, pruning and error states if the backup experience has a Storybook host by then (T-VZA1-backup-storybook-enrolment); otherwise integration tests cover those states, and T-VZA1 adds the stories.
- Handle the boundary and pruned-snapshot cases in the history and changes views.
| Area | Note |
|---|---|
solutions/determined/crates/determined-platform/src/backup_schedule.rs | Retention config, the prune gates, passing prune_since |
solutions/determined/crates/determined-core/src/wire/ | Retention config, preview and status types |
solutions/determined/crates/determined-core/src/ports/backup.rs | prune_since on run_snapshot |
solutions/determined/crates/determined-backup/src/ | The prune in the snapshot job, last-prune.json, preview (dry run) and status fields |
solutions/determined/experiences/backup/app/components/ | Retention control in the settings section; boundary and pruned states in history views |
Acceptance criteria
Section titled “Acceptance criteria”- AC-1: With retention off, no prune ever runs and the store keeps all history. A value under one day is refused.
- AC-2: With “keep 30 days” on, a scheduled pass prunes, and the list then holds every snapshot from the last 30 days plus the newest one before that.
- AC-3: A scheduled pass prunes even when the vault is unchanged and no commit is made.
- AC-4: With the hourly schedule, the scheduled prune runs at most once in 24 hours, and a pass with nothing to drop does not clone the store.
- AC-5: Settings shows the store size, the oldest kept snapshot, and the last prune’s result, and they survive a restart.
- AC-6: “Prune now” shows the snapshots it would drop and the size after, and does nothing until the user confirms.
- AC-7: After a prune, a link to a dropped snapshot says it may have been removed by retention. In a never-pruned store, an unknown sha gives the plain error.
- AC-8: The new settings UI has stories for its states, per S-0019, if the experience has a Storybook host when this task lands; otherwise integration tests cover the same states.
Out of scope
Section titled “Out of scope”- Pruning by snapshot count, or keeping thinned-out older snapshots (one a week, and so on).
- Pruning on Back Up Now.
- Pruning while the app is closed.
Dependencies
Section titled “Dependencies”- T-A97O: the prune operation,
prune::pruneand the boundary flag.