T-UPPP-backup-bundle-git

State: planning/draft · Impact: low · Complexity: medium

Goal

Determined Backup.app runs git from PATH. On a Mac without the Xcode Command Line Tools, /usr/bin/git is a stub that opens the “install developer tools” prompt, so the installed app only works on machines with developer tools. This task ships a pinned git inside the app, so backup, restore and prune work on any Mac.

Today

crates/determined-backup/src/git.rs is the only module that spawns a process. It builds every child with Command::new("git") in four places: command (the --git-dir/--work-tree pairing), init_bare, clone_shallow and the test-only available.
A spawn failure maps to BackupError::GitUnavailable. The /usr/bin/git stub does not fail to spawn. It exits non-zero after showing the install prompt, so the user sees a GitFailed error.
A Finder-launched app gets a minimal PATH, so a git installed by Homebrew or elsewhere is not found either.
BackupConfig::git_env adds env to every child. It is empty in production. Tests set GIT_CONFIG_GLOBAL and GIT_CONFIG_SYSTEM to a missing file and TZ=UTC. So production git reads the user’s ~/.gitconfig and the system config.
ensure_store pins user.name, user.email, commit.gpgsign=false, core.autocrlf=false and core.quotepath=false in the store’s own config. Other user settings still apply, such as core.fsmonitor and core.hooksPath. The user’s XDG ignore and attributes files and init.templateDir also apply. That gap affects production now and is fixed separately in T-KAU0-backup-isolate-store-git.
A git built with RUNTIME_PREFIX finds its exec-path by removing bin or libexec/git-core from its own path. Contents/MacOS/git matches neither, so it falls back to the prefix compiled in on the build machine. The same compiled-in prefix decides where init looks for templates.
lib.rs explains why the crate shells out rather than linking gix or git2. One reason is “every machine that ran backups already has git”. That is no longer true for the installed app.
The app already bundles one sidecar. products/backup/src-tauri/tauri.bundle.conf.json lists externalBin: ["binaries/family-cli"], and its beforeBuildCommand runs scripts/stage-family-cli.ts. That script builds into src-tauri/binaries/ with the host target-triple suffix Tauri expects. Tauri strips the suffix and copies the binary to Contents/MacOS/. determined-platform/src/sidecar.rs finds it with exe_sibling, beside current_exe().
api/v1/diagnostics.rs::get_paths reports the backup store path. It reports nothing about git.
The app is macOS-only.

Git subcommands backup runs

Subcommand	Caller	Builtin
`init --bare`	`config::ensure_store` via `git::init_bare`	yes
`config`	`config::ensure_store`	yes
`add -A`, `add -f`	`snapshot`	yes
`diff --cached`	`snapshot`	yes
`commit -m`	`snapshot::commit`	yes
`rev-parse`, `rev-list --count`	`restore`, `snapshot`, `prune`, `git::has_commits`	yes
`log`	`history`, `status`, `prune`	yes
`diff-tree`	`history::changes_for`	yes
`ls-tree`	`index`, `restore`	yes
`show`, `cat-file blob`	`index`, `restore`	yes
`reset --no-refresh`	`prune`	yes
`clone --bare --depth file://`	`prune` via `git::clone_shallow`	yes, but see below

Every subcommand is a builtin of the main binary. One helper is needed: a file:// clone runs git-upload-pack as a separate process and finds it on git’s exec-path. In a git install, libexec/git-core/git-upload-pack is a link to the main binary. So the bundle needs the main binary plus an exec-path directory with a git-upload-pack entry. A future HTTPS remote (B-NNS4-backup-off-machine-remote-with-full-history) would also need git-remote-http, which is a separate program of about 1-2 MB. A push to a local or SSH remote would need git-receive-pack on the far side, not in this bundle.

Proposed

Piece	Shape
Build	Build git from a pinned release tag with its Makefile: `NO_GETTEXT=1 NO_PERL=1 NO_TCLTK=1 NO_PYTHON=1`, no PCRE, no `RUNTIME_PREFIX`. It links only macOS system libraries (libz, libiconv, CoreFoundation). Check with `otool -L`. Building needs a C compiler, so the Command Line Tools, on the build host. That is fine for local builds. A CI build would fetch a prebuilt pinned artifact instead.
Arch	arm64 only, from `rustc -vV`’s host triple, like `stage-family-cli.ts`. The app itself is built for the host triple and is not universal, so a universal git adds about 3.5 MB for nothing. Revisit if the app goes universal.
Stage	`scripts/stage-git.ts`, run from `beforeBuildCommand` beside `stage-family-cli.ts`. It builds the pinned version once and caches it under fleet’s cache home (`@sksizer/fleet`, `cache()` scoped by git version and triple), per S-0011. It then copies it into `src-tauri/binaries/` with the triple suffix. Shell-outs go through the command seam.
Bundle	`git` as a second `externalBin`, so it lands in `Contents/MacOS/git`. Its exec-path directory (`libexec/git-core/` with a `git-upload-pack` entry) and git’s `COPYING` ship through `bundle.resources`.
Exec-path	Decided: no `RUNTIME_PREFIX`. Whenever `git.rs` uses the bundled git, it sets `GIT_EXEC_PATH` to the bundled `libexec/git-core` on every child. The prune’s `file://` clone then finds `git-upload-pack` there. Rejected alternative: pass `--upload-pack=<bundled git-upload-pack>` to the shallow clone only; it fixes one call and leaves any other helper lookup on the build-machine prefix.
Templates	No bundled templates. T-KAU0-backup-isolate-store-git makes `init_bare` pass `--template=`, so the compiled-in template path is never read.
Resolve	`git.rs` resolves the program once, in a `OnceLock`, used by all four spawn sites, and spawns it by absolute path. Order: an explicit override env var (`DETERMINED_BACKUP_GIT`), then the bundled git beside `current_exe()`, then `git` on `PATH` (dev and tests).
Isolate	Bundling-specific env on every child: `GIT_CONFIG_NOSYSTEM=1` and `GIT_CONFIG_GLOBAL` pointed at a missing file, so the user’s `~/.gitconfig` and the system config no longer apply, and `GIT_TERMINAL_PROMPT=0`. This applies to PATH git too, so dev matches the app. The store-config pins for ignore, attributes and templates come from T-KAU0-backup-isolate-store-git.
Diagnose	The diagnostics report shows the resolved git path, where it came from (override, bundled, PATH) and `git --version`.
License	`COPYING` from the pinned git source ships in the bundle. The source for the shipped version is recorded (tag and any patches).

Isolation is a behavior change. User settings in ~/.gitconfig, such as core.fsmonitor or core.hooksPath, stop applying to the store. That is the intent: the store’s own config and info/exclude decide behavior, not machine config. Note it in the change.

Approach

Write scripts/stage-git.ts. Pin the git version in one constant. Build into the fleet cache, keyed by version and triple, and stage binaries/git-<triple> plus the exec-path directory and COPYING. Check otool -L shows only /usr/lib and /System libraries.
Add binaries/git to externalBin and the exec-path directory and COPYING to bundle.resources. Build the app and confirm the layout inside Determined Backup.app.
In git.rs, add the one resolution point and use it in all four spawn sites. Spawn by absolute path. Set GIT_EXEC_PATH when the source is bundled. Keep git.rs the only module that names Command.
Add the isolation env to every child in git.rs. Fold the test-only GIT_CONFIG_* entries in scratch() into it and remove the duplicates.
Add the git path, source and version to the diagnostics report.
Update lib.rs’s “every machine already has git” reason and the git_env doc comment in config.rs.
Add the tests in AC-1 and AC-2. Record the app size before and after.

Areas

Area	Note
`solutions/determined/crates/determined-backup/src/git.rs`	One resolution point; isolation env
`solutions/determined/crates/determined-backup/src/lib.rs`	Update the shell-out rationale
`solutions/determined/crates/determined-backup/src/config.rs`	`git_env` doc comment
`solutions/determined/crates/determined-platform/src/api/v1/diagnostics.rs`	Report git path, source and version
`solutions/determined/scripts/stage-git.ts`	New: build, cache and stage pinned git
`solutions/determined/products/backup/src-tauri/tauri.bundle.conf.json`	`externalBin`, resources, `beforeBuildCommand`

Acceptance criteria

AC-1: git.rs spawns the resolved git by absolute path and sets GIT_EXEC_PATH when the source is bundled. A test points the override at a staged git with its own exec-path directory, runs with no git on PATH, and does a snapshot, a restore and a prune. The prune’s file:// shallow clone succeeds, which proves git-upload-pack came from the bundled exec-path.
AC-2: A unit test covers the resolution order: override beats bundled, bundled beats PATH, and PATH is the fallback. The diagnostics report shows the matching source in each case.
AC-3: cargo test -p determined-backup and the parity cases pass with system git on PATH and no bundled git present.
AC-4: The git version is pinned in one place. The diagnostics report shows the resolved git path, its source (override, bundled or PATH) and its git --version output.
AC-5: otool -L on the bundled git lists only /usr/lib and /System libraries.
AC-6: The user’s global and system git config no longer reach backup’s git children. A test with a global config that sets commit.gpgsign=true and core.hooksPath still snapshots without signing or running hooks. The change notes this behavior change.
AC-7: COPYING from the shipped git version is inside the app bundle. The task’s completion note records the git tag and patches (none expected).
AC-8: The app size increase is recorded in the completion note.
AC-9: Before any public distribution of the app, someone reviews the GPLv2 obligations: ship COPYING, and provide or offer the corresponding source for the exact shipped version.

Out of scope

Signing and notarization, as for the app. When they land, the bundled git must be signed with the app.
Linux and Windows. The app is macOS-only. The resolution order stays generic so another platform only needs its own staged binary.
HTTPS remotes and git-remote-http. They come with B-NNS4-backup-off-machine-remote-with-full-history.
Replacing the binary with a library. See B-ZDX9-backup-embed-a-git-library-instead-of-a-binary.

Dependencies

Ordered after the retention and pruning tasks: T-A97O-backup-prune-operation, T-94OR-backup-retention-setting and T-YON4-backup-prune-tests. This is scheduling order, not a technical need. No other task depends on this one.
T-KAU0-backup-isolate-store-git for the store-config pins (ignore, attributes) and the empty init template. The bundled git relies on --template=, since its compiled-in template path does not exist on a user’s machine.

Discovery context

Exec audit note (not an AC): once the app builds on Linux CI, strace -f -e trace=execve can list every program it runs without sudo. On macOS the same check needs fs_usage or eslogger, which need sudo.
Licensing note (not legal advice): git is GPLv2. Shipping it as a separate program beside the app is mere aggregation, so the app’s own source is not affected. Personal installs trigger no obligations. Distribution does, which is AC-8.
Size guide from existing builds: Apple’s Xcode git 2.54 arm64 main binary is 3.7 MB, with libexec/git-core at 9.8 MB, but most of that directory is links to the main binary. The Command Line Tools git is a 7.3 MB universal binary. Homebrew’s git links Homebrew pcre2 and gettext, so it cannot be copied into the app as it is.

Depends on

T-A97O-backup-prune-operation, T-94OR-backup-retention-setting, T-YON4-backup-prune-tests, T-KAU0-backup-isolate-store-git

← Back to Tasks