Face detection and photo identity matching
State: open/building · Kind: module · Audience: system
Summary
Section titled “Summary”- Finds faces in photos, embeds them, clusters them into people and matches them to vault identities.
- Writes sidecar and identity stores that stay byte-identical to the Node implementation.
Statement
Section titled “Statement”One face pipeline (detect, align, embed, cluster or match) serves two surfaces: the photo annotator over one attachments directory, and photo-match over the vault plus Apple Photos. On the Rust arm photo-match serves scan-photos, embed-apple and match. embed-vault and run-all are refused on purpose, because they write stores the merge retires. The crate is an importable library with no bin of its own. It keeps on-disk JSON stores byte-exact so hours of earlier CPU inference are not redone.
What it provides
Section titled “What it provides”- SCRFD detection preprocessing, FPN decode and NMS
- Similarity-transform alignment and ArcFace embedding via ONNX
- L2 normalisation, cosine, DBSCAN and k-NN vote clustering
- Git-committed JSON sidecars and derived identity and embedding stores
- Apple Photos person to vault Human linking, person pages and photo pages (scan-photos, embed-apple and match; embed-vault and run-all are refused)
- Model-presence gate that refuses rather than returning empty face lists
Underlying implementation
Section titled “Underlying implementation”- solutions/determined/crates/determined-photos/src/inference.rs holds the ONNX face detector and embedder
- solutions/determined/crates/determined-photos/src/cluster.rs holds DBSCAN and vote clustering
- solutions/determined/crates/determined-photos/src/port.rs implements the photos and photo-match ports
Locations
Section titled “Locations”solutions/determined/crates/determined-photos/solutions/determined/crates/determined-photos/src/lib.rssolutions/determined/crates/determined-photos/src/inference.rs#FaceDetectorsolutions/determined/crates/determined-photos/src/inference.rs#FaceEmbeddersolutions/determined/crates/determined-photos/src/cluster.rs#dbscansolutions/determined/crates/determined-photos/src/port.rs#RustPhotossolutions/determined/crates/determined-photos/src/port.rs#RustPhotoMatch