FHE Benchmarking Results - Face Recognition (Large Batch - 1,024)
+
Timing and batched quality values are averages across 3 measurement run(s). Single-pair scores and labels are listed per run. Durations are wall time unless labeled as worker time.
FHE Benchmarking Results - Face Recognition (Medium Batch - 256)
+
Timing and batched quality values are averages across 3 measurement run(s). Single-pair scores and labels are listed per run. Durations are wall time unless labeled as worker time.
FHE Benchmarking Results - Face Recognition (Single - 1)
+
Timing and batched quality values are averages across 1 measurement run(s). Single-pair scores and labels are listed per run. Durations are wall time unless labeled as worker time.
FHE Benchmarking Results - Face Recognition (Small Batch - 128)
+
Timing and batched quality values are averages across 3 measurement run(s). Single-pair scores and labels are listed per run. Durations are wall time unless labeled as worker time.
The face recognition workload implements an encrypted face verification functionality.
+The input is a pair of in-the-wild face images drawn from the CelebA dataset.
+The goal is a binary decision: determine whether the two images depict the same person (a genuine pair) or two different people (an impostor pair).
+For each pair the model computes, under homomorphic encryption, a similarity score between the two faces; only that score is decrypted. A genuine/impostor decision corresponds to thresholding this score, and the harness evaluates quality by sweeping thresholds to compute the Equal Error Rate and TAR@FAR (see below).
+
+
+
The workload includes two interfaces for benchmark submitters to implement:
+
+
+
Single Inference: On a single pair of face images. The submitter is expected to implement the verification functionality for one pair.
+
Batched Inference: On a batch of face-image pairs. The submitter is expected to implement the verification functionality for a batch of pairs. This can be more efficient than single inference for large datasets leveraging parallelism.
+
+
+
+
+
+
Size
+
Pairs (N)
+
+
+
+
+
Small
+
128
+
+
+
Medium
+
256
+
+
+
Large
+
1,024
+
+
+
+
+
Each variant samples N pairs from a fixed master set of 1,024 screened CelebA pairs (512 genuine, 512 impostor); the Large variant uses the full set.
+Hence, there are a total of four variants of this workload: single inference and batch inference for each one of the three sizes.
+Submitters need not implement all four, instead each submitter can implement and report the results of any subset.
+
+
Submission to the benchmarking suite must set the implementation parameters so as to achieve security level of at least 128 bits (against a semi-honest server).
+Submitters must document their choice of parameters and explain why they believe that it meets the 128-bit security mandate.
+The reference CryptoFace submission uses RNS-CKKS with ring dimension N = 216 and a sparse ternary secret (Hamming weight 192); its parameter selection and security analysis are described in [AB25].
+
+
Submissions are also required to meet a quality bar defined relative to a plaintext ArcFace[DGX+19] baseline that the harness evaluates on the same sampled pairs.
+For batched inference, the encrypted model's Equal Error Rate (EER) must be no more than 0.15 above the ArcFace baseline EER on the same pairs.
+EER is used because it is threshold-free and stable across batch sizes; the harness additionally reports the True Accept Rate at fixed False Accept Rates (TAR@FAR = 1% and 0.1%).
+For single inference, the harness reports the encrypted similarity score for the pair.
+
+
+
The face-recognition harness contains a script that can be called to run the implementation of submitters, that script accepts command-line arguments to specify which interface of what instance size to run.
+The harness currently supports verification on CelebA pairs with a reference CryptoFace[AB25] submission (a patch-wise CNN with polynomial activations, evaluated under RNS-CKKS). In the future, this workload may be updated to support more models and more datasets.
+
+
+
+$ uv run python harness/run_submission.py -h
+usage: run_submission.py [-h] [--num_runs NUM_RUNS] [--seed SEED]
+ [--clrtxt CLRTXT]
+ {0,1,2,3}
+
+Run Face Verification FHE benchmark.
+
+positional arguments:
+ {0,1,2,3} Instance size (0-single/1-small/2-medium/3-large)
+
+options:
+ -h, --help show this help message and exit
+ --num_runs NUM_RUNS Number of times to run stages 4-10 (default: 1)
+ --seed SEED Random seed for reproducible pair sampling (default: 42).
+ Fixed by default so all submissions sample identical pairs.
+ --clrtxt CLRTXT Set to 1 to force rerun of the cleartext reference
+
+
+
+
Because the harness generates the evaluation pairs (submission-agnostic) before invoking any submission, and the seed defaults to a fixed value, every submission run with the default configuration is evaluated on the same pairs and is therefore directly comparable.