Introduction
In real-world surveillance and security applications, face recognition systems frequently encounter a significant resolution mismatch. While state-of-the-art models are typically trained on high-resolution (HR) images (e.g., 112x112 pixels), the actual input from long-range cameras or cropped surveillance footage can be as small as 7x7 or 14x14 pixels. The industry standard for bridging this gap has long been up-sampling low-resolution (LR) images via interpolation or super-resolution. However, this approach often falls into the "up-sampling trap," where artificial artifacts and identity-blurring noise are introduced, leading to degraded recognition accuracy.
This paper introduces the Branch-to-Trunk Network (BTNet), a novel architecture designed to learn unified representations across varying resolutions without relying on traditional up-sampling. By utilizing resolution-specific adapters, BTNet achieves superior performance on low-resolution benchmarks while significantly reducing computational overhead.
The Up-sampling Trap and the 32-Pixel Threshold
The authors identify a critical flaw in the standard rescaling pipeline: interpolation error. When an image is smaller than 32x32 pixels, the mathematical error introduced by up-sampling to 112x112 "soars," obscuring discriminative identity features. While super-resolution techniques can produce visually pleasing images, they often "hallucinate" features from other identities, which is unacceptable in high-stakes security contexts. BTNet addresses this by processing LR images at their native resolution for as long as possible, avoiding the injection of noise from empty up-sampled pixels.
Methodology: The BTNet Architecture
BTNet moves away from a "one-size-fits-all" approach in favor of a modular, resolution-aware design consisting of four key components:
- Branch Networks (BNets): These serve as resolution-specific adapters. Instead of forcing a 7x7 image into a 112x112 input layer, a dedicated branch processes the 7x7 input and "implants" the resulting feature maps into the appropriate depth of the main network where the spatial dimensions match.
- Trunk Network (TNet): This is a shared encoder that receives implanted features from the various BNets. By sharing the trunk, the model ensures that regardless of the input resolution, the final high-dimensional embedding lives in a compatible metric space.
- Branch Distillation: To enhance the discriminative power of LR branches, the authors employ a teacher-student training strategy. The HR features from the trunk act as a guide, using Mean Squared Error (MSE) loss to distill knowledge from the HR domain into the LR branches.
- Backward-Compatible Training (BCT): To ensure practical utility, BTNet is trained to be interoperable with existing face databases. This prevents the need for "re-enrollment," allowing new multi-resolution models to match against legacy HR galleries.
Experimental Results
BTNet was evaluated using the MS1Mv3 dataset for training and several challenging benchmarks for verification and identification.
Surveillance Performance
On the QMUL-SurvFace dataset—a benchmark consisting of "native" low-resolution surveillance faces (average 24x20 pixels)—BTNet established a new state-of-the-art. The "BTNet (max+ceil)" variant achieved a TPIR20@FPIR=0.1 of 20.6%, a substantial improvement over standard models like AdaFace (16.5%) and ArcFace (10.1%).
Cross-Resolution Accuracy
In tests matching 112x112 images against 7x7 or 14x14 crops, BTNet demonstrated a "Cross-Resolution Gain" of +2.02 to +5.00 over baseline models. This confirms that the unified representation effectively bridges the gap between high-quality gallery images and low-quality probes.
Efficiency and Storage
Beyond accuracy, BTNet offers significant practical advantages in terms of deployment:
- Computational Savings: Because LR images bypass the early, high-resolution layers of the network, the model performs fewer Floating Point Operations (FLOPs). For a 7x7 input, BTNet requires approximately 3.0G FLOPs, compared to 12.6G for a standard rescaled model—a 75% reduction.
- Minimal Overhead: BTNet does not require storing multiple full models for different resolutions. By only storing the small branch parameters and resolution-aware Batch Normalization statistics, the additional storage requirement ranges from only 1.1% to 48.9% of a single full model.
Conclusion and Future Work
BTNet provides a robust solution to the multi-resolution face recognition problem by replacing problematic up-sampling with resolution-specific feature implantation. While the model currently selects branches based on pixel dimensions, the authors note a potential "dislocation" where a large but blurry image might be better served by an LR branch. Future research will focus on quality-aware branch selection, where the model automatically determines the optimal processing path based on the actual information content of the image rather than just its spatial resolution.
Summary updated