Pre-Capture Privacy via Adaptive Single-Pixel Imaging

Yoko Sogabe, Shiori Sugimoto, Ayumi Matsumoto, Masaki KitaharaWACV 2025

TL;DR

This paper introduces a novel "pre-capture" privacy method that optically eliminates specific "anonymized targets" (like faces or license plates) directly during image acquisition, rather than post-processing. By adaptively controlling aperture patterns in a single-pixel imaging setup, it achieves privacy without sacrificing overall image quality for other objects, enabling broader computer vision applications.

What Problem Does This Paper Address?

As cameras become ubiquitous, privacy concerns are escalating. Traditional privacy solutions involve removing personal information after an image is captured (post-capture privacy). However, this approach carries the risk of the original, sensitive data being leaked. This paper tackles the challenge of achieving privacy at the point of capture (pre-capture privacy) in a way that is flexible, doesn't globally degrade the image, and can anonymize various types of objects, not just faces. The goal is to strike a better balance between privacy and the utility of captured images for downstream tasks.

Methods

The core innovation lies in an Adaptive Aperture Pattern Generator (APG) integrated into a Single-Pixel Imaging (SPI) framework.

  • Single-Pixel Imaging (SPI): Instead of a traditional sensor array, SPI uses a photodetector and a Digital Micromirror Device (DMD) to modulate incoming light with an aperture pattern. The image is reconstructed from many such modulated measurements, gradually becoming clearer with each acquisition.
  • Adaptive Aperture Pattern Generator (APG): This is the novel component. The APG acts as a feedback mechanism. It takes the partially reconstructed image from previous acquisitions as input and adaptively generates the next aperture pattern. The goal of this pattern is to avoid sampling the region identified as the anonymized target. This is achieved using a U-Net-like deep learning model.
  • Feedback Loop: The APG implicitly estimates the location of the anonymized target from the current unclear reconstructed image and then outputs an aperture pattern designed to minimize the light captured from that specific area. This process is repeated iteratively.
  • Loss Function: The APG and a reconstructor network are jointly trained. The loss function has two components: one for image quality (Lmse) and one for anonymity (Lanony). Lanony is specifically designed to penalize the presence of the anonymized target (e.g., faces, license plates) in the reconstructed image.
  • Block-based CS: For computational efficiency, the image is processed in blocks, and the reconstruction is performed jointly across these blocks.

Datasets

  • Training: The paper mentions using the BSDS500 and DIV2K datasets for general image reconstruction training, and CelebA for face anonymization training. For license plate anonymization, the Cars dataset was used.
  • Evaluation:
    • Face Anonymization: LFW (Labeled Faces in the Wild) and AgeDB-30 datasets were used for evaluating anonymity via face recognition tests.
  • Object Detection: PASCAL VOC 2007 was used to evaluate the image quality of non-anonymized objects.
  • License Plate Anonymization: The ALPR-Unconstrained test condition was followed for anonymity evaluation.

Key Insights and Ideas

  • Privacy at the Source: The most significant insight is demonstrating that privacy can be enforced optically during image capture, rather than relying on software-based post-processing. This fundamentally shifts the privacy paradigm.
  • Adaptive Sampling for Anonymity: The APG's ability to learn and adapt its sampling strategy based on partially acquired data is crucial. It's not just about masking; it's about intelligently avoiding the capture of specific features.
  • Generalizability: The framework is designed to be adaptable to different anonymized targets (faces, license plates) by retraining the APG, without hardware changes. This is a key advantage over methods specifically designed for faces.
  • Balancing Privacy and Utility: The method successfully demonstrates that anonymized targets can be obscured while maintaining high image quality for other scene elements, making the images usable for various computer vision tasks.

Key Results

  • Face Anonymization: The proposed method effectively conceals facial details, making individuals difficult to identify, as evidenced by high AUC scores on LFW and AgeDB-30 (e.g., AUC of 0.675 for LFW, compared to 0.999 for original images).
  • License Plate Anonymization: Detailed license plate information was successfully concealed, with anonymity scores indicating no correct recognition of license plates.
  • Image Quality: Despite anonymization, the image quality for non-target objects remained high. For face anonymization, PSNR was comparable to or better than defocus blurring, and object detection (PASCAL VOC 2007 mAP) was significantly better than defocus blurring (0.6117 vs. 0.2535).
  • Robustness: The method showed strong robustness against image restoration attacks, where a "Defocus" method was easily compromised, but the proposed method maintained anonymity.
  • Computational Efficiency: While SPI inherently requires multiple acquisitions, the APG's computation is fast (less than 0.5 seconds per image), and the overall capture time for the prototype was around 30 seconds.

Limitations and Future Work

  • Real-time Imaging: A significant limitation is that SPI, even with the proposed method, is not yet real-time due to the sequential nature of acquisitions. The authors suggest that advancements in DMD modulation speed and mechanical scanning could enable real-time performance.
  • Anonymity Against Temporal Attacks: The current evaluation focuses on single-image reconstruction. The authors acknowledge that for video, attackers might exploit temporal information across frames, and this aspect needs further evaluation.
  • Reconstruction-Free Inference: The paper primarily evaluates anonymity against reconstruction-based attacks. Future work could explore anonymity from the perspective of reconstruction-free inference methods.
  • Simultaneous Multiple Targets: The current system focuses on one type of anonymized target at a time. Future work aims to expand the system to handle multiple types of anonymized targets simultaneously.

Summary updated

BibTeX
@inproceedings{sogabe2025pre,
  author = {Yoko Sogabe and Shiori Sugimoto and Ayumi Matsumoto and Masaki Kitahara},
  title = {{Pre-Capture Privacy via Adaptive Single-Pixel Imaging}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2025},
}
Figure