Abstract
Preserving privacy is a growing concern in our society where sensors and cameras are ubiquitous. In this work, for the first time, we propose a trainable image acquisition method that removes the sensitive identity revealing information in the optical domain before it reaches the image sensor. The method benefits from a trainable optical convolution kernel which transmits the desired information while filters out the sensitive content. As the sensitive content is suppressed before it reaches the image sensor, it does not enter the digital domain therefore is unretrievable by any sort of privacy attack. This is in contrast with the current digital privacy-preserving methods that are all vulnerable to direct access attack. Also, in contrast with the previous optical privacy-preserving methods that cannot be trained, our method is data-driven and optimized for the specific application at hand. Moreover, there is no additional computation, memory, or power burden on the acquisition system since this processing happens passively in the optical domain and can even be used together and on top of the fully digital privacy-preserving systems. The proposed approach is adaptable to different digital neural networks and content. We demonstrate it for several scenarios such as smile detection as the desired attribute while the gender is filtered out as the sensitive content. We trained the optical kernel in conjunction with two adversarial neural networks where the analysis network tries to detect the desired attribute and the adversarial network tries to detect the sensitive content. We show that this method can reduce 65.1% of sensitive content when it is selected to be the gender and it only loses 7.3% of the desired content. Moreover, we reconstruct the original faces using the deep reconstruction method that confirms the ineffectiveness of reconstruction attacks to obtain the sensitive content.
Introduction
In an era of ubiquitous sensing, privacy has become a paramount concern. Traditional privacy-preserving methods typically operate in the digital domain, sanitizing data only after it has been captured and stored. This approach leaves a critical window of vulnerability: if an attacker gains direct access to the raw sensor data or device memory, these digital safeguards can be bypassed. Furthermore, digital filters like adversarial noise or feature masking impose significant computational and power burdens, which are often prohibitive for resource-constrained edge devices.
Proposed Methodology: Hybrid Optical-Digital Network
This work introduces a novel "privatizing" camera front-end that utilizes a trainable optical kernel to strip sensitive information from a scene before it reaches the digital image sensor. By performing this filtering passively in the optical domain, the system is physically invulnerable to digital-access attacks and incurs zero computational overhead.
The architecture consists of two primary components:
- Trainable Optical Kernel: An amplitude-varying mask (implemented via a Spatial Light Modulator) that performs a mathematical convolution on incoming light. This passive process requires no electricity or processing time.
- Digital Back-end: A lightweight digital network (e.g., VGG11 or MobileNetV2) that performs classification on the "privatized" feature maps captured by the sensor.
Training Strategies
The optical kernel is optimized using two distinct data-driven strategies:
- Generative Adversarial Privacy (GAP): The kernel is trained alongside an "analyzer" (detecting desired attributes) and an "adversary" (attempting to extract sensitive info). The kernel learns to pass information necessary for the analyzer while obfuscating data for the adversary.
- Inverse Siamese (IS): Utilizing a contrastive loss, this method forces the kernel to produce feature maps that are indistinguishable across different sensitive categories (e.g., gender) while remaining distinct for the target task.
Experimental Results
The system was evaluated using the CelebA dataset, focusing on tasks such as smile detection (desired attribute) while filtering out gender (sensitive attribute).
Key findings include:
- Privacy-Utility Trade-off: Using the VGG11 architecture with the Inverse Siamese strategy, the system reduced sensitive gender information by 65.1%, while smile detection accuracy dropped by only 7.3% compared to a non-private baseline.
- Zero-Cost Processing: The physical nature of the optical convolution ensures no added latency or power consumption.
- Reconstruction Resilience: Deep learning-based reconstruction attacks (Visual Representation Inverters) failed to recover identifiable faces from the privatized sensor data, resulting only in unidentifiable blobs.
Conclusion and Future Work
This approach demonstrates that privacy can be effectively integrated into the image acquisition process at the speed of light. While the current implementation is limited to grayscale information, future research will focus on color integration, ensuring mathematical irreversibility of the optical kernel, and combining this front-end with advanced digital privacy techniques like homomorphic encryption.
Summary updated