Abstracted Representations for Privacy-Preserving Computer Vision

41 concepts41 papers1 proposal

The central challenge in visual monitoring is a fundamental tension between utility and privacy. To detect a patient falling in a hospital or to analyze student engagement in a classroom, a system needs high-fidelity data. However, standard RGB video captures identifiable facial features, skin tones, and private environments, creating significant ethical and legal barriers to deployment.

Research in this field focuses on decoupling situational awareness (what is happening) from individual identity (who is doing it). This is achieved by moving away from raw pixel data toward abstracted representations that retain geometric and temporal signals while discarding identifiable information.

Alternative Modalities and Geometric Abstraction

One of the most effective ways to preserve privacy is to avoid capturing high-resolution visible light altogether. By using sensors that record physical properties rather than appearance, researchers can extract necessary clinical or safety data without risking identity exposure.

In intensive care units (ICUs), where patient dignity is paramount, Liu et al. and Rinaldo et al. demonstrate that 3D point clouds from depth sensors provide sufficient geometric detail to recognize mobility care activities—such as "turning in bed"—without recording facial features. Similarly, Hamoud et al. use depth maps to train self-supervised models for operating room awareness, significantly reducing the privacy risks associated with standard video.

Thermal imaging offers another layer of abstraction by capturing heat signatures. Armin et al. show that pose estimation can be performed on Long-Wave Infrared (LWIR) data by "hallucinating" features typically found in RGB images, allowing for accurate monitoring even under blankets. For extreme privacy, Hanosh utilizes ultra-low-resolution $8 \times 8$ thermopile arrays to detect convulsive seizures, proving that even highly granular data can be medically useful if the temporal patterns are preserved.

Skeletal and Pose-Based Anonymization

A recurring mechanism for privacy is the reduction of a human subject to a "skeleton"—a set of 2D or 3D coordinates representing body joints. This format is inherently anonymized, as it strips away all textures and facial details.

  • Healthcare: Asif et al. and Chen et al. use pose keypoints to detect falls, arguing that this representation generalizes better across different home environments because it is independent of background clutter.
  • Education: Yu et al. transform raw classroom footage into skeletal data to analyze student behaviors like "reading" or "raising hands," providing pedagogical insights while strictly protecting student identities.
  • Sign Language: In Sign Language Recognition (SLR), Selvaraj et al. and Gokul NC et al. utilize pose-based pretraining to build large-scale datasets. This approach bypasses the privacy concerns of sharing raw video of signers, which has historically hindered the growth of SLR research.

However, skeletal abstraction is not a perfect substitute. Moryossef et al. note that current pose estimation tools often struggle with the complex hand-face interactions essential for sign language, suggesting that while skeletons provide privacy, they may currently sacrifice some linguistic nuance.

Transform-Domain and Mathematical Obfuscation

When raw video must be used, researchers apply mathematical transformations to the data stream to hide sensitive regions while maintaining the intelligibility of the scene for human operators.

Dufaux et al. propose scrambling video by pseudo-randomly flipping the signs of DCT or wavelet coefficients within specific regions of interest. This makes individuals unidentifiable but allows security personnel to see the background and general actions. A different approach by Thorpe et al. uses "coprime blur," where two mathematically distinct kernels blur the video. The details can only be recovered by combining both streams, creating a tiered access system where public viewers see only general activity.

Balancing Privacy and Utility in Analytics

The final frontier of this research is ensuring that even aggregate data cannot be used to re-identify individuals. Cangialosi et al. introduce Privid, which applies differential privacy to video analytics. Instead of trying to perfectly blur every face—which is prone to failure—they bound the duration an individual's presence can influence a query result. This provides formal privacy guarantees for urban analytics even when using "black-box" deep learning models.

As O’Toole highlights in the context of road safety, the goal is to reach a point where we can de-identify drivers while preserving their facial actions and emotions. By shifting the focus from pixels to semantic and geometric abstractions, the field aims to unlock sensitive data for research without compromising the privacy of the individuals within the frame.

Key Foundational Work

Scrambling for Video Surveillance with Privacy (2006) — contributed 1 concept elements to this direction.
A Human Behavior Analysis based on Motion Trajectory Measurement for Guiding Services (2004) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_visual_privacy_preservation_6482f34c,
  title = {visual privacy preservation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/visual-privacy-preservation/rdr_6482f34c3f4b4907984dc93e98ad62f5},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_visual_privacy_preservation_6482f34c_v1,
  title = {visual privacy preservation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/visual-privacy-preservation/rdr_6482f34c3f4b4907984dc93e98ad62f5/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake