Introduction
Forensic suspect identification is a critical yet flawed process, often relying on a witness's ability to recall and describe complex facial features. Traditional methods, such as sketches or rigid search queries, struggle with the subjectivity of human memory. This paper presents FaIRCoP (Facial Image Retrieval using Contrastive Personalization), an automated framework that leverages iterative user feedback to refine suspect searches. By combining unsupervised disentangled representations with a novel contrastive loss function, FaIRCoP learns a witness's unique "mental model" of a suspect in real-time.
The Challenge of Subjective Retrieval
Identifying a suspect from a large dossier presents two primary hurdles:
- High Cognitive Load: Describing specific facial features (e.g., "narrow eyes") is difficult when relying on fuzzy visual memories.
- Subjectivity: "Similarity" is perceived differently by every individual. Fixed retrieval metrics fail to adapt to how a specific witness prioritizes certain facial characteristics over others.
FaIRCoP addresses these issues through a "human-in-the-loop" approach, requiring only simple binary feedback—selecting images that look "similar" or "dissimilar" to the memory.
The FaIRCoP Framework
The architecture of FaIRCoP is designed to be both robust and adaptive. It consists of three main pillars:
1. Unsupervised Disentangled Representations
Instead of standard deep features, the framework utilizes the MIX method to decompose facial images into independent "chunks" of data. Each chunk represents a distinct factor (e.g., face shape, complexion) without requiring manual labels. This disentanglement makes the system more interpretable and resilient to noise.
2. Separating Cluster Loss (SCLoss)
The "secret sauce" of the system is the SCLoss, based on the N-pair loss paradigm. During each search iteration, images selected by the user are treated as positive samples, while unselected ones are negative. The projection network is trained to cluster these positive samples together in a latent space while pushing negative samples away.
3. Online Optimization and Anchoring
The system updates its projection network in real-time. To maintain consistency and prevent the model from "forgetting" earlier feedback, an "anchoring" mechanism includes a subset of previously chosen images in every new training iteration.
Experimental Evaluation
The framework was tested on two significant datasets: the Criminal Dataset (39,196 real-world mugshots) and the CelebA Dataset (a benchmark of over 200,000 celebrity images).
Iterative Search Performance
FaIRCoP significantly outperformed traditional baselines like the Rocchio Algorithm and FaceFetch. In the Criminal Dataset, it achieved the target image in far fewer iterations, reaching an Average Convergence Iteration (ACI) of 0.44, compared to 0.28 for FaceFetch and 0.08 for Rocchio.
Key Findings
- Weak Supervision Efficacy: Simple "like/not like" labels provide sufficient signal for the contrastive learner to rapidly narrow the search space.
- Fairness and Bias: A study using "Group Demographic Parity" revealed that FaIRCoP’s disentangled representations were less biased than standard pre-trained models, a crucial factor for law enforcement applications.
- User Preference: A study with 20 participants showed that users found FaIRCoP more responsive and preferable compared to existing retrieval systems, with an Average Relevance score of 0.72.
Conclusion
FaIRCoP represents a significant step forward in forensic technology by bridging the gap between human perception and automated retrieval. While challenges remain regarding data privacy and the "cold start" problem, the framework's ability to learn personalized similarity metrics offers a faster, more accurate, and fairer way to identify suspects. Future work will focus on refining the real-time user experience and exploring more diverse feedback mechanisms.
Summary updated