An Analytical Workflow for Clustering Forensic Images (Student Abstract)

Sara Mousavi, Dylan Lee, Tatianna Griffin, Dawnie W. Steadman, Audris MockusAAAI 2020 · DOI

Abstract

Researchers at the University of Tennessee have developed an unsupervised workflow to organize massive, graphic datasets of human decomposition images. By integrating ResNet50 deep learning features with environmental metadata—specifically Accumulated Degree Days (ADD)—the team improved clustering precision from 64% to 89%. This approach provides a structured framework for forensic experts to navigate and label sensitive, high-volume datasets.

Introduction

Forensic anthropology relies on the systematic documentation of human decomposition to support law enforcement and medical research. The University of Tennessee’s Anthropology Research Center (ARC) maintains a repository of over 1 million images collected over eight years. Managing this volume is a significant challenge; manual labeling is impractical due to the scale and the graphic nature of the content.

Standard computer vision models, typically trained on datasets like ImageNet, struggle with forensic imagery. As a body decays, the human form deforms significantly, rendering traditional "human part detection" ineffective. The visual textures of decaying tissue are often outside the distribution of standard training data.

A subject in an advanced stage of decomposition, illustrating the visual complexity and deformation that challenges standard computer vision models.

Methodology

The researchers proposed a hybrid unsupervised workflow that combines deep learning with domain-specific environmental science to bridge the gap between pixel data and biological reality.

The proposed analytical workflow, integrating ResNet feature extraction with weather-based metadata for improved clustering.
  1. Feature Extraction: A ResNet50 model extracts 2048-dimensional feature vectors, which are then reduced to 256 dimensions using Principal Component Analysis (PCA).
  2. Domain Knowledge Integration: To capture decomposition-related nuances missed by standard models, the team incorporated Accumulated Degree Days (ADD)—a forensic metric calculating heat, humidity, and wind speed to estimate postmortem intervals.
  3. Feature Fusion: A 3-value ADD vector is appended to the image features, creating a 259-length vector for each image.
  4. Clustering and Visualization: The data is projected into 2D space using t-SNE to estimate natural cluster counts, followed by K-Means clustering.
  5. Human-in-the-Loop: A custom web interface allows experts to quickly identify "miss-clustered" images and refine the groups manually.

Results and Discussion

The integration of environmental metadata proved critical. Using image features alone yielded a clustering precision of 64%. Adding weather-based ADD metadata—acting as a biological "proxy" for the decay process—boosted precision to 89%. The workflow successfully organized a subset of 8,507 images into 15 meaningful clusters representing various body parts and stages of decay.

This study highlights that pixel data alone is insufficient for specialized forensic tasks. ImageNet-trained models have significant "blind spots" regarding biological decay. By providing the model with a proxy for the biological "clock" of the body, the researchers have created a scalable path toward training more robust supervised models for forensic pathology.

Conclusion and Future Work

While the workflow currently requires human-in-the-loop refinement to merge clusters, it represents a major step in managing large-scale forensic datasets. These high-purity clusters can now be used to generate labels for training supervised models for object detection and semantic segmentation in forensic contexts. Future challenges include the scalability of metadata synchronization for non-controlled crime scene environments.

Summary updated

BibTeX
@inproceedings{mousavi2020analytical,
  author = {Sara Mousavi and Dylan Lee and Tatianna Griffin and Dawnie W. Steadman and Audris Mockus},
  title = {{An Analytical Workflow for Clustering Forensic Images (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i10.7212},
}

Loading related content...

Figure