Abstract
Deep detection networks trained with a large amount of annotated data achieve high accuracy in detecting various objects. However, disaster victim detectors suffer from a lack of real-world training data, as images of victims partially buried by debris are difficult to obtain. This paper proposes generating realistic synthetic images for training. We extract human body parts from open-source datasets and paste them into ruins background images. An unsupervised generative adversarial network (GAN) then harmonizes these parts to fit the background's style (illumination, texture, and color). Fine-tuning a YOLOv5 model on these images significantly improves detection performance, increasing AP@[.5 : .95] from 19.5% to 35.7% and [email protected] from 32.4% to 59.5%.
Introduction
Object detection models have reached impressive levels of maturity for common categories like pedestrians and vehicles. However, in search-and-rescue scenarios following earthquakes or building collapses, these models often fail. Victims are frequently partially buried, with only limbs or torsos visible, and their appearance is obscured by the gray dust and debris of the disaster site. This "domain gap" between standard datasets (e.g., COCO) and real-world disaster scenes necessitates specialized training data that is currently scarce.
Methodology
The proposed framework addresses the data scarcity problem through a three-stage synthetic data generation and training pipeline.
1. Image Composition
Human body parts (limbs, torsos, or full bodies) are extracted from the LIP (Look Into Person) dataset using semantic masks. These parts are then randomly placed onto background images of earthquake ruins and collapsed structures.
2. Unsupervised Image Harmonization
A simple "cut-and-paste" approach results in visual inconsistencies that hinder model generalization. To bridge this gap, we employ an unsupervised GAN to harmonize the foreground body parts with the background environment.
The GAN architecture utilizes a U-Net generator with attention layers and dual discriminators (Global and Local). The Global Discriminator ensures overall image realism, while the Local Discriminator focuses on the boundary between the pasted limb and the debris. This process effectively simulates the "dust factor," adjusting the color and texture of the body parts to match the surrounding ruins.
3. Model Fine-tuning
The final harmonized images are used to fine-tune a YOLOv5 detector, enabling it to recognize partially obscured human features in disaster-specific contexts.
Results and Discussion
The effectiveness of the harmonization process was evaluated using three versions of YOLOv5. The YOLOv5l (Large) model showed the most significant gains. While the baseline model pre-trained on COCO achieved only 19.5% AP@[.5:.95], fine-tuning on simple composite data raised this to 33.6%. The addition of the unsupervised harmonization step further boosted performance to 35.7% AP@[.5:.95] and 59.5% [email protected].
Key insights from the study include:
- The Dust Effect: The GAN's ability to automatically apply "gray" tones to limbs mimics real-world conditions where victims are covered in debris dust.
- Local Context: The local discriminator is vital for ensuring that small, partially visible body parts are not ignored by the network.
- Limb Proxies: Training the model to detect body parts rather than whole persons is a more robust strategy for disaster scenarios.
Conclusion
This research demonstrates that unsupervised image harmonization can effectively synthesize high-quality training data for niche applications where real data is unavailable. By bridging the domain gap between standard person detection and disaster-specific environments, we can significantly enhance the capabilities of search-and-rescue technology. Future work will explore night-time detection using thermal sensors and further refine the model's ability to detect both partial and whole bodies.
Summary updated