Introduction
In remote sensing applications, such as satellite or drone-based surveillance, detecting small and densely packed objects against complex backgrounds remains a significant challenge. To enhance detection performance, researchers often leverage multimodal data, integrating standard RGB imagery with contextual "a priori" information like land-use maps or segmentation masks. However, a persistent issue is the spatial misalignment between these modalities caused by varying sensor angles or low-resolution data sources. Rigid binary masks often lead to performance degradation when shifted by even a few pixels. This paper proposes a method to mitigate these effects by converting misaligned contextual information into probabilistic maps.
Methodology
The core contribution of this work is the transformation of binary contextual masks into "soft" probability gradients. This approach replaces the binary 0/1 representation with a continuous range that reflects the likelihood of an object's presence, providing a "search area" for the detection model rather than a strict, potentially incorrect boundary.
The authors propose two primary methods for generating these probability maps:
- Euclidean Distance Transformation: This method calculates the distance of each pixel to the nearest object boundary and normalizes it, creating a gradient where probability decreases with distance from the mask boundary.
- Neighborhood-based Probability: This approach considers the density of neighboring cells, offering greater resilience to multi-axis misalignments.
The resulting maps are integrated via an early fusion strategy, where they are concatenated with RGB channels and processed through a Faster-RCNN detector with a Feature Pyramid Network (FPN). The approach was validated using the DOTA and iSAID datasets, with synthetic noise (5-10% spatial shifts) injected to simulate real-world misalignment.
Results and Discussion
The proposed probabilistic integration consistently outperformed both RGB-only baselines and standard binary fusion techniques. The baseline RGB model achieved a mean Average Precision (mAP) of 0.48, which improved to 0.53 with the best probabilistic fusion models.
Key findings include:
- Robustness to Noise: Probability maps successfully mitigated the performance drop typically caused by the 5-10% spatial shift in binary masks.
- Contextual Gains: Significant improvements were noted in specific classes when relevant priors were used. For instance, ship detection improved from 0.47 to 0.50 mAP when using harbor masks.
- Spatial Co-occurrence: The model identified non-obvious spatial patterns, such as "plane" masks aiding the detection of "basketball courts," suggesting that the network learns complex geographic layouts beyond simple semantic labels.
Conclusion
By shifting from rigid binary masks to probabilistic representations, this research provides a robust framework for multimodal fusion in the presence of spatial uncertainty. While the current study focused on early fusion, future work will explore mid-fusion strategies and investigate the generalizability of these probabilistic maps across diverse sensor types and resolutions.
Summary updated