RGBD Salient Object Detection via Disentangled Cross-Modal Fusion

Hao Chen, Yongjian Deng, Youfu Li, Tzu-Yi Hung, Guosheng Lin

Abstract

Depth is beneficial for salient object detection (SOD) for its additional saliency cues. Existing RGBD SOD methods focus on tailoring complicated cross-modal fusion topologies, which although achieve encouraging performance, are with a high risk of over-fitting and ambiguous in studying cross-modal complementarity. Different from these conventional approaches combining cross-modal features entirely without differentiating, we concentrate our attention on decoupling the diverse cross-modal complements to simplify the fusion process and enhance the fusion sufficiency. We argue that if cross-modal heterogeneous representations can be disentangled explicitly, the cross-modal fusion process can hold less uncertainty, while enjoying better adaptability. To this end, we design a disentangled cross-modal fusion network to expose structural and content representations from both modalities by cross-modal reconstruction. For different scenes, the disentangled representations allow the fusion module to easily identify and incorporate desired complements for informative multi-modal fusion. Extensive experiments show the effectiveness of our designs and a large outperformance over state-of-the-art methods.
BibTeX
@article{chen2020rgbd,
  author = {Hao Chen and Yongjian Deng and Youfu Li and Tzu-Yi Hung and Guosheng Lin},
  title = {{RGBD Salient Object Detection via Disentangled Cross-Modal Fusion}},
  journal = {IEEE Trans. Image Process.},
  year = {2020},
  url = {https://openreview.net/forum?id=DOYIIMM86R},
}

Loading related content...

Figure