Rendered 2D Semantic and Generative Priors Guided 3D Multi-Object Grounding
Lists
Only you see this
Abstract
3D multi-object visual grounding aims to identify and localize all objects in a 3D scene that correspond to a given text description. Unlike traditional single-object grounding, this task presents additional challenges as point clouds inherently lack fine-grained details, making it difficult to capture subtle object features and contextual information. Moreover, textual descriptions are inherently limited in perceiving and understanding complex 3D environments, especially in scenarios with high object similarity or intricate spatial arrangements. To tackle the above challenges, we propose SGMG, a Rendered 2D Semantic and Generative priors guided 3D Multi-object Grounding Framework. The SGMG framework introduces two key innovations that work cohesively to enhance grounding accuracy. First, a Generative-Assistant(GA) Module leverages the capabilities of a generative model to provide enriched scene prior information and capture the fine-grained scene details. Second, the Semantic-Augment Fusion(SAF) Module is designed to improve the representation of text features from the vision features, thereby boosting the accuracy of multimodal information interactions. Furthermore, we introduce a multi-level fusion mechanism, ensuring that semantic and spatial relationships between objects are preserved and effectively leveraged during the grounding process. Experimental results demonstrate that SGMG achieves state-of-the-art performance in multi-object 3D grounding and competitive results in traditional single-object tasks, highlighting its effectiveness in diverse scenarios.
BibTeX
@article{guo2026rendered,
author = {Peng Guo and Hongyuan Zhu and Hancheng Ye and Yihang Yang and Fukun Yin and Tao Chen},
title = {{Rendered 2D Semantic and Generative Priors Guided 3D Multi-Object Grounding}},
journal = {IEEE Transactions on Multimedia},
year = {2026},
doi = {10.1109/tmm.2026.3668488},
}Loading related content...