COGS: A Diagnostic Benchmark for Attribute-Aware Object-Level 3D Gaussian Splatting Perception

Zheng Xiao, Yifan Liu, Weijian Deng, Jiashu Li, Ziqin Liu, Zhenjun Han, Jianbin JiaoarXiv 2026 · arXiv

TL;DR

The COGS paper introduces a large-scale, curated dataset and diagnostic benchmark designed to reveal how 3D perception models actually use the unique attributes of 3D Gaussian Splatting (3DGS). The authors find that simply adding more Gaussian attributes often hurts performance in standard architectures, leading them to propose HALO, a reference model that achieves state-of-the-art results by processing these heterogeneous attributes in separate, role-preserving streams.

What Problem Does This Paper Address?

3D Gaussian Splatting (3DGS) has revolutionized real-time rendering, but its potential for 3D perception (like object classification) is still being mapped out. Unlike simple point clouds, 3DGS primitives are "heterogeneous": each Gaussian has a position, opacity, scale, rotation, and color coefficients.

Current 3D architectures usually treat these extra attributes as just more input channels. This creates a "representation-level ambiguity": are models actually learning from the geometry and appearance of the Gaussians, or are they just finding shortcuts? Existing datasets don't help much—they are either derived from "clean" CAD models with no texture (ModelNet40GS) or captured from the real world with messy reconstruction artifacts (MACGS).

The "Dataset Gap" between existing 3DGS datasets and the proposed COGS.

Methods

The authors tackle this by creating COGS (Common Objects of 3D Gaussian Splatting), a dataset of 15,675 objects across 40 categories. They used a generative pipeline (TRELLIS) to create high-quality 3DGS assets from web images, followed by a rigorous multi-stage filtering process to remove "hallucinated" geometry and artifacts.

Figure 2. Overview of the COGS construction pipeline. Starting from large-scale web image collection, we perform semantic localization, foreground-aware filtering and deduplication, manual image inspection, generative 3DGS reconstruction with TRELLIS, and viewer-based 3DGS quality inspection to obtain curated object-centric 3DGS assets.

To solve the attribute-usage problem, they introduced HALO (Hierarchical Aggregation with Locally-decOupled MLP). HALO's key innovation is the Locally-decOupled MLP (LOM). Instead of mashing all attributes together immediately (early fusion), HALO:

  1. Decouples: Processes attribute groups (like position, geometry, and color) in separate streams.
  2. Extracts: Uses LOM to treat local neighborhoods as sequences, making multi-stream modeling efficient.
  3. Aggregates: Fuses these features hierarchically, preserving the specific "role" of each attribute until higher levels of the network.

Datasets

  • COGS (New): 15,675 artifact-filtered, appearance-rich 3DGS objects.
  • COGS-Bench (New): A diagnostic protocol testing 8 different attribute combinations (e.g., Position only, Position + Opacity, etc.) to see which ones actually help a given model.
  • ModelNet40GS: Used for external validation (CAD-derived).
  • MACGS: Used for external validation (Real-world capture-derived).
  • ScanNet: Used for scene-level semantic segmentation validation.

Key Insights and Ideas

  • The "More is Less" Paradox: For many standard models like PointNet and DGCNN, adding more attributes (like color) actually decreased accuracy. This suggests that heterogeneous attributes can introduce "fusion conflicts" if collapsed into a single feature space too early.
  • Attribute Sensitivity: Different architectures are sensitive to different things. For example, some models gained significantly from opacity (O), while others were confused by it.
  • Artifacts Matter: Figure 3 shows common 3DGS failures like "Structural Duplication" or "Unintended Fusion." By filtering these out, COGS provides a "clean" signal for diagnostic testing that capture-based datasets lack.
Examples of discarded 3DGS samples showing structural duplication and incomplete geometry.
  • Pre-training Power: Even though COGS is generated from images, pre-training on it significantly boosts performance on real-world captured data (MACGS), proving the features learned are transferable.

Key Results

  • State-of-the-Art Performance: HALO-L achieved 94.70% Overall Accuracy (OA) on COGS-Bench, significantly outperforming standard backbones like PTv3 (89.11%) and PointNet (79.53%).
  • Attribute-Aware Gains: Unlike baselines, HALO's performance improved monotonically as more attributes were added. It reached its peak only when using the full set of attributes (POSQC), validating its "decoupled" design.
  • Transfer Learning: Using COGS for pre-training improved PointNet's accuracy on the MACGS dataset by a massive 26.98 points when only 10% of the target data was available.

Limitations and Future Work

The authors note that COGS is currently object-centric. While they performed a "proof-of-concept" on the ScanNet dataset for scenes, the primary benchmark focuses on isolated objects.

Future work could expand COGS-Bench to include:

  • Controlled Corruptions: Testing how models handle noisy or "broken" Gaussians.
  • Compositional Scenes: Moving from single objects to complex environments.
  • Dense Annotations: Adding part-level or point-level labels for more granular perception tasks.

Summary updated

Abstract

3D Gaussian Splatting (3DGS) is increasingly explored as an input representation for 3D perception. Yet it remains unclear how current 3D architectures exploit heterogeneous Gaussian attributes. This ambiguity makes accuracy hard to interpret and limits principled architecture design for 3DGS perception. However, existing object-level 3DGS datasets lack the appearance variation or artifact control needed for systematic attribute-level analysis. We introduce COGS, a large-scale 3DGS dataset with 15,675 artifact-filtered and appearance-rich objects across 40 categories. Built on COGS, we establish COGS-Bench for object classification and attribute-usage diagnosis, complemented by external validation. Our benchmark shows that Gaussian attributes are not uniformly beneficial: selected subsets can outperform the full attribute set, and the effects of appearance, opacity, and covariance-related attribute combinations vary substantially across architectures. We further study this finding with HALO, a reference model that preserves attribute-specific evidence before hierarchical aggregation. Experiments on COGS-Bench and external 3DGS datasets demonstrate that COGS supports systematic diagnosis of 3DGS attribute usage, while HALO highlights the value of role-preserving attribute organization. Our dataset is available at https://huggingface.co/datasets/XiaoZhengCV/COGS.
BibTeX
@misc{xiao2026cogs,
  author = {Zheng Xiao and Yifan Liu and Weijian Deng and Jiashu Li and Ziqin Liu and Zhenjun Han and Jianbin Jiao},
  title = {{COGS: A Diagnostic Benchmark for Attribute-Aware Object-Level 3D Gaussian Splatting Perception}},
  year = {2026},
  eprint = {2610.39201},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.39201},
}
Figure