Introduction
Object-Centric Learning (OCL) aims to decompose complex visual scenes into discrete "slots," each representing an individual object. The primary mechanism for this task is Slot Attention (SA), an iterative cross-attention process where query vectors compete to extract information from image features. Despite its success, standard SA implementations suffer from two significant inefficiencies: the "cold-start" problem and transform homogeneity.
The Cold-Start Issue
In the initial frame of a video or a static image, queries are typically sampled from a global Gaussian distribution. These "cold" queries lack any scene-specific context, requiring multiple iterations of the attention mechanism simply to locate objects within the feature space.
Transform Homogeneity
While subsequent video frames benefit from "warm" queries transitioned from previous slots, standard models apply the same number of iterations and identical transformation parameters across all frames. This approach is computationally expensive and fails to leverage the high-quality information already present in temporal recurrences.
Methods: The SmoothSA Framework
To address these bottlenecks, the authors introduce SmoothSA, featuring two primary architectural innovations: Query Preheating and Differentiated Recurrences.
Query Preheating
SmoothSA incorporates a "preheater" module—a lightweight Transformer decoder—that processes initial cold queries alongside image features. This module produces "preheated" queries that are already conditioned on the specific sample. The preheater is trained via self-distillation, learning to predict the final slot outputs of the main SA module. This bootstrapping process allows the model to reach high-quality object representations much faster without requiring external labels.
Differentiated Recurrences
SmoothSA optimizes video processing by varying the computational effort based on query quality. The model utilizes three iterations of SA for the first frame to refine preheated queries but reduces this to a single iteration for all subsequent frames. Because transitioned queries are already highly informative, a single update is sufficient, significantly reducing computational overhead without sacrificing accuracy.
Experimental Evaluation
The model was evaluated on diverse benchmarks, including synthetic (ClevrTex) and real-world (COCO, VOC) images, as well as video datasets (YTVIS, MOVi-C). Downstream reasoning was tested using Visual Question Answering (VQA) tasks on GQA and CLEVRER.
Key Results and Insights
SmoothSA achieves new state-of-the-art performance in object discovery and downstream reasoning:
- Superior Discovery: On the YTVIS video dataset, SmoothSA improved the Adjusted Rand Index (ARI) from 42.0 to 44.1. Similar gains were observed on the VOC dataset, with a 4-point ARI increase over previous methods.
- Enhanced Downstream Reasoning: Integrating SmoothSA slots into the Aloe VQA model led to a 4.4% accuracy boost on the GQA dataset, indicating that the learned slots capture more semantic detail.
- Efficiency Gains: By reducing iterations on non-first frames, the model cuts the computational cost of the aggregation stage by approximately 2/3 for the majority of video sequences.
Conclusion and Future Work
SmoothSA demonstrates that sample-specific initialization and adaptive processing are key to efficient and effective Object-Centric Learning. While the model significantly improves performance, it still relies on a fixed number of slots. Future research will explore dynamic slot allocation to better handle scenes with varying object counts.
Summary updated