Abstract
In complex visual environments, contextual information is vital for accurate image segmentation and labeling. While the auto-context (AC) algorithm has proven effective by iteratively refining probability maps, its reliance on fixed spatial sampling radii makes it sensitive to significant scale variations in objects. This paper presents the Scale Invariant Auto-context (SIAC) algorithm, which achieves scale invariance by iteratively estimating the optimal scale of an image and adapting the context sampling radius accordingly. Experimental results on horse segmentation and human body labeling demonstrate that SIAC consistently outperforms standard auto-context when objects vary in size.
Introduction: The Context Challenge
Contextual cues—such as the spatial relationship between a horse's head and its torso—are essential for robust computer vision models. The standard Auto-context (AC) algorithm captures these cues using a sequence of classifiers where each learns from both the original image features and the probability maps generated by the preceding step.
However, AC suffers from a "ruler problem." It samples context at fixed distances (radii) from a target pixel. If an object appears at different scales across images, these fixed radii will land on different anatomical parts, leading to feature inconsistency. For instance, a fixed radius might capture the saddle on a small horse but only the neck on a larger one, confusing the classifier.
Methodology: Scale Invariant Auto-context (SIAC)
To overcome scale sensitivity, the SIAC algorithm introduces an iterative feedback loop that makes the model "scale-aware."
1. Initial Classification and Scale Estimation
The process begins with a standard appearance-based classifier to generate a rough probability map. From this map, the algorithm estimates the image scale by counting foreground pixels ($N_F$). By comparing this count to a learned average foreground size ($\theta$), the image is categorized into discrete scale buckets: "small," "medium," or "large."
2. Adaptive Sampling
Once the scale is estimated, the algorithm selects a corresponding radius sequence for context sampling.
- Small Scale: Sampling distances are halved to maintain relative spatial context.
- Large Scale: Sampling distances are doubled to capture the full extent of the larger object.
3. Iterative Refinement
A new classifier is then trained or applied using these scale-corrected features. This process repeats, with the classification map and the estimated scale refining each other until convergence. This approach effectively allows the model to "zoom" its contextual field of view to match the object's physical size in the image.
Experimental Results
The researchers evaluated SIAC on two challenging datasets:
- Weizmann Horse Dataset: Using a version of the dataset manually scaled to create high size variety, SIAC achieved a higher F-measure than standard AC. Qualitatively, SIAC produced cleaner and more complete object masks, avoiding the fragmentation common in fixed-scale models.
- Baseball Player Dataset: In a complex 14-part labeling task (head, torso, limbs, etc.), SIAC reached a pixel-wise accuracy of 78.9%, significantly higher than the 75.2% achieved by standard AC.
The algorithm typically converges within 3–5 iterations, proving that the self-correcting scale estimation is both stable and efficient.
Conclusion and Future Work
SIAC addresses a fundamental weakness in context-based segmentation by making the spatial sampling adaptive. By reducing intra-class variation caused by scale, the learning task becomes simpler and more accurate. Future research aims to improve the coarse scale estimation (currently based on simple pixel counts) to handle occlusions and multiple objects, and to incorporate orientation invariance to handle rotated objects.
Summary updated