Abstract
Scene recognition presents a significant challenge in computer vision due to high intra-class variance and low inter-class variance. This paper proposes a novel framework that bridges the gap between global image context and local object relationships. By mining "essential scene sub-graphs"—the key objects and interactions that define a category—and employing an iterative, bi-enhanced learning process, the framework significantly improves classification accuracy on complex datasets.
Introduction
Visual scene recognition is inherently more complex than standard object detection. While identifying a "bride" is an object-level task, recognizing a "wedding" requires a holistic understanding of specific objects (e.g., bride, groom, altar) and their spatial and semantic relationships. The primary challenge lies in balancing high intra-class variance—where two instances of the same scene type look different—and low inter-class variance—where different scene types share many features. The authors aim to solve this by explicitly modeling the "knowledge" of what makes a scene unique.
Methodology
The authors propose a two-stage framework designed to model the knowledge inherent in a scene's composition.
1. Essential Scene Sub-graph Mining
Rather than processing every object within an image, the system identifies the most "salient" objects using a probabilistic framework inspired by human visual attention. Statistical analysis across datasets revealed that an "upper limit" of three essential objects per scene provides the optimal balance of semantic depth and computational efficiency. For instance, a "tennis court" is effectively defined by a player, a racket, and the court surface.
2. Bi-enhanced Knowledge Space Learning
The architecture employs two parallel networks that interact iteratively:
- Object-stacked Network: Processes cropped patches of essential objects and their relationships.
- Global Scene Network: Captures the overall image context.
This "bi-enhanced" approach uses iterative training where the global network provides supervision to refine local representations, while the object network guides the global network toward discriminative features.
Datasets and Knowledge Representation
The model was evaluated on two benchmarks:
- Scene30: A large-scale dataset constructed from Visual Genome, containing 4,608 images across 30 categories. It includes detailed scene graphs with over 10,000 objects and 30,000 relation types.
- Scene15: A standard benchmark with 4,485 grayscale images across 15 categories.
Key Insights
- The Power of Three: Statistical evidence suggests that using more than three essential objects leads to diminishing returns, as most scenes are semantically defined by a small core of interactions.
- Iterative Mutual Supervision: Unlike simple "late fusion" techniques, the iterative process allows each network to act as a teacher for the other, ensuring the global model understands the importance of local components.
- Relational Importance: The model prioritizes objects with the highest number of relations (e.g., a "man holding a racket") to emphasize interaction data over isolated object features.
Results
The proposed method outperformed standard CNN baselines and previous state-of-the-art techniques:
- Scene30: Achieved 89.72% accuracy, compared to 82.51% for a standard VGG-16.
- Scene15: Reached 94.35% accuracy, surpassing models like DAG-CNN (92.90%) and Object Bank (80.90%).
- Ablation Study: The bi-enhanced iterative learning contributed a nearly 2% accuracy gain over single-pass training, validating the effectiveness of mutual reinforcement.
Conclusion
This research demonstrates that explicitly modeling object relationships and global context through iterative learning significantly enhances scene recognition. Future work will explore more complex relationship modeling using Markov Random Fields and the integration of human-understandable visual guides to further refine essential object discovery.
Summary updated