Evaluating information contributions of bottom-up and top-down processes

Xiong Yang, Tianfu Wu, Song Chun Zhu

Introduction

Computer vision systems traditionally rely on two primary informational pathways: bottom-up processes, which aggregate low-level features into objects, and top-down processes, which leverage high-level context to guide detection. While the qualitative importance of both is well-recognized, researchers at HUST, UCLA, and the Lotus Hill Research Institute have developed a framework to quantitatively "price" these individual contributions. This research provides a rigorous information-theoretic roadmap for designing more efficient recognition systems that mirror the flexibility of human perception.

The Hierarchical And-Or Graph Model

The study utilizes a Hierarchical And-Or Graph (AoG) to represent object categories. Within this structure, the authors define three distinct computational "channels" that contribute to the final recognition task:

  1. The $\alpha$-channel (Holistic Bottom-up): Direct recognition of the object as a whole based on local image patches (e.g., standard holistic detectors).
  2. The $\beta$-channel (Part-based Bottom-up): Recognition through the detection and "binding" of constituent parts (e.g., identifying a face by finding eyes, nose, and mouth).
  3. The $\gamma$-channel (Top-down): Prediction of an object's presence based on its parent or sibling nodes in the hierarchy (e.g., using a head-shoulder silhouette to infer face location).
Hierarchical representation of the 'face' category, illustrating the alpha, beta, and gamma channels.

Quantifying Information Contribution (IC)

To evaluate these channels independently, the researchers implemented a blocking control strategy. By isolating specific inputs—such as cropping an object from its context to test the $\alpha$-channel or hiding the object itself to test the $\gamma$-channel—they could measure the Information Contribution (IC) using Shannon entropy. This metric quantifies how much each channel reduces uncertainty regarding the object's presence, allowing for a direct numerical comparison between holistic, part-based, and contextual processes.

Empirical Findings: Scale and Category Variations

The research reveals that the value of each channel is highly dependent on the object category and the image scale.

  • Faces vs. Cars: Human faces exhibit a dominant $\alpha$-channel; they are easily recognized as a whole. In contrast, cars often rely on the $\beta$-channel. Because cars are frequently occluded in urban environments, detecting a single part (like a wheel) often provides more certain information than attempting to match a full silhouette.
  • Low-Level Primitives: For features like T-junctions or L-junctions, local filters ($\alpha$-channel) are surprisingly ineffective. The $\beta$-channel, which binds two edges, provides significantly higher information gain.
  • Scale Inversion: As image resolution decreases, bottom-up channels ($\alpha$ and $\beta$) lose efficacy as details vanish. In these low-resolution scenarios, the $\gamma$-channel (top-down context) becomes the primary source of information.
An example of the beta-channel in action, where occlusion requires part-based recognition.

Results and Implications

The integration of all three channels ($\alpha + \beta + \gamma$) yielded significantly improved ROC curves compared to single-channel methods like the Viola-Jones detector. Furthermore, the "information gain" calculated by the model closely aligned with human performance in visual perception studies. This suggests that the IC metric is a valid proxy for the prioritization strategies used by biological vision systems.

Conclusion

By moving beyond intuition and providing a numerical framework for evaluating information pathways, this work paves the way for "optimal schedulers" in computer vision. Such systems could dynamically prioritize channels based on image quality and context, achieving the fluid efficiency characteristic of human sight.

Summary updated

Abstract

This paper presents a method to quantitatively evaluate information contributions of individual bottom-up and top-down computing processes in object recognition. Our objective is to start a discovery on how to schedule bottom-up and top-down processes. (1) We identify two bottom-up processes and one top-down process in hierarchical models, termed α, β and γ channels respectively ; (2) We formulate the three channels under an unified Bayesian framework; (3) We use a blocking control strategy to isolate the three channels to separately train them and individually measure their information contributions in typical recognition tasks; (4) Based on the evaluated results, we integrate the three channels to detect objects with performance improvements obtained. Our experiments are performed in both low-middle level tasks, such as detecting edges/bars and junctions, and high level tasks, such as detecting human faces and cars, together with a group of human study designed to compare computer and human perception.
BibTeX
@inproceedings{yang2009evaluating,
  author = {Xiong Yang and Tianfu Wu and Song Chun Zhu},
  title = {{Evaluating information contributions of bottom-up and top-down processes}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2009},
  doi = {10.1109/iccv.2009.5459386},
}
Figure