Brain-inspired automated visual object discovery and detection

Lichao Chen, Sudhir Kumar Singh, T. Kailath, Vwani Roychowdhury

TL;DR

This paper introduces a novel brain-inspired framework called Structural Unsupervised Viewlets Models (SUVMs) for automatically discovering and detecting objects in images without requiring labeled data. SUVMs learn flexible, part-based representations of objects by mimicking how the human brain processes visual information, achieving competitive performance with supervised methods.

What Problem Does This Paper Address?

Current computer vision systems, despite impressive progress, still fall short of biological vision systems in terms of robustness, scalability, and the ability to learn from unlabeled data. Humans can effortlessly discover objects, understand their constituent parts and their relationships, and robustly detect them even under challenging conditions like occlusion or varying viewpoints. Existing machine learning approaches, particularly deep learning, often rely heavily on massive amounts of manually labeled data, which is expensive and time-consuming to acquire. Furthermore, they struggle to achieve the same level of contextual understanding and flexible representation that biological systems exhibit. This paper aims to bridge this gap by developing an unsupervised framework that learns object representations inspired by principles of biological vision.

Methods

The core of the proposed approach is the Structural Unsupervised Viewlets Model (SUVM), which is inspired by cognitive science theories of object perception. An SUVM is composed of three key interacting parts:

  • Viewlets: These are multi-scale visual cues or "building blocks" that represent different appearances of object parts under various conditions. They are modeled as distributions over feature spaces, allowing for variations.
  • Spatial Relationship Network (SRN): This network captures the geometric organization of viewlets within an object. It uses a probabilistic model (specifically, a variation of a spring network and Gaussian Markov Random Field) to represent the relative spatial locations and scale constraints between viewlets. This allows for scale and translation invariance.
  • Semantic Structure: This is captured by two components:
  • Configuration-Independent Parts Clustering (CIPC): This groups viewlets with distinct appearances but stable geometric relationships into higher-level concepts of "parts" (e.g., "left arm").
  • Global Positional Embedding (GPE): This assigns 2D locations and scale values to viewlets, revealing the hierarchical semantic structure of the object and how parts are organized.

The learning process is positive-only, meaning it learns from unlabeled data without needing explicit negative examples. The framework uses a Maximum Likelihood Estimation (MLE) approach with sparsity constraints for computational tractability. For detection, it employs a probabilistic interpretation and a heuristic search algorithm that leverages the learned SUVM structure for efficient and robust object localization.

Datasets

The authors experimented with two datasets:

  • CalTech-4 dataset: This dataset includes images of Faces, Motorbikes, Airplanes, and Cars. It was chosen for comparison with existing unsupervised/weakly-supervised methods and to explore the framework's limits, including experiments with small training sets.
  • Celebrity dataset: This dataset was created by crawling 12,047 high-quality images from the web. It features natural settings with diverse backgrounds, resolutions, and multiple instances of individuals, making it well-suited for the unsupervised perceptual learning framework. For evaluation purposes, this dataset was manually annotated with main body part locations, but this information was not used during the learning process.

Key Insights and Ideas

  • Brain-inspired "Viewlets" as Flexible Building Blocks: The concept of viewlets as multi-scale, potentially overlapping visual cues, rather than rigid parts, is crucial for representing deformable objects and their variations.
  • Unsupervised Learning from the Internet: The paper demonstrates the feasibility of learning robust object models directly from vast, unlabeled internet data, mimicking how humans learn.
  • Positive-Only Learning: Eliminating the need for explicit negative examples simplifies the learning process and aligns better with biological learning, where negative examples are not always explicitly provided.
  • Geometric and Semantic Structure for Robustness: Combining spatial relationships (SRN) with semantic part groupings (CIPC) and global embeddings (GPE) leads to more robust object detection, even in the presence of occlusion or varying viewpoints.
  • Scalability through Structure: The proposed framework is designed to be computationally scalable, allowing for models with hundreds of viewlets and efficient detection algorithms.

Key Results

  • Competitive Performance: The SUVM framework achieved performance comparable to, and in some cases superior to, existing weakly-supervised and supervised part-aware approaches on the CalTech-4 dataset for tasks like face and torso detection.
  • High Precision in Face Detection: The face detector derived from the human SUVM showed significantly higher precision than the Viola-Jones algorithm, even with less manual training.
  • Robust Torso Detection: The SUVM outperformed strongly-supervised models like Poselets and DPM in precision for torso detection, while maintaining solid recall.
  • Scalability Demonstrated: The framework's ability to handle large datasets and construct complex object models efficiently was highlighted.

Limitations and Future Work

  • Weak Classifiers for Viewlets: The current implementation uses k-Nearest-Neighbor (kNN) classifiers for viewlet detection, which are acknowledged as weak. The authors suggest that replacing these with more sophisticated supervised classifiers (like DNNs) or incorporating negative examples could further improve performance, especially for low-resolution images.
  • Improving SUVMs: Future work could involve explicitly assigning prior probabilities to viewlets and parts to enhance detection robustness. Relaxing the assumption of unimodal pairwise relative distances could also improve modeling of highly flexible objects.
  • Scaling to Multiple Categories: Integrating SUVMs for multiple categories and mapping them to a common feature space is a key challenge. The authors suggest leveraging deep learning paradigms for this integration, potentially automating both object discovery and detection.
  • Modeling Scenes: Extending the framework to model scenes by capturing the co-occurrence and relative locations of different object instances is a promising direction.
  • Deep Learning Integration: The authors plan to incorporate deep learning techniques, such as deep autoencoders for better feature extraction and potentially generative adversarial networks (GANs) or sequence prediction models, to enhance model accuracy and expressiveness.

Summary updated

Abstract

Despite significant recent progress, machine vision systems lag considerably behind their biological counterparts in performance, scalability, and robustness. A distinctive hallmark of the brain is its ability to automatically discover and model objects, at multiscale resolutions, from repeated exposures to unlabeled contextual data and then to be able to robustly detect the learned objects under various nonideal circumstances, such as partial occlusion and different view angles. Replication of such capabilities in a machine would require three key ingredients: (<i>i</i>) access to large-scale perceptual data of the kind that humans experience, (<i>ii</i>) flexible representations of objects, and (<i>iii</i>) an efficient unsupervised learning algorithm. The Internet fortunately provides unprecedented access to vast amounts of visual data. This paper leverages the availability of such data to develop a scalable framework for unsupervised learning of object prototypes-brain-inspired flexible, scale, and shift invariant representations of deformable objects (e.g., humans, motorcycles, cars, airplanes) comprised of parts, their different configurations and views, and their spatial relationships. Computationally, the object prototypes are represented as geometric associative networks using probabilistic constructs such as Markov random fields. We apply our framework to various datasets and show that our approach is computationally scalable and can construct accurate and operational part-aware object models much more efficiently than in much of the recent computer vision literature. We also present efficient algorithms for detection and localization in new scenes of objects and their partial views.
BibTeX
@inproceedings{chen2018brain,
  author = {Lichao Chen and Sudhir Kumar Singh and T. Kailath and Vwani Roychowdhury},
  title = {{Brain-inspired automated visual object discovery and detection}},
  booktitle = {Proceedings of the National Academy of Sciences},
  year = {2018},
  doi = {10.1073/pnas.1802103115},
}
Figure