TRAFFIC: Recognizing Objects Using Hierarchical Reference Frame Transformations

Richard S. Zemel, Michael Mozer, Geoffrey E. HintonNeurIPS 1989 · OpenReview

TL;DR

TRAFFIC is a hierarchical connectionist model designed to recognize 2D objects by encoding the spatial relationships between parts and wholes directly within the network's weights as coordinate transformations. By enforcing "viewpoint consistency"—the principle that all parts of an object must agree on the object's overall position, orientation, and scale—the model achieves robust, pose-invariant recognition in unsegmented, noisy images.

The Challenge of Viewpoint Invariance

A fundamental challenge in machine vision is recognizing familiar objects regardless of their orientation, position, or scale (viewpoint invariance) within "unsegmented" images, where the object has not been pre-isolated from the background.

Historically, models approached this by either discarding spatial information to achieve invariance (which loses the ability to verify if parts are in the correct structural arrangement) or by using "Hough transform" techniques. The latter required a massive number of neurons to represent every possible viewpoint, leading to a hardware explosion of discrete parameter bins. The authors sought a method to maintain precise spatial relationships without this combinatorial explosion.

The TRAFFIC Model

The core contribution of this paper is the TRAFFIC model (TRAnsforming Feature Function InstanCes). Its architecture relies on three foundational pillars:

1. Instantiation Units

Instead of utilizing a grid of neurons where each neuron represents a specific spatial location, TRAFFIC uses a compact set of real-valued units to represent the pose of a feature or object. For 2D shapes, four units are used: two for $(x, y)$ position, and two values $(c, s)$ that encode scale and orientation (specifically, $\text{scale} \times \cos\theta$ and $\text{scale} \times \sin\theta$).

2. Hard-coded Transformations

The spatial relationship between a part (feature) and the whole (object) is treated as a fixed geometric transformation. The authors demonstrate that this transformation can be implemented as a simple matrix multiplication, where the "weights" of the network are the coefficients of that matrix.

Network diagram showing how coordinate transformations are encoded in the weights between feature and object instantiation units.

3. Hierarchical Abstraction

The model operates hierarchically. Simple features (e.g., star triplets) predict intermediate features, which subsequently predict complex objects. At each level, the network averages the predictions from lower levels, weighted by a "confidence" unit. If the predictions exhibit low variance—meaning they all agree on the object's location and pose—the confidence in the presence of that object increases.

Experimental Setup: Star Constellations

The model was evaluated on a synthetic dataset of star constellations.

  • Input: Triples of stars arranged in various configurations.
  • Task: Recognize one of eight target constellations (such as the Big Dipper) embedded in an image containing other "noise" stars.
  • Variations: The training and testing sets included images where constellations were subjected to translation, rotation, and scaling.

Key Insights

  • Weights as Geometry: Rather than acting merely as "learned correlations," the weights in TRAFFIC represent specific coordinate transformations. This allows the network to mathematically "project" where an object should be based on observing just a single constituent part.
  • Viewpoint Consistency: The critical insight is that a true object is detected when multiple independent features all predict the exact same object pose. If one feature predicts the object is at $(10, 10)$ and another predicts $(50, 50)$, they lack viewpoint consistency and are not grouped as part of the same object.
  • Real-Valued Parameters vs. Bins: By utilizing real-valued units to represent pose, the model circumvents the "curse of dimensionality" inherent in Hough transforms, which require separate neurons for every possible pixel location and rotation degree.
  • Hybrid Learning: The training process is accelerated by using a mix of unsupervised competitive learning (to discover common feature types) and supervised backpropagation (to map those features to specific object labels).

Key Results

  • Fast Convergence: The network successfully learned to identify target constellations in fewer than 150 epochs.
  • Pose Accuracy: Beyond simple classification, experiments demonstrated the network could accurately output the precise $(x, y, \text{scale}, \text{angle})$ parameters of the detected object.
  • Noise Robustness: In a "scrambled feature" test, providing the correct features in an incorrect spatial arrangement caused the "confidence" unit to correctly drop. This confirms the network actively verifies geometry rather than acting as a simple "bag of features" classifier.

Limitations and Future Directions

  • 2D Constraint: The current paper focuses entirely on 2D transformations. Extending this framework to 3D would require a more complex set of instantiation units and transformations to account for perspective and depth.
  • Feature Extraction: The model assumes a pre-processor has already identified basic features like "star triplets." A more robust front-end is needed to extract reference frames directly from raw pixel data.
  • Rigidity: The model assumes objects are perfectly rigid. Handling "deformable" objects, where parts might shift slightly relative to one another, would require a probabilistic framework.

Summary updated

Abstract

We describe a model that can recognize two-dimensional shapes in an unsegmented image, independent of their orientation, position, and scale. The model, called TRAFFIC, efficiently represents the structural relation between an object and each of its component features by encoding the fixed viewpoint-invariant transformation from the feature's reference frame to the object's in the weights of a connectionist network. Using a hierarchy of such transformations, with increasing complexity of features at each successive layer, the network can recognize multiple objects in parallel. An implemen(cid:173) tation of TRAFFIC is described, along with experimental results demonstrating the network's ability to recognize constellations of stars in a viewpoint-invariant manner.
BibTeX
@inproceedings{zemel1989traffic,
  author = {Richard S. Zemel and Michael Mozer and Geoffrey E. Hinton},
  title = {{TRAFFIC: Recognizing Objects Using Hierarchical Reference Frame Transformations}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {1989},
  url = {https://openreview.net/forum?id=Sy4NbD-dWB},
}
Figure