Abstract
This paper introduces a "descriptor-free" approach to computer vision by training a neural network to detect interest points that are already implicitly matched. By assigning specific visual features to dedicated output channels, the system eliminates the need to calculate, store, or transmit heavy descriptor vectors, drastically reducing the data footprint for visual localization and multi-agent systems.
1. Introduction
The standard "detect-then-describe" pipeline has dominated computer vision for decades. To find correspondences between two images, researchers typically identify "interesting" pixels (e.g., corners, blobs), calculate a mathematical summary of the local neighborhood (a descriptor like SIFT or ORB), and then perform a nearest-neighbor search in the descriptor space of a second image.
While effective, this process is resource-intensive. Descriptors are often large vectors (e.g., 128 floating-point numbers), consuming significant memory and bandwidth—a major bottleneck for low-power robots or swarms of drones that need to share map data. This work addresses a fundamental question: Can we match points without descriptors at all?
2. Methodology: Implicit Matching via Specialized Channels
The authors propose a single, convolution-only neural network that collapses the detection and matching pipeline into a single step. Instead of a single output channel representing a generic "interest point" score, the network outputs multiple channels ($n=128$). Each channel acts as a specialized detector for a specific type of visual feature (e.g., a specific window corner or a unique texture).
The Key Innovation
The core innovation lies in the "implicit matching" mechanism. The interest point for Channel $i$ in Image A is defined as the pixel with the maximum response (the argmax) in that channel. This point is implicitly matched to the global maximum of the same Channel $i$ in Image B. This eliminates the need for explicit descriptor calculation and matching.
Training is conducted in a self-supervised manner using three primary loss functions:
- Inlier Reinforcement: Enhances the response of channels that yield correct matches.
- Redundancy Suppression: Prevents multiple channels from converging on the same visual features.
- Correspondence Reinforcement: Corrects outliers by pulling channel responses toward ground-truth matching locations.
3. Experimental Evaluation
The method was evaluated across several benchmarks, including HPatches (viewpoint changes), KITTI (large-scale outdoor driving), and EuRoC (indoor drone flight).
Key results indicate that while traditional methods like SIFT or SuperPoint may achieve higher raw matching scores, IMIPs offer a superior accuracy-to-representation-size trade-off. By representing observations as ordered $(x, y)$ coordinates (requiring as little as 3 bytes per point), the system enables high-quality pose estimation with minimal data overhead. On KITTI and EuRoC, the method achieved relative pose estimates with rotation errors under 1° and translation errors under 30cm, provided at least 10 inliers were detected.
4. Discussion and Future Work
The primary strength of IMIPs is extreme data compression and efficiency in narrow-baseline scenarios, making it ideal for high-frequency visual odometry or SLAM. However, the current architecture lacks explicit scale and rotation invariance, leading to performance degradation in wide-baseline settings. Future research will focus on integrating geometric invariance into the network architecture and improving localization precision through specialized "peakedness" loss functions to refine interest point localization.
Summary updated