Abstract
Mismatch removal is a critical step in computer vision tasks such as 3D reconstruction and SLAM, where distinguishing correct feature matches (inliers) from outliers is essential. GLA-Net addresses the limitations of existing deep learning methods—namely class imbalance and context pollution—by introducing a Guided Loss function and an Inlier Attention Block. The Guided Loss directly optimizes for the F-score by dynamically adjusting loss weights, while the Inlier Attention Block filters out outlier noise during global context aggregation. Experimental results on benchmarks like St.Brown, WIDE, and COLMAP demonstrate that GLA-Net achieves state-of-the-art performance, significantly improving F1-scores through a robust coarse-to-fine architecture.
Introduction
In computer vision, finding correspondences between images is the foundation for spatial reasoning. However, initial "putative" matches are often contaminated by mismatches (outliers) caused by repetitive textures, lighting variations, or extreme viewpoint changes. While deep learning has framed this as a binary classification problem, two significant challenges persist:
- Class Imbalance: Outliers frequently outnumber inliers, leading standard loss functions like cross-entropy to produce models biased toward one class, sacrificing either precision or recall.
- Context Pollution: Traditional "Context Normalization" (CN) layers aggregate statistics across all matches. Because outliers are essentially noise, they pollute these global statistics, hindering the network's ability to isolate true inliers.
Methodology
GLA-Net (Guided Loss Attention Network) introduces two primary innovations to overcome these hurdles within a coarse-to-fine framework.
Guided Loss
Rather than relying on static weights for positive and negative classes, GLA-Net utilizes a mathematically derived Guided Loss. The authors established a direct link between the loss function and the $F_n$-score, a metric that balances precision and recall. During training, the algorithm calculates the derivatives of the $F_n$-score and dynamically adjusts loss weights ($\lambda$ and $\mu$). This ensures that minimizing the loss directly correlates with maximizing the $F_n$-score, guiding the model toward the optimal decision hyperplane.
Inlier Attention Block (IA Block)
To solve the problem of context pollution, the Inlier Attention Block (IA Block) replaces standard Context Normalization. Unlike CN, which treats every match equally, the IA Block learns an "indicating matrix" that assigns higher weights to potential inliers and lower weights to outliers.
When calculating global context (mean and variance), the network effectively ignores the noise from outliers. This results in cleaner feature extraction and more robust classification.
Coarse-to-Fine Architecture
The overall architecture is split into two stages: a Crude Context Subnet that performs an initial pass to generate a preliminary indicating matrix, and a Fine Optimization Subnet that refines the matches using the weighted context from the first stage, mimicking the iterative refinement found in traditional methods like RANSAC.
Experimental Results
GLA-Net was evaluated on three challenging benchmarks: St.Brown (low inlier ratio ~7.6%), WIDE (viewpoint changes), and COLMAP (complex rotations).
The model consistently outperformed previous state-of-the-art methods such as LFGC-Net and NM-Net. On the St.Brown dataset, GLA-Net achieved an F1-score of 59.95%, a significant improvement over LFGC-Net's 49.83%. On the WIDE dataset, it reached a near-perfect 95.21% F1-score. Analysis of the IA Block revealed that by the final layers, the network assigned weights to inliers that were 20-25 times higher than those assigned to outliers, confirming the effectiveness of the attention mechanism in "silencing" noise.
Conclusion
GLA-Net demonstrates that mismatch removal can be significantly improved by aligning the training objective with the evaluation metric and using attention as a robust statistical filter. By preventing outlier pollution and directly optimizing for the F-score, the network finds a superior balance between precision and recall. Future work may focus on reducing the computational overhead of the attention blocks and exploring integration into real-time SLAM systems for mobile applications.
Summary updated