SCAR: Bridging Coarse Retrieval and Fine Localization through Candidate Reranking

Shuhan Dong, Tianyi Shang, Zhenyu LiarXiv 2026 · arXiv

TL;DR

SCAR introduces a lightweight reranking stage for natural-language point-cloud localization that evaluates candidate positions based on their spatial consistency with the query. By shifting the final selection criteria from semantic region relevance to actual coordinate accuracy, it achieves significant performance gains across multiple backbones and datasets without retraining the underlying localization models.

What Problem Does This Paper Address?

Language-driven point-cloud localization—finding a specific spot in a 3D map based on a description like "east of the gray lamp post"—typically follows a two-stage "coarse-to-fine" pipeline. First, a model retrieves the most relevant map cells (coarse); then, it predicts a precise coordinate within those cells (fine).

The authors identify a critical "selection problem": the final output is almost always determined by the retrieval order. However, a highly relevant region might produce a poorly estimated coordinate, while a lower-ranked region might contain a highly accurate one. As shown in Figure 1, there is massive "headroom"—on one benchmark, the top-ranked prediction is correct only 54.47% of the time, but a successful prediction exists somewhere in the top 25 candidates 93.22% of the time.

Fig. 1. Candidate availability and selection on MNCL×KITTI VAL with a fixed Top-25 prediction pool. (a) Original and Oracle@25 Top-1@5 m. (b) Queries corrected and broken by two training-free rankings relative to Original. The oracle uses GT only to measure recoverable coverage. Query counts, net gains, and the paired interval are discussed in the text.

Methods

The proposed solution is SCAR (Spatial Consistency-Aware Reranking). Instead of trusting the initial retrieval order, SCAR treats every (cell, predicted coordinate) pair as a hypothesis to be re-evaluated.

The core innovation is Fine-conditioned Evidence Readout. Traditional methods might check if a region contains the right objects. SCAR goes further by checking if the specific predicted coordinate satisfies the directional constraints of the query (e.g., is this point actually "east of the pole"?).

The process involves:

  1. Query Parsing: Breaking the text into semantic and directional relations.
  2. Evidence Construction: Building a 23-D feature vector for each candidate. This includes retrieval scores, semantic presence, and "spatial support" measured specifically at the predicted fine coordinate.
  3. Shared Scoring: A tiny, three-layer MLP (only 3,649 parameters) scores these vectors. It is trained using a listwise cross-entropy loss, where "success" is defined as being within 5 meters of the ground truth.
  4. Reranking: The candidates are sorted by the MLP score, and the coordinate from the new top-ranked candidate is returned.
Fig. 2. SCAR overview. A frozen Coarse–Fine backbone supplies 25 cell–position hypotheses. Query relations are matched to map objects, spatial support is read at each existing Fine prediction, and the resulting 23-D representation is normalized with TRAIN statistics and scored by a shared MLP. SCAR reranks the fixed hypotheses and returns the existing Fine coordinate of the selected candidate. Point-cloud layouts, relations, support fields, and scores are illustrative.

Datasets

The authors evaluated SCAR across five diverse domains using four public backbones (Text2Loc, Text2Loc++, MNCL, MambaPlace) and a reimplementation of Des4Pos. The benchmarks include:

  • KITTI360Pose: Urban driving scenes.
  • PandaSetPose: Urban scenes with high-density LiDAR.
  • nuScenesPose Full: Large-scale autonomous driving dataset.
  • S3DISPose: Indoor office environments.
  • ParisLillePose/NPM3D: Urban point clouds.

The splits and unit definitions for these benchmarks are detailed in the paper's experimental setup: Benchmark split details

Benchmark split details

Key Insights and Ideas

  • Position Matters for Truth: A directional relation like "east of the pole" doesn't have a fixed truth value for a whole region; it only becomes verifiable once a specific test position is chosen.
  • Separating Relevance from Accuracy: Semantic relevance (retrieval) and metric success (localization) are distinct. SCAR explicitly bridges this gap by using geometric verification as a second filter.
  • Readout Location is Crucial: The authors found that reading spatial evidence at the predicted fine point is significantly more effective than reading it at the cell center. This "fine-conditioned" approach ensures the model verifies the exact coordinate it intends to output.
  • Efficiency: Because the scorer is a tiny MLP and the backbones remain frozen, SCAR adds negligible computational overhead (roughly 5ms) to the localization pipeline.

Key Results

SCAR demonstrated nearly universal improvements across 25 different backbone/dataset configurations.

  • Localization Accuracy: On the primary Top-1@5m metric, SCAR outperformed the original ranking in 23 out of 24 configurations.
  • Massive Gains: For the Text2Loc backbone on KITTI, accuracy jumped from 35.14% to 59.45% (a 24.31 percentage point increase). On S3DIS, it rose from 13.84% to 41.33%.
  • Retrieval Success: Cell Recall@1 (finding the correct map cell) improved in all 24 configurations.

Detailed success rates across different distance thresholds (1m, 3m, 5m, 10m) show consistent gains: Test localization success rates

Test localization success rates

Limitations and Future Work

  • Candidate Pool Dependency: SCAR is a reranker; it cannot "fix" a query if no successful coordinate exists within the initial pool of candidates provided by the backbone.
  • Template-Based Parsing: The current spatial evidence construction relies on a rule-based parser for queries. Moving toward more flexible, natural language understanding for complex descriptions is a logical next step.
  • Exploratory Nature: The authors note that many TEST set results are exploratory. Future work should involve larger-scale confirmatory studies and testing on even more diverse "in-the-wild" point cloud data.

Summary updated

Abstract

Natural-language point-cloud localization typically retrieves candidate map cells and then predicts a position inside each candidate, yet the final output is still determined by retrieval order. A retrieval score measures region-description relevance, not whether the coordinates a region yields are accurate: a relevant region may produce a displaced prediction, and a lower-ranked region may contain a successful one. We study the final selection problem after the candidate pool and Fine predictions are fixed---how to choose the coordinate most likely to localize successfully, and under which conditions the gain holds. We propose SCAR (Spatial Consistency-Aware Reranking). SCAR treats each cell--Fine-coordinate pair as a localization hypothesis, interprets directional relations as functions of position, and reads spatial support at the backbone's actual Fine point (Fine-conditioned Evidence Readout). It organizes this readout with center support, semantic presence, retrieval score, and geometric offsets into a 23-D representation, trains a shared scorer with localization-success labels, and at inference reranks existing hypotheses and returns their original coordinates. Experiments cover four public backbones and a corrected Des4Pos reimplementation across five domains. Excluding one protocol-reference configuration, exploratory TEST point estimates on the remaining 24 configurations show Top-1@5\,m above Original on 23 and Cell R@1 above Original on all 24. On development-set VAL, the Fine readout improves Top-1@5\,m by 5.77 percentage points (pp) over a center surrogate on corrected Des4Pos*KITTI, and the complete scoring scheme improves it by 13.98\,pp over FSR on MambaPlace$\times$nuScenesFull; the latter measures the complete scheme, not learned ranking alone.
BibTeX
@misc{dong2026scar,
  author = {Shuhan Dong and Tianyi Shang and Zhenyu Li},
  title = {{SCAR: Bridging Coarse Retrieval and Fine Localization through Candidate Reranking}},
  year = {2026},
  eprint = {2611.18240},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.18240},
}
Figure