Overcoming the Proposal Bottleneck in Two-Stage Vision Systems

This direction is merged into another direction: cascaded detection bottleneck. Its last version stays readable and citable here.

Version 1 of 128 papers29 conceptsfrozen 2026-10-04

In computer vision, many complex tasks—like finding an object based on a description or identifying how a person is interacting with a tool—have traditionally been solved in two distinct steps. First, a "proposal" stage scans the image to find all potential objects. Second, a "reasoning" stage analyzes those candidates to pick the right one or describe their relationships.

While intuitive, this architecture suffers from a structural flaw known as the proposal bottleneck. Across domains ranging from 3D LiDAR detection to natural language grounding, researchers have identified that the performance of the entire system is strictly capped by the first stage. If the initial detector misses an object, the second stage never even sees it, leading to an irreversible "cascading failure."

The Recall Ceiling

The fundamental problem is one of recall. In a two-stage pipeline, the second stage can only refine or rank what the first stage provides. As Zhichao Li and colleagues observe in their work on LiDAR R-CNN, if the primary stage fails to identify a candidate region, the subsequent refinement stage has no opportunity to recover the missing object, regardless of how accurate that refinement might be.

This creates a "performance ceiling." In the PLAN framework, Bohan Zhuang and Qi Wu note that models relying on pre-extracted proposals are fundamentally limited by the initial detector's recall. This is particularly problematic in cluttered scenes or for small objects that generic detectors might overlook.

The Problem of Task-Agnostic Proposals

The bottleneck is most severe when the first stage is "blind" to the final goal. In Referring Expression Grounding (REG)—where a model must find an object described by a phrase like "the small blue chair behind the table"—the initial proposal stage usually looks for "objects" in a general sense, ignoring the text entirely.

Long Chen and his team highlight this in their research on Ref-NMS. Standard object detectors use Non-Maximum Suppression (NMS) to remove redundant boxes based on generic confidence scores. However, a box that looks "unlikely" to a generic detector might be exactly what the text is describing. When the first stage discards these boxes, it creates a bottleneck that the grounding model cannot fix. Yulei Niu argues in Variational Context that this acts as a "hard filter," where missing the target or even its surrounding context prevents the model from performing the task at all.

Breaking the Bottleneck

To solve this, research has moved in two primary directions: making the first stage "smarter" or removing it entirely.

1. Task-Aware Proposals Instead of using a generic detector, researchers like Long Chen propose Ref-NMS, which uses nouns from the input text to guide the proposal filtering. By making the first stage "expression-aware," the system ensures that relevant candidates survive to be analyzed by the second stage.

2. One-Stage End-to-End Architectures The more radical solution is to ditch the proposal stage altogether. Vivek Mittal introduced AttnGrounder, a single-stage model that uses attention mechanisms to look at the image and text simultaneously. Similarly, Wei Suo developed PFOS, which maps words directly to image grids. By removing the "anchor boxes" and proposal steps, these models eliminate the cascading failure point entirely, often resulting in faster and more robust performance.

Beyond Simple Detection

This shift is also transforming how we detect complex interactions. In Human-Object Interaction (HOI) detection, Mingfei Chen and colleagues developed AS-Net, which treats interaction as a set prediction problem rather than a sequential detection-then-classification task. Even in zero-shot settings, where models encounter new objects, Sanjay Subramanian shows with ReCLIP that while we can repurpose powerful models like CLIP, we are still ultimately "constrained by the performance of the initial region proposal mechanism."

The consensus across these diverse papers is clear: the most sophisticated reasoning engine in the world is useless if the initial "filter" of the system discards the data it needs to see. By integrating task-specific knowledge earlier in the pipeline or moving toward unified, one-stage architectures, the field is successfully bypassing the recall ceiling of the proposal bottleneck.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_overcoming_the_proposal_bottleneck_in_tw_6198b405,
  title = {Overcoming the Proposal Bottleneck in Two-Stage Vision Systems},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/overcoming-the-proposal-bottleneck-in-two-stage-vision-systems/rdr_6198b405288644a78f7e3ab1a51e17db},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_overcoming_the_proposal_bottleneck_in_tw_6198b405_v1,
  title = {Overcoming the Proposal Bottleneck in Two-Stage Vision Systems},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/overcoming-the-proposal-bottleneck-in-two-stage-vision-systems/rdr_6198b405288644a78f7e3ab1a51e17db/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}