Overcoming the Non-Maximum Suppression Bottleneck in Crowded Scenes

266 concepts202 papers2 proposals

Most modern object detection pipelines follow a "detect-then-prune" strategy. A neural network generates thousands of candidate bounding boxes, many of which are redundant predictions for the same object. To clean this up, researchers traditionally rely on Non-Maximum Suppression (NMS), a greedy algorithm that keeps the highest-scoring box and deletes any nearby boxes that overlap significantly with it.

While NMS works well for isolated objects, it becomes a critical failure point in high-density environments. As Kishan Sharma et al. note, standard NMS frequently fails in crowded scenarios because it cannot distinguish between redundant predictions for one object and valid, overlapping detections of distinct objects.

The Spatial Overlap Ambiguity

The fundamental logic of NMS relies on a proxy: Intersection-over-Union (IoU). The algorithm assumes that if two boxes have a high IoU, they must represent the same physical entity. However, in "densely packed" scenes—such as supermarket shelves, parking lots, or biological cell clusters—this assumption collapses.

In these environments, distinct objects naturally overlap in the 2D image plane. When NMS is applied, it "over-suppresses," incorrectly identifying these neighboring instances as duplicates and deleting them. This leads to several systemic issues:

  • Under-counting: In agricultural tasks like counting maize tassels, Hongwei Zou et al. found that NMS leads to systematic underestimation because it filters out physically clustered plants.
  • Missed Detections in Infrared: Wenbo Li et al. observe that in infrared imaging, where targets appear as indistinct thermal blobs, NMS often ignores distinct objects in close proximity, causing high miss rates.
  • Occlusion Sensitivity: For autonomous driving, Tianyu Li et al. argue that NMS suppresses valid boxes of occluded pedestrians, as the visible portion of a hidden person often overlaps heavily with the person in front.

Geometric and Representation Failures

The limitations of NMS are often exacerbated by the way we represent objects. Standard axis-aligned bounding boxes are particularly prone to "false overlap." Gui-Song Xia et al. demonstrate that in aerial imagery, axis-aligned boxes for adjacent, tilted objects (like ships in a harbor) will overlap significantly even if the objects themselves do not. This causes NMS to delete valid detections simply because the rectangular containers overlap.

To combat this, Lianwen Jin et al. proposed using flexible quadrilaterals instead of rectangles, while Zixuan Xu et al. introduced a "Beta Representation" to move away from rigid boxes entirely, allowing the system to better isolate individuals in dense crowds.

Beyond Greedy Pruning

Researchers are increasingly moving away from fixed-threshold NMS toward more "intelligent" post-processing. One direction is Adaptive NMS, where the suppression threshold is not a constant but changes based on local density. Dang Lin et al. show that an adaptive approach can prevent the omission of valid detections in clustered scenarios like rice pest identification.

Other researchers aim to replace the "detect-then-prune" paradigm with sequential prediction. Instead of predicting everything at once and then deleting, these models predict objects one by one, accounting for what has already been found:

  • Visual Chunking: Nicholas Rhinehart et al. treat detection as a list prediction problem, iteratively selecting "chunks" of the image to avoid the redundancy issues of parallel proposal generation.
  • Autoregressive Models: The authors of Falcon Perception-HD use reinforcement learning to train models that generate sequences of detections, eliminating the need for NMS heuristics entirely even in ultra-dense scenes with up to 500 objects.
  • Relational Learning: Rather than looking at boxes in isolation, JointDet uses a Relationship Discriminating Module to learn the spatial correlation between different body parts (like heads and torsos), helping the system distinguish between two overlapping people and two boxes for the same person.

By treating detection as a reconstruction or sequential task rather than a simple filtering problem, these methods bypass the "NMS bottleneck," allowing AI to maintain high recall even in the most cluttered environments.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_non_maximum_suppression_limitations_9e520a09,
  title = {non-maximum suppression limitations},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/non-maximum-suppression-limitations/rdr_9e520a0929c54f4596b62c428befea69},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_non_maximum_suppression_limitations_9e520a09_v1,
  title = {non-maximum suppression limitations},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/non-maximum-suppression-limitations/rdr_9e520a0929c54f4596b62c428befea69/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake