Mining Negative Temporal Contexts for False Positive Suppression in Real-Time Ultrasound Lesion Detection

Haojun Yu, Youcheng Li, QuanLin Wu, Ziwei Zhao, Dengbo Chen, Dong Wang, Liwei Wang

TL;DR

This paper introduces UltraDet, a novel real-time ultrasound lesion detection system that significantly reduces false positives by leveraging "negative temporal contexts" (NTC) from previous video frames. By mimicking how experienced radiologists use past observations to rule out non-lesions, UltraDet achieves state-of-the-art performance and real-time inference speeds, making it a promising tool for clinical assistance.

What Problem Does This Paper Address?

Real-time lesion detection in ultrasound videos is crucial for assisting radiologists in cancer diagnosis. However, existing general-purpose object detection models struggle with this task, often producing false positives (FPs) by misinterpreting non-lesion anatomical structures as lesions. This is particularly problematic for junior radiologists who may be misled by these errors. The core issue identified is the failure of these models to utilize "negative temporal contexts" (NTC) – information from previous frames where a potential lesion-like area exhibits characteristics that rule it out as a true lesion. Experienced radiologists naturally use this temporal information, but it has been largely ignored in prior automated systems.

Methods

The proposed solution is the UltraDet model, which integrates a novel Negative Temporal Context Aggregation (NTCA) module into a basic real-time detector. The NTCA module's key innovation is its ability to extract and leverage NTC.

Here's how it works:

  • Inverse Optical Flow Align (IOF Align): To account for the motion between frames, the model uses inverse optical flow to align features from previous "context" frames with the current frame. This ensures that corresponding anatomical regions are accurately compared.
  • Temporal Context Extraction: For each region of interest (ROI) proposed by a base detector, the NTCA module extracts features from sampled previous frames, aligned using IOF Align.
  • Temporal Aggregation: These extracted temporal contexts (including NTC) are then aggregated and fused with the current frame's proposal features. This fusion process uses an attention mechanism to effectively combine information from multiple temporal contexts.
  • UltraDet Architecture: The NTCA module is integrated into the head of a basic real-time detector (referred to as BasicDet). This head consists of stacked NTCA and relation modules, allowing for sophisticated temporal reasoning.

The model is trained using regression and classification losses on the current frame, with auxiliary losses applied to previous frames to improve training efficiency. Crucially, during inference, UltraDet uses both the current and previous frames but generates predictions only for the current frame, enabling real-time performance.

Datasets

The primary dataset used is the CVA-BUS dataset [9].

  • The authors split the dataset into train-val (154 videos) and test (32 videos) sets.
  • The train-val split contains 21,423 frames with 170 lesions.
  • The test split contains 3,849 frames with 32 lesions.
  • The authors focus solely on the lesion detection task and do not use the benign/malignant classification labels.
  • They also mention providing high-quality labels for the CVA-BUS dataset, which are re-annotated by experienced radiologists to address issues of label unsteadiness and inaccuracy in the original dataset.

Key Insights and Ideas

  • The Power of "Negative" Information: The paper highlights that simply detecting positive visual cues isn't enough. The absence of lesion-like characteristics in previous frames (NTC) is a critical signal for suppressing false positives, a concept well-understood by human experts but overlooked in automated systems.
  • Mimicking Radiologist Expertise: The NTCA module is explicitly designed to emulate how experienced radiologists use temporal information to disambiguate potential lesions. This human-centric approach leads to more robust and interpretable detection.
  • Inverse Optical Flow for Temporal Alignment: Using inverse optical flow to align features is a clever way to handle inter-frame motion in ultrasound videos, ensuring that the temporal context is relevant to the current frame's analysis.
  • RoI-Level Aggregation is Key: Ablation studies suggest that aggregating temporal context at the Region of Interest (RoI) level is more effective than feature-level aggregation for this task, aligning with the idea that radiologists focus on specific areas of concern.

Key Results

  • Significant FP Reduction: UltraDet achieves a substantial reduction in false positives. Specifically, it reduces FP90 (lesion-level FP rate at 0.90 recall) to 5.7 seq/min, which is about a 50% reduction compared to the best competitor (PTSEFormer).
  • State-of-the-Art Performance: The model significantly outperforms previous state-of-the-art methods in terms of precision (Pr90 of 90.8%) and FP rates.
  • Real-Time Inference: UltraDet achieves an inference speed of 30.4 FPS on a single NVIDIA GeForce RTX 3090 GPU, meeting the requirement for real-time applications. This speed can be further optimized to 35.2 FPS using TensorRT.
  • Qualitative Improvements: Visual comparisons show UltraDet successfully suppressing FPs that are present in the baseline detector (BasicDet), particularly in scenarios where temporal context is crucial.

Limitations and Future Work

  • Dataset Size: While the CVA-BUS dataset is used, the number of lesions in the test set (32 lesions) is relatively small, which might limit the generalizability of findings.
  • Reliance on Optical Flow: The performance is dependent on the accuracy of the optical flow estimation. While FlowNetS is used and shown to generalize well, challenging ultrasound artifacts could still impact flow accuracy.
  • Future Work: The paper implicitly suggests that further exploration of temporal context aggregation strategies and potentially larger, more diverse ultrasound datasets could lead to even greater improvements. The authors also release their code and high-quality labels, which will facilitate future research in this area.

Summary updated

Abstract

During ultrasonic scanning processes, real-time lesion detection can assist radiologists in accurate cancer diagnosis. However, this essential task remains challenging and underexplored. General-purpose real-time object detection models can mistakenly report obvious false positives (FPs) when applied to ultrasound videos, potentially misleading junior radiologists. One key issue is their failure to utilize negative symptoms in previous frames, denoted as negative temporal contexts (NTC) [15]. To address this issue, we propose to extract contexts from previous frames, including NTC, with the guidance of inverse optical flow. By aggregating extracted contexts, we endow the model with the ability to suppress FPs by leveraging NTC. We call the resulting model UltraDet. The proposed UltraDet demonstrates significant improvement over previous state-of-the-arts and achieves real-time inference speed. We release the code, checkpoints, and high-quality labels of the CVA-BUS dataset [9] in https://github.com/HaojunYu1998/UltraDet .
BibTeX
@inproceedings{yu2023mining,
  author = {Haojun Yu and Youcheng Li and QuanLin Wu and Ziwei Zhao and Dengbo Chen and Dong Wang and Liwei Wang},
  title = {{Mining Negative Temporal Contexts for False Positive Suppression in Real-Time Ultrasound Lesion Detection}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2023},
  url = {https://openreview.net/forum?id=TWfB8YK8Ad},
}
Figure