Cell Detection from Imperfect Annotation by Pseudo Label Selection Using P-classification

Kazuma Fujii, Daiki Suehiro, Kazuya Nishimura, Ryoma Bise

TL;DR

Annotating every single cell in dense microscopy images is labor-intensive, but training detectors on partially labeled images causes standard models to mistake unannotated cells for background. This paper introduces an iterative pseudo-labeling pipeline that couples masked loss detection, Positive-and-Unlabeled (PU) feature learning, and P-classification ranking to harvest reliable cell (positive) and background (negative) pseudo-labels. With only 10% of cells annotated, the method dramatically outperforms standard baselines and approaches fully supervised accuracy.

What Problem Does This Paper Address?

Deep learning models for cell detection typically rely on dense, pixel-level ground truth to distinguish foreground cells from the background. In biological microscopy, however, an image can contain hundreds or thousands of cells, making exhaustive annotation expensive and impractical. As a result, public benchmarks and real-world datasets often feature imperfect annotations, where only a fraction of cells are marked.

When a standard convolutional neural network (CNN) is trained on incomplete annotations, any unmarked cell is mistakenly treated as background (negative). This penalizes the model for detecting real cells and causes performance to collapse. While a naive fix is to compute loss only within masked regions around known cells, this eliminates negative feedback, causing the detector to severely over-detect and produce high false-positive rates.

Methods

The authors propose a 5-step iterative framework to progressively identify missing cells and background regions without requiring negative labels up front:

Partially Labeled Images 
       │
       ▼
[1] Train Detector D with Masked Loss (Heatmap regression)
       │
       ▼
[2] Extract Candidate Patches (Positives X_P vs. Unlabeled detections X_U)
       │
       ▼
[3] Learn Features via PU-Learning (Non-negative risk formulation)
       │
       ▼
[4] Rank & Select via P-Classification (Top α% as cells, bottom β% as background)
       │
       ▼
[5] Add Pseudo-Labels to Masks/Heatmaps ──► [Repeat Step 1]
  1. Cell Detection with Masked Loss: A U-Net predicts centroid heatmaps (Gaussian peaks at cell centers). The loss is masked so that it is only computed within a radius $r$ around annotated cells, ignoring unannotated regions.
  2. Patch Extraction: Local maxima in the predicted heatmaps above a threshold identify candidate cells. Annotated centroids form the positive set ($X_P$), while detected but unannotated locations form the unlabeled set ($X_U$).
  3. Feature Extraction via PU-Learning: A classification CNN is trained on $X_P$ and $X_U$ using a non-negative risk estimator. The classifier's penultimate layer ($\hat{g}$) is extracted as a robust feature representation that separates cells from background without needing true negative labels.
  4. Pseudo-Label Selection via P-Classification: Rather than drawing a hard decision boundary (which is error-prone when class priors are uncertain), the authors train a ranking function $f_\lambda$ via P-classification. By tuning a parameter $p \ge 1$ (set to 4), the ranking focuses strictly on precision at the top of the list. The top $\alpha\%$ (5%) of $X_U$ candidates are accepted as positive pseudo-labels, while the bottom $\beta\%$ (5%) are designated as negative pseudo-labels (background).
  5. Iterative Retraining: Selected pseudo-labels update the target heatmaps and loss masks, and the detector is retrained over successive iterations (typically 2 iterations).

Datasets

The authors evaluated their framework across five imaging conditions drawn from two standard benchmarks:

  • BF-C2DL-HSC (ISBI Cell Tracking Challenge): Brightfield microscopy of hematopoietic stem cells ($1010 \times 1010$ resolution). Evaluated on 264 dense images (13 train, 251 test) where training images contained only 10% labeled cells (125 total annotated cells).
  • C2C12: Phase-contrast microscopy of myoblast cells ($1040 \times 1392$ resolution) cultured under four biological conditions:
  • Control (10 train / 90 test)
  • FGF2 (38 train / 342 test)
  • BMP2 (10 train / 90 test)
  • FGF2 + BMP2 (10 train / 90 test)

No new datasets were introduced.

Key Insights and Ideas

  • Ranking beats hard classification under unknown priors: Standard PU-learning requires knowing the positive class prior $\pi_p$. In dense microscopy, this prior fluctuates wildly between images. By using PU-learning solely for feature extraction and delegating label selection to P-classification, the framework optimizes for extreme top/bottom rank precision rather than global accuracy.
  • Dual pseudo-labeling curbs over-detection: Adding reliable negative pseudo-labels from the bottom-ranked pool is just as crucial as adding missed cells. It gradually restores the background suppression that the masked loss intentionally suppressed.
  • Rapid convergence: The model's recall starts near 0.999 while precision is initially low (~0.25–0.34). After just two rounds of pseudo-labeling, precision catches up without sacrificing recall.

Key Results

Detection performance was evaluated using the F-score with a 15-pixel assignment radius:

Method C2C12 Control FGF2 BMP2 FGF2+BMP2 HSC Average
Nishimura et al. (Standard supervised) 0.477 0.318 0.609 0.105 0.157 0.333
Masked Loss Only 0.239 0.361 0.217 0.257 0.393 0.293
Vicar et al. (Image processing) 0.802 0.648 0.739 0.539 0.475 0.641
Yang et al. (Direct PU-learning) 0.239 0.768 0.809 0.257 0.108 0.436
Ours (Iterative Pseudo-labeling) 0.834 0.768 0.769 0.578 0.952 0.780
Fully Supervised (Upper Bound) 0.922 0.924 0.978 0.962 0.998 0.971
  • Substantial gains over baselines: The proposed method reached an average F-score of 0.780, significantly surpassing standard supervision trained on missing labels (0.333) and direct PU object detection (0.436).
  • Near fully supervised on HSC: On the challenging HSC dataset, the method jumped from an initial F-score of 0.394 (iteration 0) to 0.952 (iteration 2), closely matching the fully supervised benchmark (0.998).

Limitations and Future Work

  • Static selection heuristics: The hyperparameter percentages ($\alpha = 5\%$, $\beta = 5\%$) and number of iterations (2) were manually fixed rather than dynamically scheduled based on model confidence or dataset density.
  • Double-detection artifacts: Qualitative results reveal occasional false positives where a single large or irregularly shaped cell produces two distinct heatmap peaks.
  • Future work: The authors plan to design adaptive iteration stopping criteria by analyzing changes in the learned feature space and tracking the stability of pseudo-label assignments.

Summary updated

Abstract

Cell detection is an essential task in cell image analysis. Recent deep learning-based detection methods have achieved very promising results. In general, these methods require exhaustively annotating the cells in an entire image. If some of the cells are not annotated (imperfect annotation), the detection performance significantly degrades due to noisy labels. This often occurs in real collaborations with biologists and even in public data-sets. Our proposed method takes a pseudo labeling approach for cell detection from imperfect annotated data. A detection convolutional neural network (CNN) trained using such missing labeled data often produces over-detection. We treat partially labeled cells as positive samples and the detected positions except for the labeled cell as unlabeled samples. Then we select reliable pseudo labels from unlabeled data using recent machine learning techniques; positive-and-unlabeled (PU) learning and P-classification. Experiments using microscopy images for five different conditions demonstrate the effectiveness of the proposed method. Our code is available at https://github.com/FujiiKazuma/CDFIAPLSUP.git.
BibTeX
@inproceedings{fujii2021cell,
  author = {Kazuma Fujii and Daiki Suehiro and Kazuya Nishimura and Ryoma Bise},
  title = {{Cell Detection from Imperfect Annotation by Pseudo Label Selection Using P-classification}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2021},
  url = {https://openreview.net/forum?id=cJycKrpRLb},
}

Loading related content...

Figure