Abstract We introduce software assistants—"bots"—designed to automate the analysis of image-based transcriptomic data. The primary challenges of detecting nuclei and counting associated RNA puncta are addressed through a user-friendly interface and a novel stacked Random Forest model. By incorporating circularity features as geometric priors, our model achieves segmentation accuracy comparable to state-of-the-art deep learning architectures like U-Net, while remaining trainable on standard CPU hardware. This approach democratizes advanced image analysis for laboratories without specialized computational infrastructure.
Introduction
Image-based transcriptomics provides high-resolution spatial mapping of gene expression in situ. However, the analysis of these datasets is often a significant bottleneck. Researchers must accurately segment cell nuclei and count tiny fluorescent dots (puncta) representing RNA molecules. A major technical hurdle is the segmentation of "clumped" or touching nuclei; traditional algorithms often fail to separate these clusters, leading to inaccurate counts. While deep learning models like U-Net offer high performance, their requirement for expensive GPUs and large annotated datasets limits their accessibility.
The "Bot" Framework
To bridge the gap between manual analysis and hardware-intensive deep learning, we developed two Matlab-based tools: SpotsInNucleiBot (SNB) for general workflows and NucleiSegmentationBot (NSB) for training custom models. These tools are designed as "assistants" with minimalistic graphical user interfaces (GUIs) that guide users through parameter setting and model training, making automated analysis accessible to biologists.
Technical Innovation: Circularity Features
The technical standout of our framework is the Stacked Random Forest for nuclei segmentation. Unlike standard classifiers, a stacked model uses the output of one layer as input for the next to refine predictions. To specifically address touching nuclei, we introduced circularity features derived from a variation of the Circular Hough Transform. These features generate "likelihood maps" for the centers and boundaries of circular objects, providing the model with geometric priors that allow it to "see" boundaries that traditional edge-detection methods miss.
For puncta detection, the bots offer two methods: a standard Laplacian of Gaussian (LoG) filter and an "Advanced LoG" that utilizes Gaussian fitting to handle noisier data, ensuring robust performance across varying image qualities.
Key Results and Performance
We evaluated the bots using a publicly available dataset of approximately 2,000 nuclei from mouse cerebral cortex slices. The Random Forest model with circularity features (RF+) achieved a Normalized Coverage Score matching that of the popular U-Net model. This performance parity is significant because the RF+ model can be trained and deployed on a modern CPU-only computer. Furthermore, we demonstrated the versatility of these bots by successfully applying them to protein detection in human fibroblasts, proving their utility for general "spots-in-nuclei" problems.
Conclusion
By combining geometric priors with accessible machine learning, our software bots provide a powerful alternative to hardware-demanding deep learning models. They empower researchers to perform high-throughput transcriptomics and protein detection using standard laboratory hardware, effectively democratizing advanced image-based analysis.
Summary updated