MExConn: A Mechanistically Interpretable Multi-Expert Framework for Multi-Organelle Segmentation in Connectomics

Abrar Rahman Abir, Anik Saha, Shashata Sawmya, Thomas L. Athey, Nir Shavit, Md. Shamsuzzoha BayzidbioRxiv 2025 · DOI

TL;DR

This paper introduces MExConn, a novel multi-expert U-Net architecture that simultaneously segments multiple organelles from electron microscopy images using a shared encoder and specialized decoders. MExConn not only achieves state-of-the-art segmentation performance but also offers mechanistic interpretability, revealing how the shared encoder learns crucial representations for accurate multi-organelle analysis.

What Problem Does This Paper Address?

Connectomics, the study of neural wiring, relies heavily on segmenting subcellular organelles from electron microscopy (EM) images. This process is incredibly labor-intensive and time-consuming when done manually. While computational methods exist, most focus on segmenting only one organelle at a time. This approach misses out on valuable shared information present within EM images that contain multiple organelles simultaneously. The authors aim to bridge this gap by developing a method that can efficiently and accurately segment multiple organelles at once, leveraging the inherent relationships between them.

Methods

The core of MExConn is a U-Net-inspired architecture featuring a shared encoder and multiple decoder heads. The shared encoder processes the input EM image and generates a set of hierarchical feature maps. These feature maps are then fed into individual decoder heads, each specialized for segmenting a specific type of organelle (e.g., mitochondria, membranes, synapses). This multi-head design allows for simultaneous segmentation of different organelles from a single input.

A key innovation is MExConn's mechanistic interpretability (MI). The authors employ a gradient-based approach to analyze the encoder's internal workings. By examining how decoder outputs change with respect to individual encoder channels, they can quantify the importance of each channel for segmenting specific organelles. This analysis reveals whether channels are universally important for all organelles (shared) or specific to certain ones.

Datasets

The study utilizes three established EM datasets for training and evaluation:

  • Urocell [23]: Focuses on mouse urinary bladder, containing mitochondria, lysosomes, and fusiform-vesicles.
  • Multiclass-EPFL [21]: From mouse CA1 hippocampus, includes membranes and mitochondria.
  • Drosophila-VNC [22]: From Drosophila ventral nerve cord, containing membranes, mitochondria, and synapses.

The datasets were split into training, validation, and testing sets, with images processed as overlapping patches.

Key Insights and Ideas

  • Leveraging Shared Information is Crucial: The most significant insight is that joint training for multiple organelles, by sharing an encoder, allows the model to learn representations that are beneficial for all target organelles. This is a more efficient and effective approach than training separate models for each.
  • Mechanistic Interpretability Unlocks Understanding: The ability to dissect the encoder's learned features is a game-changer. It moves beyond just achieving high accuracy to understanding why the model works, revealing the interplay between shared and organelle-specific features.
  • Shared Channels are Foundational: The analysis shows that a significant number of encoder channels are crucial for segmenting all organelles. Ablating these shared channels leads to substantial performance degradation, underscoring their fundamental role.
  • Organelle-Specific Channels Provide Nuance: While shared channels are vital, the existence of organelle-specific channels allows the model to capture the unique characteristics of each organelle, leading to more refined segmentation.
  • Joint Training Scales to 3D: The effectiveness of MExConn isn't limited to 2D; it demonstrates strong performance in 3D volumetric segmentation, indicating its broad applicability.

Key Results

  • Superior Segmentation Performance: MExConn consistently outperformed five baselines (single-organelle models, PyTC, FusionNet, RETINA, and SAM) across all evaluation metrics (VI, IoU, Precision, Recall, Dice) on the three benchmark datasets.
  • Significant Reduction in Variation of Information (VI): MExConn reduced VI by an average of 10.4% compared to single-organelle models. In the Urocell dataset, this reduction reached up to 33.54% on average across organelles.
  • Mechanistic Interpretability Findings:
    • Venn diagrams revealed a mix of shared and organelle-specific channels. For example, in Multiclass-EPFL, 16 channels were shared among the top 100 for each organelle.
  • Ablation studies showed that removing even a small number of shared channels (e.g., 8-16) caused a much larger performance drop (average VI increase of 170.05%) than removing a larger number of random channels (average VI increase of 56.71%).
  • Ablating organelle-specific channels also led to significant degradation (average VI increase of 212.37%), highlighting their importance.
  • Qualitative Improvements: Visual comparisons showed MExConn producing more accurate, contiguous segmentations, especially for challenging cases, compared to single-organelle models which often yielded porous or fragmented results.

Limitations and Future Work

  • Fixed Organelle Set: The current MExConn model is trained for a predefined set of organelles. It cannot segment previously unseen structures.
  • Annotation Dependency: The approach relies on sufficient labeled data for each organelle, limiting its use in low-annotation or unlabeled scenarios.

Future work suggested includes:

  • Extending MExConn to support a dynamic set of organelles.
  • Integrating semi-supervised or self-supervised learning to reduce reliance on labeled data and improve scalability.
  • Exploring domain adaptation and active learning pipelines for enhanced cross-domain generalization.

Summary updated

Abstract

Abstract Electron microscopy (EM) provides subcellular resolution which has made it a critical tool in fields such as cellular biology and connectomics. However, manual annotation of subcellular organelles in these EM images is extremely labor-intensive and impractical at scale. While computational segmentation methods have been developed, most existing approaches are limited to segmenting a single organelle at a time, neglecting the inherent shared information present in EM images containing multiple organelles. To address this, we present MExConn, the first known interpretable multi-expert U-Net architecture in the connectomics field that employs a shared encoder and multiple decoder heads to simultaneously segment multiple organelles from the same input EM image. MExConn significantly outperforms five baselines, including single-organelle model and four state-of-the-art connectomics segmentation models in all evaluation metrics, reducing the Variation of Information by up to 33.54% on average across organelles. A key novelty of our approach is that MExConn offers mechanistic interpretability by revealing that the shared encoder learns shared representations essential for accurately segmenting multiple organelles. Through systematic analysis of encoder gradients with respect to each decoder output, we identify channel-wise importance profiles and reveal that many encoder channels are jointly essential for all organelles, while others are organelle-specific. Rigorous experiments on three connectomics datasets demonstrate the effectiveness of MExConn in both segmentation performance and interpretability, establishing it as a principled approach for multi-organelle analysis in connectomics. The source code is publicly available at https://github.com/abrarrahmanabir/MExConn .
BibTeX
@misc{abir2025mexconn,
  author = {Abrar Rahman Abir and Anik Saha and Shashata Sawmya and Thomas L. Athey and Nir Shavit and Md. Shamsuzzoha Bayzid},
  title = {{MExConn: A Mechanistically Interpretable Multi-Expert Framework for Multi-Organelle Segmentation in Connectomics}},
  year = {2025},
  doi = {10.1101/2025.11.19.689394},
  howpublished = {bioRxiv},
}
Figure