Regional Localization of Mouse Brain Slices Based on Unified Modal Transformation

Songwei Wang, Yuhang Wang, Ke Niu, Qian Li, Xiaoping Rao, Hui Zhao, Liwei Chen, Li ShiSymmetry 2021 · DOI

TL;DR

This paper introduces a novel deep learning framework, JEMI-Net, for accurately and efficiently localizing regions within mouse brain slices by registering them to the Allen Reference Atlas (ARA). It tackles the challenge of "non-unified modal images" by transforming them into a common representation, significantly improving registration accuracy and speed compared to existing methods.

What Problem Does This Paper Address?

Understanding the brain requires precisely identifying and analyzing neuronal activity across different brain regions. This process hinges on accurately mapping experimental brain slices to a standardized reference, like the Allen Reference Atlas (ARA). However, a major hurdle is the significant difference in appearance (modality) between real brain slices and the ARA, compounded by distortions and noise introduced during sample preparation. Traditional image registration methods struggle with these "non-unified modal images," leading to inaccurate localization. Existing solutions often involve complex preprocessing or still fall short due to these inherent challenges.

Methods

The core of this work is the Joint Enhancement of Multimodal Information (JEMI) network. This approach tackles the problem in two main stages:

  1. Rough Registration: To handle large displacements and individual animal variations, an affine transformation is first applied. This is a linear transformation that aligns the brain slice and the ARA roughly, making subsequent fine registration more effective. The authors found affine transformation to be the most suitable for this initial alignment due to its speed and good performance.
  2. Modal Transformation and Fine Registration: This is where JEMI-Net shines. Instead of directly registering the disparate brain slice and ARA images, JEMI-Net first transforms them into a unified modality. This is achieved by treating the problem as an image segmentation task. The JEMI network, built on a symmetric encoder-decoder architecture (similar to U-Net), takes both the brain slice and the ARA as input simultaneously. It learns to extract and enhance common key features from both, effectively creating a shared representation. This unified representation then significantly simplifies the subsequent diffeomorphic registration, which ensures that the topological structure of the brain is preserved during the deformation. The diffeomorphic registration is performed using a deep learning framework that optimizes a spatial transformation layer.

Datasets

The study utilized:

  • Mouse brain slices: Acquired through immunohistochemistry and confocal microscopy.
  • Allen Reference Atlas (ARA): Specifically, both the Allen Digital Atlas (ADA) and the Average Template Atlas (ATA) were used.

The authors mention that the amount of data was limited, necessitating data augmentation techniques within the JEMI network to improve its generalization ability and prevent overfitting. They used 64 groups of images for training, 18 for verification, and 18 for testing.

Key Insights and Ideas

  • Transforming the problem: The most significant insight is reframing the multi-modal registration challenge as a segmentation problem. By converting disparate images into a unified modality before registration, the complexity is drastically reduced.
  • Joint enhancement is key: Training the JEMI network with both the brain slice and ARA simultaneously allows the network to learn complementary features and mutually constrain their representations, leading to a more robust unified modality.
  • Diffeomorphism preserves structure: Combining the unified modality approach with diffeomorphic registration is crucial for maintaining the integrity of the brain's topology, which is vital for accurate anatomical analysis.
  • Human intuition inspires AI: The authors were inspired by how human experts can "ignore noise and establish correspondence based on key regions," a principle they embedded into their JEMI network's design.

Key Results

  • Improved Accuracy: The proposed JEMI-Net method achieved significantly better regional localization results compared to existing methods, including those without modal transformation, PCA-based modal transformation, and other registration algorithms.
  • Faster Registration: The JEMI method demonstrated a substantial reduction in processing time. For instance, it was 66.66% faster than the PCA + Reg-Net method.
  • Enhanced Similarity Metrics: The method led to increased NCC (Normalized Correlation Coefficient) and NMI (Normalized Mutual Information) values, indicating better image correlation and information overlap between the registered images.
  • Qualitative Improvements: Visualizations show that the JEMI method accurately aligns brain slice features with the ARA, preserving topological structures and effectively handling noise and complex feature correspondences.

Limitations and Future Work

  • Data Augmentation Necessity: The authors acknowledge that the dataset size was limited, requiring extensive data augmentation. This suggests that performance might be further improved with larger, more diverse datasets.
  • Future Directions: The authors plan to explore methods for extracting smaller features within brain slices to further enhance registration performance. They also suggest that the JEMI framework can be extended to other multi-modal image registration tasks beyond brain slices.

Summary updated

Abstract

Brain science research often requires accurate localization and quantitative analysis of neuronal activity in different brain regions. The premise of related analysis is to determine the brain region of each site on the brain slice by referring to the Allen Reference Atlas (ARA), namely the regional localization of the brain slice. The image registration methodology can be used to solve the problem of regional localization. However, the conventional multi-modal image registration method is not satisfactory because of the complexity of modality between the brain slice and the ARA. Inspired by the idea that people can automatically ignore noise and establish correspondence based on key regions, we proposed a novel method known as the Joint Enhancement of Multimodal Information (JEMI) network, which is based on a symmetric encoder–decoder. In this way, the brain slice and the ARA are converted into a segmentation map with unified modality, which greatly reduces the difficulty of registration. Furthermore, combined with the diffeomorphic registration algorithm, the existing topological structure was preserved. The results indicate that, compared with the existing methods, the method proposed in this study can effectively overcome the influence of non-unified modal images and achieve accurate and rapid localization of the brain slice.
BibTeX
@inproceedings{wang2021regional,
  author = {Songwei Wang and Yuhang Wang and Ke Niu and Qian Li and Xiaoping Rao and Hui Zhao and Liwei Chen and Li Shi},
  title = {{Regional Localization of Mouse Brain Slices Based on Unified Modal Transformation}},
  booktitle = {Symmetry},
  year = {2021},
  doi = {10.3390/sym13060929},
}
Figure