TL;DR
This paper introduces "Unrolled-DOT," a novel deep learning approach for diffuse optical tomography (DOT) that significantly speeds up image reconstruction and improves its quality. By "unrolling" a traditional iterative solver into a neural network, Unrolled-DOT learns to account for model mismatches and hyperparameters directly from data, achieving state-of-the-art results on both simulated and real-world datasets.
What Problem Does This Paper Address?
Imaging through scattering biological tissues is crucial for applications like tumor detection and neuroimaging. Diffuse optical tomography (DOT) is a promising technique for this, but its image reconstruction algorithms often suffer from long computation times and sensitivity to model mismatches (e.g., inaccurate system calibration). Traditional methods require extensive hyperparameter tuning, and even fast solvers can take minutes per image, precluding real-time applications. This paper aims to overcome these limitations by developing a faster, more accurate, and interpretable DOT inverse solver.
Methods
The core of Unrolled-DOT is an "algorithm unrolling" technique. This involves taking the iterative steps of a standard linear inverse solver (specifically, the Iterative Shrinkage-Thresholding Algorithm, ISTA) and transforming them into layers of a neural network. This allows the network to learn the parameters of the solver, including regularization terms and potentially even aspects of the forward model, directly from data.
Key methodological contributions include:
- Unrolled ISTA: The ISTA algorithm's gradient steps are converted into fully connected (FC) layers, and its proximal operator (shrinkage function) becomes a learnable activation function. This makes the network interpretable, as its learned parameters correspond to physical quantities.
- Refinement U-Net and VGG Perceptual Loss: To further enhance reconstruction quality, a U-Net architecture is used as a refinement stage after the unrolled network. Additionally, a Visual Geometry Group (VGG) perceptual loss is incorporated, which encourages the network to match high-level structural features of the ground truth image, not just pixel-wise accuracy.
- Data-Driven Calibration: By training on real-world data, Unrolled-DOT can implicitly learn and correct for system parameters (like source-detector positions) that are often difficult to calibrate precisely, thus mitigating model mismatch.
Datasets
- Simulated Data: Generated using an in-house Monte Carlo simulator. This includes datasets derived from MNIST and fashion-MNIST images, scaled to represent absorption coefficients. For generalizability tests, a 10x10 array of sources and detectors was used.
- Real-World Data: The authors collected and made publicly available a novel dataset of 5000 ToF-DOT measurements. This dataset was created using a prototype system with an E-ink display to programmatically control target shapes and placements, providing accurate ground truth. This is highlighted as the first dataset of its kind with thousands of real-world measurements.
Key Insights and Ideas
- Interpretability through Unrolling: Unrolling traditional algorithms into neural networks offers a bridge between deep learning's power and the interpretability of physics-based models. The learned parameters in Unrolled-DOT directly relate to physical concepts like sensitivity and regularization.
- Learning to Mitigate Model Mismatch: The ability of Unrolled-DOT to learn from real-world data allows it to implicitly calibrate system parameters and account for discrepancies between the assumed forward model and the actual system, leading to more robust reconstructions.
- Synergy of Unrolling and Refinement: Combining the unrolled network (for efficient initial reconstruction) with a U-Net and VGG loss (for fine-tuning and perceptual quality) yields superior results compared to either component alone.
- Data-Driven Hyperparameter Tuning: Unrolled-DOT directly learns optimal hyperparameters, bypassing the tedious and often suboptimal manual tuning required by traditional solvers.
Key Results
- Speed: Unrolled-DOT achieved over a 10x reduction in runtime compared to traditional physics-based solvers.
- Accuracy: On real-world data, Unrolled-DOT outperformed other learning-based algorithms and achieved over a 10x reduction in mean-squared error (MSE) compared to traditional physics-based solvers.
- For synthetic data, adding U-Net and VGG loss reduced MSE from 6.40 x 10⁻² to 1.50 x 10⁻².
- On real-world data with sparse measurements (5x5 confocal scanning), Unrolled-DOT achieved a higher SSIM (0.72) and lower MSE (0.042) than FISTA and ADMM.
- With full ToF-DOT measurements (625 source-detector pairs), Unrolled-DOT achieved an SSIM of 0.88 (compared to 0.60 for ADMM/FISTA) and a significantly lower MSE.
- Generalizability: The model demonstrated good generalizability, performing well on unseen data even when trained on different image classes.
- Efficiency: High-quality reconstructions were achieved with just a few layers in the unrolled network, corresponding to a few iterations of the original solver.
Limitations and Future Work
- Prototype Limitations: The current prototype uses a single laser source and detector, requiring raster scanning. Future work could involve systems with on-chip arrays for faster acquisition.
- Dimensionality: The current work focuses on 2D imaging targets, with extensions to 3D being a future direction.
- In-Vivo Imaging: Applying these algorithms to in-vivo targets is reserved for future work.
- Transfer Learning: The authors suggest that transfer learning could be used to adapt the model to other imaging modalities or to learn only the apparatus-specific parameters for calibration.
Summary updated