Abstract
Multimode fibers (MMFs) are promising for ultra-thin medical endoscopy due to their high spatial information capacity. However, light propagation is scrambled by fiber bending and temperature changes, requiring a dynamic Transmission Matrix (TM) for correction. Traditional deep learning models like CNNs often fail to capture the long-range correlations inherent in TMs. This work introduces a self-attention-based non-linear basis transformation that treats TMs as sequences. The approach maps complex, dynamic TMs into a highly sparse, low-dimensional latent space. Our results demonstrate that this method achieves less than 10% reconstruction error and significantly outperforms standard linear and convolutional models, particularly for dense round-trip TMs.
Introduction
Multimode fibers (MMFs) can carry massive amounts of spatial information within a footprint the width of a human hair. This makes them ideal for minimally invasive medical imaging. However, the "scrambling" of light during propagation results in complex speckle patterns. This process is mathematically described by a Transmission Matrix (TM). While TMs can be calibrated for static fibers, real-world applications involve dynamic perturbations (e.g., movement, heating) that shift the TM.
Standard computer vision tools, such as Convolutional Neural Networks (CNNs), are optimized for local pixel correlations. TMs, however, exhibit long-range mathematical links that CNNs struggle to model. There is a critical need for a representation that can capture these shifting complexities in a compact, real-time trackable form.
Methods: Self-Attention and Non-Linear Transformations
The proposed architecture moves away from the "image" paradigm, treating the TM as a sequence of data. This allows the use of self-attention mechanisms, which can relate distant elements within the matrix regardless of their spatial proximity.
The framework consists of three primary innovations:
- Self-Attention Layers: These layers learn to "attend" to the most critical elements defining the fiber's state, capturing long-range correlations that define wave propagation.
- Non-linear Basis Transformation: Unlike rigid linear transformations, this model learns an input-dependent mapping to a new coordinate system where the data representation is sparse.
- Dual-Constraint Autoencoder: The model is trained using an invertibility constraint to ensure the original TM can be recovered, and a sparsity constraint to force the discovery of a compact latent representation.
Results
The performance of the self-attention-based model was evaluated against linear models, Fully Connected Neural Networks (FCNNs), and CNNs across three datasets: sparse forward TMs, dense round-trip TMs, and physically modeled perturbed TMs.
Key findings include:
- Superior Sparsity: The self-attention FCNN achieved a participation ratio ($p$) as low as 0.01 to 0.11, indicating a highly compressed representation.
- High Fidelity: Reconstruction errors remained $\le 10\%$ across all test cases.
- Robustness to Density: On dense round-trip matrices where CNNs exhibited 32% error, the self-attention model maintained a low 8% error, proving its ability to handle complex physical priors.
Discussion and Conclusion
By breaking the "image" bias and treating TMs as sequences, this research demonstrates that "modeling is compression." The ability to represent a $78 \times 78$ matrix (over 6,000 elements) with a few latent parameters suggests the model has successfully captured the underlying physics of fiber perturbations.
Future work will focus on reducing the "data hunger" of these models through simulation and domain transfer, as well as scaling the self-attention mechanism to handle higher-resolution TMs. Additionally, exploring broad-spectrum light handling will be essential for practical multi-color fluorescence imaging applications.
Summary updated