Deep Learning-Assisted Simultaneous Targets Sensing and Super-Resolution Imaging

Jin Zhao, Huang Zhao Zhang, Ming-Zhe Chong, Yueyi Zhang, Zi-Wen Zhang, Zong-Kun Zhang, Chao-Hai Du, Pu-Kun LiuarXiv 2023 · OpenReview

Abstract

Recently, metasurfaces have experienced revolutionary growth in the sensing and superresolution imaging field, due to their enabling of subwavelength manipulation of electromagnetic waves. However, the addition of metasurfaces multiplies the complexity of retrieving target information from the detected fields. Besides, although the deep learning method affords a compelling platform for a series of electromagnetic problems, many studies mainly concentrate on resolving one single function and limit the research's versatility. In this study, a multifunctional deep neural network is demonstrated to reconstruct target information in a metasurface targets interactive system. Firstly, the interactive scenario is confirmed to tolerate the system noises in a primary verification experiment. Then, fed with the electric field distributions, the multitask deep neural network can not only sense the quantity and permittivity of targets but also generate superresolution images with high precision. The deep learning method provides another way to recover targets' diverse information in metasurface based target detection, accelerating the progression of target reconstruction areas. This methodology may also hold promise for inverse reconstruction or forward prediction problems in other electromagnetic scenarios.

Introduction

In the field of sensing and imaging, metasurfaces—surfaces engineered with sub-wavelength structures—function as "super-lenses" capable of manipulating electromagnetic waves beyond the capabilities of natural materials. They are particularly effective for Structural Illumination Microscopy (SIM), a technique used to achieve super-resolution by surpassing the classical diffraction limit (typically half the wavelength, $0.5\lambda$).

However, integrating metasurfaces introduces a "complexity tax." Traditional methods for reconstructing images from metasurface-scattered fields are computationally intensive, sensitive to noise, and often require multiple raw images taken at varying angles or frequencies. Furthermore, most existing deep learning (DL) solutions are single-task oriented, focusing on either denoising, classification, or reconstruction in isolation. This research proposes a multitask deep learning framework that simultaneously identifies the number of targets, their material properties (permittivity), and generates super-resolution images from a single frame of raw electromagnetic intensity data.

Overview of the multitask deep learning framework for simultaneous target sensing and super-resolution imaging.

Methodology and Physical Mechanism

The proposed system integrates a multitask deep neural network with a custom-designed 1D periodic circular metal grating metasurface. The metasurface is engineered to generate a complex "multi-harmonic" illumination pattern consisting of both surface waves and propagating waves. When dielectric targets (such as ceramic cylinders) are placed near this surface, they distort the local electromagnetic field.

Unlike traditional SIM, which requires multiple exposures, this model utilizes just one intensity curve—the electric field distribution along a single detection line. The physical mechanism relies on the metasurface's ability to provide high-frequency spatial harmonics that interact with the targets, as shown in the spectral analysis of the field distributions.

Metasurface design and spatial frequency analysis. (a) Unit cell geometry. (b-h) Electric field distributions and corresponding spatial frequency spectra for different periods $P$.

Neural Network Architecture

The network employs a shared Encoder to extract general features from the input field data, which then branches into three specialized heads:

  • Classifier: A multilayer perceptron (MLP) that predicts the number of targets (1, 2, or 3).
  • Regressor: An MLP that estimates the relative permittivity (dielectric constant) of the targets.
  • Decoder: A trans-convolutional network that reconstructs the final super-resolution image.

The training process utilizes a weighted loss function combining Peak Signal-to-Noise Ratio (PSNR), Mean Squared Error (MSE), and Cross-Entropy (CE) to optimize all three tasks concurrently.

Results and Discussion

The model was trained and validated using a dataset of 5,100 simulated samples. The targets consisted of single, double, and triple cylindrical targets with relative permittivity ranging from 10 to 20. Positions and inter-target distances were randomized, including "super-resolution" scenarios with separations as small as $0.2\lambda$.

Key performance metrics include:

  • Sensing Accuracy: The model achieved nearly 100% accuracy in target counting and approximately 95% accuracy in permittivity prediction.
  • Imaging Precision: Reconstructed images reached a PSNR exceeding 42dB.
  • Super-Resolution: The system successfully resolved targets separated by only $0.2\lambda$, significantly outperforming the classical diffraction limit of $0.5\lambda$.

By leveraging the nonlinear mapping power of deep learning, the system extracts quantity, material, and spatial data from a single frame, drastically reducing data acquisition time compared to iterative algorithms like Gerchberg-Saxton. Furthermore, the researchers demonstrated a "forward" model capable of predicting scattered fields from target information, which can generate synthetic training data much faster than traditional physics simulations.

Conclusion

This study demonstrates a robust multitask deep learning approach for metasurface-based sensing and imaging. While currently demonstrated in a 1D environment with simple geometries, the methodology holds significant promise for expansion into 2D and 3D scenarios. Future applications may include high-speed medical imaging or non-destructive testing of integrated circuits, where rapid multi-parameter sensing is essential.

Summary updated

Abstract

Recently, metasurfaces have experienced revolutionary growth in the sensing and superresolution imaging field, due to their enabling of subwavelength manipulation of electromagnetic waves. However, the addition of metasurfaces multiplies the complexity of retrieving target information from the detected fields. Besides, although the deep learning method affords a compelling platform for a series of electromagnetic problems, many studies mainly concentrate on resolving one single function and limit the research's versatility. In this study, a multifunctional deep neural network is demonstrated to reconstruct target information in a metasurface targets interactive system. Firstly, the interactive scenario is confirmed to tolerate the system noises in a primary verification experiment. Then, fed with the electric field distributions, the multitask deep neural network can not only sense the quantity and permittivity of targets but also generate superresolution images with high precision. The deep learning method provides another way to recover targets' diverse information in metasurface based target detection, accelerating the progression of target reconstruction areas. This methodology may also hold promise for inverse reconstruction or forward prediction problems in other electromagnetic scenarios.
BibTeX
@misc{zhao2023deep,
  author = {Jin Zhao and Huang Zhao Zhang and Ming-Zhe Chong and Yueyi Zhang and Zi-Wen Zhang and Zong-Kun Zhang and Chao-Hai Du and Pu-Kun Liu},
  title = {{Deep Learning-Assisted Simultaneous Targets Sensing and Super-Resolution Imaging}},
  year = {2023},
  eprint = {2305.03177},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2305.03177},
}
Figure