Research on key technology of cooled infrared bionic compound eye camera based on small lens array

Xiaoyu Wang, Linhan Li, Yinghao Chi, Jie Liu, Juan Yue, Sili Gao, Xiupeng Yuan, Yang YuScientific 2024 · DOI

TL;DR

This paper presents an innovative infrared bionic compound eye camera inspired by insect vision, designed for wide-field, all-day detection. By integrating a cooled mid-wave infrared detector with a novel optical system featuring 61 small lenses on a curved shell, the camera achieves a 108° x 108° field of view with improved signal-to-noise ratio and reduced distortion compared to traditional systems.

What Problem Does This Paper Address?

Traditional 2D imaging technologies struggle with achieving a large field of view (FOV) while maintaining sensitivity to small targets, especially in low-light or night conditions. Existing wide-angle lenses often suffer from significant edge distortion, compromising image accuracy. While biomimetic compound eye cameras offer advantages like rapid response and reduced edge distortion, they have historically faced challenges with lower resolution and limited applicability in infrared spectrum for all-day detection. This work aims to bridge this gap by developing a compound eye camera specifically for the mid-wave infrared spectrum, leveraging insect eye principles to overcome these limitations.

Methods

The core of this research is the design and implementation of an infrared bionic compound eye camera. The system is inspired by the overlapping structure of insect compound eyes. Key methodological aspects include:

  • Biomimetic Design: Mimicking the compound eye structure with 61 small lens arrays arranged on a curved spherical shell.
  • Optical System Design:
    • A spherical lens array as the first-stage imaging system, with each small lens independently imaging a portion of the scene.
  • A relay optical system to transform the curved focal surface from the lens array into a flat focal plane suitable for the detector. This system incorporates higher-order aspheric surfaces and modified geometric lens surfaces to optimize for large-field aberrations like curvature and astigmatism.
  • Achieving a 100% cold-membrane match between the infrared optical system and the cooled detector, crucial for efficient infrared detection.
  • Detector: Utilization of a high-performance cooled mid-wave infrared detector to enable all-day detection capabilities and minimize thermal noise.
  • Image Processing: Development of an energy correction algorithm tailored for the compound eye structure, addressing non-uniformity and attenuation issues, particularly at the periphery of the field of view. This includes a generalized one-point and two-point correction, and a linear compensation for energy attenuation.
  • Stitching Algorithm: A novel ommatidium information extraction and fixed-pixel alignment stitching method is proposed to reconstruct the large field of view from individual sub-images, overcoming the computational intensity and feature point limitations of traditional stitching algorithms.

Datasets

The paper does not explicitly mention the use of pre-existing datasets for training. Instead, it focuses on experimental validation using custom-built targets and real-world scenes.

  • Experimental Targets: Checkerboard targets, point sources, single and multi-letter targets, and a heating strip were used for calibration and testing.
  • Scene Images: Laboratory and outdoor scene images were captured to demonstrate the camera's performance.

Key Insights and Ideas

  • Insect Eyes as a Blueprint for Infrared Imaging: The research demonstrates that the principles of insect compound eyes can be effectively applied to design advanced infrared imaging systems, offering solutions to limitations of traditional cameras.
  • Wide-Field, All-Day Detection: The combination of a biomimetic compound eye structure and a cooled mid-wave infrared detector is a promising approach for achieving both a wide field of view and robust performance in varying light conditions, including nighttime.
  • Aberration Correction in Compound Eyes: Optimizing optical design with aspheric surfaces and modified lens geometries is crucial for mitigating large-field aberrations inherent in compound eye systems, especially when dealing with curved focal surfaces.
  • Tailored Image Correction: Standard image correction algorithms are insufficient for compound eye cameras due to their unique structure. Developing specialized algorithms that account for factors like peripheral energy attenuation is essential for accurate image reconstruction.
  • Stitching for Large FOV Reconstruction: A simplified stitching approach, leveraging overlap rate and spatial transformations, is more suitable for compound eye cameras than traditional methods, enabling efficient reconstruction of a large, seamless field of view.

Key Results

  • Wide Field of View: The system achieves a total observation field angle of 108° x 108°.
  • Improved Signal-to-Noise Ratio (SNR): Experiments showed an improved SNR in various scenes due to the overlapping nature of the ommatidia and the developed correction algorithms.
  • Reduced Distortion: The overall optical system design achieved a total field distortion of less than 6%.
  • Effective Ommatidia Functionality: Each ommatidium was found to be effective within the imaging range, contributing to the overall image quality.
  • SNR Improvement with Stitching: Table 3 shows a significant improvement in SNR when stitching multiple sub-images compared to individual sub-images, with stitched SNRs ranging from 1.5348 to 1.9906 for different targets.

Limitations and Future Work

  • Energy Attenuation: The paper acknowledges that the energy intensity of the mid-wave infrared compound-eye camera decreases with the hierarchical level of ommatidia from the center toward the periphery. This is attributed to differences in target-ommatidia positional relationships and the special spherical lens design, making consistent light flux calibration challenging.
  • Calibration Complexity: Traditional calibration methods are not fully sufficient, and further research into calibration algorithms is needed to compensate for light flux loss in each ommatidium.
  • Assembly Errors: The authors mention that errors were encountered during manual assembly, which may have impacted image quality.
  • Future Work: The authors suggest that the developed scene reconstruction algorithm, leveraging the camera's special structure, helps achieve real-time target positioning. Further exploration into energy correction methods for the special compound eye camera structure is also indicated.

Summary updated

Abstract

Traditional 2D imaging technologies are limited by the need for a large field of view and their sensitivity to small target motion. Inspired by the characteristics of insect compound eye structure, we propose an infrared bionic compound eye camera based on a small lens array. The camera is composed of 61 small lens arrays mounted on a curved spherical shell and a relay optical system. The imaging device is a high-performance cooled mid-wave infrared detector. This is an innovative design for an infrared biomimetic compound eye camera system that provides a wide field of view and all-day detection capability. Aimed to meet the specified requirements. The optical system achieves a 100% cold-membrane match between the infrared optical system and the cooled detector, and the relay optical system optimizes the large-field aberration by introducing a higher-order aspheric surface and modifying the geometric surface of the lenses. Our entire system enables an observation field angle of <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow><mml:msup><mml:mn>108</mml:mn> <mml:mo>∘</mml:mo></mml:msup> <mml:mo>×</mml:mo> <mml:msup><mml:mn>108</mml:mn> <mml:mo>∘</mml:mo></mml:msup> </mml:mrow> </mml:math> . The experiments showed that the image quality of the system is high, each ommatidium was effective within the imaging range of the compound eye camera, resulting in an improved signal-to-noise ratio in various scenes.
BibTeX
@inproceedings{wang2024research,
  author = {Xiaoyu Wang and Linhan Li and Yinghao Chi and Jie Liu and Juan Yue and Sili Gao and Xiupeng Yuan and Yang Yu},
  title = {{Research on key technology of cooled infrared bionic compound eye camera based on small lens array}},
  booktitle = {Scientific},
  year = {2024},
  doi = {10.1038/s41598-024-61606-1},
}
Figure