Image Recovery in the Infrared Domain via Path-Augmented Compressive Sampling Matching Pursuit

Tegan H. Emerson, Colin C. Olson, Anthony LutzWorkshops @ CVPR 2019 · DOI

Introduction: The Challenge of High-Resolution Infrared Imaging

High-resolution (HR) infrared (IR) sensors are prohibitively expensive, creating a significant barrier for many applications. While Compressive Sensing (CS) offers a theoretical framework to reconstruct HR images from low-resolution (LR) data, practical implementations often suffer from basis mismatch.

Standard CS algorithms represent images using a discrete dictionary of predefined "atoms" or shapes. If the features in the target image do not align perfectly with these atoms, the reconstruction process introduces artifacts. While deep learning approaches like Convolutional Neural Networks (CNNs) can mitigate this, they require extensive HR IR training datasets, which are costly to acquire. This paper introduces PAIR (Path-Augmented Image Recovery), an algorithm designed to improve reconstruction accuracy without the need for massive training sets or perfectly matched dictionaries.

The Acquisition Model

The proposed system utilizes a compressive sensing framework where the unknown HR image $\vec{x}$ is observed through an optical train. This process involves a random mask $H$ in the Fourier plane (equivalent to convolution with a random mask) followed by downsampling and random demodulation $D$. The resulting observed LR signal $\vec{y}$ is a compressed representation of the original scene.

The acquisition model for compressive sensing in the infrared domain.

The PAIR Algorithm: Path-Augmented Image Recovery

The core of the PAIR algorithm is an enhancement of the CoSaMP (Compressive Sampling Matching Pursuit) greedy algorithm. The primary innovation is the Path-Augmentation step, which leverages optimal transport theory to bridge the gap between discrete dictionary atoms.

1. Baseline: CoSaMP

In traditional CoSaMP, the algorithm iteratively identifies the dictionary atoms that best match the residual (the part of the image not yet reconstructed) to build the final image.

2. Innovation: Path-Augmentation and Geodesics

PAIR moves beyond selecting single atoms. Instead, it identifies the two most correlated shapes and calculates a 2-Wasserstein geodesic between them. This geodesic represents a continuous "morphing" path. For example, if the two atoms are lines at different angles, the geodesic contains all intermediate rotations. By searching along this continuous path, PAIR can synthesize a new atom that matches the image features more precisely than any single item in the original static dictionary.

3. Patch-Based Recovery

To manage computational complexity and local variations, the image is processed in small patches. Each patch is reconstructed individually using the path-augmented support and then stitched together to form the final HR output.

Comprehensive overview of the PAIR recovery process, integrating the acquisition model with patch-based dictionary matching and the path-augmentation step.

Experimental Results

The researchers evaluated PAIR using a dataset of 30 high-resolution IR maritime scenes. They tested the algorithm using both a standard Discrete Cosine Transform (DCT) dictionary and a kSVD learned dictionary.

Key Performance Metrics

  • PSNR (Peak Signal-to-Noise Ratio): PAIR achieved gains of approximately 1.1 dB with the DCT dictionary and 0.5 dB with the learned kSVD dictionary over standard CoSaMP.
  • SSIM (Structural Similarity Index): The algorithm maintained higher structural integrity across various sparsity levels.
  • PEA (Percent Edge Agreement): This metric specifically measures the recovery of sharp edges, which is critical for object identification (e.g., ships). PAIR significantly outperformed the baseline in preserving these edges, especially at low sparsity levels.

Conclusion and Future Work

PAIR demonstrates that incorporating geometric "imagination" via optimal transport into greedy algorithms can significantly enhance the performance of low-cost IR sensors. By turning a discrete dictionary search into a continuous one, the algorithm effectively overcomes basis mismatch without the heavy data requirements of deep learning.

Future work will focus on improving the "stitching" process for overlapping patches to reduce blurring and exploring the impact of different patch sizes and strides on reconstruction quality.

Summary updated

BibTeX
@inproceedings{emerson2019image,
  author = {Tegan H. Emerson and Colin C. Olson and Anthony Lutz},
  title = {{Image Recovery in the Infrared Domain via Path-Augmented Compressive Sampling Matching Pursuit}},
  booktitle = {Workshops @ CVPR},
  year = {2019},
  doi = {10.1109/cvprw.2019.00116},
}
Figure