Vision-Informed Flow Image Super-Resolution with Quaternion Spatial Modeling and Dynamic Flow Convolution

Qinglong Cao, Zhengqin Xu, Chao Ma, Xiaokang Yang, Yuntian ChenarXiv 2024 · DOI · arXiv

Abstract

Flow Image Super-Resolution (FISR) is essential for recovering high-resolution turbulent velocity fields from low-resolution data. Traditional super-resolution methods often fail in this domain because they treat fluid velocity components as independent color channels, ignoring the underlying physical principles of fluid dynamics. This paper proposes a vision-informed architecture that incorporates Quaternion Spatial Modeling (QSM) to represent orthogonal velocity components and Dynamic Flow Convolution (DFC) to capture fluid-specific morphological properties like viscosity and surface tension. Experimental results on Direct Numerical Simulation (DNS) datasets demonstrate that the proposed method significantly outperforms existing state-of-the-art models, achieving up to a 5.35 dB improvement in PSNR for multi-velocity tasks.

Introduction

Super-resolution (SR) techniques have seen massive success in natural image processing. However, applying these models to fluid flow data—Flow Image Super-Resolution (FISR)—reveals a significant domain gap. In fluid dynamics, the "channels" of an image represent orthogonal velocity components (U, V, W) governed by the Navier-Stokes equations, not independent color intensities. Furthermore, fluids exhibit unique morphological structures driven by physical properties like viscosity. Standard convolutional kernels, designed for rigid or natural textures, struggle to reconstruct these chaotic, droplet-like turbulent structures accurately.

Methodology

The proposed "vision-informed" framework bridges this gap by encoding physical properties directly into the network architecture.

  1. Quaternion Spatial Modeling (QSM): Instead of standard 2D convolutions on RGB channels, the model maps U, V, and W velocities to the imaginary axes ($i, j, k$) of a quaternion field. This allows the network to learn the inherent spatial rotations and orthogonal dependencies between velocity components.
  2. Dynamic Flow Convolution (DFC): To model fluid morphology, the authors developed a specialized deformable convolution. DFC uses a chain decision procedure to determine sampling offsets, mimicking the effects of viscosity, and applies constraints based on surface tension to maintain physical plausibility.
  3. Hybrid Architecture: The overall pipeline uses a series of Flow Feature extraction Blocks (FFB). These blocks combine Swin Transformer layers for global context with DFC for local morphology and QSM for spatial relationships.
The proposed vision-informed architecture featuring Flow Feature Extraction Blocks (FFB), Quaternion Spatial Modeling (QSM), and Flow Image Reconstruction (FIR).

Experimental Setup

The model was evaluated using a dataset derived from Direct Numerical Simulation (DNS) of incompressible isotropic turbulence ($Re_\lambda \approx 250$). The dataset includes high-resolution $512 \times 512$ images and low-resolution versions at $2\times, 4\times,$ and $8\times$ downsampling. Two experimental settings were used: Single-velocity (29,491 training images) and Multi-velocities (9,750 training images).

Results and Discussion

The proposed method consistently achieves state-of-the-art performance across all metrics, including Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), Root Mean Square Error (RMSE), and Mean Absolute Error (MAE). Notably, in the $4\times$ multi-velocity task, the model achieved a PSNR of 44.82 dB, representing a 5.35 dB improvement over the previous best model (DAT).

Visual comparison of super-resolution results on various flow images. The proposed method (Ours) recovers sharper and more accurate turbulent structures compared to baseline models like RCAN, SwinIR, HAT, and DAT.

Visual comparisons show that the proposed method recovers much sharper and more accurate turbulent structures (vortices and filaments) that other models either blur out or distort. Ablation studies highlighted that while both QSM and DFC contribute to performance, the DFC module provides the most significant gains, underscoring the importance of capturing local turbulent morphology.

Conclusion

By integrating physical principles—orthogonality, viscosity, and surface tension—into the neural network's mathematical operations, this work sets a new benchmark for Flow Image Super-Resolution. While currently focused on isotropic turbulence, this "vision-informed" approach could be extended to other physical domains where imaging principles and morphology are well-defined but currently ignored by general-purpose AI models.

Summary updated

Abstract

Flow image super-resolution (FISR) aims at recovering high-resolution turbulent velocity fields from low-resolution flow images. Existing FISR methods mainly process the flow images in natural image patterns, while the critical and distinct flow visual properties are rarely considered. This negligence would cause the significant domain gap between flow and natural images to severely hamper the accurate perception of flow turbulence, thereby undermining super-resolution performance. To tackle this dilemma, we comprehensively consider the flow visual properties, including the unique flow imaging principle and morphological information, and propose the first flow visual property-informed FISR algorithm. Particularly, different from natural images that are constructed by independent RGB channels in the light field, flow images build on the orthogonal UVW velocities in the flow field. To empower the FISR network with an awareness of the flow imaging principle, we propose quaternion spatial modeling to model this orthogonal spatial relationship for improved FISR. Moreover, due to viscosity and surface tension characteristics, fluids often exhibit a droplet-like morphology in flow images. Inspired by this morphological property, we design the dynamic flow convolution to effectively mine the morphological information to enhance FISR. Extensive experiments on the newly acquired flow image datasets demonstrate the state-of-the-art performance of our method. Code and data will be made available.
BibTeX
@misc{cao2024vision,
  author = {Qinglong Cao and Zhengqin Xu and Chao Ma and Xiaokang Yang and Yuntian Chen},
  title = {{Vision-Informed Flow Image Super-Resolution with Quaternion Spatial Modeling and Dynamic Flow Convolution}},
  year = {2024},
  eprint = {2401.15913},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2401.15913},
}
Figure