HHW-Ego: DSLR-Quality Enhancement for Multi-Source Wearable Ego Imaging

Yunfei Tong, Qianqian Sun, Jixiang Luo, Dell Zhang, Hao SunarXiv 2026 · arXiv

TL;DR

The researchers introduce HHW-Ego, a novel framework and dataset designed to bridge the massive quality gap between wearable cameras (like smart glasses) and professional DSLRs. By combining a customized "aligned degradation" training pipeline with hyperspectral-guided color calibration, they achieve state-of-the-art image enhancement that significantly outperforms existing super-resolution models in both detail and color fidelity.

What Problem Does This Paper Address?

Wearable egocentric cameras are essential for applications like AR/VR, life-logging, and embodied AI. However, their physical constraints—tiny lenses, small sensors, and limited dynamic range—result in images that are often blurry, noisy, and color-distorted compared to smartphones or DSLRs.

The core challenge isn't just the hardware; it's the data bottleneck. Most deep-learning enhancement models require "paired" data (a low-quality image and its perfect high-quality counterpart) to learn. For wearable cameras, such pairs don't naturally exist. Standard synthetic degradations (like simple downsampling) fail to mimic the complex artifacts of real wearable sensors, leading to models that perform poorly in the real world.

Methods

The authors propose a two-stage enhancement framework:

  1. Stage 1: Aligned Degradation & Super-Resolution. To create realistic training pairs, the team developed an "aligned degradation" pipeline. Instead of random noise, they estimate blur kernels and noise distributions directly from real wearable devices and apply them to high-quality HDR images. This ensures the model learns to fix the actual problems of wearable sensors.
  2. Stage 2: HSI-Guided Color Calibration. Even with better resolution, color remains a struggle. The authors leverage Hyperspectral Imaging (HSI)—which captures light across dozens of narrow spectral bands rather than just Red, Green, and Blue—to perform precise color temperature refinement. This stage includes white balance correction, spectral matching, and a clever overexposure recovery technique that uses neighboring pixel statistics to "fill in" saturated areas.
Fig. 1. Overview of our two-stage enhancement pipeline. Stage 1 utilizes an aligned degradation model to train a super-resolution network for detail recovery. Stage 2 integrates hyperspectral (HSI) information via XYZ/LAB color space transformation to achieve precise color calibration and overexposure recovery.

Datasets

The paper introduces the HHW-Ego dataset, a massive multi-source collection of 5,000 image groups. Each group contains:

  • LQ (Low-Quality): Images from consumer-grade wearable devices (EmdoorVR and RayNeo AI glasses).
  • HQ (High-Quality): 14-bit HDR images from a Canon EOS 77D DSLR.
  • HSI (Hyperspectral): Spectral cube data (380-980 nm) from a Seetrum sensor.

The data spans indoor/outdoor and day/night scenes, all strictly aligned via a multi-stage co-registration workflow to ensure pixel-level correspondence.

Fig. 2. (a) Schematic of the HHW-Ego data acquisition platform. (b) The image co-registration workflow, including cropping, resizing, and feature-based matching. (c) The sub-pixel target enhancement pipeline integrating resized HSI information for the final output.

Key Insights and Ideas

  • The "Aligned" Advantage: The authors found that simply training on more data isn't enough; the style of degradation matters. By aligning the synthetic training data's noise and color space (Lab/RGB) to the target device, they significantly lowered the Fréchet Distance (FD), meaning the model's output looked much more "real."
  • HSI as a Ground Truth for Color: While RGB cameras are prone to "color casts" under tricky lighting, hyperspectral data provides a physical reference for what the colors should be. Using HSI to guide Correlated Color Temperature (CCT) mapping allows for professional-grade color correction without needing to retrain the underlying SR model.
  • Overexposure Recovery: One of the most impressive "aha" moments is the use of XYZ color space masks to identify overexposed regions and then using dilated masks to pull valid color information from surrounding pixels, effectively "healing" blown-out highlights.

Key Results

The framework was tested against heavyweights like Real-ESRGAN, StableSR, and SUPIR.

  • Quantitative Gains: In Stage 1, fine-tuning on the HHW-Ego dataset improved PSNR by 4.26 dB for Real-ESRGAN and 1.46 dB for PFT-SR.
  • Qualitative Superiority: In human preference studies, the proposed method was preferred 93% of the time over baselines.
  • Color Fidelity: Gamut analysis showed that HSI-guided correction significantly broadened chromaticity coverage, particularly in night scenes where artificial lights often cause severe color clipping in standard wearable cameras.
Fig. 3. Visual comparison with state-of-the-art super-resolution methods on diverse real-world scenes.

Limitations and Future Work

While the results are impressive, the authors note that the computational cost of some backbones (like Diffusion or Transformer-based models) remains a hurdle for real-time deployment on resource-constrained smart glasses. Future work will likely focus on optimizing these models for low-latency inference and exploring even more extreme lighting conditions to further push the boundaries of wearable imaging.

Summary updated

Abstract

Wearable ego-centric images are now in high demand for scenarios ranging from daily smart glass usage to embodied intelligence. But the image quality is far behind smart phone due to the lacking paired high-quality reference images and the dedicated enhancement systems. To solve this, a customized degradation pipeline is designed to generate paired samples matching wearable camera images, boosting the upper limit of enhancement performance. Besides a two-stage enhancement framework is further built: first, tuning an efficient model on the paired dataset to enhance real wearable images; second, using hyperspectral data to refine color temperature for better quality. Moreover we present HHW-Ego, a multi-source paired dataset integrating with Hyperspectral, High-dynamic range and Wearable Ego-centric data, which includes 5,000 image groups spanning indoor/outdoor and day/night scenes. On held-out paired data, Stage-1 fine-tuning improves PSNR, SSIM, Lab distance, and LPIPS for Real-ESRGAN and PFT-SR. On real wearable images, no-reference metrics, human preference, gamut analysis, and visual comparisons characterize the effects of Stage 2.
BibTeX
@misc{tong2026hhw,
  author = {Yunfei Tong and Qianqian Sun and Jixiang Luo and Dell Zhang and Hao Sun},
  title = {{HHW-Ego: DSLR-Quality Enhancement for Multi-Source Wearable Ego Imaging}},
  year = {2026},
  eprint = {2611.24010},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.24010},
}
Figure