Abstract
Super-resolution techniques are crucial in improving image granularity, particularly in complex urban scenes, where preserving geometric structures is vital for data-informed cultural heritage applications. In this paper, we propose a city scene super-resolution method via geometric error minimization. The geometric-consistent mechanism leverages the Hough Transform to extract regular geometric features in city scenes, enabling the computation of geometric errors between low-resolution and high-resolution images. By minimizing mixed mean square error and geometric align error during the super-resolution process, the proposed method efficiently restores details and geometric regularities. Extensive validations on the SET14, BSD300, Cityscapes and GSV-Cities datasets demonstrate that the proposed method outperforms existing state-of-the-art methods, especially in urban scenes.
Introduction
Single Image Super-Resolution (SISR) aims to reconstruct a high-resolution (HR) image from a single low-resolution (LR) input. While modern deep learning models excel at generating textures, they often fall into a "pixel-centric" trap by focusing primarily on minimizing Mean Squared Error (MSE) between individual pixels. In urban environments characterized by skyscrapers, windows, and street grids, this approach frequently results in geometric distortions. Models may achieve accurate average color values but produce "wobbly" or disconnected edges where crisp, straight lines are required. For applications such as urban planning and cultural heritage preservation, these distortions represent a significant loss of structural data.
Methodology
To address these challenges, we introduce GeoSR, a framework built upon a modified UnetSR architecture. The core innovation lies in treating geometry as a mathematical constraint during the training process.
Geometric Feature Extraction
GeoSR utilizes a combination of Canny edge detection and the Hough Transform to identify regular geometric features. The Hough Transform is particularly effective at detecting lines by analyzing patterns in edge pixels across various urban contexts.
A critical component of this extraction is the Hough Transform threshold ($T_h$). Ablation studies revealed that a threshold of 0.4 provides the optimal balance for urban scenes; lower values capture chaotic noise, while higher values ignore essential architectural details.
Geometric Error Minimization
The model is trained using a hybrid loss function ($L$) designed to ensure both pixel-level accuracy and structural integrity:
- $L_{MSE}$: Standard pixel-level accuracy.
- Classic Geometric Error ($L_c$): The difference between the geometric map of the reconstructed image and the ground truth.
- Geometric Align Error ($L_d$): Ensures the reconstructed HR geometry is consistent with a proportional projection of the original LR geometry.
Experimental Evaluation
The model was evaluated on four datasets: SET14 and BSD300 (standard benchmarks), and Cityscapes and GSV-Cities (urban street scenes). The latter two datasets are essential for testing the model's ability to handle rich geometric regularities like buildings and traffic signs.
Results and Discussion
GeoSR consistently outperforms state-of-the-art (SOTA) methods, including NLSAN, DBPN, and SRGAN, particularly in urban contexts.
- Quantitative Performance: On the GSV-Cities dataset with an $8\times$ upscaling factor, GeoSR achieved a PSNR of 26.0288, notably higher than UnetSR's 24.8691.
- Qualitative Quality: Visual analysis confirms that architectural elements like window edges and balcony shadows are significantly sharper and more regular in GeoSR outputs compared to other models, which often produce jagged edges.
- Computational Efficiency: Despite the added geometric constraints, GeoSR maintains a competitive parameter count of 8.50M, making it more efficient than heavier models like DBPN (23.21M) or NLSAN (10.12M).
Conclusion
GeoSR demonstrates that integrating geometric priors into the super-resolution process significantly improves the reconstruction of urban environments. While the current focus is on geometric structures, future work will explore domain-specific priors for different architectural styles and self-supervised learning to enhance generalization to "in-the-wild" imagery. This approach holds promise for broader applications in environmental monitoring and virtual tourism.
Summary updated