What’s Wrong with the Murals at the Mogao Grottoes: A Near-Infrared Hyperspectral Imaging Method

Meijun Sun, Dong Zhang, Zheng Wang, Jinchang Ren, Bolong Chai, Jizhou SunScientific 2015 · DOI

TL;DR

This paper introduces a non-contact, automated method using near-infrared (NIR) hyperspectral imaging and deep belief networks (DBNs) to assess the degree of flaking in ancient murals at the Mogao Grottoes. This approach offers a more objective and efficient way to monitor mural degradation compared to traditional visual inspection, aiding in preservation efforts.

What Problem Does This Paper Address?

The Mogao Grottoes, a UNESCO World Heritage site, house invaluable ancient murals that are susceptible to degradation over time due to environmental factors and human activity. Flaking, a common form of damage where pigment layers detach and warp, is a critical indicator of mural deterioration. While preservation efforts are ongoing, there's a need for non-contact, objective, and automated methods to accurately assess the extent and severity of flaking. Current visual assessments are subjective and time-consuming, making it difficult to consistently monitor the effectiveness of preservation techniques or detect early signs of damage.

Methods

The researchers employed a novel approach combining NIR hyperspectral imaging with advanced machine learning algorithms.

  • Hyperspectral Imaging: They used a NIR hyperspectral camera (900-1700 nm) to capture detailed spectral information from the murals. This technique allows for the analysis of material composition and physical state based on how different wavelengths of light are reflected or absorbed.
  • Data Acquisition and Labeling: Hyperspectral images were acquired from murals in Cave No.
  • Regions of Interest (ROIs) were manually identified and labeled into four categories: normal, slight, moderate, and severe flaking.
  • Classification Models: Four algorithms were trained and compared for their ability to predict flaking levels:
  • Deep Belief Networks (DBNs)
    • Partial Least Squares Regression (PLSR)
    • Principal Component Analysis with Support Vector Machine (PCA + SVM)
    • Principal Component Analysis with Artificial Neural Network (PCA + ANN)
  • Evaluation Metrics: The performance of these models was evaluated using R-squared coefficient of determination and Root Mean Square Error (RMSE).

Datasets

The study utilized hyperspectral images of murals from Mogao Grottoes, specifically from Cave No.

126.

  • Training/Testing Data: Manually labeled ROIs from two hyperspectral images were used. Each level of flaking was represented by multiple ROIs. A total of 12,000 sample pixels, each with 216 spectral bands, were used for training and testing.
  • New Benchmark: While not explicitly stated as a new benchmark, the paper presents a dataset of hyperspectral images with detailed flaking level annotations, which can serve as a valuable resource for future research in this domain.

Key Insights and Ideas

  • Spectral Signatures of Flaking: Different degrees of flaking exhibit distinct spectral profiles in the NIR range, allowing for differentiation between normal, slight, moderate, and severe damage.
  • DBNs Outperform: Deep Belief Networks (DBNs) demonstrated superior performance in predicting flaking levels, especially when the training data contained striping noise, a common artifact in hyperspectral imaging.
  • Non-Destructive Assessment: NIR hyperspectral imaging offers a non-contact and non-destructive means to assess mural condition, crucial for preserving delicate historical artifacts.
  • Visualizing Damage Distribution: The DBN-based model can generate distribution maps that visually represent the spatial extent and severity of flaking, providing actionable insights for conservationists.
  • Pigment Spectral Differences: The study highlights how different mineral pigments used in the murals have unique spectral signatures, which can be leveraged for material analysis alongside damage assessment.

Key Results

  • The DBN-based model achieved the highest R-squared value (0.5409) and the lowest RMSE (0.2482) among the tested algorithms, indicating better prediction accuracy for flaking levels.
  • While ANN and DBNs performed well in identifying normal and slight flaking, SVM and PLSR showed better results for moderate and severe flaking. However, DBNs provided the best overall performance.
  • The generated flaking distribution maps showed good consistency with human visual assessments, with values between 0.05 and 0.3 suggesting areas at risk of flaking.

Limitations and Future Work

  • Striping Noise: The paper acknowledges that striping noise in hyperspectral data can affect results and suggests future work to reduce it.
  • Feature Selection: While DBNs are less reliant on explicit feature selection, the authors note that conventional algorithms like SVM and PLSR are sensitive to feature selection.
  • Fusion of Models: The authors suggest that fusing different approaches might lead to further improvements in identifying various flaking levels.
  • Future Directions: Future research could focus on real-time monitoring, evaluating protection methods, and developing early warning systems for mural damage. Further investigation into reducing noise, selecting discriminative spectral bands, and tuning DBN parameters is also proposed.

Summary updated

Abstract

Although a significant amount of work has been performed to preserve the ancient murals in the Mogao Grottoes by Dunhuang Cultural Research, non-contact methods need to be developed to effectively evaluate the degree of flaking of the murals. In this study, we propose to evaluate the flaking by automatically analyzing hyperspectral images that were scanned at the site. Murals with various degrees of flaking were scanned in the 126th cave using a near-infrared (NIR) hyperspectral camera with a spectral range of approximately 900 to 1700 nm. The regions of interest (ROIs) of the murals were manually labeled and grouped into four levels: normal, slight, moderate, and severe. The average spectral data from each ROI and its group label were used to train our classification model. To predict the degree of flaking, we adopted four algorithms: deep belief networks (DBNs), partial least squares regression (PLSR), principal component analysis with a support vector machine (PCA + SVM) and principal component analysis with an artificial neural network (PCA + ANN). The experimental results show the effectiveness of our method. In particular, better results are obtained using DBNs when the training data contain a significant amount of striping noise.
BibTeX
@inproceedings{sun2015whats,
  author = {Meijun Sun and Dong Zhang and Zheng Wang and Jinchang Ren and Bolong Chai and Jizhou Sun},
  title = {{What’s Wrong with the Murals at the Mogao Grottoes: A Near-Infrared Hyperspectral Imaging Method}},
  booktitle = {Scientific},
  year = {2015},
  doi = {10.1038/srep14371},
}
Figure