TL;DR
This paper introduces a novel Rank-R tensor-based deep learning model for automatically detecting and classifying defects on cultural heritage (CH) monuments using hyperspectral imagery. The proposed method achieves higher accuracy and better robustness against overfitting compared to conventional deep learning approaches, especially when training data is limited.
What Problem Does This Paper Address?
Cultural heritage (CH) assets are vulnerable to damage from human activities and natural disasters, necessitating regular inspections for defects, deterioration, and structural deformation. Traditional inspection methods are manual, time-consuming, and costly. Hyperspectral imaging offers rich material information, but processing its high-dimensional data for defect detection in CH is challenging. Existing deep learning methods often require large amounts of labeled data, which is expensive and difficult to obtain for CH applications. This paper aims to bridge this gap by developing an accurate, robust, and data-efficient automated inspection system for CH monuments.
Methods
The core of this work is the Rank-R Feed Forward Tensor-based Neural Network (Rank-R FNN). Unlike standard deep learning models that treat input data as vectors or matrices, the Rank-R FNN leverages the inherent tensor structure of hyperspectral data. Hyperspectral images are processed as 3D tensors (height x width x spectral bands), capturing both spatial and spectral information.
The Rank-R FNN utilizes a Rank-R Canonical Polyadic decomposition for its weights. This decomposition significantly reduces the number of trainable parameters, which is key to overcoming the data scarcity problem. By decomposing the weight tensors, the model can learn effectively with fewer training samples without sacrificing accuracy. The model is trained using a cross-entropy loss function and optimized with the Adam gradient-based optimizer. For comparison, a standard Convolutional Neural Network (CNN) is also implemented and evaluated.
Datasets
The paper introduces a new dataset collected from the ancient walls of Saint Nicolas fortress in the UNESCO Heritage Medieval city of Rhodes, Greece. This dataset consists of 6 hyperspectral images, each with 1016 x 1820 pixels and 42 spectral bands. The dataset is accompanied by pixel-based ground truth annotations provided by CH experts, identifying four classes of defects: salt defects, non-significantly defected areas, minor deterioration, and major deterioration.
Key Insights and Ideas
- Tensor-based learning is crucial for hyperspectral data: By treating hyperspectral data as tensors, the Rank-R FNN can exploit the rich spatio-spectral correlations that are often lost in traditional tensorization or flattening approaches.
- Reduced model complexity leads to data efficiency: The Rank-R decomposition significantly reduces the number of parameters, making the model less prone to overfitting and enabling effective learning from smaller datasets. This is a critical advantage for CH applications where data labeling is costly.
- Robustness to limited data: The proposed tensor-based approach demonstrates superior performance and lower variance compared to CNNs, particularly when training with a limited number of samples. This suggests its practical applicability in real-world CH scenarios.
- Improved discrimination for CH defects: The model's ability to achieve high accuracy even with small training sets highlights its potential for fine-grained defect classification, which is essential for assessing the stability of CH structures.
Key Results
The Rank-R FNN consistently outperformed the state-of-the-art CNN model across various experimental settings.
- Overall Accuracy: For a Tensor Window Size (TWS) of 9 and a small number of training samples (TS), the Rank-R FNN achieved accuracies ranging from 65% to 78%, while the CNN achieved only 45% to 65%. As TS increased, the CNN's accuracy approached that of the tensor model, but the tensor model maintained better performance.
- Robustness: The Rank-R FNN exhibited a significantly lower standard deviation across multiple experimental runs compared to the CNN, indicating greater robustness.
- Defect Class Accuracy: The proposed tensor model outperformed the CNN across all defect classes.
- Qualitative Results: Visual comparisons showed that even the worst-case scenario for the Rank-R FNN (small TS) was more accurate than the best-case scenario for the CNN.
Limitations and Future Work
- Limitations: The paper acknowledges that as the number of training samples increases, the performance gap between the CNN and the tensor model narrows. However, the CNN still shows higher variance, implying lower robustness. The scale of analysis is on the order of centimeters, which is manageable, but the computational demands for larger areas might still be a consideration.
- Future Work: The authors suggest that their proposed system offers civil engineers a non-invasive, real-time method to assess the impact of defects on CH monuments. Further exploration of the Rank-R FNN's capabilities on larger and more diverse CH datasets would be beneficial. Investigating different tensor decomposition techniques or network architectures could also lead to further improvements.
Summary updated