Abstract
In support of art investigation, researchers have proposed a novel source separation method to unmix single X-ray scans acquired from double-sided paintings. Traditional methods struggle when sources have similar morphological characteristics, such as overlapping brushstrokes. This approach utilizes high-resolution photographs of the front and back of a panel as side information to drive a coupled dictionary learning framework. By capturing both common features across modalities and modality-specific innovations, the method enables accurate separation of complex X-ray mixtures, significantly outperforming state-of-the-art techniques.
The Challenge of Double-Sided Masterpieces
In art conservation, X-ray radiography is a vital tool for seeing beneath the surface of a painting. However, many masterpieces—like the panels of the famous Ghent Altarpiece (1432)—are painted on both sides of a single wooden support. Because X-rays penetrate the entire structure, the resulting image is a confusing jumble of the front painting, the back painting, and the internal wood grain.
Standard "blind" source separation techniques, such as Morphological Component Analysis (MCA), usually rely on the different layers having distinct textures (e.g., smooth vs. grainy). In this case, the two sources are both paintings with similar brushstroke characteristics, making them nearly impossible to distinguish without extra help. This research addresses how to use visible-light photographs—which only capture one side at a time—to guide the separation of the complex X-ray mixture.
Methodology: Coupled Dictionary Learning
The proposed framework relies on Coupled Dictionary Learning. The core idea is to represent images as a combination of "common" and "innovation" components:
- Common Component: Captures features shared between the visible photograph and the X-ray (like the shapes and structures of the figures).
- Innovation Component: Captures features unique to the X-ray modality (like the wood grain of the panel or internal support structures).
To solve the sparse coding problem across these dictionaries, the authors developed a variation of the Orthogonal Matching Pursuit algorithm (mOMP). Additionally, the algorithm uses binary masks to identify "craquelure" (the fine network of cracks on old paintings). By assigning a weight of zero to these pixels during training, the dictionaries avoid being contaminated by high-frequency noise, leading to cleaner reconstructions.
The Multi-Scale Advantage
A critical finding of the study was that single-scale dictionary learning often leaves "ghosts" of the other side in the separated images because it cannot handle the DC (average brightness) values of patches correctly. The authors implemented a recursive pyramid decomposition to address this.
This multi-scale approach effectively "drains" the low-frequency content down through the pyramid, allowing the model to capture global structures at coarse scales and fine details at high resolutions. This was found to be essential for achieving high-quality visual separation.
Key Results and Insights
Experimentation on both synthetic data and the 2TB digital acquisition of the Ghent Altarpiece confirmed the superiority of this method:
- Quantitative Gains: On simulated mixtures, the proposed method outperformed state-of-the-art MCA by up to 1.41 dB in PSNR and significantly improved the Structural Similarity Index (SSIM).
- Qualitative Improvement: Visual results on the Adam and Eve panels show a dramatic improvement. Where previous methods left faces visible on both separated sides, this method successfully isolated the face to one side and the fabric textures to the other. It also successfully isolated the vertical wood grain into the "innovation" layer.
Limitations and Future Work
The method relies heavily on the photographs and X-rays being perfectly registered (aligned pixel-for-pixel). Any misalignment can lead to artifacts in the separation. Furthermore, dictionary learning and multi-scale sparse coding are computationally expensive, especially for ultra-high-resolution art scans.
Future directions include extending this framework to other multi-modal problems, such as medical imaging (combining MRI and CT scans) or remote sensing, and automating the "innovation vs. common" selection process to prevent brushstrokes from being misclassified as wood grain.
Summary updated