Advancing Textile Damage Segmentation: A Novel RGBT Dataset and Thermal Frequency Normalization

Farshid Rayhan, Jitesh Joshi, Guangyu Ren, Luis M. Hernández, Bruna Petreca, Sharon Baurley, Nadia Bianchi‐Berthouze, Youngjun ChoSensors 2025 · DOI

TL;DR

This paper introduces RGBT-Textile, a new dataset for segmenting textile damage using RGB and thermal (RGBT) imagery, and a novel normalization technique called ThermoFreq. ThermoFreq effectively reduces thermal noise, significantly improving the performance of state-of-the-art RGBT semantic segmentation models on this challenging task, which is crucial for sustainable textile repair.

What Problem Does This Paper Address?

The textile industry faces pressure to reduce waste and embrace sustainability. A key aspect of this is extending the lifespan of garments through repair. Automating the detection of textile damage is a critical step towards this goal, and RGB-Thermal (RGBT) imaging offers a promising approach. Thermal imaging can reveal subtle material properties and damages that might be missed by RGB cameras alone, especially in varying lighting conditions. However, a significant challenge in RGBT semantic segmentation, particularly for textile damage, is the scarcity of dedicated, publicly available datasets. Furthermore, thermal images are prone to noise and variations due to ambient temperature, which can degrade segmentation performance. Existing normalization techniques often fall short in addressing these specific thermal image challenges.

Methods

The authors introduce two main contributions:

  • RGBT-Textile Dataset: This dataset was meticulously collected and annotated for textile damage segmentation. It comprises 1441 RGBT image pairs, each with ground-truth segmentation masks for three classes: background, garment, and damage. The data collection involved a specific protocol to capture variations in garment types and damage, including scenarios with and without induced thermal variations.
  • ThermoFreq Normalization: This is a novel frequency-domain thermal normalization technique. It works by transforming the thermal image into the frequency domain using a Fourier transform. High-frequency components, often associated with noise, are then selectively filtered or attenuated. The modified frequency domain representation is then transformed back to the spatial domain. This process aims to enhance salient thermal features while mitigating noise, without significantly altering the overall temperature distribution.

Datasets

  • RGBT-Textile Dataset: A newly introduced benchmark dataset specifically for textile damage segmentation. It contains 1441 RGBT image pairs with detailed annotations.
  • Existing RGBT Datasets: The paper also evaluates their proposed method on six other publicly available RGBT datasets: KAIST Multispectral Pedestrian, LLVIP, MFNet, OSU Thermal Pedestrian, McubeS, and PST900. This allows for a broader assessment of ThermoFreq's effectiveness.

Key Insights and Ideas

  • Thermal Imaging's Potential for Textiles: Thermal imaging is not just for detecting heat signatures; it can reveal material properties and subtle damages that are invisible to RGB cameras, making it valuable for textile inspection.
  • The Challenge of Thermal Noise: Ambient temperature fluctuations and sensor noise significantly impact the quality of thermal images, posing a major hurdle for accurate semantic segmentation.
  • Frequency Domain for Thermal Normalization: By operating in the frequency domain, ThermoFreq can selectively target and suppress noise components that are often problematic in thermal imagery, offering a more sophisticated approach than traditional spatial-domain methods.
  • Dataset Specificity Matters: The optimal parameters for thermal normalization (like the tolerance threshold in ThermoFreq) can vary significantly depending on the dataset's characteristics, particularly its temperature range and the nature of the thermal variations.

Key Results

  • ThermoFreq Improves Performance: Experiments show that applying ThermoFreq consistently improves the performance of state-of-the-art RGBT segmentation models (CRM_RGBTSeg, MMSFormer, and UNet variants) across various datasets, including the new RGBT-Textile dataset. Improvements in mean Intersection over Union (mIoU) range from 1% to 2.2% compared to baseline models.
  • Transformer Models Excel: Transformer-based models (CRM_RGBTSeg, MMSFormer) generally outperform CNN-based models (UNet) by 1-2% in mIoU, demonstrating their superior ability to capture global context and multimodal dependencies.
  • RGBT-Textile is Challenging: The RGBT-Textile dataset is moderately challenging, with the "damaged area" class being the most difficult to segment, achieving mIoU values around 0.645-0.675.
  • Robustness to Noise: ThermoFreq-enhanced models show greater robustness to synthetic thermal noise and occlusions, with CNN-based models exhibiting larger performance drops when not normalized.

Limitations and Future Work

  • Parameter Sensitivity: The effectiveness of ThermoFreq is sensitive to its tolerance threshold parameter (t), which needs to be tuned for different datasets. Datasets with higher thermal temperature ranges benefit from higher 't' values, while those with lower ranges require lower 't' values to avoid losing important information.
  • Damage Area Segmentation: While improved, segmenting damaged areas remains the most challenging aspect. Future work could focus on enhancing the normalization technique to better highlight these features or augmenting the dataset with more diverse damage examples.
  • Dataset Expansion: The authors plan to expand the RGBT-Textile dataset with more samples and labels to further improve model training and evaluation.
  • Broader Applications: Investigating the application of ThermoFreq to other domains where thermal imaging is used for segmentation could be a valuable future direction.

Summary updated

Abstract

RGB-Thermal (RGBT) semantic segmentation is an emerging technology for identifying objects and materials in high dynamic range scenes. Thermal imaging particularly enhances feature extraction at close range for applications such as textile damage detection. In this paper, we present RGBT-Textile, a novel dataset specifically developed for close-range textile and damage segmentation. We meticulously designed the data collection protocol, software tools, and labeling process in collaboration with textile scientists. Additionally, we introduce ThermoFreq, a novel thermal frequency normalization method that reduces temperature noise effects in segmentation tasks. We evaluate our dataset alongside six existing RGBT datasets using state-of-the-art (SOTA) models. Experimental results demonstrate the superior performance of the SOTA models with ThermoFreq, highlighting its effectiveness in addressing noise challenges inherent in RGBT semantic segmentation across diverse environmental conditions. We make our dataset publicly accessible to foster further research and collaborations.
BibTeX
@inproceedings{rayhan2025advancing,
  author = {Farshid Rayhan and Jitesh Joshi and Guangyu Ren and Luis M. Hernández and Bruna Petreca and Sharon Baurley and Nadia Bianchi‐Berthouze and Youngjun Cho},
  title = {{Advancing Textile Damage Segmentation: A Novel RGBT Dataset and Thermal Frequency Normalization}},
  booktitle = {Sensors},
  year = {2025},
  doi = {10.3390/s25072306},
}

Loading related content...

Figure