Fusing Expert Knowledge with Monitoring Data for Condition Assessment of Railway Welds

Cyprien Hoelzl, Giacomo Arcieri, Lucian Ancu, Stanislaw Banaszak, Aurelia Kollros, Vasilis K. Dertimanis, Eleni N. Chatzi

TL;DR

This paper tackles the challenge of accurately assessing the condition of railway welds, which is crucial for safety and efficiency. By fusing data from on-board sensors (Axle Box Accelerations - ABAs) with expert knowledge, the authors developed a more robust method for identifying defective welds, outperforming traditional data-driven approaches.

What Problem Does This Paper Address?

Railway infrastructure, particularly welds, requires constant monitoring to ensure safety and prevent costly failures. While automated monitoring systems using sensors like Axle Box Accelerations (ABAs) can collect vast amounts of data, interpreting this data for weld condition assessment is challenging. ABA data can be noisy, affected by complex vehicle-track interactions, and environmental factors. Existing methods often struggle with these uncertainties, leading to potential misclassifications. This paper addresses the need for a more reliable and accurate method to assess weld condition by integrating human expertise with sensor data.

Methods

The authors propose a multi-stage framework:

  1. Feature Extraction: They extract a range of features from ABA time series data, including statistical indicators (mean, max, min, standard deviation), time-frequency analysis (STFT, DWT), and geometric features (longitudinal level).
  2. Unsupervised Outlier Identification: Extreme Value Analysis (EVA) is used to identify potential "outlier" welds based on these ABA features. This step aims to flag welds that deviate significantly from normal behavior.
  3. Expert Evaluation and Labeling: The identified outlier welds, along with their corresponding rail-head images, are presented to human experts (from Swiss Federal Railways - SBB). These experts provide "ground truth" labels (defective/healthy) for the welds.
  4. Supervised Classification: The expert-labeled data is then used to train and evaluate three supervised machine learning models:
  5. Binary Choice (BC) models: A baseline approach using individual features.
  6. Random Forest (RF): An ensemble method known for its robustness.
  7. Bayesian Logistic Regression (BLR): A probabilistic model that also provides an estimate of prediction uncertainty.

The novelty lies in the fusion of ABA-derived indicators with expert feedback to refine the classification process, particularly by using expert labels to improve the models' accuracy and reliability.

Datasets

The study utilized data collected from the Swiss Federal Railways (SBB) diagnostic vehicle (gDFZ). This included:

  • ABA data: Collected from vertical and lateral accelerometers on axles 1 and 4 of the vehicle.
  • Rail-head images: Captured by the VCUBE system on the diagnostic vehicle.
  • ZMON database: The SBB's condition monitoring database, containing historical data on weld inspections and defects.

A dataset of approximately 25,000 unique welds was initially compiled. After the EVA and expert evaluation process, a labeled dataset was generated for training the classification models.

Key Insights and Ideas

  • Expert knowledge is crucial: Sensor data alone, even with advanced analysis, struggles with the inherent uncertainties in railway monitoring. Expert feedback acts as a vital "ground truth" to refine these models.
  • Early detection is possible: ABA indicators can show significant growth before a defect is officially recognized by traditional inspection methods, highlighting the potential for proactive maintenance.
  • Probabilistic models offer confidence: The BLR model's ability to provide a probability of prediction is invaluable for understanding the confidence level in a given classification, allowing for more informed decision-making.
  • Not all welds are equal: The study emphasizes that a binary "healthy/defective" classification might be too simplistic. The progression of damage and severity are important factors that ABA data can potentially capture over time.

Key Results

  • The Random Forest (RF) and Bayesian Logistic Regression (BLR) models significantly outperformed the simpler Binary Choice (BC) models.
  • The best performing RF model achieved an F1-score of around 48%, while the BLR model achieved around 43%.
  • Including the vehicle speed as an input feature further improved the performance of both RF and BLR models.
  • The BLR model provided a valuable indication of predictive uncertainty, which could be further leveraged in future work.
  • The study demonstrated that ABA indicators could track the evolution of weld condition over time, providing a more continuous assessment than discrete labels.

Limitations and Future Work

  • Label quality and uncertainty: The authors acknowledge that expert labels can be subjective and that the ZMON database might not always be perfectly up-to-date or accurate in its positional data. This uncertainty in the ground truth impacts classification performance.
  • Binary classification: The current models primarily focus on a binary "healthy/defective" classification. The authors suggest that a more nuanced approach considering damage severity and progression would be beneficial.
  • Expert time constraints: The expert evaluation process is time-consuming, limiting the number of samples that can be assessed.
  • Future work: The authors propose extending the framework with unsupervised approaches like Mahalanobis distance for label assignment beyond one-class classification, further incorporating expert uncertainty into probabilistic models, and developing sequential decision-making frameworks for optimal maintenance planning. They also suggest exploring the use of high-frequency sensors to better capture the impulse-type vibration responses characteristic of weld faults.

Summary updated

Abstract

Monitoring information can facilitate the condition assessment of railway infrastructure, via delivery of data that is informative on condition. A primary instance of such data is found in Axle Box Accelerations (ABAs), which track the dynamic vehicle/track interaction. Such sensors have been installed on specialized monitoring trains, as well as on in-service On-Board Monitoring (OBM) vehicles across Europe, enabling a continuous assessment of railway track condition. However, ABA measurements come with uncertainties that stem from noise corrupt data and the non-linear rail–wheel contact dynamics, as well as variations in environmental and operational conditions. These uncertainties pose a challenge for the condition assessment of rail welds through existing assessment tools. In this work, we use expert feedback as a complementary information source, which allows the narrowing down of these uncertainties, and, ultimately, refines assessment. Over the past year, with the support of the Swiss Federal Railways (SBB), we have assembled a database of expert evaluations on the condition of rail weld samples that have been diagnosed as critical via ABA monitoring. In this work, we fuse features derived from the ABA data with expert feedback, in order to refine defection of faulty (defect) welds. Three models are employed to this end; Binary Classification and Random Forest (RF) models, as well as a Bayesian Logistic Regression (BLR) scheme. The RF and BLR models proved superior to the Binary Classification model, while the BLR model further delivered a probability of prediction, quantifying the confidence we might attribute to the assigned labels. We explain that the classification task necessarily suffers high uncertainty, which is a result of faulty ground truth labels, and explain the value of continuously tracking the weld condition.
BibTeX
@inproceedings{hoelzl2023fusing,
  author = {Cyprien Hoelzl and Giacomo Arcieri and Lucian Ancu and Stanislaw Banaszak and Aurelia Kollros and Vasilis K. Dertimanis and Eleni N. Chatzi},
  title = {{Fusing Expert Knowledge with Monitoring Data for Condition Assessment of Railway Welds}},
  booktitle = {Sensors},
  year = {2023},
  url = {https://openreview.net/forum?id=F2W15nfKh2},
}

Loading related content...

Figure