Rubber Material Property Prediction Using Electron Microscope Images of Internal Structures Taken under Multiple Conditions

Ren Togo, Naoki Saito, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama

TL;DR

This paper introduces a novel method for predicting rubber material properties using electron microscope images. It addresses the challenge that different imaging conditions are optimal for different materials by intelligently selecting reliable predictions from various conditions and integrating them using Dempster-Shafer evidence theory, achieving under 10% MAPE.

What Problem Does This Paper Address?

Predicting the properties of materials like rubber is crucial for developing high-performance applications. Traditionally, this involves time-consuming physical measurements or relying on mix proportions. Electron microscopy offers a way to visualize internal structures, which are linked to material properties. However, a significant challenge arises because the effectiveness of electron microscope images for prediction varies depending on the imaging conditions (e.g., magnification, mode) and the specific rubber material. Previous methods often assumed consistent imaging conditions, limiting their applicability. This paper tackles the problem of how to leverage electron microscope images taken under multiple, potentially varying conditions to achieve accurate property prediction, even when the optimal conditions differ per material.

Methods

The proposed method is a three-step process:

  1. Property Prediction for Each Imaging Condition: For each distinct imaging condition (defined by magnification and mode), Support Vector Regression (SVR) models are trained. These models use a combination of visual features extracted from electron microscope images (co-occurrence matrix, Gabor wavelet, adaptive local binary patterns) and the material's mix proportion information. This step generates multiple prediction results, one for each imaging condition.
  2. Selection of Reliable Prediction Results: Not all predictions from Step 1 are equally trustworthy. This step uses prediction intervals to identify reliable results. The core idea is that a smaller prediction interval indicates higher confidence and lower variance in the prediction. By analyzing the prediction errors and their empirical distribution, the method calculates prediction intervals for each prediction. Predictions with smaller intervals are deemed more reliable.
  3. Integration of Reliable Results: The selected reliable predictions from Step 2 are then integrated using Dempster-Shafer (DS) evidence theory. This theory is well-suited for combining evidence from multiple, potentially conflicting, sources. The method first determines Basic Probability Assignment (BPA) functions for each reliable prediction and then fuses them using DS evidence theory to arrive at a final, robust prediction. This approach allows for integration even if the underlying imaging conditions were different for each source of evidence.

Datasets

The study utilized a dataset comprising electron microscope images and mix proportions from 75 rubber material samples. These samples were imaged under eight different conditions (combinations of magnification and imaging mode). The paper notes that the number of samples and images varied across these conditions. The dataset was split into training (74 samples) and testing (1 sample) sets for leave-one-out cross-validation.

Key Insights and Ideas

  • Condition-Specific Optimality: The paper highlights that the "best" imaging conditions for predicting rubber properties are not universal; they vary from one material to another. This is a critical observation that drives the need for a multi-condition approach.
  • Prediction Intervals as Reliability Indicators: Using prediction intervals to filter reliable predictions is a clever way to leverage the inherent uncertainty in regression models. A smaller interval directly translates to higher confidence in the prediction.
  • DS Evidence Theory for Heterogeneous Data: The application of Dempster-Shafer theory is particularly powerful here. It allows for the meaningful integration of predictions derived from different imaging conditions, which would be problematic for simpler averaging methods. This makes the method robust to variations in data acquisition.
  • Handcrafted Features vs. Deep Learning: The authors opt for handcrafted visual features (co-occurrence matrix, Gabor, LBP) combined with mix proportions, rather than deep learning features. They justify this by the limited number of electron microscope images available and their unique characteristics, which might not be well-captured by pre-trained deep networks.

Key Results

The proposed method achieved a Mean Absolute Percent Error (MAPE) under 10% on the rubber material property prediction task. This performance significantly outperformed several comparative methods, including a Convolutional Neural Network (CNN)-based approach (CM9). The paper demonstrates that:

  • Using electron microscope images from multiple conditions (compared to CM1) improves accuracy.
  • Integrating reliable results using DS evidence theory (compared to simple averaging or weighted averaging in CM4, CM5) is effective.
  • The selection of reliable results based on prediction intervals (compared to CM6, CM7, CM8) further enhances accuracy.

Limitations and Future Work

The authors acknowledge several limitations:

  • Data Restriction: The current method has only been validated on rubber materials. Its effectiveness on other material types needs to be confirmed.
  • Sample Size: The dataset of 75 samples, while sufficient for demonstrating the method's potential, is considered small for real-world applications.

Future work suggested includes:

  • Extending the method to other materials.
  • Improving the accuracy of property estimation by enhancing image feature extraction techniques.
  • Further investigation into the versatility of the method, especially its ability to handle situations where image capturing conditions are not fully known.

Summary updated

Abstract

A method for prediction of properties of rubber materials utilizing electron microscope images of internal structures taken under multiple conditions is presented in this paper. Electron microscope images of rubber materials are taken under several conditions, and effective conditions for the prediction of properties are different for each rubber material. Novel approaches for the selection and integration of reliable prediction results are used in the proposed method. The proposed method enables selection of reliable results based on prediction intervals that can be derived by the predictors that are each constructed from electron microscope images taken under each condition. By monitoring the relationship between prediction results and prediction intervals derived from the corresponding predictors, it can be determined whether the target prediction results are reliable. Furthermore, the proposed method integrates the selected reliable results based on Dempster–Shafer (DS) evidence theory, and this integration result is regarded as a final prediction result. The DS evidence theory enables integration of multiple prediction results, even if the results are obtained from different imaging conditions. This means that integration can even be realized if electron microscope images of each material are taken under different conditions and even if these conditions are different for target materials. This nonconventional approach is suitable for our application, i.e., property prediction. Experiments on rubber material data showed that the evaluation index mean absolute percent error (MAPE) was under 10% by the proposed method. The performance of the proposed method outperformed conventional comparative property estimation methods. Consequently, the proposed method can realize accurate prediction of the properties with consideration of the characteristic of electron microscope images described above.
BibTeX
@inproceedings{togo2021rubber,
  author = {Ren Togo and Naoki Saito and Keisuke Maeda and Takahiro Ogawa and Miki Haseyama},
  title = {{Rubber Material Property Prediction Using Electron Microscope Images of Internal Structures Taken under Multiple Conditions}},
  booktitle = {Sensors},
  year = {2021},
  url = {https://openreview.net/forum?id=8oWLgQ9hFm3},
}
Figure