Visual Understanding of Multiple Attributes Learning Model of X-Ray Scattering Images

Wei (Celia) Xu, Xinyi Huang, Suphanut Jamonnak, Ye Zhao, Boyu Wang, Minh Hoai, Kevin G. Yager, Wei Xu 0020arXiv 2019 · arXiv

Abstract

Modern X-ray scattering experiments generate massive datasets that require automated analysis via deep learning. However, these models often function as "black boxes," making it difficult for scientists to trust or refine them. This paper presents a specialized visual analytics system that projects high-dimensional model data into intuitive "attribute-flower" visualizations. By comparing ground-truth labels, internal feature representations, and final predictions, the system enables experts to identify model successes, uncover hidden physical patterns, and correct labeling errors.

Introduction

X-ray scattering is a cornerstone of materials science, providing critical insights into the nano-level structures of diverse materials, from advanced polymers to biological proteins. With the advent of high-speed detectors, experimental facilities can now generate up to a million images daily. This deluge of data has created a significant bottleneck, necessitating the use of deep learning models for automated attribute identification. While models like ResNet-50 can achieve high accuracy in identifying features such as "rings," "halos," or "diffuse scattering," they often lack interpretability. For scientists, understanding the "why" behind a classification is as important as the classification itself, particularly when multiple overlapping structural features are present. This research addresses the need for domain-specific tools that allow experts to debug and trust multi-label scientific image classification models.

System Design and Methodology

The developed visual analytics system bridges the gap between raw experimental data and complex model logic. The core methodology involves the comparative analysis of three distinct high-dimensional vector spaces:

  1. Actual Labeling Space (ACT): This space is constructed based on the ground-truth labels provided by human experts.
  2. Feature Space (FEA): This represents the high-dimensional "hidden" features extracted by the final layer of the ResNet-50 model before classification.
  3. Prediction Space (PRD): This space is defined by the final output probabilities generated by the model for each attribute.

To facilitate human exploration, these high-dimensional spaces are projected into 2D maps using t-distributed Stochastic Neighbor Embedding (t-SNE). This allows scientists to observe how images cluster differently across the three stages of the pipeline.

The visual analytics interface showing the Actual (ACT), Feature (FEA), and Prediction (PRD) spaces alongside image-level details.

The Attribute-Flower Visualization

A key innovation of the system is the "Attribute-Flower" glyph. In this visualization, each image is represented as a flower where each of the 17 "petals" corresponds to a specific structural attribute or instrumental artifact (e.g., "Circular Beamstop," "High Background").

The visual encoding of the petals provides immediate feedback on model performance:

  • Color and Border: These indicate whether a prediction is a True Positive (TP), False Positive (FP), True Negative (TN), or False Negative (FN).
  • Central Dot: Represents the image ID and overall error state.

This granular representation allows scientists to quickly identify which specific attributes are causing confusion for the model in complex, multi-attribute scenarios.

Detailed breakdown of the Attribute-Flower glyph and its application in identifying model performance and data inconsistencies.

Key Insights and Case Studies

The system's utility was demonstrated through several case studies that highlighted the model's behavior beyond simple accuracy metrics.

The Model as a "Super-Scanner"

One of the most significant findings was the model's ability to act as a "super-scanner." In several instances, the model identified "weak ring" attributes that human experts had initially overlooked. By using the visualization tool to zoom into these specific cases, scientists confirmed that the model was indeed correct, demonstrating a higher sensitivity to faint signals than the original human labeling process.

Label Debugging and Data Cleaning

The comparison between the ACT and PRD spaces revealed numerous instances where the model appeared to fail, but the "error" was actually due to incorrect ground-truth labels. This capability transforms the visualization tool into a powerful data-cleaning utility, allowing experts to refine their training sets and improve future model performance.

Attribute Co-existence and Interference

The tool also sheds light on how different attributes interact. For example, the presence of a "Circular Beamstop" or "High Background" noise can significantly impact the model's confidence in detecting other physical structures. Understanding these interferences is crucial for developing more robust models that can handle the "noisy" reality of experimental data.

Results and Conclusion

Quantitatively, the underlying ResNet-50 model achieved a mean average precision (mAP) of approximately 77% across 17 attributes. Qualitatively, the visual analytics system proved effective in helping scientists identify outliers, differentiate between model and labeling errors, and understand the impact of instrumental artifacts.

While the current system focuses on end-to-end behavior, future work aims to "open the hood" further. The research team plans to visualize individual neurons and intermediate layers to understand how specific network components contribute to final predictions. This granular insight could eventually allow scientists to tune model logic to better align with the fundamental physical laws governing X-ray scattering.

Summary updated

BibTeX
@misc{xu2019visual,
  author = {Wei (Celia) Xu and Xinyi Huang and Suphanut Jamonnak and Ye Zhao and Boyu Wang and Minh Hoai and Kevin G. Yager and Wei Xu},
  title = {{Visual Understanding of Multiple Attributes Learning Model of X-Ray Scattering Images}},
  year = {2019},
  eprint = {1910.04357},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1910.04357},
}
Figure