Research on Abnormal State Detection of CZ Silicon Single Crystal Based on Multimodal Fusion

Lei Jiang, H. X. Wei, Ding LiuSensors 2024 · DOI

TL;DR

This paper introduces a novel multimodal fusion deep learning model to detect and classify four common abnormal states (loss, twisting, swinging, and squareness) during the Czochralski (CZ) silicon single-crystal growth process. By combining time-frequency features from sensor data with visual features from meniscus images, the model achieves a remarkable 99.2% classification accuracy, significantly improving upon single-modality approaches.

What Problem Does This Paper Address?

The production of high-quality silicon single crystals is crucial for numerous industries, and the Czochralski (CZ) method is a dominant technique. However, this process is prone to various anomalous states, such as crystal loss, twisting, swinging, and squareness. These anomalies not only degrade the quality of the final product but also reduce production efficiency and can damage expensive equipment. Traditionally, detecting these issues relies heavily on the experience of human operators, leading to subjective judgments, delayed detection, and potential inaccuracies. Existing automated methods often focus on single data sources (e.g., only visual data or only sensor data), failing to capture the full complexity of the growth process and its potential failure modes. This paper aims to bridge this gap by developing a robust, multimodal approach for comprehensive abnormal state detection and classification.

Methods

The core of this research is a multimodal fusion classification network. The approach involves several key steps:

  • One-Dimensional Signal Processing: One-dimensional sensor data (diameter, temperature, and pulling speed) are transformed into time-frequency domain images using Continuous Wavelet Transform (CWT). These images are then fed into a custom-designed Dense-ECA-SwinTransformer network. This network combines dense blocks for feature reuse, SwinTransformer modules for efficient attention across windows, and an ECA (Efficient Channel Attention) mechanism to focus on important cross-channel dependencies.
  • Video Signal Processing: Meniscus video feeds are processed to extract two types of visual information: static meniscus images and inter-frame difference images. These visual inputs are then fused at the channel level and processed by a ConvNeXt network for feature extraction. ConvNeXt is a modern convolutional neural network architecture known for its strong performance.
  • Multimodal Fusion: The extracted features from the one-dimensional signals (via Dense-ECA-SwinTransformer) and the visual signals (via ConvNeXt) are concatenated. This fused feature representation is then passed through an ECA attention mechanism to further refine channel-wise importance before being fed into fully connected layers for final multi-class classification using a Softmax layer.

Datasets

The study utilizes a dataset collected from a real-world 12-inch CZ silicon single-crystal furnace. The data includes:

  • One-dimensional sensor data: Diameter, temperature, and pulling speed.
  • Visual data: Meniscus images and inter-frame difference images captured by a camera.

The paper does not explicitly state the creation of a new benchmark dataset but rather uses data collected from an operational industrial furnace. The dataset comprises thousands of groups of normal and abnormal growth images, each associated with corresponding one-dimensional sensor data.

Key Insights and Ideas

  • Multimodal Fusion is Key: The most significant insight is that combining diverse data sources (sensor readings and visual cues) is far more effective than relying on any single modality for detecting complex anomalies in the CZ growth process.
  • Time-Frequency Representation for Sensor Data: Transforming 1D sensor signals into time-frequency images using CWT effectively captures dynamic changes and patterns that might be missed in the raw time-domain signals.
  • Synergy of Vision Transformers and CNNs: The paper demonstrates the power of combining SwinTransformers (for global context and efficient attention) with ConvNeXt (for local feature extraction) within a multimodal framework.
  • Attention Mechanisms Enhance Robustness: The integration of ECA attention at various stages helps the model focus on the most discriminative features, leading to improved accuracy and robustness against noise or subtle variations.
  • Beyond Loss Detection: While many studies focus solely on "loss" detection, this work addresses a broader spectrum of critical anomalies (twisting, swinging, squareness), offering a more comprehensive solution.

Key Results

The proposed multimodal fusion model achieved an impressive overall classification accuracy of 99.2% for detecting and classifying abnormal growth states.

  • The Dense-ECA-SwinTransformer for one-dimensional signals achieved an accuracy of 96.8% after fusion, a significant improvement over individual signals.
  • The fusion of meniscus images and inter-frame difference images yielded an accuracy of 94.2%, outperforming individual image types.
  • When comparing the final multimodal fusion model against existing methods like MMFN, the proposed approach showed superior performance across precision, recall, F1-score, and accuracy, achieving 0.996, 0.989, 0.993, and 0.993 respectively, compared to MMFN's 0.993, 0.978, 0.985, and 0.986.

Limitations and Future Work

  • Data Availability: While the paper states that data is available upon request, it's not publicly shared, which can limit independent verification and further research.
  • Real-time vs. Offline: The current study focuses on offline detection. Adapting this model for real-time monitoring and control would be a significant next step.
  • Generalizability: The model was trained and tested on data from a specific furnace. Further validation on data from different equipment or under varying operational conditions would be beneficial.
  • Interpretability: While attention mechanisms offer some insight, deeper interpretability of why certain anomalies are detected could aid operators in understanding and intervening more effectively.

The authors suggest that their multimodal fusion approach provides a robust and accurate solution for abnormal state detection, which can help operators make better judgments, formulate personalized treatment plans, and ultimately improve production efficiency and resource utilization.

Summary updated

Abstract

The Czochralski method is the primary technique for single-crystal silicon production. However, anomalous states such as crystal loss, twisting, swinging, and squareness frequently occur during crystal growth, adversely affecting product quality and production efficiency. To address this challenge, we propose an enhanced multimodal fusion classification model for detecting and categorizing these four anomalous states. Our model initially transforms one-dimensional signals (diameter, temperature, and pulling speed) into time-frequency domain images via continuous wavelet transform. These images are then processed using a Dense-ECA-SwinTransformer network for feature extraction. Concurrently, meniscus images and inter-frame difference images are obtained from the growth system's meniscus video feed. These visual inputs are fused at the channel level and subsequently processed through a ConvNeXt network for feature extraction. Finally, the time-frequency domain features are combined with the meniscus image features and fed into fully connected layers for multi-class classification. The experimental results show that the method can effectively detect various abnormal states, help the staff to make a more accurate judgment, and formulate a personalized treatment plan for the abnormal state, which can improve the production efficiency, save production resources, and protect the extraction equipment.
BibTeX
@inproceedings{jiang2024research,
  author = {Lei Jiang and H. X. Wei and Ding Liu},
  title = {{Research on Abnormal State Detection of CZ Silicon Single Crystal Based on Multimodal Fusion}},
  booktitle = {Sensors},
  year = {2024},
  doi = {10.3390/s24216819},
}
Figure