Analysis and Applications of Deep Learning with Finite Samples in Full Life-Cycle Intelligence of Nuclear Power Generation

Chenwei Tang, Wenqiang Zhou, Dong Wang, Caiyang Yu, Zhenan He, Jizhe Zhou, Shudong Huang, Yi Gao, Jianming Chen, Wentao Feng, Jiancheng LvarXiv 2023 · DOI · OpenReview

TL;DR

This paper explores the application of Deep Learning (DL) with limited data (finite samples) to enhance the intelligence of Nuclear Power Generation (NPG) across its entire lifecycle. It addresses challenges like data imbalance and domain shift common in NPG, proposing and evaluating DL techniques for tasks like material metallography analysis and sensor signal diagnosis.

What Problem Does This Paper Address?

The nuclear power industry, like many industrial sectors, is increasingly looking to Artificial Intelligence (AI) and Deep Learning (DL) for intelligent manufacturing, operation, and maintenance. However, NPG environments present unique data challenges:

  • Long-tailed class distribution: Some events or conditions are very common, while others are rare.
  • Sample imbalance: Datasets often have vastly more "normal" operational data than "abnormal" or "fault" data.
  • Domain shift: Data collected under one set of conditions might not accurately represent data from another, even on the same equipment.

These issues make standard DL models, which often assume abundant and well-distributed data, perform poorly. The paper aims to bridge this gap by investigating how DL techniques designed for "finite samples" can be effectively applied to NPG's complex, data-constrained scenarios.

Methods

The paper focuses on DL techniques suited for limited data, categorizing them into:

  • Small-Sample Learning (SSL): Training models with very few labeled examples. This involves knowledge transfer (leveraging pre-trained models), data augmentation (creating more training data), and algorithmic improvements.
  • Any-Shot Learning (including Zero-Shot Learning (ZSL) and Few-Shot Learning (FSL)):
    • ZSL: Recognizing classes that were not seen during training, by learning a shared semantic space between visual and label information.
  • FSL: Learning from a tiny number of examples (e.g., <10 per class), often by leveraging knowledge from base classes.
  • Open-Set Recognition (OSR): Identifying not only known classes but also distinguishing unknown or novel classes. This is crucial for safety-critical systems where unexpected events can occur.

Two specific DL approaches are detailed:

  1. An improved Transformer-based network (EDTR) for edge detection and metallographic analysis, incorporating multi-scale feature extraction and a novel loss function to handle class imbalance.
  2. A Deep Variational Encoder-Classifier (DVEC) framework for signal diagnosis, enhanced with an open-set discriminator that uses Extreme Value Theory (EVT) and Shannon's entropy to better identify unknown fault signals.

Datasets

The paper utilizes several datasets for its case studies:

  • Zirconium Alloy Metallography: A custom dataset was created by acquiring 50 high-resolution images from a scanning electron microscope. These were pre-processed and augmented to create 4200 labeled image pairs.
  • Edge Detection Benchmarks: For training the Transformer model, standard datasets like BSDS500, NYUDv2, and Pascal-Context were used.
  • Signal Diagnosis: The Primary Circuit of the Nuclear Reactor (PCNR) dataset, consisting of vibration signals from sensors during actual reactor operation, was used. This dataset includes six categories of signals. Publicly available fault diagnosis datasets like CWRU (Bearing Dataset), MFPT (Bearing Fault Dataset), and PU (Bearing Dataset) were also used for validation.

Key Insights and Ideas

  • NPG's Data is Inherently Challenging: The paper strongly emphasizes that NPG data is not just "big data" but often "difficult data" due to long tails, imbalance, and domain shifts, making standard DL approaches insufficient.
  • Finite Sample DL is Crucial for NPG: Techniques like SSL, FSL, ZSL, and OSR are not just academic curiosities but essential tools for making DL practical and reliable in NPG.
  • Material Science Meets DL: The zirconium alloy metallography case study demonstrates how DL can automate complex visual analysis tasks that are currently manual, time-consuming, and prone to error.
  • Safety Through Open-Set Recognition: The signal diagnosis case study highlights how OSR can significantly improve safety by detecting unknown failure modes that might be missed by traditional methods, thus preventing critical incidents.
  • Hybrid Approaches are Powerful: Combining techniques like transfer learning, data augmentation, and specialized network architectures (Transformers, VAEs) is key to overcoming data limitations.

Key Results

  • Zirconium Alloy Metallography: The proposed DL approach successfully automated the recognition of zirconium alloy microstructures, enabling efficient identification of phases and grain sizes, which is critical for material quality and safety. The method effectively handled the small sample size and high annotation cost.
  • Signal Diagnosis (OSR): The DVEC framework with its open-set discriminator achieved strong performance on the PCNR dataset, outperforming baseline methods like CROSR and OpenMax in various openness scenarios. For instance, in one task, DVEC achieved an accuracy of 0.9840, demonstrating its ability to reliably distinguish known and unknown fault signals. The entropy-based discriminator showed better performance in low-openness scenarios.

Limitations and Future Work

  • Data Accessibility and Annotation: The paper acknowledges that acquiring and annotating data in NPG is challenging due to security and cost factors. This remains a significant hurdle for applying DL.
  • Generalization to Extreme Scenarios: While the OSR framework shows promise, further work is needed to ensure robust performance across all possible "openness" levels and highly complex, unseen failure modes.
  • Interpretability: As with many DL models, the interpretability of the decisions made by these complex networks in safety-critical applications is an ongoing area of research.
  • Broader Lifecycle Application: The paper presents two case studies, but the full potential of finite-sample DL across the entire NPG lifecycle (from mining to waste management) is vast and requires further exploration. Future work could focus on expanding the range of application scenarios and developing more specific methods for each stage.

Summary updated

Abstract

The advent of Industry 4.0 has precipitated the incorporation of Artificial Intelligence (AI) methods within industrial contexts, aiming to realize intelligent manufacturing, operation as well as maintenance, also known as industrial intelligence. However, intricate industrial milieus, particularly those relating to energy exploration and production, frequently encompass data characterized by long-tailed class distribution, sample imbalance, and domain shift. These attributes pose noteworthy challenges to data-centric Deep Learning (DL) techniques, crucial for the realization of industrial intelligence. The present study centers on the intricate and distinctive industrial scenarios of Nuclear Power Generation (NPG), meticulously scrutinizing the application of DL techniques under the constraints of finite data samples. Initially, the paper expounds on potential employment scenarios for AI across the full life-cycle of NPG. Subsequently, we delve into an evaluative exposition of DL's advancement, grounded in the finite sample perspective. This encompasses aspects such as small-sample learning, few-shot learning, zero-shot learning, and open-set recognition, also referring to the unique data characteristics of NPG. The paper then proceeds to present two specific case studies. The first revolves around the automatic recognition of zirconium alloy metallography, while the second pertains to open-set recognition for signal diagnosis of machinery sensors. These cases, spanning the entirety of NPG's life-cycle, are accompanied by constructive outcomes and insightful deliberations. By exploring and applying DL methodologies within the constraints of finite sample availability, this paper not only furnishes a robust technical foundation but also introduces a fresh perspective toward the secure and efficient advancement and exploitation of this advanced energy source.
BibTeX
@misc{tang2023analysis,
  author = {Chenwei Tang and Wenqiang Zhou and Dong Wang and Caiyang Yu and Zhenan He and Jizhe Zhou and Shudong Huang and Yi Gao and Jianming Chen and Wentao Feng and Jiancheng Lv},
  title = {{Analysis and Applications of Deep Learning with Finite Samples in Full Life-Cycle Intelligence of Nuclear Power Generation}},
  year = {2023},
  eprint = {2311.04247},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2311.04247},
}
Figure