Automatic Crane-Related Workflow Control for Nuclear Plant Outages through Computer Vision and Simulation

Cheng Zhang, Pingbo Tang, Alper Yılmaz, Nancy J. Cooke, Verica Buchanan, Alan Chasey, Ronald L. Boring, Shawn W St Germain, Timothy Vaughn, Samuel JonesProceedings of the ... ISARC 2016 · DOI

Abstract

Nuclear power plant (NPP) outages involve a large number of maintenance activities with a tight schedule and zero-tolerance for accidents. Outage projects thus need real-time control to ensure safety and productivity. During outages, crane lifting is critical for outage control and risk management. An effective outage control method should monitor detailed interactions between human and workspaces, and streamline the workflows of cranes to control both productivity and risks. Unfortunately, current approaches of outage control rely heavily on tedious and error-prone manual inspection that can hardly achieve detailed spatiotemporal monitoring.

Introduction

Nuclear Power Plant (NPP) outages are high-stakes maintenance windows where reactors are shut down for refueling and repairs. These outages are incredibly complex, involving thousands of activities and personnel packed into a tight 20-to-30-day window. Because every idle day can cost an energy company $1–2 million, there is zero tolerance for delays or accidents.

Crane operations represent the "critical path" of these outages, often involving "very heavy lifts" exceeding 30 tons. Currently, monitoring these workflows relies on manual inspection, which is tedious, error-prone, and slow. A significant gap exists in systematically understanding how small human errors or minor task delays propagate through the schedule to cause major project overruns or safety hazards. This paper proposes an automated framework that combines real-time computer vision (2D/3D human tracking) with Monte Carlo simulations to monitor crane-related workflows and identify potential "domino effects."

Methodology

The proposed framework consists of two primary components: a task-duration anomaly propagation model and a computer-vision (CV) based monitoring system.

Task-Duration Anomaly Propagation Model

The mathematical model categorizes task relationships into three fundamental types:

  1. Chain: Linear sequences where one task must finish before the next begins.
  2. Co-prerequisite: Multiple tasks that must all be completed before a subsequent task can start.
  3. Resource Sharing: Tasks that compete for the same resource, such as a single crane. This is the most volatile relationship, as "first-come, first-served" logic introduces significant uncertainty.
Workflow model for Sites A and B illustrating task dependencies and shared crane resources.

Computer-Vision Based Monitoring

The system utilizes GigE cameras for wide-area monitoring and Microsoft Kinect V2 sensors for precise 3D human tracking. The CV system automatically counts personnel in specific zones (e.g., waiting lines or working areas) and identifies task start and end times by observing human movement trajectories. This allows the system to infer the experience level of a team or identify bottlenecks before they impact critical maintenance work.

Simulation Integration

Observed "as-is" data from the CV system is fed into a Monte Carlo simulation. This allows for the real-time prediction of how detected anomalies will impact the overall project duration, comparing current progress against the original "as-planned" schedule.

Experimental Validation

To validate the framework, the authors designed a custom indoor laboratory experiment. The setup included a physical site (Site B) where participants performed valve maintenance tasks, and virtual sites (Sites A & C) to test delay propagation. The study utilized raw 2D video and 3D point cloud data to track participant movements and validate the accuracy of the automated task detection.

Key Results and Insights

The research highlights the critical role of the "human factor," noting that human behavior is responsible for over 80% of crane-related accidents. Monitoring human activity in the workspace is therefore as vital as monitoring the crane itself.

Simulation Findings

In a theoretical 50-minute workflow, the introduction of resource sharing and a 10% rework risk increased the average duration to 56 minutes. Notably, the results exhibited a "bimodal" distribution, indicating high unpredictability in project outcomes when resources are shared and communication is imperfect.

Simulation results showing a bimodal distribution of workflow durations due to resource sharing and rework risks.

Experimental Impact

During lab experiments, a single communication failure—where a participant representing a crane operator finished a task but failed to inform the next team—delayed the overall workflow by 30%. This single error influenced the start times of 60% of all subsequent tasks, demonstrating the massive "domino effect" inherent in shared resource workflows.

Limitations and Future Work

While promising, the current system has limitations. The preliminary experiment tracked only one person at a time, whereas real NPP outages involve thousands of workers. Future work must address multi-human tracking in complex, "construction-like" environments with significant occlusions and metallic interference. Additionally, while the system detects delays, future iterations aim to automatically diagnose root causes, such as distinguishing between training gaps and equipment failure.

Summary updated

Abstract

Nuclear power plant (NPP) outages involve a large number of maintenance activities with a tight schedule and zero-tolerance for accidents. Outage projects thus need real-time control to ensure safety and productivity. During outages, crane lifting is critical for outage control and risk management. An effective outage control method should monitor detailed interactions between human and workspaces, and streamline the workflows of cranes to control both productivity and risks. Unfortunately, current approaches of outage control rely heavily on tedious and error-prone manual inspection that can hardly achieve detailed spatiotemporal monitoring.
BibTeX
@inproceedings{zhang2016automatic,
  author = {Cheng Zhang and Pingbo Tang and Alper Yılmaz and Nancy J. Cooke and Verica Buchanan and Alan Chasey and Ronald L. Boring and Shawn W St Germain and Timothy Vaughn and Samuel Jones},
  title = {{Automatic Crane-Related Workflow Control for Nuclear Plant Outages through Computer Vision and Simulation}},
  booktitle = {Proceedings of the ... ISARC},
  year = {2016},
  doi = {10.22260/isarc2016/0107},
}
Figure