Abstract
This report summarizes the findings of DOE NEUP Project 15-8121, which investigated the use of real-time imagery-based object tracking and human behavior modeling to improve the efficiency of nuclear power plant (NPP) outages. Typical NPP outages involve thousands of maintenance activities and a large workforce operating under tight schedules with zero-tolerance for accidents. Delays in these environments are extremely costly, averaging approximately $1.5 million in losses per day. This research established methods for real-time object tracking, human factor modeling, and spatiotemporal analysis to detect productivity anomalies and improve coordination between workers and control centers. By integrating computer vision with agent-based simulations, the project demonstrated that automated imagery-driven methods can proactively enhance workflow efficiency and safety in complex maintenance environments.
The Challenge of NPP Outages
Nuclear power plant outages are high-stakes logistical marathons occurring every 18 to 24 months. These periods are dedicated to refueling and critical repairs, involving a massive temporary workforce and thousands of simultaneous activities. Because these environments operate on razor-thin schedules, any delay is catastrophic for both the utility budget and the stability of the power grid.
Currently, outage management relies heavily on manual inspections and verbal reports. These traditional methods are inherently slow and prone to "human-related issues," such as miscommunication and a lack of real-time situational awareness. There is a critical need for systems that can monitor human behavior in real-time, predict how minor delays might cascade through a complex schedule, and provide proactive alerts to managers before small issues become major setbacks.
Methodology: The Virtual-Physical Loop
The researchers addressed these challenges through an integrated framework that connects digital modeling with real-world physical behaviors. This "Virtual-Physical" loop allows for continuous monitoring and proactive adjustment of maintenance workflows.
As illustrated in the framework above, the system consists of three primary components:
- Computer Vision (CV): To track workers in crowded indoor spaces without privacy-invasive sensors, the team developed a deep-learning approach using single fixed cameras. They utilized a 2D human pose predictor to identify joints and introduced "Anthropometric Planes"—virtual parallel planes based on average human height—to project 2D video coordinates onto a 3D layout map, solving the common "loss of depth" problem in single-camera setups.
- Human Factors (HF) Analysis: The team analyzed a decade of Licensee Event Reports (LERs) from the Nuclear Regulatory Commission. They discovered that nearly 44% of incidents were rooted in team cognition and communication failures rather than technical glitches, highlighting the need for better behavioral monitoring.
- Agent-Based Simulation: Using the NetLogo platform, the researchers built a "digital twin" of the maintenance workflow. This allowed them to test "what-if" scenarios, such as the impact of automated communication systems or specific protocol changes on the overall schedule.
Key Insights and Results
The study revealed that the human supervisor is often a communication bottleneck. Manual reporting chains (worker to supervisor to coordinator to control center) are often too slow for the 10-minute accuracy required in modern outages.
Efficiency Gains
In laboratory settings, replacing a human-mediated reporting system with an automated "blackboard" system—where task status updates automatically trigger notifications for the next team—reduced average workflow duration from 79.97 minutes to 68.39 minutes, a 14.5% improvement.
The "Early-Call" Strategy
One of the most effective strategies identified was the "early-call" protocol: requiring workers to notify supervisors 15–20 minutes before completing a task. Simulation results showed this could reduce overall schedule delays by approximately 3.1% to 3.2%. While seemingly small, this represents tens of thousands of dollars saved daily in a million-dollar-per-day operation.
Tracking Performance
The multi-worker tracking algorithm achieved an average precision of 70%. However, recall was lower (38%) due to the extreme difficulty of tracking individuals in highly occluded, "packed" indoor environments typical of NPPs. The analysis also identified Mode 5 (Cold Shutdown) as the most error-prone phase for human factors in nuclear plants.
Conclusion and Future Work
This research demonstrates the significant potential of integrating computer vision and simulation for NPP outage control. While technical hurdles remain—specifically regarding "ID switching" when workers cross paths and reflections from stainless steel surfaces—the framework provides a clear path toward reducing waste and ensuring safety.
Future work will focus on increasing the robustness of tracking algorithms in compact indoor workspaces and integrating more complex human behavior models, such as "forgetting curves," to predict when fatigue or distraction might lead to critical safety omissions.
Summary updated