Abstract
Wave energy remains a largely untapped renewable resource due to the high Levelized Cost of Energy (LCOE) and the mechanical complexity of capturing energy from chaotic ocean environments. This paper presents a novel Multi-Agent Reinforcement Learning (MARL) controller designed for the CETO 6 wave energy converter. By employing Proximal Policy Optimization (PPO) and recurrent neural networks, the proposed system achieves significant improvements in energy capture efficiency—up to 19.9%—while drastically reducing mechanical stress. These results demonstrate the potential of AI-driven control systems to enhance the commercial viability of offshore renewable energy.
Introduction
The transition to a sustainable energy future requires diversifying renewable sources beyond solar and wind. Wave energy offers a dense and predictable power source, yet it remains a difficult engineering challenge. The primary hurdle is the Levelized Cost of Energy (LCOE); to be competitive, Wave Energy Converters (WECs) must maximize energy extraction from variable ocean swells while surviving extreme conditions that can damage sensitive machinery.
Modern WECs, such as the three-legged CETO 6 platform shown above, are structurally complex. They utilize multiple "legs" or tethers connected to independent generators (Power Take-Off units) that interact in highly non-linear ways. Traditional control strategies, often based on rigid "Spring Damper" models, struggle to account for these complex interactions, leading to suboptimal energy capture and increased mechanical wear.
The Complexity of Wave Energy Control
Traditional physics-based controllers are often too inflexible to handle the stochastic nature of ocean waves. In a multi-generator system like the CETO 6, the motion of the buoyant actuator is influenced by the combined reactive forces of all three legs. A key issue is "yaw"—the twisting motion of the buoy. Excessive yaw not only reduces the efficiency of energy capture but also causes significant mechanical stress on the tethers and foundations, leading to high maintenance costs and shorter operational lifespans.
The research team from HPE and Carnegie Clean Energy sought an AI-driven approach capable of "learning" the optimal control policy through interaction with the environment. The goal was to maximize power output while simultaneously minimizing the rotational motions that lead to mechanical fatigue.
Multi-Agent Reinforcement Learning Approach
The researchers implemented a Multi-Agent Reinforcement Learning (MARL) framework utilizing the Proximal Policy Optimization (PPO) algorithm. This architecture treats each of the three generators as an independent agent (or groups them into two agents to exploit symmetry), each controlling the reactive force of a specific leg.
Technical Innovations
Several key technical components were integrated to ensure the controller's effectiveness:
- Recurrent Neural Networks (LSTM): Because ocean waves are inherently time-series data, the agents utilize Long Short-Term Memory (LSTM) layers. This allows the system to "remember" previous states and anticipate future wave motions using a 10-second look-ahead provided by wave sensors.
- Reward Shaping and Team Coefficients: To balance individual generator performance with the overall health of the system, a "team coefficient" was introduced. This parameter allows agents to cooperate or compete for power capture depending on the incoming wave direction, ensuring the system acts as a cohesive unit.
- Safety and Stability: To prevent the policy from reaching unsafe mechanical limits, "action clipping" was implemented. Furthermore, the researchers found that a smaller-than-standard clipping parameter in the PPO algorithm was essential to maintain training stability in the sensitive marine environment.
Experimental Methodology
The study utilized a high-fidelity simulator of the CETO 6 WEC platform. To ensure the results were representative of real-world conditions, the wave environment was modeled using statistical distributions (height, frequency, and spectrum) collected from three distinct locations: Albany (Western Australia), Armintza (Spain), and Wave Hub (UK).
The standard Jonswap Spectrum was used to generate heterogeneous wave episodes. The evaluation set consisted of 1,000 episodes per principal wave period, ranging from 6 to 16 seconds, covering a wide variety of sea states.
Performance and Efficiency Gains
The MARL controller demonstrated a clear advantage over the baseline Spring Damper controller across all tested scenarios.
Energy Capture
The system achieved an average 19.9% gain in energy capture for frontal waves. Even in more complex "angled wave" configurations (30°), the controller maintained a 13.9% gain. Interestingly, the RL agent often took a "long-term view," occasionally compromising immediate power gains to better position the buoy for larger energy capture later in the wave cycle—a behavior not seen in traditional controllers.
Stress Reduction
One of the most significant findings was the impact on mechanical health. The RL controller reduced yaw movement to between 1.2% and 6.3% of the baseline levels. Initially, the researchers feared that penalizing yaw would hurt power production; however, they discovered that stabilizing the buoy's rotation actually allowed the agent to enter a more stable regime, facilitating higher energy capture.
Survivability
In simulated "disaster" scenarios with 7-meter high waves, the RL controller successfully mitigated yaw build-up that caused the baseline controller to fail. It maintained positive power production while keeping the system within safe operational limits.
Conclusion and Future Outlook
The application of MARL to wave energy conversion represents a significant step toward making offshore renewables cost-competitive. The PPO-based controller proved to be both highly efficient and mechanically protective, with a computational footprint small enough to run on embedded edge devices at sea.
Future research will focus on scaling these results to "WEC farms." Deploying multiple units in close proximity introduces "cross-talk"—hydrodynamic interference between units—that will require even more sophisticated multi-agent coordination. Additionally, extending the wave prediction horizon beyond 10 seconds could unlock further efficiency gains, bringing the rhythm of the ocean closer to a reliable, global power source.
Summary updated