Abstract
Ocean state estimation, specifically the prediction of wave height and period, is fundamental for marine engineering and autonomous navigation. Traditional sensor-based methods are costly and prone to failure in harsh environments, while existing computer vision approaches often fail to capture temporal dynamics. This paper introduces iOceanSee, a novel scheme utilizing a 3D Mobile Convolutional Neural Network (3D MCNN) to estimate wave parameters from simple RGB video. By employing 3D depthwise separable convolutions, iOceanSee achieves high accuracy with significantly reduced computational overhead, making it suitable for real-time deployment on edge devices like the NVIDIA Jetson TX2.
Introduction
Accurate estimation of ocean wave height and period is critical for the safety of autonomous ships, the design of offshore energy systems, and coastal management. Historically, this data has been gathered using expensive physical sensors like buoys, X-band radar, or satellite remote sensing. However, these methods face significant challenges: physical sensors are easily damaged by saltwater and storms, while radar systems require frequent, expensive calibration.
Recent advances in computer vision have attempted to solve this using 2D images, but these models often struggle because they ignore the temporal "motion" information inherent in waves. While standard 3D Convolutional Neural Networks (CNNs) can capture these spatiotemporal features, they are typically too computationally intensive for the remote, low-power environments found at sea. iOceanSee bridges this gap by providing a lightweight, high-accuracy detection scheme capable of running on mobile terminals without a stable network connection.
Methodology: The 3D Mobile CNN
The core of the iOceanSee system is the 3D Mobile Convolutional Neural Network (3D MCNN). To enable deployment on edge hardware, the architecture replaces standard 3D convolutions with 3D depthwise separable convolutions. This technique factorizes a standard convolution into a depthwise convolution (applying one filter per input channel) and a pointwise convolution (using $1 \times 1 \times 1$ kernels to combine features). This approach drastically reduces the parameter count from 375,900 in a standard 3D CNN to just 21,700—a 17x reduction—with minimal impact on accuracy.
Spatiotemporal Feature Extraction
The pipeline utilizes a "hardwired" pre-processing layer that extracts grayscale values, spatial gradients, and optical flow from seven consecutive video frames. This provides the network with immediate physical priors regarding fluid motion. By analyzing these adjacent frames simultaneously, the model can effectively "feel" the rhythm and velocity of the waves, which is essential for predicting temporal features like wave period.
Multi-Task Learning
iOceanSee employs a joint loss function to simultaneously predict wave height and wave period. This multi-task approach allows the model to learn shared features that benefit both estimations, improving overall robustness.
Experimental Results
The researchers validated iOceanSee using real-world data collected from research vessels (YUMING and XIANGYANGHONG81) across eight different oceanic stations. The input consisted of 1080p RGB video patches, with ground truth provided by high-precision underwater pressure wave meters.
Accuracy and Efficiency
In comparative tests against traditional methods (such as EOFWNN) and standard 3D CNNs, iOceanSee demonstrated exceptional performance:
- Accuracy: The model achieved a correlation coefficient ($R^2$) of 0.9986 for wave height and 0.9945 for wave period, nearly matching the performance of much larger, non-mobile 3D models.
- Inference Speed: When deployed on an NVIDIA Jetson TX2, iOceanSee achieved an inference time of just 8.6 ms. This is approximately 9 times faster than a standard 3D CNN (78.5 ms), enabling true real-time monitoring.
- Error Reduction: Compared to non-deep learning methods, iOceanSee improved the Root Mean Square Error (RMSE) for wave height by over 66%.
Conclusion and Future Work
iOceanSee demonstrates that lightweight 3D CNNs can provide high-fidelity ocean state estimation on low-cost edge hardware. While the system currently relies on visible light—making it dependent on environmental visibility—it offers a significant step forward in non-interference marine monitoring. Future research will focus on incorporating attention mechanisms to better identify wave crests and expanding the training dataset to include a wider variety of global sea states and extreme weather conditions.
Summary updated