iOceanSee: A Novel Scheme for Ocean State Estimation Using 3D Mobile Convolutional Neural Network

Huafeng Wu, Yuanyuan Zhang, Jun Wang, Weijun Wang, Jiangfeng Xian, Jing Chen, Xiangyi Zou, Prasant MohapatraIEEE Access 2020 · OpenReview

Abstract

Ocean state estimation, specifically the prediction of wave height and period, is fundamental for marine engineering and autonomous navigation. Traditional sensor-based methods are costly and prone to failure in harsh environments, while existing computer vision approaches often fail to capture temporal dynamics. This paper introduces iOceanSee, a novel scheme utilizing a 3D Mobile Convolutional Neural Network (3D MCNN) to estimate wave parameters from simple RGB video. By employing 3D depthwise separable convolutions, iOceanSee achieves high accuracy with significantly reduced computational overhead, making it suitable for real-time deployment on edge devices like the NVIDIA Jetson TX2.

Introduction

Accurate estimation of ocean wave height and period is critical for the safety of autonomous ships, the design of offshore energy systems, and coastal management. Historically, this data has been gathered using expensive physical sensors like buoys, X-band radar, or satellite remote sensing. However, these methods face significant challenges: physical sensors are easily damaged by saltwater and storms, while radar systems require frequent, expensive calibration.

Recent advances in computer vision have attempted to solve this using 2D images, but these models often struggle because they ignore the temporal "motion" information inherent in waves. While standard 3D Convolutional Neural Networks (CNNs) can capture these spatiotemporal features, they are typically too computationally intensive for the remote, low-power environments found at sea. iOceanSee bridges this gap by providing a lightweight, high-accuracy detection scheme capable of running on mobile terminals without a stable network connection.

Methodology: The 3D Mobile CNN

The core of the iOceanSee system is the 3D Mobile Convolutional Neural Network (3D MCNN). To enable deployment on edge hardware, the architecture replaces standard 3D convolutions with 3D depthwise separable convolutions. This technique factorizes a standard convolution into a depthwise convolution (applying one filter per input channel) and a pointwise convolution (using $1 \times 1 \times 1$ kernels to combine features). This approach drastically reduces the parameter count from 375,900 in a standard 3D CNN to just 21,700—a 17x reduction—with minimal impact on accuracy.

Overview of the iOceanSee system architecture, illustrating the transition from high-performance training to real-time edge inference.

Spatiotemporal Feature Extraction

The pipeline utilizes a "hardwired" pre-processing layer that extracts grayscale values, spatial gradients, and optical flow from seven consecutive video frames. This provides the network with immediate physical priors regarding fluid motion. By analyzing these adjacent frames simultaneously, the model can effectively "feel" the rhythm and velocity of the waves, which is essential for predicting temporal features like wave period.

Multi-Task Learning

iOceanSee employs a joint loss function to simultaneously predict wave height and wave period. This multi-task approach allows the model to learn shared features that benefit both estimations, improving overall robustness.

Experimental Results

The researchers validated iOceanSee using real-world data collected from research vessels (YUMING and XIANGYANGHONG81) across eight different oceanic stations. The input consisted of 1080p RGB video patches, with ground truth provided by high-precision underwater pressure wave meters.

Accuracy and Efficiency

In comparative tests against traditional methods (such as EOFWNN) and standard 3D CNNs, iOceanSee demonstrated exceptional performance:

  • Accuracy: The model achieved a correlation coefficient ($R^2$) of 0.9986 for wave height and 0.9945 for wave period, nearly matching the performance of much larger, non-mobile 3D models.
  • Inference Speed: When deployed on an NVIDIA Jetson TX2, iOceanSee achieved an inference time of just 8.6 ms. This is approximately 9 times faster than a standard 3D CNN (78.5 ms), enabling true real-time monitoring.
  • Error Reduction: Compared to non-deep learning methods, iOceanSee improved the Root Mean Square Error (RMSE) for wave height by over 66%.

Conclusion and Future Work

iOceanSee demonstrates that lightweight 3D CNNs can provide high-fidelity ocean state estimation on low-cost edge hardware. While the system currently relies on visible light—making it dependent on environmental visibility—it offers a significant step forward in non-interference marine monitoring. Future research will focus on incorporating attention mechanisms to better identify wave crests and expanding the training dataset to include a wider variety of global sea states and extreme weather conditions.

Summary updated

Abstract

Ocean state estimation is a basic problem in the field of ocean engineering. Under the trend of data-driven, the development of intelligent ship decision-making, ocean energy system design and other aspects, are inseparable from the estimation of wave parameters in the ocean area. In recent years, researchers have developed remote sensing technology to monitor ocean waves. However, sensor-based methods all have a key limitation, which is high cost and fault worry. More importantly, one major limitation exists in current research: due to lack of change information and relying on a single feature of spatial data, the final predictive results are inaccurate. Adopting a 3D Convolutional Neural Network is a possible solution to improve the detection accuracy. Unfortunately, it cannot be deployed in the ocean environment due to lack of physical network connections. To resolve these issues, we develop a light-weight version of 3D Convolutional Neural Network, namely a low-cost, high-accuracy detection scheme to foresee ocean wave parameters using a 3D Mobile Convolutional Neural Network technique called iOceanSee in the marine environment. iOceanSee employs a mobile terminal composed of low-cost measuring equipment and non-interference (except light) device-an RGB camera to collect video data in real time. It extracts both space and time features through three-dimensional depthwise separable convolutions. More specifically, iOceanSee is able to capture the encoding motion information from multiple adjacent frames of the video, according to which period and height of the waves being evaluated. Our experimental results conclude that iOceanSee obtains comparable performance to 3D Convolutional Neural Network and outperforms other models in terms of measurement accuracy in the marine environments.
BibTeX
@inproceedings{wu2020ioceansee,
  author = {Huafeng Wu and Yuanyuan Zhang and Jun Wang and Weijun Wang and Jiangfeng Xian and Jing Chen and Xiangyi Zou and Prasant Mohapatra},
  title = {{iOceanSee: A Novel Scheme for Ocean State Estimation Using 3D Mobile Convolutional Neural Network}},
  booktitle = {IEEE Access},
  year = {2020},
  url = {https://openreview.net/forum?id=aBxXeqWo3s},
}
Figure