Motion Degeneracy in Self-supervised Learning of Elevation Angle Estimation for 2D Forward-Looking Sonar

Yusheng Wang, Yonghoon Ji, Chujie Wu, Hiroshi Tsuchiya, Hajime Asama, Atsushi Yamashita

Introduction

2D Forward-Looking Sonar (FLS) is an essential sensor for underwater robotics, providing visibility in turbid environments where optical cameras fail. However, FLS technology inherently suffers from a loss of information: it projects a 3D acoustic volume onto a 2D image plane, effectively discarding the elevation angle. This ambiguity makes it difficult to determine the vertical position of objects, a critical requirement for 3D mapping and obstacle avoidance.

While deep learning has shown promise in recovering this missing dimension, supervised methods are limited by the extreme difficulty of obtaining 3D ground-truth labels underwater. Self-supervised approaches offer a potential alternative, yet they frequently struggle with convergence or rely heavily on synthetic pre-training that fails to generalize to real-world noise. This paper investigates the root cause of these failures—identifying "motion degeneracy" as a primary obstacle—and proposes a self-supervised framework that succeeds by leveraging specific sensor movements.

The Theory of Motion Degeneracy

The core contribution of this research is the mathematical analysis of the sonar motion field. By deriving equations that describe how pixels shift between frames based on sensor movement, the authors identified that not all motions are created equal for the purpose of elevation estimation.

In many common robotic movements, such as horizontal translation ($x$ or $y$ axes), the resulting pixel displacement is independent of the elevation angle. This state is termed motion degeneracy. In these scenarios, the network cannot learn to estimate height because the visual evidence does not change relative to the object's vertical position. Conversely, the analysis identifies "effective" motions—specifically $x$-axis rotation ($\omega_x$) and $z$-axis translation ($t_z$)—where pixel displacement is highly sensitive to elevation. These "golden" motions provide the necessary geometric signal for self-supervised convergence.

Methodology: EAE-Net and Warping Loss

To exploit these effective motions, the authors developed EAE-Net, a UNet-based architecture designed to predict an elevation map from a single sonar image. The network is trained using a self-supervised warping loss.

Given a source frame ($I_s$), a target frame ($I_t$), and the known relative motion between them, the network predicts the elevation for $I_t$. Using this prediction and the known motion, the system "warps" $I_s$ to synthesize a reconstructed version of $I_t$. The training objective is to minimize the photometric difference between the real $I_t$ and the synthesized version using a combination of L1 and Structural Similarity Index (SSIM) losses.

To handle the significant noise and multi-path reflections (echoes) typical of sonar data, the authors implemented a signal masking strategy. A pre-trained segmentation network identifies the actual acoustic returns, allowing the warping loss to focus on meaningful signals and ignore disruptive noise, which significantly stabilizes the training process.

Experimental Results

Simulation Studies

Using a Blender-based sonar simulator, the researchers tested the network across six basic motion types. The results confirmed the degeneracy theory:

  • Effective Motions ($\omega_x, t_z$): Achieved a Mean Absolute Error (MAE) of approximately 0.03 rad, nearly matching the performance of supervised models.
  • Degenerate Motions ($t_x, t_y$): Failed to converge, resulting in MAEs three to four times higher and near-zero f-scores.

Real-World Validation

The system was further validated using an ARIS EXPLORER 3000 sonar in a controlled water tank. Using a robotic arm to execute precise $x$-axis rotations, the team collected a dataset of 1,071 images. The self-supervised EAE-Net achieved an f-score of 83.18% at a 3mm threshold. This performance is comparable to state-of-the-art supervised models like A2FNet, demonstrating that geometric constraints can effectively replace manual labels.

Conclusion

This work demonstrates that the success of self-supervised elevation estimation in FLS depends less on architecture and more on the geometric properties of the training data. By identifying and utilizing effective motions like $x$-axis rotation, researchers can train high-performing models without the need for synthetic pre-training or difficult-to-obtain 3D labels.

Future work will focus on relaxing the requirement for known motion by learning odometry and elevation simultaneously, as well as incorporating temporal information from multi-frame sequences to further improve robustness against multi-path reflections.

Summary updated

Abstract

2D forward-looking sonar is a crucial sensor for underwater robotic perception. A well-known problem in this field is estimating missing information in the elevation direction during sonar imaging. There are demands to estimate 3D information per image for 3D mapping and robot navigation during fly-through missions. Recent learning-based methods have demonstrated their strengths, but there are still drawbacks. Supervised learning methods have achieved high-quality results but may require further efforts to acquire 3D ground-truth labels. The existing self-supervised method requires pretraining using synthetic images with 3D supervision. This study aims to realize stable self-supervised learning of elevation angle estimation without pretraining using synthetic images. Failures during self-supervised learning may be caused by motion degeneracy problems. We first analyze the motion field of 2D forward-looking sonar, which is related to the main supervision signal. We utilize a modern learning framework and prove that if the training dataset is built with effective motions, the network can be trained in a self-supervised manner without the knowledge of synthetic data. Both simulation and real experiments validate the proposed method.
BibTeX
@inproceedings{wang2023motion,
  author = {Yusheng Wang and Yonghoon Ji and Chujie Wu and Hiroshi Tsuchiya and Hajime Asama and Atsushi Yamashita},
  title = {{Motion Degeneracy in Self-supervised Learning of Elevation Angle Estimation for 2D Forward-Looking Sonar}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2023},
  doi = {10.1109/iros55552.2023.10341601},
}
Figure