Edge-Guided Multi-Scale Fusion and Importance-Aware Learning for Real-Time Semantic Segmentation in Waterborne Navigation

Lijia Chen, Jianzhou Zou, Yao Huang, Yang Zhou, Guozhu Hao, Yi ZhangTransNav 2025 · DOI

TL;DR

Researchers have developed MSIA-Net, a specialized deep learning model for real-time semantic segmentation in waterborne navigation. By combining edge-guided feature extraction with a safety-first "importance-aware" loss function, the model achieves a high-accuracy 83.1% mIoU at a blistering 69.1 FPS, outperforming standard models in detecting small obstacles and complex shoreline boundaries.

What Problem Does This Paper Address?

Navigating a vessel autonomously is a massive computer vision challenge. Unlike self-driving cars on structured roads, unmanned surface vehicles (USVs) face "waterborne" chaos: blinding sun reflections, fog, dynamic waves, and a lack of clear lane markings.

Existing semantic segmentation models (which classify every pixel in an image, e.g., "this pixel is a ship," "this is water") often struggle with two things in maritime settings:

  1. Scale Disparity: A tiny buoy and a massive cargo ship might appear in the same frame, requiring the model to "see" at multiple scales simultaneously.
  2. Edge Blurring: Reflections and spray make it hard to tell where a boat ends and the water begins.
  3. Hardware Constraints: USVs often have limited onboard computing power, making heavy, state-of-the-art models too slow for real-time collision avoidance.

Methods

The authors introduce MSIA-Net (Multi-Scale Importance-Aware Network), built on the lightweight DDRNet-23-slim backbone. They added three clever architectural "upgrades":

  • Edge Guidance Module (EGM): This branch uses a traditional Sobel operator to explicitly find edges in the image. It then uses a Squeeze-and-Excitation (SE) module to tell the network which edges are actually important, helping the model "sharpen" the boundaries of ships and docks.
  • Dynamic Feature Fusion Module (DFFM): Instead of just stacking features from different layers, this module uses a lightweight attention mechanism to adaptively weight features. It ensures that the model doesn't lose track of small targets while processing large-scale background elements like the sky.
  • SimAM (Simple Parameter-Free Attention): In the upsampling (decoder) stage, the authors integrated SimAM. This is an "energy-based" attention mechanism that highlights important neurons without adding any extra learnable parameters, keeping the model fast and lightweight.
  • Importance-Weighted Loss: Not all pixels are equal. Missing a pixel of "sky" is fine; missing a pixel of an "obstacle" is a crash. The authors modified the loss function to penalize the model more heavily if it misclassifies critical safety objects like ships and buoys.

Datasets

The model was evaluated on two primary benchmarks:

  1. On_Water Dataset: A custom-curated dataset of 1,200 high-resolution images (1920×1080) combining footage from the Yangtze River in Wuhan with established datasets like MODD and SMD. It features five classes: water, ships, navigation aids, ports, and obstacles.
  2. SeaShips Dataset: A publicly available dataset focused on ship recognition. The authors selected 200 representative images to test the model's generalization across different aquatic environments.

Key Insights and Ideas

  • Safety as a Loss Function: The "aha moment" here is the Weight-aware Ranking. By categorizing objects into three ranks (Rank 1: Boats/Obstacles; Rank 2: Buoys; Rank 3: Water/Sky) and scaling the loss function accordingly, the model learns to prioritize the objects that matter most for navigation safety.
  • Efficiency via "Parameter-Free" Attention: Usually, adding "attention" to a model makes it slower. By using SimAM, the researchers gained the benefits of spatial and channel attention (focusing on where and what) without the computational tax of extra convolutional layers.
  • Edges Matter in Low Contrast: In maritime scenes where the horizon or a shoreline might blend into the water, the explicit Edge Guidance Module acts as a "stencil," forcing the network to respect physical boundaries that purely data-driven approaches might overlook.

Key Results

MSIA-Net strikes an impressive balance between speed and precision:

  • Accuracy: Achieved a mean Intersection over Union (mIoU) of 83.1% on the On_Water dataset. Specifically, ship segmentation accuracy hit 88.2%.
  • Speed: Clocked in at 69.1 FPS on an NVIDIA 4060 laptop (at 1080p resolution), which is well above the threshold for real-time navigation.
  • Efficiency: The model uses only 5.9 million parameters, making it significantly smaller than competitors like WaSR (13.23M) or STDC2 (11.45M) while providing better accuracy in complex inland river scenarios.

Limitations and Future Work

While MSIA-Net is a leap forward for vision-based navigation, the authors note a few areas for improvement:

  • Sensor Fusion: The current model relies solely on RGB cameras. Future versions should integrate multi-modal data (like LiDAR or Radar) to handle extreme weather where cameras fail entirely.
  • Temporal Context: The model currently processes images frame-by-frame. The authors suggest that incorporating temporal information (video consistency) could help the model better track dynamic objects and small targets over time.

Summary updated

Abstract

Effective multi-scale feature representation and focused attention on critical objects are essential for accurate perception of waterborne navigation scenes. To address the insufficient exploitation of multi-scale information in existing methods that leads to imprecise segmentation, this study proposes a real-time semantic segmentation method for waterborne navigation scenes through multi-scale information enhancement and importance-weighted optimization. First, DDRNet-23-slim is selected as the backbone network for feature extraction. An edge-guided branch is embedded into its shallow layers, and a Dynamic Feature Fusion Module (DFFM) is constructed by integrating a lightweight hybrid attention mechanism, effectively enhancing multi-scale feature interaction capabilities. Second, the loss function is improved using an importance-weighted strategy to prioritize critical objects during training. Finally, a parameter-free attention mechanism is introduced in the upsampling stage, maintaining real-time performance while ensuring segmentation stability for key objects under complex background interference. Evaluations on the On_Water and Seaships datasets demonstrate that the proposed method achieves mIoU scores of 83.1% and 73.2%, respectively, with ship segmentation accuracy reaching 88.2% on On_Water. The inference speed attains 69.1 FPS, outperforming mainstream real-time segmentation models (e.g., DDRNet, STDC) in balancing accuracy and efficiency. Notably, it exhibits stronger robustness in complex inland river scenarios with dense shore structures and numerous small targets.
BibTeX
@inproceedings{chen2025edge,
  author = {Lijia Chen and Jianzhou Zou and Yao Huang and Yang Zhou and Guozhu Hao and Yi Zhang},
  title = {{Edge-Guided Multi-Scale Fusion and Importance-Aware Learning for Real-Time Semantic Segmentation in Waterborne Navigation}},
  booktitle = {TransNav},
  year = {2025},
  doi = {10.12716/1001.19.02.30},
}
Figure