TL;DR
This paper introduces EcoSense, a novel edge-cloud collaborative system for in-shore ship detection that intelligently routes tasks based on object difficulty. By splitting detection into localization and fine-grained classification, EcoSense significantly reduces data transmission and energy consumption while achieving state-of-the-art accuracy, making it ideal for resource-constrained maritime surveillance.
What Problem Does This Paper Address?
Detecting ships in coastal waters (Inshore Marine Object Detection - IMOD) is crucial for safety, navigation, and security. However, existing methods face significant challenges. AIS systems are unreliable for vessels without active transmitters, and RADAR has limitations in detail and blind spots. While RGB cameras offer rich data, applying deep learning models directly to IMOD is difficult due to the subtle visual differences between ship types and the demanding computational resources required for real-time processing on edge devices. Furthermore, deploying sensors along vast coastlines creates a need for efficient data transmission and low energy consumption, which traditional centralized or fully distributed approaches struggle to balance.
Methods
EcoSense tackles these challenges with a two-tiered, difficulty-aware edge-cloud collaboration framework.
- Edge-Side (Front-end Model): This lightweight model performs object localization, difficulty estimation, and a preliminary classification. It uses a modified GhostNet backbone for efficiency and incorporates a "coordinate attention" mechanism to capture both channel-wise and spatial information. Crucially, it includes a difficulty estimator that assesses how challenging a detected object is to classify.
- Cloud-Side (Back-end Model): For objects deemed "hard" by the front-end, a more powerful model is employed in the cloud. This back-end model utilizes a Swin Transformer combined with a Graph Convolutional Network (GCN) for robust fine-grained classification. It also incorporates a Background Suppression module to improve feature discrimination and a "high-temperature refinement" technique to leverage multi-scale context.
The core innovation lies in the difficulty-aware routing: easy-to-classify objects are handled entirely on the edge, while hard ones are sent to the cloud. This intelligent split minimizes data transfer and cloud computation.
Datasets
The paper evaluates EcoSense on two publicly available datasets:
- SeaShips: A large-scale dataset with over 31,455 images of ships, annotated with 6 ship types.
- Singapore Maritime Dataset - Plus (SMD-Plus): An extension of the SMD dataset, featuring corrected labels and improved bounding box annotations for small maritime objects.
Key Insights and Ideas
- Difficulty-Aware Routing is Key: The most significant insight is that not all objects require the full power of a cloud-based model. By intelligently estimating and routing based on difficulty, substantial computational and communication savings can be achieved.
- Edge-Cloud Synergy: The paper demonstrates that a well-designed collaboration between resource-constrained edge devices and powerful cloud servers can outperform either approach in isolation, achieving a balance of accuracy, efficiency, and low latency.
- Attention for Efficiency: The use of "coordinate attention" in the front-end model is crucial for capturing essential spatial and channel information without incurring excessive computational overhead, making it suitable for edge deployment.
- Fine-grained Classification for Maritime: The challenge of distinguishing between visually similar ship types is addressed by a sophisticated back-end model that leverages advanced architectures like Swin Transformers and GCNs, along with techniques to suppress irrelevant background information.
Key Results
EcoSense achieves impressive results across the board:
- Accuracy: Outperforms baselines with a +4.3% improvement in [email protected] on SeaShips and +1.9% on SMD-Plus.
- Data Transmission Reduction: Achieves a remarkable 95.43% reduction in data transmission volume on SeaShips and 92.83% on SMD-Plus compared to a centralized approach.
- Energy Efficiency: Consumes only 27.3% of the energy on SeaShips and 31.6% on SMD-Plus compared to a centralized system.
- Real-world Deployment: Successfully validated on embedded platforms (edge GPU, TPU, FPGA) and demonstrated in a real-world drone deployment scenario, showing accurate detection of ships and even some smaller objects like buoys.
Limitations and Future Work
- False Positives on Similar Objects: The drone deployment example highlights that smaller objects with shapes similar to ships (like buoys) can be misclassified. The authors attribute this partly to the lack of bird's-eye view data in their training set.
- Future Work: The authors suggest that collecting more diverse data, particularly from various viewpoints, could further boost performance. They also implicitly point towards continued optimization of edge hardware and model architectures for even greater efficiency.
Summary updated