Introduction
Self-supervised monocular depth estimation typically relies on the static scene assumption, where image warping between frames is attributed solely to camera motion. However, dynamic objects like moving vehicles or pedestrians violate this assumption, leading to high reconstruction errors that degrade depth map quality. While joint training of depth, pose, and optical flow networks has been proposed to handle these dynamic regions, a fundamental intrinsic bias exists: optical flow is naturally easier for networks to learn than complex 3D geometry. This often leads to a competitive scenario where the flow network dominates early on, "starving" the depth network of training data and resulting in degenerate outputs like infinite-depth "black stains."
The CoopNet Approach: From Competition to Cooperation
To address these instabilities, the authors introduce CoopNet, a framework that replaces competition with a cooperative training strategy. Instead of forcing networks to compete for pixels, CoopNet dynamically rebalances gradients to ensure each network specializes in the regions it is best suited for.
Quantile-Based Error Splitting
The core of CoopNet is a dynamic rebalancing mechanism based on the distribution of the difference ($\Delta$) between depth-based and flow-based reconstruction errors. Using the $P^2$ algorithm, the model tracks the quantiles of this distribution in real-time throughout the 30-epoch training cycle.
As illustrated in the figure above, the training process identifies an Agreement Area—a tight neighborhood around the mean error where both depth and flow networks produce similar results. The depth and pose networks are exclusively trained on pixels within this area, ensuring they learn from reliable, static regions of the scene. This allows the stronger network (flow) to act as a filter for the weaker one (depth), similar to teacher-student techniques.
Specialized Training and Regularization
While the depth network focuses on the agreement area, the optical flow network is trained on the entire image. However, higher weights are assigned to the "tails" of the error distribution—pixels where the networks disagree. This encourages the flow network to specialize in modeling moving objects. Additionally, a flow regularization term ($\Delta_{flow}$) is introduced to ensure geometric consistency in textureless areas where photometric losses are uninformative.
Experiments and Results
CoopNet was evaluated on the KITTI and Cityscapes benchmarks, focusing on depth estimation and camera odometry.
Performance Metrics
- Depth Estimation: On the KITTI dataset, CoopNet achieved an Absolute Relative (Abs Rel) error of 0.113, surpassing established baselines such as Monodepth2 (0.115) and SGDepth (0.117).
- Odometry: The method demonstrated significant improvements in camera trajectory accuracy. On KITTI Sequence 09, it reduced translation error to 8.42%, a substantial improvement over the 17.84% error of the Zhou et al. baseline.
- Qualitative Sharpness: As seen in the visual results, CoopNet produces significantly sharper depth maps with fewer "bleeding" edges around thin objects and high-texture regions compared to competitive methods.
Conclusion
CoopNet demonstrates that managing the intrinsic bias between different self-supervised tasks is crucial for stable training. By shifting from a competitive to a cooperative paradigm via quantile-based gradient rebalancing, the model achieves state-of-the-art performance in depth and odometry without the need for external supervision like pre-trained semantic segmentation. While challenges remain in homogeneous areas, future work may explore combining this strategy with explicit residual motion correction to further refine the modeling of dynamic scene elements.
Summary updated