Eff-UNet: A Novel Architecture for Semantic Segmentation in Unstructured Environment

Bhakti Baheti, Shubham Innani, Suhas S. Gajre, Sanjay N. TalbarWorkshops @ CVPR 2020 · DOI

Introduction

Autonomous driving research has historically focused on "structured" environments, such as those found in the Cityscapes or KITTI datasets. These environments feature well-defined lane markings, clear boundaries, and predictable traffic patterns typical of North American and European roads. However, a significant portion of the global driving landscape consists of unstructured environments. These areas are characterized by ambiguous road boundaries, diverse vehicle types like auto-rickshaws, and varying terrains that blend seamlessly into the drivable path, making semantic segmentation a significant challenge.

Comparison of unstructured road scenes (top) versus structured environments (bottom).

Methodology: The Eff-UNet Architecture

To address the complexities of unstructured scenes, the authors propose Eff-UNet. This hybrid architecture is designed to balance high-level contextual understanding ("what is this?") with precise low-level spatial localization ("where is this?").

The model utilizes EfficientNet-B7 as its encoder backbone. EfficientNet is distinguished by its use of compound scaling, which simultaneously balances network depth, width, and image resolution to achieve superior feature extraction efficiency. Unlike traditional symmetric UNets, Eff-UNet is asymmetric; the EfficientNet-B7 encoder is significantly deeper and more complex than the expansion path, allowing for the extraction of incredibly rich features.

The decoder follows a UNet-style structure, employing skip connections to concatenate low-level spatial information from the encoder with upsampled high-level features. This architectural choice ensures that fine-grained spatial details, which are often lost during the downsampling process in standard CNNs, are preserved and utilized in the final reconstruction of the segmentation map.

Detailed architecture of Eff-UNet illustrating the encoder-decoder structure, feature map dimensions, and skip connections.

Datasets and Experimental Setup

The primary benchmark for evaluating Eff-UNet is the India Driving Dataset (IDD) Lite. IDD Lite is a subsampled version of the full IDD, specifically curated for resource-constrained training while maintaining the inherent complexity of Indian road conditions. The dataset consists of 1,404 training images, 204 validation images, and 408 test images, categorized into seven classes: Drivable, Non-drivable, Living things, Vehicles, Roadside objects, Far objects, and Sky.

Results and Discussion

Eff-UNet demonstrated state-of-the-art performance, winning first prize in the NCVPRIPG 2019 IDD Lite segmentation challenge. The model achieved a mean Intersection over Union (mIoU) of 0.7376 on the validation set and 0.6276 on the test set, significantly outperforming baseline models such as Dilated ResNet18 (0.55 mIoU) and ERFNet (0.66 mIoU).

Key insights from the experimental results include:

  • Encoder Scaling: The transition from EfficientNet-B5 to B7 yielded a substantial boost in mIoU, suggesting that the quality of initial feature extraction is a primary bottleneck in complex, messy scenes.
  • Decoder Efficacy: The UNet decoder outperformed the popular DeepLabV3+ framework in this specific task. The authors attribute this to UNet’s superior ability to incorporate low-level features, which are vital when road boundaries are visually ambiguous.
  • The "Non-Drivable" Challenge: The model struggled most with the "Non-drivable" class (0.50 IoU). This confirms the difficulty of segmenting unstructured environments where the transition from road to dirt or mud is often a gradient rather than a distinct line.

Conclusion

Eff-UNet provides a robust solution for semantic segmentation in challenging, unstructured environments. By leveraging compound-scaled encoders and skip-connection-based decoders, it effectively balances high-level context with spatial precision. Future work will focus on scaling the architecture to high-resolution datasets and addressing class imbalances through specialized loss functions to better handle "long-tail" objects like roadside poles and signs.

Summary updated

BibTeX
@inproceedings{baheti2020eff,
  author = {Bhakti Baheti and Shubham Innani and Suhas S. Gajre and Sanjay N. Talbar},
  title = {{Eff-UNet: A Novel Architecture for Semantic Segmentation in Unstructured Environment}},
  booktitle = {Workshops @ CVPR},
  year = {2020},
  doi = {10.1109/cvprw50498.2020.00187},
}
Figure