EdgeMaskFormer: Adapting Mask Transformer for Semantic Edge Detection

Longjun Dong, Wei Ma, Hongbin ZhaIEEE Trans. Multimedia 2025 · DOI

Abstract

Semantic Edge Segmentation (SED) is crucial for intelligent agents to understand and interact with their environments, as it enables them to locate and recognize semantic boundaries. The prevailing framework in the field of SED is multi-label learning, which identifies edges and their semantics by learning to assign multiple labels that indicate the categories of the objects forming the edges. However, this framework has demonstrated limited performance when dealing with complex scenarios. In this paper, we propose a mask classification framework specifically tailored for the SED task, termed EdgeMaskFormer. Within this framework, we develop a query-based edge semantic extractor to learn semantic embeddings for edge mask classification with assistance from regional semantic supervision. Additionally, we design a context-aware hierarchical edge extractor to serve as an edge mask head, which can capture multi-scale edges of different categories under the guidance from the semantic embeddings via dynamic convolution. Furthermore, we develop matching and supervision mechanisms specifically for edge mask classification in order to reduce edge noise and address the imbalance between edge and non-edge samples. Our extensive experiments on three public datasets demonstrate that the proposed approach achieves outstanding performance in semantic edge detection, particularly on those datasets with complex scenarios.
BibTeX
@article{dong2025edgemaskformer,
  author = {Longjun Dong and Wei Ma and Hongbin Zha},
  title = {{EdgeMaskFormer: Adapting Mask Transformer for Semantic Edge Detection}},
  journal = {IEEE Transactions on Multimedia},
  year = {2025},
  doi = {10.1109/tmm.2025.3586134},
}

Loading related content...

Figure