ODOV: Towards Open-Domain Open-Vocabulary Object Detection

Yupeng Zhang, Ruize Han, Fangnan Zhou, Song Wang, Wei Feng, Liang Wan

Introduction

Standard object detection models typically excel in controlled environments or within a fixed set of categories. However, real-world applications demand models that can handle two distinct types of shifts simultaneously: Category Shift and Domain Shift. Open-Vocabulary Detection (OVD) focuses on identifying novel object categories not seen during training, while Domain Generalization (DG) aims to maintain performance across varying visual conditions like weather or artistic styles.

The intersection of these challenges—detecting novel objects in unseen domains—is termed Open-Domain Open-Vocabulary (ODOV) detection. This "compound shift" is particularly difficult because models often struggle to distinguish between an object's intrinsic features and the environmental "noise" or stylistic distortions of a new domain.

The ODOV framework addresses compound shifts in both category and domain.

Methodology

To address the ODOV problem, the authors propose a framework that leverages Vision-Language Models (VLMs) like CLIP, enhanced by three core components:

1. Domain-Agnostic Category Prompt (DAPmt)

Traditional prompts like "a photo of a [category]" are often insufficient for complex domains. ODOV utilizes Large Language Models (LLMs), specifically GPT-4o, to generate detailed, attribute-based descriptions. These descriptions focus on the physical essence of an object (e.g., "An airplane is a large vehicle with long wings and a streamlined body") while remaining agnostic to domain-specific attributes like color or texture. This ensures the model learns a robust representation of the object's identity.

2. Domain Projection and Grafting (DP&G)

Rather than attempting to ignore domain information, the ODOV framework explicitly extracts "style" features (mean and standard deviation of feature maps) from the input image. This domain information is then "grafted" onto the text embeddings, allowing the model to adapt its internal representation of a category to the specific visual context of the test image.

3. Orthogonality Constraint

To prevent domain information from overwhelming category identity, the model employs an orthogonality loss ($L_{orth}$). This mathematical constraint ensures that category-specific and domain-specific features remain distinct, preventing the "style" of an image from washing out the "identity" of the object during the detection process.

The ODOV architecture featuring Domain-Agnostic Category Prompts (DAPmt) and Domain Projection and Grafting (DP&G).

The OD-LVIS Benchmark

The researchers introduced OD-LVIS, a massive new benchmark designed to evaluate ODOV performance. Built upon the LVIS validation set, it includes:

  • Scale: 46,949 images covering 1,203 categories.
  • Diversity: 18 distinct domains, including 9 imaging conditions (e.g., rain, haze, low-resolution) and 9 artistic styles (e.g., oil painting, pencil sketch, watercolor).
  • Evaluation Protocol: Models are trained on "base" categories in clear, natural images and tested on "novel" categories across all 18 challenging domains.

Key Results and Insights

The ODOV framework was benchmarked against state-of-the-art models including CLIPSelf, OWL-ViT, and YOLO-World.

  • Performance Gains: Using a ViT-L/14 backbone, the ODOV method achieved an overall Average Precision (AP) of 23.1%, outperforming the previous best (CLIPSelf) by 1.5%.
  • Artistic Robustness: The model demonstrated significant improvements in abstract domains like "lyrical abstraction" and "oil paintings," where traditional CLIP-based models frequently fail.
  • Consistency: Improvements were observed across frequent, common, and rare categories, indicating that domain adaptation does not compromise the model's vocabulary breadth.

Limitations and Future Work

While ODOV sets a new state-of-the-art baseline, challenges remain. The model still struggles with severe motion and Gaussian blur, where the physical structure of the object is significantly degraded. Furthermore, the absolute AP of 23.1% highlights the inherent difficulty of the ODOV task compared to standard detection. Future research may focus on integrating more sophisticated LLM knowledge or developing "restoration-aware" detection heads to better handle extreme visual distortions.

Summary updated

BibTeX
@inproceedings{zhang2026odov,
  author = {Yupeng Zhang and Ruize Han and Fangnan Zhou and Song Wang and Wei Feng and Liang Wan},
  title = {{ODOV: Towards Open-Domain Open-Vocabulary Object Detection}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2026},
  url = {https://openreview.net/forum?id=SvxfR4x7Uy},
}
Figure