Abstract
Our key innovation lies in bridging the semantic gap between object detection and event understanding without requiring expensive task-specific training. The proposed plug-and-play framework interfaces with any open-vocabulary detector while extending their inherent capabilities across architectures. At its core, our approach combines (i) a symbolic regression mechanism exploring relationship patterns among detected entities and (ii) a LLM-guided strategically guiding the search toward meaningful expressions. These discovered symbolic rules transform low-level visual perception into interpretable event understanding, providing a transparent reasoning path from objects to events with strong transferability across domains. We compared our training-free framework against specialized event recognition systems across diverse application domains. Experiments demonstrate that our framework enhances multiple object detector architectures to recognize complex events such as illegal fishing activities (75% AUROC, +8.36% improvement), construction safety violations (+15.77%), and abnormal crowd behaviors (+23.16%).
Introduction
Modern object detectors excel at identifying discrete entities within an image, such as a "person" or a "helmet." However, transitioning from simple object recognition to complex event understanding—answering "what is happening?"—remains a significant challenge. Traditional approaches to event recognition often rely on "black-box" neural networks trained on massive, task-specific datasets. These models are not only computationally expensive to train but also lack interpretability and struggle to adapt to new scenarios.
SymbolicDet addresses these limitations by introducing a training-free framework that transforms standard object detectors into sophisticated event recognizers. By bridging the gap between perception and reasoning, the system identifies complex events through human-readable logical rules without requiring additional training data.
Methodology
SymbolicDet is a "plug-and-play" framework designed to sit atop any open-vocabulary object detector. The process is divided into three primary stages:
- Entity Extraction: An off-the-shelf detector (e.g., APE, GLIP, or YOLO-World) identifies objects and their attributes, including counts and spatial relationships.
- Symbolic Logic Search: The system employs an evolutionary algorithm to navigate a search space of logical expressions. It seeks the optimal combination of operators (AND, OR, NOT, >, <) to distinguish target events from baseline behavior.
- LLM-Guided Reasoning: To avoid the inefficiencies of brute-force search, a Large Language Model (LLM) acts as a "Visual Logic Architect." Leveraging its inherent world knowledge and Chain-of-Thought (CoT) prompting, the LLM suggests and refines meaningful patterns (e.g., identifying that a safety violation involves a
Personand aHeadbut the absence of aHelmet).
The output of this process is a Symbolic Pattern, a transparent logical formula that provides a clear reasoning path for the detection decision.
Experimental Evaluation
The framework was evaluated across several diverse benchmarks, including the newly introduced Helmet-Mac Dataset (12,213 samples for construction safety), the Multi-Event Dataset (for illegal fishing detection), and the ERA Dataset (for aerial crowd analysis).
Results indicate that SymbolicDet significantly enhances the performance of various detector backbones. For instance, in construction safety compliance, the framework improved detection by +15.77% over the baseline. In the domain of illegal fishing, it achieved a 75.16% AUROC, representing an +8.36% improvement. Notably, on the UCSD Ped2 crowd anomaly benchmark, the training-free framework reached 98.7% AUROC, nearly matching state-of-the-art models that require intensive training.
Discussion and Conclusion
SymbolicDet demonstrates that training-free event recognition is not only viable but highly effective when leveraging the common-sense reasoning of LLMs. By treating object detectors as "visual sensors" and applying a symbolic reasoning layer, the framework provides interpretability by design—a critical requirement for safety-regulated industries.
While the system shows strong performance, it remains dependent on the quality of the underlying entity detection. Future work will focus on optimizing computational latency associated with LLM API calls and extending the framework's capabilities to include temporal reasoning for behavioral analysis in video sequences.
Summary updated