Training-free Anomaly Event Detection via LLM-guided Symbolic Pattern Discovery

Yuhui Zeng, Haoxiang Wu, Wenjie Nie, Guangyao Chen, Xiawu Zheng, Yunhang Shen, Guilin Li, Yixiong Zou, Yonghong Tian, Rongrong Ji

Abstract

Our key innovation lies in bridging the semantic gap between object detection and event understanding without requiring expensive task-specific training. The proposed plug-and-play framework interfaces with any open-vocabulary detector while extending their inherent capabilities across architectures. At its core, our approach combines (i) a symbolic regression mechanism exploring relationship patterns among detected entities and (ii) a LLM-guided strategically guiding the search toward meaningful expressions. These discovered symbolic rules transform low-level visual perception into interpretable event understanding, providing a transparent reasoning path from objects to events with strong transferability across domains. We compared our training-free framework against specialized event recognition systems across diverse application domains. Experiments demonstrate that our framework enhances multiple object detector architectures to recognize complex events such as illegal fishing activities (75% AUROC, +8.36% improvement), construction safety violations (+15.77%), and abnormal crowd behaviors (+23.16%).

Introduction

Modern object detectors excel at identifying discrete entities within an image, such as a "person" or a "helmet." However, transitioning from simple object recognition to complex event understanding—answering "what is happening?"—remains a significant challenge. Traditional approaches to event recognition often rely on "black-box" neural networks trained on massive, task-specific datasets. These models are not only computationally expensive to train but also lack interpretability and struggle to adapt to new scenarios.

SymbolicDet addresses these limitations by introducing a training-free framework that transforms standard object detectors into sophisticated event recognizers. By bridging the gap between perception and reasoning, the system identifies complex events through human-readable logical rules without requiring additional training data.

Methodology

SymbolicDet is a "plug-and-play" framework designed to sit atop any open-vocabulary object detector. The process is divided into three primary stages:

  1. Entity Extraction: An off-the-shelf detector (e.g., APE, GLIP, or YOLO-World) identifies objects and their attributes, including counts and spatial relationships.
  2. Symbolic Logic Search: The system employs an evolutionary algorithm to navigate a search space of logical expressions. It seeks the optimal combination of operators (AND, OR, NOT, >, <) to distinguish target events from baseline behavior.
  3. LLM-Guided Reasoning: To avoid the inefficiencies of brute-force search, a Large Language Model (LLM) acts as a "Visual Logic Architect." Leveraging its inherent world knowledge and Chain-of-Thought (CoT) prompting, the LLM suggests and refines meaningful patterns (e.g., identifying that a safety violation involves a Person and a Head but the absence of a Helmet).
The SymbolicDet framework architecture, illustrating the flow from entity extraction to LLM-guided symbolic expression generation.

The output of this process is a Symbolic Pattern, a transparent logical formula that provides a clear reasoning path for the detection decision.

Experimental Evaluation

The framework was evaluated across several diverse benchmarks, including the newly introduced Helmet-Mac Dataset (12,213 samples for construction safety), the Multi-Event Dataset (for illegal fishing detection), and the ERA Dataset (for aerial crowd analysis).

Results indicate that SymbolicDet significantly enhances the performance of various detector backbones. For instance, in construction safety compliance, the framework improved detection by +15.77% over the baseline. In the domain of illegal fishing, it achieved a 75.16% AUROC, representing an +8.36% improvement. Notably, on the UCSD Ped2 crowd anomaly benchmark, the training-free framework reached 98.7% AUROC, nearly matching state-of-the-art models that require intensive training.

Performance comparison of various object detectors with and without the SymbolicDet framework across multiple datasets.

Discussion and Conclusion

SymbolicDet demonstrates that training-free event recognition is not only viable but highly effective when leveraging the common-sense reasoning of LLMs. By treating object detectors as "visual sensors" and applying a symbolic reasoning layer, the framework provides interpretability by design—a critical requirement for safety-regulated industries.

While the system shows strong performance, it remains dependent on the quality of the underlying entity detection. Future work will focus on optimizing computational latency associated with LLM API calls and extending the framework's capabilities to include temporal reasoning for behavioral analysis in video sequences.

Summary updated

Abstract

Our key innovation lies in bridging the semantic gap between object detection and event understanding without requiring expensive task-specific training. The proposed plug-and-play framework interfaces with any open-vocabulary detector while extending their inherent capabilities across architectures. At its core, our approach combines (i) a symbolic regression mechanism exploring relationship patterns among detected entities and (ii) a LLM-guided strategically guiding the search toward meaningful expressions. These discovered symbolic rules transform low-level visual perception into interpretable event understanding, providing a transparent reasoning path from objects to events with strong transferability across domains.We compared our training-free framework against specialized event recognition systems across diverse application domains. Experiments demonstrate that our framework enhances multiple object detector architectures to recognize complex events such as illegal fishing activities (75% AUROC, +8.36% improvement), construction safety violations (+15.77%), and abnormal crowd behaviors (+23.16%). The code will be released soon.
BibTeX
@misc{zeng2025training,
  author = {Yuhui Zeng and Haoxiang Wu and Wenjie Nie and Guangyao Chen and Xiawu Zheng and Yunhang Shen and Guilin Li and Yixiong Zou and Yonghong Tian and Rongrong Ji},
  title = {{Training-free Anomaly Event Detection via LLM-guided Symbolic Pattern Discovery}},
  year = {2025},
  eprint = {2502.05843},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2502.05843},
}
Figure