Abstract
In autonomous driving and smart city applications, real-time perception must be augmented with logical reasoning to handle complex scenarios like occlusions and sensor noise. This paper presents Semantic Stream Reasoning (SSR), a neural-symbolic framework that combines the perception capabilities of Deep Neural Networks (DNNs) with the structured logic of Answer Set Programming (ASP). By introducing an incremental reasoning algorithm and a parallel learning approach, SSR overcomes the traditional "grounding bottleneck" of symbolic solvers. Experimental results on the KITTI and AI City Challenge datasets show that SSR significantly improves tracking accuracy (MOTA and IDF1) while maintaining real-time throughput of 15–30 FPS.
Introduction
Multi-object tracking (MOT) is a cornerstone of autonomous systems, yet traditional pipelines often rely on manually tuned, brittle heuristics. While symbolic logic offers explainability and flexibility, it has historically struggled with scalability, often failing to meet the 30-FPS requirement of real-time video streams due to the computational overhead of logic grounding. This research addresses this gap by proposing a scalable neural-symbolic approach that treats stream fusion as an incremental reasoning problem.
Methodology
The SSR framework bridges the gap between raw sensor data and high-level reasoning through several key innovations.
- Semantic Stream Reasoning (SSR): The system replaces hard-coded logic with weighted logic rules. These rules, such as temporal consistency of object locations, are expressed within the Linear Analytic Reasoning over Streams (LARS) framework.
- Incremental Reasoning: To achieve real-time performance, the authors developed a "multi-shot" solving algorithm. Instead of re-evaluating the entire state for every frame, the solver reuses previous computations and updates only the changes in the semantic stream.
- Parallel Learning: Rule weights are optimized using Maximum Likelihood Estimation (MLE). A parallel learning algorithm processes data in mini-batches across multiple threads to accelerate the training phase.
Experimental Evaluation
The framework was evaluated on two major benchmarks: the KITTI autonomous driving dataset and the AI City Challenge (AIC) traffic surveillance dataset. These datasets provide diverse scenarios with varying object densities and occlusion levels.
Key Results
SSR demonstrates that symbolic logic can be both accurate and efficient enough for production environments.
- Accuracy: SSR achieved a 10–15% improvement in MOTA and IDF1 scores over the SORT baseline and outperformed DeepSORT (DSORT) by 5–7%.
- Throughput: The system maintained a processing speed of 15–30 FPS. Notably, SSR is 4–6 times faster than DeepSORT, which often struggles with computational latency.
- Efficiency: Logic reasoning accounted for only 41% of total processing time, indicating that the incremental solver successfully mitigated the symbolic grounding bottleneck.
Conclusion
The SSR framework proves that neural-symbolic integration is a viable path for real-time stream fusion. By transforming heuristics into learnable logic rules and optimizing the reasoning process for incremental updates, the authors achieved state-of-the-art performance without sacrificing speed. Future work will focus on reducing the learning time for rule weights and exploring end-to-end training where symbolic feedback directly informs DNN perception models.
Summary updated