Hidden Follower Detection: How Is the Gaze-Spacing Pattern Embodied in Frequency Domain?

Shu Li, Ruimin Hu, Suhui Li, Liang Liao

Abstract

Detecting "hidden followers"—individuals who secretly track a target—is a critical task for preventing crime, yet it remains difficult because their behavior often mimics that of normal pedestrians. Traditional detection methods rely on time-domain summaries of gaze and spacing, which often discard subtle behavioral nuances. This paper proposes the HFDF-TF framework, which leverages frequency-domain analysis via Mel-Frequency Cepstral Coefficients (MFCCs) to capture the "rhythm" of following behavior. We demonstrate that frequency-domain features maintain higher information fidelity, leading to a direct improvement in detection accuracy. Our model achieves a state-of-the-art F1 score of 94.5% on real-world datasets, with a notable 10.1% jump in gaze-based detection performance.

Introduction

Hidden following is a subtle abnormal behavior that often precedes violent crimes. Unlike overt actions like running or fighting, a hidden follower attempts to blend in, making detection by standard computer vision systems extremely challenging. Previous research has focused on time-domain features like gaze frequency and average distance. However, these methods suffer from irreversibility—the inability to reconstruct the original signal from the summary—and one-sidedness, failing to capture the complex interaction between the follower and the target.

Methodology: The HFDF-TF Framework

The authors introduce the Hidden Follower Detection based on Time-Frequency transform (HFDF-TF) framework. This approach shifts the focus from simple time-domain statistics to the frequency domain. Key components include:

  • Fidelity Estimation: Using Signal-to-Distortion Ratio (SDR), the authors prove that frequency-domain features (like MFCCs) retain significantly more information about the original behavior than time-domain summaries.
  • Feature Extraction: MFCCs, commonly used in speech processing, are extracted from continuous gaze degree scores and spacing time series. This captures the rhythmic patterns of the behavior.
  • Decision Fusion: Two independent Time Warping Invariant Echo State Network (TWIESN) classifiers process gaze and spacing data separately. Their results are fused using Dempster-Shafer (DS) evidence theory to determine the final probability of a hidden follower.

The Fidelity-Performance Correlation

A key finding of this research is the strong positive correlation between feature fidelity and detection performance. As the fidelity of the extracted features increases (measured by SDR), the F1 score of the detection model also improves. This suggests that the "essence" of following behavior is contained in the high-frequency components that time-domain methods typically ignore.

Correlation between feature fidelity (SDR) and F1 score for gaze and spacing MFCCs

Frequency Signatures of Following Behavior

Hidden followers exhibit unique frequency patterns. They must balance the need to keep the target in sight with the need to avoid being noticed. This conflict manifests as specific high-frequency aggregations in gaze and spacing signals. For instance, while a normal pedestrian's gaze might be random or steady, a follower's gaze has a distinct "rhythm" of checking and looking away, which is clearly visible in the frequency domain.

Comparison of frequency domain power distributions between hidden following and normal walking pairs

Experimental Results

The model was evaluated on two datasets: Real-HFD (real-world video) and Sim-HFD (simulated data).

  • Overall Performance: The HFDF-TF model achieved a 94.5% F1 score on Real-HFD, a 3% improvement over previous state-of-the-art methods.
  • Gaze Breakthrough: The switch to frequency-domain analysis resulted in a 10.1% improvement in the gaze module's accuracy alone.
  • Robustness: High performance across both real and simulated data indicates that these frequency-domain patterns are universal indicators of hidden following behavior.

Conclusion

By moving from the time domain to the frequency domain, this research provides a more robust and informative way to detect hidden followers. The use of MFCCs and the emphasis on feature fidelity offer a new path forward for identifying subtle abnormal behaviors in surveillance, potentially preventing crimes before they occur.

Summary updated

Abstract

Spatiotemporal social behavior analysis is a technique that studies the social behavior patterns of objects and estimates their risks based on their trajectories. In social public scenarios such as train stations, hidden following behavior has become one of the most challenging issues due to its probability of evolving into violent events, which is more than 25%. In recent years, research on hidden following detection (HFD) has focused on differences in time series between hidden followers and normal pedestrians under two temporal characteristics: gaze and spatial distance. However, the time-domain representation for time series is irreversible and usually causes the loss of critical information. In this paper, we deeply study the expression efficiency of time/frequency domain features of time series, by exploring the recovery mechanism of features to source time series, we establish a fidelity estimation method for feature expression and a selection model for frequency-domain features based on the signal-to-distortion ratio (SDR). Experimental results demonstrate the feature fidelity of time series and HFD performance are positively correlated, and the fidelity of frequency-domain features and HFD performance are significantly better than the time-domain features. On both real and simulated datasets, the accuracy of the proposed method is increased by 3%, and the gaze-only module is improved by 10%. Related research has explored new methods for optimal feature selection based on fidelity, new patterns for efficient feature expression of hidden following behavior, and the mechanism of multimodal collaborative identification.
BibTeX
@inproceedings{li2024hidden,
  author = {Shu Li and Ruimin Hu and Suhui Li and Liang Liao},
  title = {{Hidden Follower Detection: How Is the Gaze-Spacing Pattern Embodied in Frequency Domain?}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i1.27809},
}
Figure