Low Pass Filter for Anti-aliasing in Temporal Action Localization

Cece Jin, Yuanqi Chen, Ge Li, Tao Zhang, Thomas H. LiarXiv 2021 · arXiv

Introduction

Temporal Action Localization (TAL) is a critical task in computer vision, requiring the precise identification of action boundaries within long, untrimmed videos. To manage the high computational cost of processing video data, researchers frequently employ temporal downsampling techniques, such as pooling or frame skipping. However, standard downsampling often ignores the fundamental Nyquist-Shannon sampling theorem, leading to "aliasing" errors. Aliasing occurs when high-frequency information "folds over" into lower frequencies, distorting the underlying features and potentially making distinct actions indistinguishable.

Figure 1: Comparison of signal amplitudes showing how a Low Pass Filter (LPF) helps the sampled signal more closely match the original input by reducing aliasing artifacts.

The Aliasing Problem in Video Models

In the context of TAL, aliasing manifests as the loss of fine-grained temporal details or the introduction of noise that muddies feature representations. While anti-aliasing is a standard procedure in classical signal processing and 2D image resizing, its application to the 1D temporal dimension in video AI has been largely overlooked. Without proper filtering, the downsampled features used for action boundary prediction may contain artifacts that hinder the model's ability to localize actions accurately.

Proposed Methodology: Dynamic Low Pass Filtering

To mitigate aliasing, this research proposes integrating a Low Pass Filter (LPF) before any downsampling operation. The core of the approach is the use of a filter based on the Sinc function, the ideal mathematical tool for frequency cutoff in the time domain.

Figure 6: Frequency domain representation of down-sampling. (a) Without anti-aliasing, high-frequency components overlap (aliasing). (b) With a low-pass filter, the signal is band-limited before sampling, preventing overlap.

A key innovation is the introduction of a Dynamic Cutoff Frequency ($f_C$). Recognizing that different actions possess varying motion complexities—for instance, a "Basketball Dunk" contains higher frequency components than a "Discus Throw"—the authors utilize a small sub-network to learn an optimal $f_C$ for each video proposal. This allows the model to strike a balance between removing aliasing noise and preserving essential high-frequency details.

The LPF is designed to be modular and can be integrated into three common operations:

  1. Pooling: Replacing standard average or max pooling with an LPF followed by pooling.
  2. Frame Sampling: Filtering the temporal sequence before selecting every $k$-th frame.
  3. Convolutional Layers: Implementing a "Truncated Low Pass Mixture" (TLPM) that functions as an anti-aliased convolution.

Experimental Results

The proposed method was evaluated across several benchmarks, including THUMOS’14, ActivityNet 1.3, Charades, and Kinetics 400.

  • THUMOS’14: The "LPF + Average Pooling" configuration yielded a significant improvement of 8.02% [email protected] over naive average pooling. The full model achieved a state-of-the-art 52.6% mAP.
  • ActivityNet: Integration into the Boundary-Matching Network (BMN) improved the Area Under the Curve (AUC) from 67.10% to 67.53%.
  • Charades: The anti-aliased convolution (TLPM) significantly boosted per-frame mAP from 18.4% to 23.1%.

Qualitative analysis via t-SNE visualizations confirmed that features processed with LPF are more linearly separable, indicating that anti-aliasing produces cleaner, more discriminative representations.

Conclusion

By addressing the neglected issue of temporal aliasing, this work demonstrates that classical signal processing principles can significantly enhance modern deep learning models for video analysis. The use of dynamic cutoff frequencies ensures that the model remains flexible across diverse action types. Future work may involve refining the truncation of Sinc filters for better efficiency and exploring their application within transformer-based architectures.

Summary updated

BibTeX
@misc{jin2021low,
  author = {Cece Jin and Yuanqi Chen and Ge Li and Tao Zhang and Thomas H. Li},
  title = {{Low Pass Filter for Anti-aliasing in Temporal Action Localization}},
  year = {2021},
  eprint = {2104.11403},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2104.11403},
}
Figure