Understanding Patch-Based Learning by Explaining Predictions

Christopher J. Anders, Grégoire Montavon, Wojciech Samek, Klaus-Robert MüllerarXiv 2018 · arXiv

Abstract

Video classification using 3D Convolutional Neural Networks (CNNs) often relies on "patch-based learning," where models are trained on short temporal snippets rather than full-length videos. This paper investigates the inherent biases introduced by this training strategy using Deep Taylor Decomposition (DTD). Our analysis reveals two distinct phenomena: the "border effect," characterized by an over-reliance on the first and last frames of a snippet, and the "lookahead effect," where models learn to ignore initial video segments. We show that these explainability-driven insights allow for zero-cost performance improvements through optimized frame sampling.

Introduction

The computational intensity of video processing necessitates training deep neural networks on short "snippets" or patches. While efficient, this approach assumes that class-relevant information is uniformly distributed and locally present—an assumption that often fails in practice. This study aims to decode the "black box" of snippet-trained models to understand how this training paradigm distorts a network's temporal focus and how these distortions can be mitigated.

Methodology: Deep Taylor Decomposition

To visualize the network's decision-making process, we employ Deep Taylor Decomposition (DTD), a technique related to Layer-wise Relevance Propagation (LRP). DTD backpropagates the model's final prediction score through the layers to assign "relevance scores" to input pixels.

Unlike standard gradient-based Sensitivity Analysis, which often produces noisy heatmaps, DTD provides a cleaner representation of the features the model considers important. We evaluated these explanations on a C3D network pre-trained on the Sports-1M dataset, comparing the results against both Sensitivity Analysis and an untrained baseline to ensure the observed patterns were learned features rather than architectural artifacts.

Comparison of explanation methods: (a) original data, (b) DTD, (c) Sensitivity Analysis, and (d) DTD on an untrained model. DTD (b) clearly highlights class-relevant features with less noise than Sensitivity Analysis (c).

Identifying Temporal Biases

Our analysis identified two primary biases in the model's temporal attention:

  1. The Border Effect: Quantitative analysis using quadratic regression confirmed that the network disproportionately focuses on the first and last frames of a 16-frame snippet. This "U-shaped" relevance distribution suggests the model is attempting to "look beyond" its limited temporal window to find context missing from the short patch.
  2. The Lookahead Effect: The model learns to assign significantly less relevance to the very beginning of a video file. This likely reflects a learned adaptation to common video structures (e.g., YouTube intros or logos) that contain no action-relevant information.

Explainability-Driven Optimization

Beyond providing diagnostic value, these insights offer a mechanism for performance tuning. By observing the border effect, we hypothesized that increasing the temporal context within a snippet could improve accuracy.

By simply increasing the frame sampling step size from 1 to 2 during the inference phase—effectively doubling the temporal span of a 16-frame snippet—we reduced the border effect's impact. This minor adjustment to the data pipeline improved top-5 classification accuracy on the Sports-1M dataset by approximately 1% (from a 60% baseline) without requiring any model retraining.

Conclusion

This work demonstrates that post-hoc explainability methods like DTD are powerful tools for identifying systematic biases in machine learning models. By uncovering the "border" and "lookahead" effects, we show that understanding why a model predicts can lead directly to actionable improvements in how it performs, paving the way for more efficient hyperparameter optimization in video and time-series analysis.

Summary updated

Abstract

Deep networks are able to learn highly predictive models of video data. Due to video length, a common strategy is to train them on small video snippets. We apply the deep Taylor / LRP technique to understand the deep network's classification decisions, and identify a "border effect": a tendency of the classifier to look mainly at the bordering frames of the input. This effect relates to the step size used to build the video snippet, which we can then tune in order to improve the classifier's accuracy without retraining the model. To our knowledge, this is the the first work to apply the deep Taylor / LRP technique on any video analyzing neural network.
BibTeX
@misc{anders2018understanding,
  author = {Christopher J. Anders and Grégoire Montavon and Wojciech Samek and Klaus-Robert Müller},
  title = {{Understanding Patch-Based Learning by Explaining Predictions}},
  year = {2018},
  eprint = {1806.06926},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1806.06926},
}
Figure