Vid2Int: Detecting Implicit Intention from Long Dialog Videos

Xiaoli Xu, Yao Lu, Zhiwu Lu, Tao XiangWACV 2021 · DOI

Introduction

Implicit Intention Detection (IID) is a critical task in computer vision with applications ranging from legal proceedings to financial risk assessment. Unlike standard action recognition, IID requires identifying subtle behavioral cues that reveal a subject's true meaning or deceptive intent. Traditional models often treat video as a single snapshot, failing to capture the complex "web of lies" that evolves over a long conversation. Researchers from Renmin University and the University of Surrey have introduced Vid2Int, a framework designed to detect these shifts in intention across multi-turn dialogs by analyzing fine-grained temporal correlations.

Methodology

The Vid2Int framework moves beyond holistic video analysis to detect deception at a segment-by-segment level. It consists of three primary components: Voice Activity Detection (VAD), a Multi-Grain Representation Module (MGRM), and a Temporal Context Module (TCM).

Detailed architecture of the Vid2Int framework, illustrating the flow from raw dialog video to segment-level and overall reports.

Multi-Grain Representation Module (MGRM)

Instead of relying on raw RGB frames, Vid2Int utilizes optical flow to capture micro-expressions and subtle movements. The MGRM extracts features at three levels of granularity: the eyes, the face, and the entire body. These features are fused using a progressive strategy where coarse-grained body features are combined with finer facial and ocular details through $1 \times 1$ convolutions, preserving critical motion cues while managing dimensionality.

Temporal Context Module (TCM)

To model how intentions evolve, the TCM employs an attentive Long Short-Term Memory (LSTM) network. A key innovation is the use of self-supervised attention. The model weights segments based on confidence; for instance, if a segment is clearly "truthful," it receives less weight when determining the overall video's deceptiveness, allowing the network to focus on the most suspicious moments.

The Vid2Int-Deception Dataset

To support this research, the authors introduced the Vid2Int-Deception benchmark. This dataset contains 292 long dialog videos of 73 volunteers, totaling 1,460 segments. Unlike previous datasets, every question-answer pair is individually labeled. Ground truth was established using a polygraph during data collection to ensure high-quality annotations.

Key Insights and Results

A central finding of the study is that intentions are highly context-dependent. The researchers observed significant temporal correlations between different parts of a dialog.

Temporal correlation matrix demonstrating how intentions in specific answer segments relate to others within the same dialog.

As shown in the correlation matrix, certain questions are "strongly correlated." If a subject provides a deceptive answer to one question, they are statistically more likely to lie in subsequent related segments to maintain consistency.

Vid2Int achieved state-of-the-art (SOTA) performance:

  • Vid2Int-Deception: The model achieved 76.53% accuracy at the segment level and 92.31% accuracy at the full dialog level.
  • Real-life Court Video Dataset: On this real-world benchmark, Vid2Int reached 97.84% accuracy and a 99.83% AUC, outperforming previous models that relied on complex multi-modal fusions.

Conclusion

Vid2Int demonstrates that fine-grained visual analysis combined with temporal context is essential for detecting implicit intentions. By focusing on optical flow across multiple grains and modeling the dependencies between conversation segments, the framework provides a robust solution for long-form deception detection. Future work aims to extend these capabilities to broader subtext detection beyond binary deception.

Summary updated

BibTeX
@inproceedings{xu2021vid,
  author = {Xiaoli Xu and Yao Lu and Zhiwu Lu and Tao Xiang},
  title = {{Vid2Int: Detecting Implicit Intention from Long Dialog Videos}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2021},
  doi = {10.1109/wacv48630.2021.00334},
}
Figure