Abstract
Video Large Language Models (VideoLLMs) have shown remarkable progress, yet they often suffer from "Motion Hallucination"—a phenomenon where models incorrectly identify actions based on static visual cues rather than temporal dynamics. This paper introduces MHBench, a benchmark designed to isolate motion understanding through adversarial video triplets, and MotionCD, a training-free decoding strategy that mitigates these hallucinations by contrasting standard outputs with motion-blurred "amateur" predictions.
Introduction: The Motion Hallucination Problem
While object hallucination in static images is well-documented, the temporal dimension of hallucination in video has been largely overlooked. Current VideoLLMs often exhibit "lazy" reasoning: if a model identifies a person and a book, it may confidently claim the person is "opening the book" even if the video shows the opposite action (closing) or no action at all. This reliance on static visual priors—the appearance of objects rather than their continuous motion—is termed Motion Hallucination (MH).
MHBench: A New Standard for Temporal Reasoning
To systematically evaluate this flaw, the researchers developed MHBench. The benchmark consists of 1,200 videos categorized into 20 action sets. Each set utilizes an "adversarial triplet" structure:
- Original Action: The target motion (e.g., rolling paper).
- Antonym Action: The semantic reverse (e.g., unrolling paper).
- Incomplete/Static: The objects are present, but the motion is absent.
By presenting the same query across these three variations, MHBench effectively tests whether a model is truly tracking temporal changes or merely guessing based on the scene's static components.
MotionCD: Motion Contrastive Decoding
To address these failures without the need for expensive retraining, the authors propose MotionCD. This training-free method utilizes a "Bidirectional Motion Elimination" (BME) technique. By averaging the probability distributions of a video played forward and backward, the model creates an "amateur" baseline where specific motion direction is neutralized while static objects remain distinct. During decoding, the amateur model's output is subtracted from the original model's output, forcing the system to prioritize the unique motion signal over static "distractors."
Evaluation and Results
Testing across 11 state-of-the-art VideoLLMs (including LLaVA-Video and VideoChat2) revealed a widespread inability to distinguish between actions and their antonyms. Most models exhibited a strong "affirmation bias," frequently answering "Yes" to incorrect actions if the relevant objects were visible.
The implementation of MotionCD yielded significant improvements. For instance, the VideoChat2-Mistral model saw a 15.1% boost in Macro F1 score for action classification and a 6.9% increase in average accuracy for binary discrimination.
Conclusion
MHBench exposes a critical weakness in current video-language modeling, where temporal understanding is often bypassed in favor of static object recognition. MotionCD offers a computationally efficient remedy, demonstrating that contrastive decoding can significantly sharpen a model's focus on actual motion. Future research will look toward expanding the benchmark's action categories and integrating motion-awareness more deeply into the pre-training phase.