Abstract: We present a method for stabilizing handheld video that simulates the camera motions cinematographers achieve with equipment like tripods, dollies, and Steadicams. We formulate a constrained convex optimization problem minimizing the $\ell_1$-norm of the first three derivatives of the stabilized motion. Our approach extends the work of Grundmann et al. [9] by solving with full homographies (rather than affinities) in order to correct perspective, preserving linearity by working in log-homography space. We also construct crop constraints that preserve field-of-view; model the problem as a quadratic (rather than linear) program to allow for an $\ell_2$ term encouraging fidelity to the original trajectory; and add constraints and objectives to reduce distortion. Furthermore, we propose new methods for handling salient objects via both inclusion constraints and centering objectives. Finally, we describe a windowing strategy to approximate the solution in linear time and bounded memory. Our method is computationally efficient, running at 300fps on an iPhone XS, and yields high-quality results, as we demonstrate with a collection of stabilized videos, quantitative and qualitative comparisons to [9] and other methods, and an ablation study.
Introduction
Most handheld smartphone video is inherently shaky, lacking the intentionality of professional cinematography. While professionals use heavy physical stabilizers like dollies and gimbals to achieve smooth pans or static "tripod" shots, digital software stabilization aims to replicate these effects algorithmically.
Existing digital solutions typically fall into two categories. 2D or 2.5D methods are computationally fast but often rely on affine transforms (translation, rotation, scale), which fail to correct perspective distortion, leading to a visible "wobble." Conversely, 3D methods using Structure-from-Motion (SfM) provide high-quality reconstruction but are too slow for mobile devices and often fail in scenes with moving objects or low texture. This research introduces a middle ground: a method that is as robust and fast as 2D techniques but utilizes full homographies to correct perspective while ensuring the resulting motion feels professional and "cinematic."
The Log-Homography Framework
The core innovation of this work is the Log-Homography Model. Standard homographies (3x3 matrices) are non-linear, which makes them difficult to integrate into convex optimization problems. By applying Lie theory, the authors map these matrices into a "log-space" vector space. In this space, the complex operation of composing transforms is simplified into vector addition, allowing the stabilization problem to remain convex and solvable via efficient quadratic programming.
To achieve a cinematic feel, the framework minimizes the $\ell_1$-norm of the first three derivatives of the camera's path. Because the $\ell_1$-norm encourages sparsity, the solution naturally produces segments where derivatives are exactly zero. As illustrated in the figure above, this results in distinct camera states: "tripod" segments (zero velocity), "constant pan" segments (zero acceleration), and "smooth transition" segments (zero jerk).
Constraints and Saliency
The model balances motion smoothing with several competing physical constraints:
- Crop Constraints: Ensures the stabilized path does not reveal empty black bars at the edges of the frame.
- Fidelity Terms: An $\ell_2$ term prevents the stabilized path from drifting too far from the user's original intent.
- Distortion Mitigation: Prevents the image from appearing "rubbery" or overly warped.
- Saliency Handling: The system can identify salient regions (like faces) and add constraints to ensure these objects are never cropped out or remain centered in the frame.
To handle long videos on mobile hardware, the authors employ an overlapping window strategy. By exploiting a "Markov property" where the future state only depends on a short history of frames, they approximate a global solution in linear time with bounded memory.
Evaluation and Results
The method was evaluated using the standard benchmark dataset from Liu et al., containing 174 videos across various motion types (e.g., parallax, crowd, running). It was compared against industry standards including Grundmann et al.’s original L1 stabilizer, Bundled Camera Paths (BCP), and Adobe Warp Stabilizer.
As shown in the comparison plots, the L1 optimization creates a "sticky" camera effect. Unlike standard Gaussian smoothing, which merely blurs shakes, the L1 approach forces the virtual camera to "lock" onto static positions or constant velocities, mimicking the physical friction of professional gear.
Quantitatively, the "Cinematic-L1" method achieved a processing speed of 300fps on an iPhone XS. In terms of stability and distortion metrics, it proved highly competitive with high-end tools like Adobe Warp Stabilizer while significantly outperforming previous affine-based L1 methods in removing perspective wobble.
Conclusion and Future Work
By linearizing homography composition through matrix logarithms, this framework brings high-quality perspective correction to the domain of fast, convex optimization. While the method is highly effective, it remains a 2D/2.5D approach and may struggle with extreme parallax where foreground and background move at vastly different speeds. Future work may involve using machine learning to automatically tune optimization weights, allowing the stabilizer to adapt its strength based on specific scene content.
Summary updated