Abstract
We present a method for stabilizing handheld video that simulates the camera motions cinematographers achieve with equipment like tripods, dollies, and Steadicams. We formulate a constrained convex optimization problem minimizing the $\ell_1$-norm of the first three derivatives of the stabilized motion. Our approach extends previous work by solving with full homographies in log-homography space to correct perspective while preserving linearity. Our method is computationally efficient, running at 300fps on an iPhone XS, and yields high-quality results across various challenging video sequences.
Introduction
Capturing professional-looking video with handheld devices is challenging due to inherent camera shake. While professional cinematographers use specialized hardware like tripods for static shots or dollies for smooth pans, casual users rely on software stabilization. Existing 2D stabilization methods often use affine transforms, which fail to correct perspective ("keystone") distortion, leading to visual artifacts. Conversely, 3D methods are often too slow for mobile real-time applications. We propose a 2D method that utilizes full homographies for perspective correction, optimized efficiently in a novel log-homography space.
Methodology: Sparse Motion Optimization
The core of our approach is the observation that professional camera moves are mathematically "sparse." By minimizing the $\ell_1$-norm of the motion's derivatives, we encourage the camera trajectory to "snap" into specific states.
The $\ell_1$ Heuristic
We minimize the $\ell_1$-norm of the first, second, and third derivatives of the camera's path. A zero first derivative corresponds to a static "tripod" shot; a zero second derivative represents a constant-velocity "pan"; and a zero third derivative indicates constant acceleration.
Log-Homography Transform
To handle perspective correction without the non-linearity of standard homographies, we employ Lie theory. By mapping homographies into a "log-space" vector field, complex compositions become simple additions. This allows us to formulate the stabilization as a Quadratic Program (QP), which can be solved efficiently. We further incorporate linearized constraints for cropping and field-of-view preservation using Jacobians.
Results and Evaluation
Our method was evaluated on a benchmark of 174 videos featuring challenges such as parallax, crowds, and quick rotations.
Performance and Stability
The algorithm achieves a processing speed of 300fps on an iPhone XS, making it suitable for real-time mobile use. Compared to standard smoothing techniques, our $\ell_1$ optimization provides a much more stable trajectory that respects the intended motion of the user while removing high-frequency jitter and perspective wobble.
Quality Comparison
By solving for full homographies rather than affine approximations, we effectively eliminate "keystone wobble." Quantitative metrics show our method is competitive with state-of-the-art desktop solutions like Adobe Warp Stabilizer while being significantly faster.
Conclusion
Cinematic-L1 stabilization bridges the gap between fast 2D methods and high-quality 3D stabilization. By leveraging the sparsity of professional camera moves and the mathematical efficiency of log-homography space, we provide a solution that is both high-performance and visually superior for mobile video applications. Future work includes automating weight tuning through machine learning to further refine the "virtual cinematographer" experience.
Summary updated