Estimating 3D scene flow—the three-dimensional motion vector of every point in a scene—is a fundamental challenge for autonomous systems. Traditionally, this was framed as a per-point regression problem: for every point in a LiDAR scan or pixel in an RGB-D frame, the model predicts a displacement vector. However, as Andreas Geiger and Moritz Menze observed in Object scene flow for autonomous vehicles, treating points as independent entities ignores the physical reality that the world is composed of solid objects.
When points are allowed to move independently, the resulting "free-form" flow often suffers from fragmentation and geometric distortion. A car might appear to "melt" as its front bumper is assigned a different velocity than its trunk, or a pedestrian might "stretch" unnaturally. Recent research has shifted toward enforcing rigidity constraints, treating the scene as a collection of rigid or articulated parts rather than a fluid cloud of points.
From Point-wise Regression to Piecewise Rigidity
The core intuition behind this shift is that the search space for motion is much smaller than it appears. Instead of predicting $3 \times N$ independent coordinates for $N$ points, we can decompose the scene into a small number of clusters. If a cluster is rigid, its entire motion can be described by just six parameters: a 3D rotation and a 3D translation (an $SE(3)$ transformation).
Or Litany and Tolga Birdal demonstrate in Weakly Supervised Learning of Rigid 3D Scene Flow that this inductive bias acts as a powerful regularizer. By enforcing that all points in a cluster follow a single rigid transformation, the model naturally filters out physically impossible results. This approach is particularly effective for self-supervised learning, where ground-truth labels are absent. In RigidFlow, the authors use "supervoxels" to generate high-quality pseudo-labels, outperforming supervised methods by simply assuming that local clusters must move cohesively.
Resolving Ambiguity with Object-Centric Constraints
Rigidity isn't just a physical constraint; it's an informational one. In sparse LiDAR data or textureless regions of an image, there is often not enough local information to determine how a specific point is moving.
By grouping points into object candidates—as seen in SeFlow by Qingwen Zhang and Olov Andersson—the system can "borrow" motion information from the most certain parts of an object. If the wheels of a car are clearly moving, the rigidity constraint dictates that the featureless car door must be moving with them. This mechanism, also explored in ICP-Flow, prevents the erratic, non-physical flow predictions common in unconstrained estimators.
Articulation and Kinematic Chains
While many objects in driving scenes are rigid (cars, poles, buildings), others are articulated. A human walking or a pair of scissors opening consists of multiple rigid parts connected by joints.
Chen Wang and Sebastian Scherer show in AirDOS: Dynamic SLAM benefits from Articulated Objects that modeling these connections allows dynamic objects to be used as landmarks rather than noise. Similarly, REArtGS and MonoMobility extend this to 3D Gaussian Splatting, using kinematic constraints (like hinges or sliders) to ensure that reconstructed objects move in physically valid ways. By parameterizing motion through these specific degrees of freedom, models can interpolate complex movements from very sparse observations.
Distinguishing Motion from Perspective
A major hurdle in scene flow is distinguishing between "true" independent motion (a car driving) and "apparent" motion caused by the camera's own movement (parallax).
Cameron Omid Smith and Vincent Sitzmann address this in SIRE: SE(3) Intrinsic Rigidity Embeddings. By lifting 2D tracks into 3D and enforcing rigid-body constraints, their model can identify which points are moving together as a 3D entity versus which are merely shifting due to perspective. This geometric regularization allows for cleaner object segmentation in dynamic scenes compared to purely 2D optical flow methods like SAMFlow, which relies on high-level segmentation priors to prevent fragmentation.
The Intuition of Structural Integrity
The common thread across these 111 papers is that structure informs motion. Whether through Continuous High-order CRFs in HCRF-Flow or surface-aware smoothness in Regularizing Self-supervised 3D Scene Flows, the goal is to maintain the structural integrity of the world.
By treating the scene as a collection of rigid parts, researchers have moved from a "cloud of points" view to an "object-centric" view. This not only makes the math more tractable by reducing the degrees of freedom but also aligns the AI’s internal representation with the fundamental laws of physics: objects do not warp, they move.