The fundamental challenge of motion estimation—whether in 2D (optical flow) or 3D (scene flow)—is the assumption of correspondence. Most algorithms operate on the principle of "brightness constancy": a pixel in Frame A must exist somewhere in Frame B. However, as Maxim Velikanov notes in A neural network approach for occlusion detection in video, this assumption fails the moment an object moves behind another.
When pixels disappear (occlusions) or new ones emerge (disocclusions or "accreted" regions, as described by Adrian Barbu in Motion Estimation by Swendsen-Wang Cuts), the mathematical link between frames is severed. Recent research has shifted from trying to find "missing" matches to inferring hidden motion through global context and structural priors.
The Correspondence Gap
Traditional two-frame methods, such as those analyzed by Ming Ye and Robert M. Haralick in Local Gradient, Global Matching, Piecewise-Smooth Optical Flow, struggle because occluded pixels have no target in the subsequent frame. This creates "ghost" artifacts or "bleeding" motion vectors, where the model incorrectly forces a match onto the background.
In 3D point clouds, this problem is even more acute. Bojun Ouyang and Dan Raviv explain in OGSF-Net that sensors are line-of-sight; when an object moves, it doesn't just change position—it changes which points are sampled. If a point disappears, standard matching algorithms like those used in FLOT by Gilles Puy et al. lose their reference, leading to significant errors in scene flow.
Borrowing Motion: Global Motion Aggregation
The breakthrough in handling these "hidden" motions lies in Global Motion Aggregation (GMA). Introduced by Dylan Campbell and colleagues in Learning to Estimate Hidden Motions with Global Motion Aggregation, GMA uses a transformer-inspired mechanism to "borrow" motion information.
The intuition is simple: if a car is driving to the right, and the front wheel is visible while the back door is occluded by a lamp-post, the back door is almost certainly moving with the same velocity as the wheel. By identifying self-similarities within a single frame, GMA allows the network to propagate motion from visible "anchors" to occluded regions. This moves the task from a local search problem to a global reasoning problem.
Transformers as Motion Anchors
Modern architectures like GMFlow and CRAFT by Xiuchao Sui and Yan Wu replace local convolutional correlations with global attention. This allows the model to look across the entire image to find correspondences, which is particularly effective for large displacements and motion blur.
To refine this, Azin Jahedi and the team behind CCMR utilize "linear" attention to maintain high resolution while performing global reasoning. Similarly, the YOIO framework introduces a "loopback judgment" to verify if a reference point is truly reliable or if it is also occluded, further cleaning the signal in complex scenes.
From Particles to Fields: The Eulerian Shift
While most methods treat motion as a Lagrangian problem—tracking individual "particles" or pixels—a new direction suggests an Eulerian perspective. In Neural Eulerian Scene Flow Fields, Kyle Vedder and Eric Eaton propose fitting a continuous-time velocity field to point clouds.
Instead of asking "where did this specific point go?", the model asks "what is the velocity of the space at this coordinate?". This bypasses the need for explicit point-to-point correspondence, naturally handling occlusions because the velocity field exists even where no points are currently visible.
Structural and Temporal Priors
When global context isn't enough, researchers turn to geometry. Daniel Maurer et al. in Structure-from-Motion-Aware PatchMatch show that if we can reconstruct the rigid 3D scene, we can predict where a pixel should be, even if it's hidden.
Other approaches, like RCP by Xiaodong Gu et al., use recurrent units (GRUs) to maintain a temporal memory of motion. By looking at the history of a point's movement, the system can "coast" through brief occlusions, maintaining a consistent trajectory until the object reappears.