SRFlow: A Dataset and Regularization Model for High-Resolution Facial Optical Flow via Splatting Rasterization

Jialin Zhang, Dong Li

TL;DR

This paper introduces SRFlow, a high-resolution facial optical flow dataset generated using 3D Gaussian Splatting, and SRFlowNet, a model featuring novel regularization losses designed to handle the unique challenges of high-resolution skin motion. By training on this specialized data and applying tailored constraints, the authors achieved a 42% reduction in flow error and a massive 48% improvement in downstream micro-expression recognition.

What Problem Does This Paper Address?

Optical flow—the pattern of apparent motion of objects in a visual scene—is a cornerstone of facial motion analysis. However, the field faces a "data bottleneck." Most optical flow datasets (like MPI Sintel or FlyingChairs) feature rigid objects or cartoonish avatars that don't capture the subtle, non-rigid deformations of human skin.

Furthermore, as we move toward high-resolution imagery, new problems emerge. High-res data introduces significant high-frequency noise and makes "texture-less" regions (like a smooth forehead or cheek) even harder for algorithms to track. Standard models often produce "spurious motion" in these areas, which ruins the accuracy of sensitive tasks like micro-expression recognition, where the signal-to-noise ratio is already incredibly low.

Methods

The authors tackle this from two angles: data generation and model regularization.

1. The Flow Rasterizer

To create the SRFlow dataset, the researchers modified the 3D Gaussian Splatting (3DGS) pipeline. Instead of just rendering RGB pixels, they developed a Flow Rasterizer that tracks the movement of 3D Gaussian primitives between frames. By projecting these 3D displacements onto a 2D image plane, they generated dense, sub-pixel accurate ground truth flow for complex facial movements.

2. SRFlowNet and Regularization

The authors built SRFlowNet using the SKFlow architecture as a backbone but added four lightweight regularization losses to stabilize training:

  • Total Variation Regularization (TVR): Uses Sobel operators to enforce smoothness.
  • Flow Difference Regularization (FDR): Uses simple pixel differences and background masks to focus on valid motion regions.
  • Mean Image Gradient Activation (MIGAR): An adaptive weighting scheme that constrains flow structure based on the first frame's gradients.
  • Image Gradient Variance Activation (IGVAR): A more robust version of MIGAR that uses the variance of gradients to emphasize structural differences within facial regions.

Datasets

  • SRFlow (New): A high-resolution dataset featuring 27 subjects, 11,161 frames, and a resolution of 2200 × 3208. It is derived from the NeRSemble dataset using the authors' 3DGS-based flow rasterizer.
  • Micro-expression Benchmarks: For downstream evaluation, they used SAMM, CASME II, and SMIC.
  • Comparison Datasets: The paper compares SRFlow against MPI Sintel, Spring, and FacialFlowNet.

Key Insights and Ideas

  • 3DGS as a Labeling Tool: Using dynamic 3D representations to generate 2D labels is a powerful way to bypass the "impossible" task of manually annotating pixel-level motion on human skin.
  • The Sobel Advantage: The authors found that Sobel-based regularization (which looks at diagonal directions) preserves subtle motion patterns better than simple axis-aligned differences, which tend to "over-smooth" and discard fine details.
  • Domain Specificity Matters: General-purpose optical flow models (like RAFT or GMA) struggle with faces because they aren't trained on human structural priors. Retraining these models on the SRFlow dataset consistently improved their performance across the board.
  • High-Res is Hard: The paper highlights that simply increasing resolution doesn't help if your loss functions can't handle the resulting noise in texture-less regions.

Key Results

  • Flow Accuracy: Training existing models on SRFlow reduced the End-Point Error (EPE) by up to 42% (from 0.5081 to 0.2953).
  • Micro-expression Recognition: When using SRFlowNet features, the Macro F1-score on a composite micro-expression dataset jumped from 0.4733 to 0.6947—a nearly 48% improvement.
  • Qualitative Superiority: Visualizations show that SRFlowNet produces much cleaner and more coherent flow fields in difficult areas like the chin and forehead, whereas traditional methods produce "blocky" or "hole-filled" estimates.

Limitations and Future Work

The authors acknowledge a few areas for growth:

  • Loss Stacking: They tested their four regularization losses individually to maintain interpretability. They suggest that combining them might further improve results but carries the risk of "over-suppressing" subtle motions.
  • Resolution Mismatch: While the flow is trained on high-res data, most current micro-expression recognition networks are designed for low-res inputs.
  • Future Direction: The authors aim to create even larger high-resolution micro-expression datasets and develop end-to-end recognition networks that can fully exploit the high-fidelity flow information provided by SRFlowNet.

Summary updated

Abstract

Facial optical flow supports a wide range of tasks in facial motion analysis. However, the lack of high-resolution facial optical flow datasets has hindered progress in this area. In this paper, we introduce Splatting Rasterization Flow (SRFlow), a high-resolution facial optical flow dataset, and Splatting Rasterization Guided FlowNet (SRFlowNet), a facial optical flow model with tailored regularization losses. These losses constrain flow predictions using masks and gradients computed via difference or Sobel operator. This effectively suppresses high-frequency noise and large-scale errors in texture-less or repetitive-pattern regions, enabling SRFlowNet to be the first model explicitly capable of capturing high-resolution skin motion guided by Gaussian splatting rasterization. Experiments show that training with the SRFlow dataset improves facial optical flow estimation across various optical flow models, reducing end-point error (EPE) by up to 42% (from 0.5081 to 0.2953). Furthermore, when coupled with the SRFlow dataset, SRFlowNet achieves up to a 48% improvement in F1-score (from 0.4733 to 0.6947) on a composite of three micro-expression datasets. These results demonstrate the value of advancing both facial optical flow estimation and micro-expression recognition.
BibTeX
@misc{zhang2026srflow,
  author = {Jialin Zhang and Dong Li},
  title = {{SRFlow: A Dataset and Regularization Model for High-Resolution Facial Optical Flow via Splatting Rasterization}},
  year = {2026},
  eprint = {2601.06479},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2601.06479},
}
Figure