Speech-driven 3D facial animation turns an audio clip into a moving face mesh. Its hardest frames are the bilabial consonants /p/, /b/ and /m/, where the lips must actually meet. Current systems judge and enforce that meeting by comparing particular upper-lip and lower-lip vertices on one fixed template mesh. This note argues that such a contact test has the same flaw computational physicists found in grid-based simulations of chemical reactions. In the reaction-diffusion master equation, a stochastic model that splits space into small boxes ("voxels") and lets two molecules react only when they sit in the same box, refining the grid past the molecules' physical reaction radius makes reactions disappear and pushes derived rates negative (Reaction-diffusion master equation in the microscopic limit; Reaction rates for mesoscopic reaction-diffusion kinetics). The fix in that field was to let neighboring boxes interact, with rates chosen so that coarse-grid timing statistics match a continuous ground truth (Reaction rates for a generalized reaction-diffusion master equation). We propose to bring both the diagnosis and the fix to lip contact.
The speaker-variability angle is what makes this matter in practice. Speakers differ in lip thickness and mouth size. A single template with a fixed vertex budget therefore samples some speakers' lips more coarsely than others, relative to their own anatomy. Game-ready and stylized meshes are often far coarser than the 5023-vertex FLAME template. If a resolution floor exists, it should be stated in units of each speaker's own contact scale, the way reaction-diffusion floors are stated in units of the reaction radius. A contact loss calibrated in those units should then hold across new speakers and new mesh densities. A loss defined by vertex identity should not.
Core Claim
We make two claims. The first is diagnostic. Bilabial closure fidelity in speech-driven animation has a measurable resolution floor. It should be expressed as the ratio $h/\sigma$, where $h$ is the mean edge length of the mesh in the lip region and $\sigma$ is a speaker-specific contact scale, namely the midline thickness of the lips measured on that speaker's neutral mesh. We predict that floors measured in absolute vertex counts scatter across speakers, but collapse onto one curve when plotted against $h/\sigma$.
The second claim is constructive. A contact operator that works between neighboring surface regions, whose weights are calibrated by matching closure timing statistics against a dense reference, keeps lip closure correct down to that floor and below it. A per-vertex contact loss on a fixed template, as used by Imitator: Personalized Speech-driven 3D Facial Animation, degrades instead. The operator is
$$ C_h(t)=\sum_{i\in U}\sum_{j\in L}\Big[k_0(h)\,\mathbf{1}\{d_{ij}(t)<h\}+k_1(h)\,\mathbf{1}\{h\le d_{ij}(t)<2h\}\Big], $$
where $U$ and $L$ are the upper- and lower-lip surface samples, $d_{ij}(t)$ is the distance between samples $i$ and $j$ at frame $t$, and $k_0,k_1$ are the near-contact and neighbor-contact weights. In practice the indicator functions are replaced by smooth versions so the loss is differentiable. As in the generalized reaction-diffusion master equation, the neighbor weight $k_1$ should shrink toward zero on coarse meshes and grow as $h$ approaches $\sigma$. Plain local contact is recovered when the mesh is coarse.
What Prior Work Does and Leaves Open
Three strands of speech-to-face work motivate the problem, and all of them hold mesh resolution fixed. Capture, Learning, and Synthesis of 3D Speaking Styles. (VOCA) conditions on subject labels across the 12 speakers of VOCASET. FaceFormer: Speech-Driven 3D Facial Animation with Transformers regresses vertex offsets from wav2vec 2.0 audio features, using biased attention for long sequences. CodeTalker: Speech-Driven 3D Facial Animation with Discrete Motion Prior replaces that regression with a learned codebook of motion primitives to avoid over-smoothed output. None of them changes vertex density.
Imitator: Personalized Speech-driven 3D Facial Animation is the closest method. It adds a lip contact loss during bilabials and reports a lip distance of 0.09 versus 0.14 for FaceFormer. However, its loss is tied to FLAME's vertex layout. Our proposal keeps Imitator's insight that contact must be supervised explicitly, and replaces its vertex-identity definition with a resolution-calibrated one.
RigAnyFace: Scaling Neural Facial Mesh Auto-Rigging with Unlabeled Data supplies one component, a backbone that does not depend on the mesh's triangulation (DiffusionNet, which spreads features over the surface with its intrinsic Laplacian). It also supplies one constraint: such backbones need a global encoder to link disconnected parts and remain sensitive at extreme discretizations. RigAnyFace produces static expression rigs, not speech-driven motion over time.
The method itself comes from the reaction-diffusion line. That work matched mean association and rebinding times to Green's-function ground truth, pushing the floor from about $3.2\sigma$ to about $1.08\sigma$ in 3D. It also showed that how dissociation is handled is essential (Reaction rates for a generalized reaction-diffusion master equation). For lips, the analogues are closure onset (association) and closure release (dissociation). The 8–16 points-per-body floor found in Low-resolution simulations of vesicle suspensions in 2D suggests that boundary contact between deformable bodies, not only point reactions, has such floors.
Research Program
Resolution-transferred ground truth. We build a resolution ladder from VOCASET by remeshing the lip region of every registered FLAME sequence. The ladder runs from heavy decimation up to subdivision denser than FLAME. Ground-truth motion is transferred by barycentric projection, which gives exact supervision at every density. BIWI, with its own native registration, serves as a second, independent topology.
Diagnostic study. For each speaker we compute $\sigma$ from the neutral mesh. We then measure closure statistics of the ground truth itself at each rung of the ladder. This separates the floor of the representation from the floor of any model.
Calibration. We fit $k_0(h)$ and $k_1(h)$ by matching three quantities on ground-truth sequences against the densest rung: mean bilabial closure duration, onset time, and release time. The fitted values are parameterized as functions of $h/\sigma$, so they transfer to held-out speakers without refitting.
Training. We use the calibrated $C_h$ as a contact loss during force-aligned bilabial frames, plus a penetration penalty in all other frames. The loss is added to two decoders. One is a FLAME-native model retrained per density on the transferred data. The other is a DiffusionNet decoder with a global encoder, conditioned on wav2vec 2.0 features, trained across all rungs at once.
Baselines
Naive reference. The do-nothing reference is FaceFormer trained on native FLAME, with its output transferred to each rung by barycentric projection. This is what a practitioner gets today without adding any contact supervision.
Strongest existing competitors. The strongest competitors are CodeTalker, because it has the best reported lip accuracy on BIWI (lip vertex error 4.7914 in CodeTalker, against 5.3077 for FaceFormer in the same paper), and Imitator's lip contact loss, because it is the only existing explicit treatment of closure (0.09 lip distance on VOCASET). We will run both on the same speaker splits, the same transferred data, and the same audio features. Each gets the same hyperparameter search budget as our method.
What counts as beating them. Lip vertex error is a maximum over vertices, so its value changes with vertex count. We therefore treat it as comparable only within a rung, and add a resolution-independent surface-distance version. Beating the competitors means matching CodeTalker's lip error at FLAME density, which shows we cost nothing there. It also means showing significantly higher bilabial closure recall and lower timing error on held-out speakers at rungs below FLAME density, and at the subdivided rungs.
Ablations that isolate the key ingredient. Two ablations isolate the calibration. The first is a fixed-millimeter contact tolerance with no calibration and no speaker normalization. The second calibrates in absolute $h$ rather than $h/\sigma$.
Why not just…?
Why not just register every mesh to FLAME and run an existing model? On our remeshed VOCASET this is a real threat. Mapping back to FLAME is nearly lossless there, so the remeshed test cannot show any advantage from topology flexibility. That is why the claim rests on contact behavior across densities and speakers, not on architecture. Registration cannot add detail beyond FLAME's density, and it cannot rescue meshes coarser than FLAME. On a coarse target, FLAME's correct closure is projected onto a polygonal seam line that may not close. The coarse rungs test exactly this case. RigAnyFace's 2.77mm versus 1.01mm gap for template-dependent rigging suggests registration struggles on real non-template heads. That evidence comes from static rigs, however, and we do not assume it transfers to speech.
Why not just use a fixed physical tolerance, say lips closer than 1mm? This is the simplest competitor and it might suffice. The fixed-millimeter ablation settles it. If closure recall and timing stay flat across rungs and speakers under that tolerance, calibration is unnecessary and we will report the floor study as the main result. The reaction-diffusion results predict otherwise: uncalibrated neighbor interaction over-counts contact on coarse grids and under-counts it near the floor.
Why not just refine the mesh? Refinement makes Imitator-style vertex pairing more arbitrary, because more candidate pairs exist and the dense lip contour must align exactly. In reaction-diffusion, refinement past the floor increases error rather than reducing it. The subdivided rungs test whether the same happens here.
Why not rely on self-supervised audio features plus speaker embeddings? Imitator's own comparison shows wav2vec 2.0 regression alone leaves lips open (0.14 versus 0.09 lip distance). The FaceFormer reference measures this at every rung.
Decisive Experiments
Does a floor exist? The first check is whether ground-truth closure recall itself drops sharply below some $h/\sigma$.
Does the floor collapse across speakers? The second check is whether held-out speakers' floor curves collapse in $h/\sigma$ but scatter in vertex count. If they do not collapse, the speaker-normalized story fails.
Does calibration help? The third check is whether calibrated $C_h$ keeps closure recall, onset and release timing error, and penetration rate flat across the ladder, while the per-vertex and fixed-tolerance losses diverge.
Does closure generalize across topologies? We train on VOCASET rungs and test on BIWI to see whether closure behavior holds on an independently registered topology. Upper-face dynamics are tracked as a sanity check that the contact loss does not damp motion elsewhere.
What the outcome means. A clean negative on the second check, or a flat fixed-tolerance result on the third, would itself be a useful finding. It would show that lip contact, unlike molecular reaction, needs no scale matching.
References
- Reaction-diffusion master equation in the microscopic limit
- Reaction rates for mesoscopic reaction-diffusion kinetics
- Reaction rates for a generalized reaction-diffusion master equation
- Imitator: Personalized Speech-driven 3D Facial Animation
- Capture, Learning, and Synthesis of 3D Speaking Styles.
- FaceFormer: Speech-Driven 3D Facial Animation with Transformers
- CodeTalker: Speech-Driven 3D Facial Animation with Discrete Motion Prior
- RigAnyFace: Scaling Neural Facial Mesh Auto-Rigging with Unlabeled Data
- Low-resolution simulations of vesicle suspensions in 2D