Abstract
Multimodal Face Anti-Spoofing (FAS) methods, which integrate multiple visual modalities, often suffer even more severe performance degradation than unimodal FAS when deployed in unseen domains. This is mainly due to two overlooked risks that affect cross-domain multimodal generalization. The first is the modal representation invariant risk, i.e., whether representations remain generalizable under domain shift. We theoretically show that the inherent class asymmetry in FAS (diverse spoofs vs. compact reals) enlarges the upper bound of generalization error, and this effect is further amplified in multimodal settings. The second is the modal synergy invariant risk, where models overfit to domain-specific inter-modal correlations. Such spurious synergy cannot generalize to unseen attacks in target domains, leading to performance drops. To solve these issues, we propose a provable framework, namely Multimodal Representation and Synergy Invariance Learning (RiSe). For representation risk, RiSe introduces Asymmetric Invariant Risk Minimization (AsyIRM), which learns an invariant spherical decision boundary in radial space to fit asymmetric distributions, while preserving domain cues in angular space. For synergy risk, RiSe employs Multimodal Synergy Disentanglement (MMSD), a self-supervised task enhancing intrinsic, generalizable modal features via cross-sample mixing and disentanglement. Theoretical analysis and experiments verify RiSe, which achieves state-of-the-art cross-domain performance.
Introduction: The Multimodal Paradox
Face Anti-Spoofing (FAS) serves as a critical security layer for facial recognition systems, distinguishing between genuine human faces and presentation attacks like high-resolution photos or 3D masks. While incorporating multiple modalities—such as RGB, Depth (DEP), and Infrared (IR)—theoretically provides richer information, a "multimodal paradox" often emerges: these models frequently exhibit poorer generalization to unseen domains than their unimodal counterparts.
This degradation stems from two primary risks identified in this work:
- Modal Representation Invariant Risk: FAS is inherently asymmetric. "Real" faces form a compact, consistent manifold, whereas "spoofs" are infinitely diverse. Standard linear classifiers struggle to maintain a stable boundary across domains due to this asymmetry.
- Modal Synergy Invariant Risk: Models often overfit to "spurious synergies"—coincidental correlations between sensors in the training data (e.g., specific lighting patterns across RGB and IR) that do not hold true for novel attack types in target domains.
The RiSe Framework
To mitigate these risks, we propose Multimodal Representation and Synergy Invariance Learning (RiSe), a framework with two core components.
Asymmetric Invariant Risk Minimization (AsyIRM)
Addressing the representation risk, AsyIRM moves away from traditional symmetric linear hyperplanes. Instead, it projects features into a spherical space where "real" faces are clustered at the center and "spoofs" are distributed externally.
- Radial Space: Captures the invariant classification boundary (real vs. spoof).
- Angular Space: Isolates domain-specific information, preventing environmental noise from corrupting the core classification logic.
Multimodal Synergy Disentanglement (MMSD)
To combat spurious synergy, RiSe introduces MMSD, a self-supervised auxiliary task. MMSD forces the model to learn the intrinsic physical properties of each modality rather than relying on inter-modal "shortcuts."
During training, MMSD randomly mixes frequency components (low vs. high frequency) from different modalities and samples, followed by spatial shuffling. The model must then "decouple" these components and identify their original source. This "synergy-breaking" process ensures the backbone focuses on generalizable, sensor-specific cues.
Experimental Results
RiSe was evaluated across four major multimodal FAS datasets: CASIA-CeFA, PADISI-Face, CASIA-SURF, and WMCA.
- State-of-the-Art Performance: In complete modality protocols, RiSe achieved a Half Total Error Rate (HTER) of 7.70%, nearly halving the error of previous advanced methods (~13-15%).
- Robustness to Missing Modalities: When sensors like Depth or IR were unavailable at test time, RiSe maintained high accuracy (e.g., 13.52% HTER with missing Depth), significantly outperforming other Invariant Risk Minimization (IRM) based methods.
- Data Efficiency: RiSe remained robust even when trained on a limited number of source domains, demonstrating superior generalization capabilities.
Conclusion
RiSe provides a theoretically grounded solution to the multimodal generalization problem in FAS. By addressing class asymmetry through AsyIRM and breaking spurious correlations via MMSD, it establishes a new benchmark for cross-domain reliability. Future work will explore test-time adaptation to further refine performance in real-time deployment scenarios.
Summary updated