KAN-Face: Efficient Resource Usage and Precision Lip-Sync in Talking Head Generation
Lists
Only you see this
Abstract
Despite significant progress in NeRF-based talking head generation, problems like poor lip synchronization and inefficient resource usage remain. To solve these, we propose KANFace, a lightweight framework. In preprocessing, we introduce a Lip-Sync Enhancement Module that uses Wav2Lip to extract high-resolution audio features and map them to an explicit intermediate representation, ensuring precise lip movement alignment with the speaker’s identity. These predicted lip-sync features are combined with fundamental audio-extracted lip features and injected into the rendering module to improve synchronization. For rendering, we introduce FastKAN to map spatial points to color values. As a variant of KAN, FastKAN’s sensitivity to 3D scenes and efficient structure enable precise, fast color prediction. Our framework reduces resource consumption while enhancing lip-sync accuracy and facial reconstruction, making it ideal for talking head generation tasks in resource-limited settings. Project: https://peterfanfan.github.io/KAN-Face/
BibTeX
@inproceedings{feng2025kan,
author = {Guanwen Feng and Siyu Jin and Zhihao Qian and Yunan Li and Qiguang Miao},
title = {{KAN-Face: Efficient Resource Usage and Precision Lip-Sync in Talking Head Generation}},
booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
year = {2025},
url = {https://openreview.net/forum?id=PwiUXDH0jb},
}Loading related content...