Punctuation Restoration: A Case Study of BERT-Based Models' Task-Specific Excellence

Qishuai Zhong, Aixin Sun

Abstract

Large Language Models (LLMs) have made remarkable strides in various tasks, yet their suitability for restoring punctuation in ASR-generated transcripts remains under-explored. Through extensive experiments, we demonstrate that LLMs tend to repeatedly use the same punctuation marks and alter input text tokens, in addition to incurring high computational costs. In contrast, a simple two-stage BERT-based method—which first identifies punctuation positions and then predicts the correct punctuation types—achieves the best accuracy with at least a 10x speed improvement.
BibTeX
@inproceedings{zhong2025punctuation,
  author = {Qishuai Zhong and Aixin Sun},
  title = {{Punctuation Restoration: A Case Study of BERT-Based Models' Task-Specific Excellence}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2025},
  doi = {10.1109/icassp49660.2025.10889374},
}

Loading related content...

Figure