Are We Scaling the Right Thing? A System Perspective on Test-Time Scaling
Lists
Only you see this
Abstract
Test-time scaling (TTS) has recently emerged as a promising direction to exploit the hidden reasoning capabilities of pre-trained large language models (LLMs). However, existing scaling methods narrowly focus on the compute-optimal Pareto-frontier, ignoring the simple fact that *compute-optimal is not always system-optimal*. In this work, we propose a system-driven perspective on TTS, analyzing how reasoning models scale against practical metrics, such as latency and cost-per-token. By evaluating the impact of popular optimizations such as tensor parallelism and speculative decoding, our preliminary analysis reveals the limitations of current methods and calls for a paradigm shift toward holistic, system-aware evaluations that capture the true essence of scaling laws at inference time.
BibTeX
@inproceedings{zhao2025are,
author = {Youpeng Zhao and Jinpeng Lv and Di Wu and Jun Wang},
title = {{Are We Scaling the Right Thing? A System Perspective on Test-Time Scaling}},
booktitle = {ER @ NeurIPS},
year = {2025},
url = {https://openreview.net/forum?id=UOHPRDlTUU},
}