Spectral Logit Sculpting: Adaptive Low-Rank Logit Transformation for Controlled Text Generation

Jin Li, Zhebo Wang, Tianliang Lu, Mohan Li, Wenpeng Xing, Meng HanarXiv 2025 · DOI · arXiv

Introduction

Large Language Models (LLMs) have demonstrated remarkable capabilities in complex reasoning, yet they frequently suffer from logical inconsistencies and "hallucinations." These issues often arise during inference when the model's output distribution becomes overly diffuse, leading to a loss of focus on the established reasoning path. Current solutions typically involve either computationally expensive fine-tuning or inference-time optimizations that ignore historical context.

Spectral Logit Sculpting (SLS) introduces a lightweight, parameter-free approach to address this. By performing on-the-fly spectral analysis on a sliding window of recent output distributions, SLS "sculpts" the model's current predictions to maintain consistency and confidence, particularly during moments of high uncertainty.

The SLS Framework

The core of SLS is a three-step pipeline that operates directly on the model's logits (raw, unnormalized scores) without requiring any updates to the model's underlying weights.

The Spectral Logit Sculpting (SLS) pipeline, illustrating the flow from logit buffering and SVD to adaptive rescaling.
  1. Sliding Logit Buffer: The system maintains a history of the "Top-K" logits from the most recently generated tokens. This buffer captures the temporal "rhythm" and logical direction of the model's reasoning.
  2. Entropy-Based Gating: To maintain efficiency, SLS is only activated when the model exhibits high uncertainty (high entropy) or when the gap between the top two token choices is narrow. If the model is confident, it proceeds with standard decoding.
  3. Spectral Transformation (SVD): When triggered, SLS performs a Singular Value Decomposition (SVD) on the logit buffer. This identifies the "dominant spectral directions"—the core logical themes of the recent text. The current logits are projected onto this low-rank subspace and rescaled, effectively dampening noise and sharpening the signal of the most likely logical path.

Experimental Evaluation

The researchers evaluated SLS across three demanding domains using several well-known benchmarks:

  • Mathematics: Math500, AMC, AIME, Minerva Math, and Olympiad Bench.
  • Code Generation: LeetCode (180 problems across varying difficulty levels).
  • Scientific Reasoning: UGPhysics (undergraduate physics benchmark).

The evaluation utilized Qwen2.5-7B-Instruct and its math-specialized variant as base models to test the method's generalizability and specialized performance.

Key Results and Insights

SLS consistently outperformed standard greedy decoding and more complex inference-time methods like Entropy Minimization Inference (EM-INF).

Performance comparison of SLS against baselines across Math, Code, and Science benchmarks.

As shown in the results table, SLS achieved a significant performance boost in mathematical reasoning, reaching an average accuracy of 42.7% on the Qwen2.5-7B-Instruct model, compared to the 40.5% baseline. In coding tasks, SLS reached a 52.2% Pass@1 score on LeetCode, a notable improvement over the baseline's 47.2%.

Beyond accuracy, SLS proved to be highly efficient. Despite the additional SVD step, it maintained a higher inference throughput (17.72 tokens/s) than EM-INF (16.81 tokens/s) because it avoids the need for multiple optimization forward passes per token.

Conclusion

Spectral Logit Sculpting demonstrates that the "correct" reasoning path often resides in a lower-dimensional subspace of the model's total vocabulary. By using SVD to isolate this subspace from recent history, SLS acts as a temporal filter that keeps the model "on track" without the need for parameter updates. While future work will explore the theoretical foundations of these spectral directions and their application to multimodal generation, SLS currently stands as a highly portable and effective tool for enhancing LLM reliability.

Summary updated

Abstract

Entropy-based inference methods have gained traction for improving the reliability of Large Language Models (LLMs). However, many existing approaches, such as entropy minimization techniques, suffer from high computational overhead and fail to leverage historical token context effectively. To address these limitations, we propose Spectral Logit Sculpting (SLS), a lightweight inference-time optimization method that dynamically modulates token distributions using spectral and entropic properties of recent logits. SLS maintains a sliding buffer of top-K logits, performs on-the-fly Singular Value Decomposition (SVD) to identify dominant spectral directions, and adaptively rescales logits based on both entropy and logit gap statistics--only activating when uncertainty is high. Without updating any model parameters, SLS effectively sharpens the output distribution while preserving contextual consistency. Experimental results on multiple public benchmarks demonstrate that SLS consistently outperforms existing baseline methods, achieving superior accuracy in mathematical, coding, and scientific reasoning tasks.
BibTeX
@misc{li2025spectral,
  author = {Jin Li and Zhebo Wang and Tianliang Lu and Mohan Li and Wenpeng Xing and Meng Han},
  title = {{Spectral Logit Sculpting: Adaptive Low-Rank Logit Transformation for Controlled Text Generation}},
  year = {2025},
  eprint = {2509.25204},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.25204},
}
Figure