TL;DR
Aspect-wise Pairwise Tournaments (APT) is a training-free method to improve LLM reasoning by rethinking how we select the "best" answer from a pool of candidates. By breaking down complex judgments into specific "aspects" (like logical soundness or domain knowledge) and using a single-elimination tournament of side-by-side comparisons, APT consistently outperforms standard selection methods like reward models and majority voting.
What Problem Does This Paper Address?
Scaling "test-time compute"—giving a model more time or attempts to solve a problem—is currently one of the most effective ways to boost LLM performance. The most common approach is best-of-n sampling: generate $n$ different solutions and pick the best one.
However, picking the "best" is surprisingly difficult. Traditional Reward Models (RMs) often score solutions in isolation, which can miss subtle errors. Furthermore, "quality" is often subjective or multi-dimensional; a solution might be mathematically correct but poorly explained, or vice versa. The authors argue that current selection methods struggle because they ask models to make holistic, absolute judgments—a task that is difficult for both humans and AI.
Methods
APT introduces a selection pipeline inspired by human decision-making, built on two core pillars:
- Pairwise Comparison: Instead of giving a solution a score from 1 to 10, APT presents two solutions side-by-side and asks, "Which one is better?" This "A vs. B" format makes subtle differences more salient to the LLM judge.
- Aspect Decomposition: Rather than asking a single judge to decide which is "better" overall, APT uses a panel of aspect verifiers. Each verifier is an off-the-shelf LLM prompted to focus on a specific criterion, such as "unit conversions," "logical soundness," or "identifying mistakes."
The process follows a Single-Elimination Tournament structure:
- Generate: Sample $n$ candidate solutions from a generator LLM.
- Matchup: For every pair in the bracket, the panel of $m$ aspect verifiers votes on a winner.
- Advance: The candidate with the most votes moves to the next round.
- Select: The final remaining candidate is returned as the output.
Crucially, APT is training-free and steerable. Users can influence the selection by adding or weighting specific aspects (e.g., "prefer concise answers") at inference time without regenerating any solutions.
Datasets
The authors evaluated APT across five diverse domains using three different generator LLMs (Gemini-3.1-Pro, GPT-5.2, and Qwen3-8B):
- AIME & HMMT: High-level competition mathematics.
- HLE (Humanity's Last Exam): A difficult multiple-choice benchmark spanning various academic fields.
- Chess Validity: A state-tracking task requiring the model to identify legal moves.
- Chess Optimality: A strategic reasoning task where models suggest the best move in grandmaster-level positions.
Key Insights and Ideas
- Comparison > Scoring: Side-by-side comparison acts as a "magnifying glass" for errors. A model might approve of two different solutions when read separately, but when compared, the logical gap in one becomes obvious.
- The Power of the Panel: The "aspect" approach provides a "wisdom of the crowd" effect. Even if one verifier makes a mistake, the others often provide a corrective signal.
- Weak-to-Strong Generalization: Interestingly, the authors found that smaller, "weaker" models (like GPT-5-mini) can effectively act as verifiers to improve the outputs of "stronger" models (like GPT-5.2). This suggests that judging is fundamentally easier than generating.
- Steerability without Retraining: By simply adding a "conciseness" aspect to the tournament, the authors could significantly reduce the length of selected math solutions with almost no loss in accuracy.
Key Results
APT achieved the best results in 14 out of 15 model-domain pairs tested.
- Strategic Reasoning: On the open-ended Chess Optimality task, APT showed massive gains over baselines, nearly doubling the "pawn score" (a measure of move quality) compared to standard reward models in some settings.
- Math Performance: On the HMMT math benchmark, APT improved accuracy by several percentage points over self-consistency and binary approval methods.
- Scaling: Performance scales reliably with both the number of candidates ($n$) and the number of verifiers ($m$), showing that APT effectively harvests the benefits of increased test-time compute.
Limitations and Future Work
- Inference Cost: APT is more computationally expensive than simple majority voting. It requires $2(n-1)k$ verifier calls (due to checking both A-B and B-A orderings to avoid position bias).
- Latency: While individual matchups can be parallelized, the rounds of a tournament are inherently sequential, which adds to the total time taken to return an answer.
- Domain Expansion: The current study focused on tasks with relatively objective "best" answers (math, chess, science). Future work is needed to see how well aspect-wise tournaments handle more subjective tasks like creative writing or code style.
Summary updated