ComplLLM: Fine-tuning LLMs to Discover Complementary Signals for Decision-making

Ziyang Guo, Yifan Wu, Jason Hartline, Kenneth Holstein, Jessica Hullman

TL;DR

ComplLLM is a novel framework that fine-tunes Large Language Models (LLMs) to act as "complementary signal extractors." Instead of just explaining why a model made a decision, ComplLLM identifies specific, interpretable information in unstructured text that an existing decision-making agent has overlooked, significantly improving the accuracy of collaborative human-AI workflows.

What Problem Does This Paper Address?

In complex fields like medicine or content moderation, decisions are rarely made in a vacuum. A clinician might use a computer vision model to screen X-rays but also has access to a written radiology report. The central challenge in these "multi-agent" workflows is complementarity: how do we know when one source of information adds unique value that isn't already captured by another?

Most current AI interpretability methods (like saliency maps or feature attribution) focus on justifying a single model's output. However, for a human supervisor, a mere restatement of the model's reasoning is often less useful than a list of features that are inconsistent with or missing from the model's recommendation. This paper addresses the gap in identifying "available-but-overlooked" evidence that can actually shift a decision toward a better outcome.

Figure 1. A graphical representation of how our work goes beyond simply identifying concepts that appear in text or signals that are predictive of a target state, by also ensuring that signals comple- ment the existing recommendation. The diagonal stripes (twill pattern) represent the target of the corresponding methods.

Methods

The authors propose ComplLLM, a post-training framework grounded in decision theory. The core idea is to define "complementary value" as the improvement in the best-attainable decision quality when a new signal is added to an existing recommendation.

The framework follows a three-step process:

  1. Estimating the Data-Generating Process: The system uses an LLM to identify a broad space of possible signals in the text and then uses a regression model to estimate how these signals, combined with existing recommendations, relate to the ground-truth state.
  2. Supervised Fine-Tuning (SFT): The LLM is trained to extract specific signals that are both present in the text and provide high complementary value (i.e., they would change the decision for the better). This step uses "Chain-of-Thought" prompting to ensure the model identifies evidence, relevance, and unique value.
  3. Reinforcement Learning (RL): The model is further refined using Group Relative Policy Optimization (GRPO). The reward function is specifically designed to favor signals that increase the "best-attainable payoff" beyond what the existing agent's recommendation achieves, while penalizing hallucinations.
Figure 2. The “two agents” setting in our framework. The recommending agent makes a recommendation Z based on their own features X, and the supervisor agent aggregates their own information T with Z to make a final decision D.

Datasets

The authors validated ComplLLM across four distinct domains:

  • Synthetic Radiology: A controlled environment based on the MIMIC-CXR dataset where complementary signals were withheld by construction to test if the framework could recover them.
  • Real-world Radiology (MIMIC-CXR): Identifying findings in radiology reports that improve upon a vision model’s assessment of cardiac dysfunction.
  • Content Moderation (DICES): Surfacing toxicity cues that specific demographic groups (e.g., Asian millennial women) weight differently than the majority-vote average.
  • Scientific Paper Reviewing (Review5K): Discovering information in human-written reviews that an LLM-based reviewer missed when judging paper acceptance.

Key Insights and Ideas

  • Complementarity $\neq$ Importance: A signal might be highly predictive of the truth, but if the existing model already knows it, its "complementary value" is zero. ComplLLM prioritizes the unexploited information.
  • Shifting the Role of Explanation: This approach moves the goalpost for LLMs from "explaining the AI" to "assisting the human" by surfacing actionable cues the human (or another model) might have missed.
  • Efficiency via GRPO: By using Group Relative Policy Optimization, the framework can optimize for complex decision-theoretic rewards without needing a separate, heavy-duty critic model.

Key Results

ComplLLM consistently outperformed standard zero-shot/few-shot prompting and existing topic/hypothesis generation methods (like BERTopic or HypotheSAE).

  • Recovery: In synthetic tests, ComplLLM achieved a surface similarity of 0.98 and an F1 score of 0.67 in recovering hidden signals, far surpassing baselines.
  • Decision Improvement: In the radiology task, ComplLLM discovered 12 significant signals (e.g., "negative pneumothorax") that improved decision accuracy from 0.819 to 0.839.
  • LLM-to-LLM Assistance: In the paper reviewing task, providing an LLM (Gemini 2.0 Flash) with complementary signals extracted by ComplLLM boosted its accuracy from 76.8% to 79.7%.
Figure 3. Expected accuracy across different methods and datasets, showing ComplLLM's superior complementary value.

Limitations and Future Work

The authors acknowledge a few key limitations:

  • Rare Signals: The framework uses a frequency threshold ($N_\tau$) to ensure statistical stability, which might cause it to miss "rare but important" signals that a domain expert would value.
  • Static Beliefs: The current model assumes a static decision-making context. In reality, showing a human a complementary signal might change their future behavior or trust in the primary model.
  • Future Direction: The authors suggest extending this to a continual learning framework that accounts for how human beliefs evolve after interacting with these surfaced signals.

Summary updated

Abstract

Multi-agent decision pipelines can outperform single agent workflows when complementarity holds, i.e., different agents bring unique information to the table to inform a final decision. We propose ComplLLM, a post-training framework based on decision theory that fine-tunes a decision-assistant LLM using complementary information as reward to output signals that complement existing agent decisions. We validate ComplLLM on synthetic and real-world tasks involving domain experts, demonstrating how the approach recovers known complementary information and produces plausible explanations of complementary signals to support downstream decision-makers.
BibTeX
@inproceedings{guo2026complllm,
  author = {Ziyang Guo and Yifan Wu and Jason Hartline and Kenneth Holstein and Jessica Hullman},
  title = {{ComplLLM: Fine-tuning LLMs to Discover Complementary Signals for Decision-making}},
  booktitle = {MALGAI @ ICLR},
  year = {2026},
  url = {https://openreview.net/forum?id=xtxEMSX9px},
}
Figure