TL;DR
RelLedger is a novel audit framework designed to detect "relational policy leakage"—the phenomenon where an LLM agent’s prior interaction history with a person (e.g., "friend" vs. "conflict") improperly influences whether it records factual events into a shared organizational ledger. The study reveals that models like Claude are up to 62 percentage points more likely to report a missed deadline if they have a "conflict" history with the target, a bias that significantly distorts downstream decision-making but can be mitigated by explicit cost-benefit rules.
What Problem Does This Paper Address?
As organizations increasingly deploy LLM agents to handle customer service, contractor oversight, and internal reporting, these agents become the "gatekeepers" of organizational memory. They decide what information is worth recording in shared databases (like a CRM or a project ledger) and what can be ignored.
The authors identify a critical vulnerability: relational policy leakage. While previous research has shown that social cues can change a model's judgment or tone, this paper asks if those same cues affect information delivery. If an agent "likes" a contractor, will it fail to log a late submission? This creates a "visibility gap" where downstream agents—and human managers—make decisions based on an incomplete or biased record.
Methods
The authors introduce RelLedger, a rigorous framework to audit the link between relational history and information recording. Key techniques include:
- Crossed Histories: To isolate the effect of a relationship, the framework holds the event, evidence, and risk fixed while crossing the sender’s history with the event target (T) and an unrelated actor (U).
- Role Reassignment: This tests whether the bias follows the specific person or the assigned role (e.g., "Contractor").
- History Rewrites: The audit repeats tests using different wording for the same relationship types to ensure the model is responding to the concept of the relationship rather than specific keywords.
- Recipient Replay: The framework doesn't just look at the sender. it "replays" the resulting record (or lack thereof) to a downstream recipient agent to measure how the "visibility gap" translates into actual organizational loss.
- Executable OPC Workflows: The authors simulated a "One-Person Company" (OPC) where agents use actual tool calls to update JSON ledgers, moving the audit from static prompts to dynamic, stateful environments.
Datasets
The evaluation was massive in scope, involving 302,764 valid records across five major models: GPT-4o (referenced as GPT-5.6-Sol), Gemini-1.5-Flash, Claude-3.5-Sonnet, DeepSeek-V2, and Qwen-2.5.
The audit utilized:
- 120 English scenarios spanning 24 organizational contexts (e.g., archive digitization, community kitchen management).
- 5 event templates covering common workplace issues like premature completion, late submissions, and unreported inconsistencies.
- Relational labels: Histories were categorized as friend, stranger, or conflict.
Key Insights and Ideas
- Model-Specific Biases: Different LLMs have different "control paths." For instance, Gemini's behavior is driven by the specific person involved, while DeepSeek and Qwen are more sensitive to designated role labels.
- The "Claude Gap": Claude showed the most dramatic sensitivity to relational context. It was 62.22 percentage points more likely to send a factual notice when in a "conflict" relationship than a "friendly" one.
- Discretionary vs. Rule-Bound Tasks: Relational effects are strongest when agents are given "room to exercise judgment." When the authors introduced explicit loss rules (e.g., "Sending when the event did not occur costs 3; not sending when it did costs 7"), the relational gaps nearly vanished.
- Downstream Cascades: The "visibility gap" is not just a theoretical bias. In executable workflows, Claude's tendency to withhold information in "warm" histories reduced downstream review rates by 45.83 points, leading to measurable increases in expected organizational loss.
Key Results
- Factual Notice Delivery: Claude’s conflict-minus-friend delivery gap reached 62.22 points.
- Actor Swap Effects: Gemini showed a -20.00 point effect, indicating its policy leakage is tied strongly to the identity of the actor.
- Workflow Impact: Under strained histories, Claude’s executed tool writes fell by 37.50 points, directly causing a 0.229 increase in expected loss for the recipient agent.
- Mitigation: Explicit cost-boundary rules reduced Gemini’s threshold gaps to below 0.47 points, suggesting that grounding agents in formal decision theory is a viable path for governance.
Limitations and Future Work
The authors acknowledge several limitations:
- Synthetic Nature: The scenarios involve fictional people and "mild" norm violations; real-world organizational politics may be more complex and high-stakes.
- Language: While the core audit was in English, the competence controls used Chinese. Expanding the full audit to more languages and cultures is a necessary next step.
- Mitigation Strategies: While explicit cost rules work, they are difficult to define for every possible organizational event. Future work should explore more generalized "relational alignment" techniques.
RelLedger provides a concrete boundary—the record-write—for auditing how social biases in LLMs can silently corrupt the "truth" within an organization's digital memory.
Summary updated