In classical artificial intelligence and formal logic, belief revision is often treated as an isolated, deterministic process: an agent receives a new piece of data, assumes it to be true, and updates its internal model of the world to accommodate it. If the new data contradicts older data, the agent resolves the conflict using predefined logical priorities.
However, this paradigm breaks down in multi-agent systems. When autonomous drones, military commanders, or Large Language Models (LLMs) operate in shared environments, incoming information is rarely an objective reflection of ground truth. Instead, it consists of reports from other agents who may be misinformed, biased, or intentionally deceptive. If an agent blindly accepts every incoming message, it quickly falls victim to context bloat, contradictions, and misinformation.
To build resilient multi-agent systems, researchers have shifted from treating information as inherently valid to treating trust as an epistemic filter. In this paradigm, an agent does not evaluate a piece of evidence purely on its own merits; instead, it uses its relationship with the source to determine whether, and to what extent, that evidence should alter its internal beliefs.
Tracking the Origin of Knowledge
Before an agent can use trust to filter information, it must be capable of distinguishing between what it knows from direct observation and what it has been told by others. Humans do this intuitively, but autonomous agents often struggle to separate perception from hearsay unless their cognitive architectures explicitly track data lineage.
To address this, Renata Vieira et al. demonstrate the necessity of decoupling message reception from internal state updates in On the Formal Semantics of Speech-Act Based Communication in an Agent-Oriented Programming Language. By implementing belief provenance annotation—tagging internal knowledge with metadata identifying its specific origin—the authors show how agents can autonomously manage their commitments. This social filtering mechanism ensures that a communication act does not automatically overwrite an agent's state; instead, the internal impact is gated by the receiver's evaluation of the sender.
Similarly, K. Suzanne Barber et al. leverage explicit information valuation policies in Multi-Agent System Development: Design, Runtime, and Analysis. By tracking source reliability, corroboration, and timeliness, their framework provides a transparent, traceable log of why specific information was trusted over alternatives, which is vital for debugging complex multi-agent deployments like UAV surveillance.
Trust as an Epistemic Gatekeeper
Once information provenance is established, trust acts as a fundamental gatekeeper in formal models of social epistemology. Rather than being a secondary variable calculated after the fact, trust is increasingly modeled as a strict prerequisite for information processing.
As Emiliano Lorini formalizes in Reasoning About Belief, Evidence and Trust in a Multi-agent Setting, an agent must first trust another's judgment before it can accept their statements as valid evidence. Without this baseline trust, information exchange cannot alter the recipient's cognitive state. This perspective is mirrored by Fenrong Liu in Reasons to Believe in Social Settings, which integrates evidence-sharing and trust into a single logic language. Here, trust acts as a primary filter, mathematically predicting how information propagates through a network based on the perceived credibility of its sources.
Real-world trust, however, is rarely binary or global; it is highly domain-specific. To capture this nuance, Aaron Hunter 0001 et al. introduce the concept of a "trust partition" in Trust-Sensitive Belief Revision. Instead of forcing agents to accept all reports from an informant, this framework maps the informant's specific areas of expertise. Agents only revise their beliefs if the incoming report falls within the informant's trusted domain, allowing them to maintain internal consistency and reject false reports from sources lacking relevant expertise.
This distinction is formalized by Junli Jiang et al. in In Data We Trust: The Logic of Trust-Based Beliefs, which provides a logical framework that explicitly separates believing a proposition from believing it because a trusted source asserted it.
Quantitative Integration and Social Weighting
When an agent decides to accept information from a trusted peer, it must reconcile that social input with its own private observations. Modern multi-agent frameworks achieve this by shifting from binary logic to quantitative degrees of certainty, weighting social influence against empirical evidence.
In PsychSim: Modeling Theory of Mind with Decision-Theoretic Agents, David V. Pynadath et al. model this integration by calculating a weighted sum of factors, including message consistency, the sender's apparent self-interest, and the recipient's trust in the source. This allows agents to dynamically adjust their degrees of belief based on the perceived reliability of communication. This quantitative approach aligns with the probabilistic principles outlined by Adam J. Grove et al. in Probability Update: Conditioning vs. Cross-Entropy, which formalizes trust by deriving an agent's final beliefs from the expectation of another agent's beliefs, weighted by its own confidence in that source.
At a network level, these weights dictate collective cognition. Paul R. Smart et al. simulate collective sensemaking in A constraint satisfaction model of collective sensemaking: insights into the effect of coalition communication environments on the dynamics of collective cognition by applying trust weights to communication links within Constraint Satisfaction Networks. Similarly, Darren Richardson et al. integrate social trust parameters into high-fidelity ACT-R cognitive architectures in Using ACT-R to Model Collective Sensemaking in Military Coalition Environments, demonstrating how trust weights influence individual decision-making and the eventual formation of group consensus.
To maintain accuracy, agents must continuously balance these weights. In the synthetic benchmark developed by Elizabeth Pavlova et al., Flag Game: Interpreting Decision Mechanisms of Bounded Social Agents, agents must balance private visual evidence against peer reports. The authors show that successful agents perform a compatibility check to ensure social input aligns with local observations, preventing the group from falling into polarization.
Systemic Vulnerabilities and Future Directions
When trust-mediated belief revision is deployed at scale, it introduces complex systemic dynamics—and unique vulnerabilities. Because trust modulates how information propagates, it can be manipulated or create unintended feedback loops.
For instance, Tianyi Qiu et al. explore the mathematical consequences of over-trusting algorithmic authorities in The Lock-in Hypothesis: Stagnation by Algorithm. Using a Bayesian framework mediated by a trust matrix, the authors demonstrate how a community's reliance on a highly trusted central model can inadvertently collapse the diversity of human thought, locking the network into ideological stagnation.
Malicious actors can also exploit these weighting mechanisms. In Using Agent-Based Modelling and Reinforcement Learning to Study Hybrid Threats, Kärt Padur et al. simulate coordinated cyber-disruptions and misinformation campaigns. Their work reveals that when systems become unstable, social influence can systematically override an agent's direct empirical observations, allowing misinformation to hijack the belief revision process.
To counter these threats, modern systems require robust frameworks to evaluate truthfulness over time. Tran Cao Son et al. address this in An Answer Set Programming Framework for Reasoning About Truthfulness of Statements by Agents, focusing on modeling what an observer believes about an agent's truthfulness given incomplete information, rather than attempting to establish absolute ground truth.
As multi-agent systems increasingly rely on LLMs, managing this information flow becomes even more critical. Yao Guan et al. note in MOC: Multi-Order Communication in LLM-based Multi-Agent Systems that traditional information distillation methods focus primarily on semantic similarity, which frequently overlooks factual reliability. The authors argue that future multi-agent communication frameworks must integrate trust-aware schemes that filter and weight information based on the proven, historical accuracy of the source agent.
Key Foundational Work
Probability Update: Conditioning vs. Cross-Entropy (1997) — contributed 1 concept elements to this direction.
Multi-Agent System Development: Design, Runtime, and Analysis (2004) — contributed 1 concept elements to this direction.