Cost Aware Communication for Autonomous Agents

32 concepts21 papers1 proposal

In classical reinforcement learning and planning, communication is often treated as a background process or a zero-cost action. However, in real-world deployments, communication is never free. For a robot navigating a household, asking a human for clarification interrupts their day and incurs a cognitive cost. In networked multi-agent systems, broadcasting data consumes bandwidth and battery power. For modern AI agents interacting with Large Language Models (LLMs), every query introduces latency and financial API costs.

If an agent communicates indiscriminately, it becomes an inefficient nuisance. If it never communicates, it risks catastrophic failure due to environmental ambiguity. The core challenge of cost-aware communication optimization is training agents to dynamically answer a single question: Is the information I am about to gain worth the cost of asking?

Communication as a Costly Action

To solve this problem, researchers integrate communication directly into the agent's decision-making space. Instead of treating communication as a separate layer, it is modeled as a discrete action with an explicit cost.

Consider a standard Markov Decision Process (MDP) or Partially Observable MDP (POMDP). The agent's action space is split into task actions (like moving forward or picking up an object) and communication actions (like asking a question or broadcasting a state update). To govern this choice, the reward function is shaped to penalize communication:

$$R(s, a) = R_{\text{task}}(s, a) - \lambda \cdot \mathbb{I}(a \in A_{\text{comm}})$$

Here, $\lambda$ represents the communication penalty. As shown by Baumann et al. in Deep Reinforcement Learning for Event-Triggered Control, incorporating this penalty allows agents to learn control policies that maintain system stability while reducing data transmissions by up to 90%.

The magnitude of $\lambda$ dictates the agent's behavior. As noted by Hu 0001 et al. in Enabling Intelligent Interactions between an Agent and an LLM: A Reinforcement Learning Approach, precise tuning of this hyperparameter is critical: if $\lambda$ is too low, the agent over-relies on expensive queries; if it is too high, the agent fails to seek help when it is genuinely lost.

Balancing Information Gain and Uncertainty

Why does this formulation work? It forces the policy to implicitly compute the Value of Information.

When an agent's internal confidence is high, the expected improvement in task reward from communicating is near zero. Because the communication action incurs a penalty $-\lambda$, the net expected utility of communicating is negative, and the agent chooses a physical task action instead. Conversely, when the agent encounters high ambiguity—such as an unseen environment or an ambiguous instruction—the probability of making a costly mistake increases. Here, the expected benefit of resolving uncertainty outweighs the penalty $\lambda$.

This dynamic manifests across several domains:

Implicit Communication and Team Interventions

Communication does not always mean explicit text or speech. Agents can also communicate implicitly through their physical actions.

Wu et al. (Actions Speak Louder Than Words: Rate-Reward Trade-off in Markov Decision Processes) formalize the fundamental information-theoretic trade-off between task reward and communication rate. When an agent alters its physical trajectory to signal information to another agent, it deviates from the most efficient path, incurring an implicit performance penalty. Similarly, Okamura et al. (Efficient and Trustworthy Social Navigation Via Explicit and Implicit Robot-Human Communication) optimize a cost-weighted combination of explicit haptic pulses and implicit physical movements, reserving explicit signaling only when physical motion is insufficient to convey intent.

In the context of human teamwork, Sangwon Seo (AI-Assisted Human Teamwork) frames the decision of an "AI Coach" to intervene as an optimization problem. The system balances the expected performance gain of realigning the team's mental models against the cognitive cost of distracting them.

Multi-Agent Coordination and LLM Planning

In decentralized multi-agent environments, communication costs fundamentally alter exploration strategies. Chakraborty et al. (Coordinated Versus Decentralized Exploration In Multi-Agent Multi-Armed Bandits) demonstrate that when broadcasting information requires sacrificing an immediate reward-generating action, agents must dynamically evaluate whether sharing information benefits the team enough to justify the opportunity cost. Kakarlapudi et al. (Decision-Theoretic Planning with Communication in Open Multiagent Systems) show that decision-theoretic planning allows agents to automatically scale down message frequency as bandwidth or energy costs rise.

Finally, as embodied agents incorporate large language models for high-level reasoning, treating communication as a costly action becomes vital for operational efficiency. Seo et al. (From Assumptions to Actions: Turning LLM Reasoning into Uncertainty-Aware Planning for Embodied Agents) show that embedding communication into the planning space allows agents to prioritize physical exploration over verbal coordination when the cost of querying or communicating exceeds its expected utility.

Key Foundational Work

Planning with Verbal Communication for Human-Robot Collaboration (2017) — contributed 2 concept elements to this direction.
Deciding to Remind during Collaborative Problem Solving: Empirical Evidence for Agent Strategies (1996) — contributed 2 concept elements to this direction.
Coordinated Versus Decentralized Exploration In Multi-Agent Multi-Armed Bandits (2017) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_cost_aware_communication_optimization_1fe6844f,
  title = {cost-aware communication optimization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/cost-aware-communication-optimization/rdr_1fe6844f527046668f9025feb607a6c9},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_cost_aware_communication_optimization_1fe6844f_v1,
  title = {cost-aware communication optimization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/cost-aware-communication-optimization/rdr_1fe6844f527046668f9025feb607a6c9/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake