Silk: A Simulation Study of Regulating Open Normative Multiagent Systems

Mehdi Mashayekhi, Hongying Du, George F. List, Munindar P. SinghIJCAI 2016

Introduction

In "open" multiagent systems (MAS)—environments like busy traffic intersections or public marketplaces where agents constantly enter and exit—coordination is a significant challenge. Traditional methods for establishing social norms often rely on "closed" systems with a fixed group of agents. This paper introduces Silk, a hybrid framework that fosters the emergence of social norms in dynamic, open environments by combining top-down recommendations with bottom-up reinforcement learning.

The Challenge of Open Multiagent Systems

Norms are the "unwritten rules" that allow autonomous agents to coordinate without constant central oversight. While well-studied in closed systems, norm emergence in open systems is difficult because agents often do not stay long enough to learn from their neighbors. Purely "bottom-up" systems, where agents learn independently, are slow to converge. Conversely, "top-down" systems with hard-coded rules are often too rigid to accommodate diverse agent preferences or changing conditions.

The Silk Framework

The authors propose a "sociotechnical" approach called Silk, which balances central monitoring with individual autonomy.

The Silk framework architecture showing the interaction between the central Generator and individual Members.

The framework consists of three primary components:

  1. The Generator (Top-Down): A central entity with a global view that monitors for potential conflicts. It "recommends" norms (soft rules) or institutes "laws" (hard constraints) based on a formal grammar.
  2. The Members (Bottom-Up): Individual agents with limited local views. They use reinforcement learning to decide whether to adopt a recommended norm based on their private payoff matrices.
  3. Experience Sharing: To solve the "cold start" problem in open systems, exiting agents pass their learned experience to incoming agents of the same type. This allows a "culture" of coordination to persist despite high population turnover.

Experimental Setup

The researchers evaluated Silk using a simulation of a 4-way grid-based traffic intersection.

The 4-way grid-based traffic intersection simulation used to evaluate Silk.

The simulation tested different reward structures:

  • Moderate Society: Balanced rewards for moving versus stopping.
  • Selfish Society: A setup where individual profit-seeking leads to collective loss (similar to a Prisoner's Dilemma).

Silk was compared against a "fully actuated control" baseline, a standard real-world traffic signal logic that adjusts timing based on vehicle detection.

Key Insights and Results

Silk demonstrates that knowledge can be a property of a role rather than an individual. By transferring experience, the system achieves rapid learning and stability.

  • Efficiency and Fairness: Silk improved average travel times by 7.4% in moderate societies and by 16.2% when fairness interventions (temporary laws to balance lane usage) were active.
  • Robustness: Stable norms emerged in nearly 100% of "Selfish Society" runs, and the system remained resilient even when 10% of agents were "disruptive" members who ignored norms.
  • Convergence: Agents typically reached a stable policy within 12,500 simulation ticks, a short duration relative to the constant turnover of the agent population.

Conclusion and Future Work

Silk provides a robust mechanism for regulating open multiagent systems by blending top-down guidance with bottom-up learning and cross-generational experience sharing. While the current study focuses on a single intersection, future research will explore its scalability across city-wide networks and its application to complex domains like collaborative cybersecurity.

Summary updated

BibTeX
@inproceedings{mashayekhi2016silk,
  author = {Mehdi Mashayekhi and Hongying Du and George F. List and Munindar P. Singh},
  title = {{Silk: A Simulation Study of Regulating Open Normative Multiagent Systems}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2016},
}
Figure