Sketching an open invisible space with multiagent reinforcement learning

Hongsheng Chen, Bei Wu, Chao Qian, Zhedong Wang, Pujing Lin, Er‐Ping LiResearch Square 2024 · DOI

Introduction

Controlling electromagnetic (EM) waves is a cornerstone of modern physics, with applications ranging from optical microcavities to quantum information processing. For decades, the pursuit of an "invisibility cloak" has focused on transformation optics and metamaterials designed to wrap around specific objects. However, these traditional approaches are inherently limited: they are "closed" systems that must physically clad the target, they are typically static and designed for specific environments, and they struggle in "open" environments where EM scattering is stochastic and disordered.

In a groundbreaking study, researchers have moved beyond these constraints to create an open, dynamic invisible space. By utilizing a swarm of reconfigurable metasurfaces controlled by a novel multi-agent reinforcement learning (RL) algorithm called MetaSeeker, the team has demonstrated that objects—such as vehicle clusters or human experimenters—can move freely within a designated area while remaining undetected by external EM sensors.

The Challenge of Open Environments

Traditional invisibility cloaks rely on precise mathematical models to guide waves around an object. In a cluttered, dynamic environment like a busy street, these models break down because the scattering becomes random. To overcome this, the researchers shifted the paradigm from a single, passive shell to a distributed, active "swarm" of intelligent surfaces.

Conceptual framework of the MetaSeeker system and the open invisible space.

The MetaSeeker Framework: Hardware and AI

The system's success relies on the synergy between advanced hardware and a sophisticated AI "brain."

Hardware: The Metasurface Swarm

The researchers deployed a team of reconfigurable metasurfaces. Each surface is composed of "meta-atoms" equipped with PIN diodes. These diodes can toggle between two phase states (0 and $\pi$), allowing the surfaces to manipulate EM wave reflections in real-time. By coordinating these surfaces, the system can actively cancel out the scattering caused by moving objects within the space.

The "Brain": MetaSeeker RL

The control mechanism, MetaSeeker, is a population-based reinforcement learning framework inspired by MuZero. Unlike traditional AI that requires a perfect physical model, MetaSeeker constructs a "narcissistic internal world" to mirror the complex physical dynamics of the environment. It learns a representation optimized specifically for its goal: invisibility.

To eliminate the need for human intervention, the system employs Population-Based Training (PBT). It trains a "population" of agents, where underperforming strategies are replaced by mutated versions of superior ones, effectively "evolving" the optimal control policy through autonomous preferment and adaptation.

Experimental Validation and Results

The system was tested in a "perception-decision-execution" loop. Using a camera and Mask-RCNN for object identification, TPUs for high-speed computation, and FPGAs for real-time control, the system achieved remarkable results:

  • Environmental Similarity: The system maintained a 99.5% similarity between the background with the hidden object and the original empty background.
  • High Precision: Detection similarity across multiple angles reached between 99.73% and 99.95%.
  • Real-time Adaptation: Once trained, the system achieved "near-perfect immune effects" within milliseconds, enabling it to hide objects during high-speed tasks like racing and tracking.

Explainability and Swarm Intelligence

A key feature of this research is the use of SHAP (Shapley Additive Explanations) to provide post-hoc explainability. By visualizing the decision-making process, the researchers proved that the metasurfaces were actively counterbalancing the scattering caused by moving objects rather than just memorizing static patterns. This "swarm intelligence" approach provides robustness; if one metasurface fails, the others can recalibrate to compensate, ensuring the integrity of the invisible space.

Conclusion

This work represents a monumental stride in the evolution of metasurfaces, moving from individual components to coordinated swarm intelligence. While currently demonstrated at microwave frequencies (5 GHz), the principles of MetaSeeker and distributed metasurface control offer a roadmap for remote management of entire EM spaces. Future developments could scale these concepts to higher frequencies or apply them to complex challenges in satellite communication and beyond.

Summary updated

Abstract

<title>Abstract</title> Controlling electromagnetic (EM) waves at will is fundamentally important for diverse applications, ranging from optical microcavities, super-resolution imaging, to quantum information processing. Decades ago, the forays into metamaterials and transformation optics have ignited?unprecedented?interest to create an invisibility cloak—a closed space with any object inside invisible. However, all features of the scattering waves become stochastic and uncontrollable when EM waves interact with an open and disordered environment, making an open invisible space almost impossible. Counterintuitively, here we for the first time present an <italic>open</italic>, <italic>cluttered</italic>, and <italic>dynamic </italic>but invisible space, wherein any freely-moving object maintains invisible. To adapt to the disordered environment, we randomly organize a swarm of reconfigurable metasurfaces, and master them by MetaSeeker, a population-based reinforcement learning (RL). MetaSeeker constructs a narcissistic internal world to mirror the stochastic physical world, capable of autonomous preferment, evolution, and adaptation. In the perception-decision-execution experiment, multiple RL agents automatically interact with the ever-changing environments and integrate a post-hoc explainability to visualize the decision-making process. The hidden objects, such as vehicle cluster and experimenter, can freely scale, race, and track in the invisible space, with the environmental similarity of 99.5%. Our results constitute a monumental stride to reshape the evolutionary landscape of metasurfaces from individual to swarm intelligence and usher in the remote management of entire EM space.
BibTeX
@inproceedings{chen2024sketching,
  author = {Hongsheng Chen and Bei Wu and Chao Qian and Zhedong Wang and Pujing Lin and Er‐Ping Li},
  title = {{Sketching an open invisible space with multiagent reinforcement learning}},
  booktitle = {Research Square},
  year = {2024},
  doi = {10.21203/rs.3.rs-4129277/v1},
}
Figure