Abstract
In this paper, we investigate remote state estimation against an intelligent denial-of-service (DoS) attack over a vulnerable wireless network whose channel undergoes attenuation and distortion caused by fading. We use the sensor to observe system states and transmit its local state estimates to the remote center. Meanwhile, the attacker injects a jamming signal to destroy the packet accepted by the remote center and causes the performance degradation. Most of the existing works are built on a time-invariant channel state information (CSI) model in which the channel fading is stationary. However, the wireless communication channels are more prone to dynamic changes. To capture this time-variant property in the channel quality of the real-world networks, we study the fading channel network whose channel model is characterized by a generalized finite-state Markov chain. With the goals of two players in infinite-time horizon, we describe the conflicting characteristic between the attacker and the sensor with a general-sum stochastic game. Moreover, the Q-learning techniques are applied to obtain an optimal strategy pair at a Nash equilibrium. Also the monotone structure of the optimal stationary strategy is constructed under a sufficient condition. Besides, when channel gain is known a priori, except for the full Channel State Information (CSI), we also investigate the partial CSI, where Bayesian games are employed. Based on the player's own channel information and the belief on the channel distribution of other players, the energy strategy at a Nash equilibrium is obtained.
Introduction
Cyber-Physical Systems (CPS), such as smart grids and autonomous vehicle fleets, rely heavily on wireless sensor networks for remote state estimation. However, these networks are inherently vulnerable to Denial-of-Service (DoS) attacks, where an intelligent adversary injects jamming signals to disrupt data transmission. A significant challenge in modeling these interactions is the non-stationary nature of wireless channels. Unlike many existing studies that assume a time-invariant channel, this research focuses on the impact of channel fading—where signal strength fluctuates due to environmental factors—on the strategic interaction between a sensor and an attacker.
System Model and Methodology
The paper investigates a scenario where a sensor observes system states and transmits local estimates to a remote center. An attacker aims to degrade estimation performance by jamming the communication link.
Channel Modeling
To capture the time-variant properties of real-world networks, the authors model the fading channel using a Finite-State Markov Chain (FSMC). This model represents the channel gain as a sequence of states $g_1, g_2, \dots, g_k$, capturing the transition between different levels of attenuation and distortion.
Game-Theoretic Framework
The interaction is framed as a general-sum stochastic game. Unlike zero-sum games, this allows for more complex reward structures where both players have their own energy constraints and objectives. The state of the game includes both the current channel quality and the "holding time" ($\tau_k$)—the duration since the last successful packet delivery.
Optimization via Nash Q-Learning
To identify the optimal power control strategies, the researchers employ Nash Q-learning. This reinforcement learning approach allows both players to learn the Nash Equilibrium (NE) through repeated interactions, adapting their power levels based on the evolving channel state and system urgency. For scenarios with Partial Channel State Information (CSI), where players have incomplete knowledge of their opponent's channel conditions, a Bayesian game framework is utilized.
Key Insights
- The Urgency Factor: The value of a packet increases with the holding time. As the estimation error grows, the sensor becomes more "desperate" to transmit, and the attacker becomes more determined to block it, leading to higher power expenditures from both sides.
- Monotonicity of Power: A key finding is that the optimal strategy is monotone. As the system state "worsens" (higher error or longer delay), both the sensor and the attacker naturally ramp up their power.
- Information Asymmetry Resilience: The study demonstrates that stable, optimal strategies can be reached even under information asymmetry, provided players have a statistical understanding of the channel's behavior.
Results and Discussion
Numerical simulations validate the theoretical framework. Using a discrete linear time-invariant system (e.g., system matrix $A=1.2$, noise covariance $Q=0.8$), the authors show that the Nash Q-learning algorithm successfully converges to stable Q-values over approximately 50,000 episodes.
- Equilibrium Behavior: In highly urgent states, the sensor and attacker converge on high-power strategies with high probability. For example, the sensor might choose high transmission power with a probability of ~83%, while the attacker counters with high jamming power at ~91%.
- CSI Trade-offs: The results quantify the performance gap between Full CSI and Partial CSI scenarios, offering insights into the value of investing in precise channel estimation hardware.
Conclusion and Future Directions
This work provides a robust framework for securing remote state estimation in dynamic wireless environments. While the Q-learning approach is effective for small-scale scenarios, future research could explore Deep Q-Learning to handle larger, multi-sensor networks. Additionally, extending the model to include coordinated multi-attacker scenarios or stealthy deception attacks remains an open area for investigation.
Summary updated