Reinforcement Learning-based Design of Side-channel Countermeasures

Jorai Rijsdijk, Lichao Wu, Guilherme Perin, Stjepan Picek

TL;DR

This paper introduces a novel framework that uses reinforcement learning (RL) to automatically design optimal combinations of side-channel countermeasures. By treating the selection of "hiding" techniques—like noise injection and clock jitter—as a search problem, the authors identify low-cost protection schemes that effectively thwart state-of-the-art deep learning attacks.

What Problem Does This Paper Address?

Side-channel analysis (SCA) allows attackers to extract secret keys from cryptographic hardware by measuring physical outputs like power consumption or electromagnetic emissions. Recently, deep learning (DL) has revolutionized SCA, enabling attackers to bypass traditional "masking" defenses with fewer traces and less manual effort.

While attack techniques are advancing rapidly, the design of defenses (countermeasures) remains largely manual and generic. Designers face a difficult trade-off: adding more protection increases the "cost" of the device (slower execution, higher power consumption), but insufficient protection leaves the device vulnerable. There is a lack of systematic methods to find the most efficient combination of countermeasures that provides maximum resilience against DL-based attacks within a specific performance budget.

Methods

The authors propose a Tabular Q-Learning framework (based on MetaQNN) to automate countermeasure selection.

  1. The Environment: The system models the selection process as a Markov Decision Process (MDP). An RL agent "chooses" from a menu of four hiding countermeasures:
  2. Desynchronization: Shifting traces in time.
  3. Uniform Noise: Adding random values to the amplitude of the signal.
  4. Clock Jitter: Introducing instability in the clock signal.
  5. Random Delay Interrupts (RDIs): Inserting dummy instructions (like nop) at random intervals.
  6. The Reward Function: This is the "brain" of the system. The agent receives a high reward if the resulting countermeasure set:
  7. Increases the Guessing Entropy (GE) (making it harder for a DL model to find the key).
  8. Prevents the attack from succeeding within a fixed number of traces (e.g., 2,000).
  9. Stays within a defined cost budget ($c_{max}$).
  10. The Attack Loop: For every combination the agent proposes, the framework simulates the countermeasures on a dataset and evaluates them against state-of-the-art Convolutional Neural Networks (CNNs) to see if the "attack" still works.

Datasets

The study utilizes the ASCAD database, the gold standard for profiling side-channel analysis. Two versions were used:

  • ASCAD Fixed Key: 50,000 traces from an 8-bit AVR microcontroller running masked AES-128 with a constant key.
  • ASCAD Random Keys: A more complex version with 300,000 traces where the keys vary, requiring the DL model to generalize better.

The authors evaluated these against two leakage models: Hamming Weight (HW) and Identity (ID).

Key Insights and Ideas

  • Time is the Best Defense: The most significant finding is that time-based countermeasures (specifically RDIs and Desynchronization) are far more effective at defeating DL attacks than simple amplitude noise. DL models are surprisingly good at "filtering" out noise but struggle significantly when the features they are looking for are shifted or delayed unpredictably.
  • RDIs are the MVP: Random Delay Interrupts consistently appeared in the top-performing, low-cost countermeasure sets. Even with a low probability of occurrence, RDIs drastically increase the complexity of the profiling task.
  • Automated "Cheating" Prevention: By including implementation cost in the reward function, the authors prevented the RL agent from simply "maxing out" every defense. This forced the agent to find clever, lightweight combinations that provide "just enough" security.
  • Generalization Matters: Attacks on the "Random Keys" dataset required more expensive countermeasures to defeat. This suggests that when a DL model is forced to learn a more general representation of leakage, it becomes more resilient to simple hiding techniques.

Key Results

  • Attack Neutralization: The RL-designed countermeasures successfully increased the Guessing Entropy to a point where the secret key could not be retrieved within the 2,000-trace limit, even by optimized CNNs.
  • Efficiency: The framework identified optimal sets with a cost budget as low as 5 (an arbitrary unit defined in the paper). For example, a simple desynchronization level of 10 was enough to defeat some attacks on the fixed-key dataset.
  • Search Performance: The RL agent showed clear learning progress. Over thousands of iterations (taking roughly 200–300 GPU hours), the average reward increased as the agent stopped picking "expensive" or "weak" combinations and converged on efficient, high-security configurations.

Limitations and Future Work

  • Simulated Costs: The "cost" of each countermeasure was defined somewhat arbitrarily for the study. In a real-world scenario, these would need to be replaced with actual hardware metrics (e.g., nanoseconds of delay or milliwatts of power).
  • Hiding vs. Masking: The study focused on evolving hiding countermeasures while keeping a standard masking scheme active. Future work could involve using RL to optimize the masking logic itself.
  • Architecture Fixedness: The countermeasures were tested against specific, pre-tuned CNN architectures. The authors acknowledge that an attacker might eventually tune a new architecture specifically to bypass these evolved defenses, suggesting a potential "cat-and-mouse" game for future RL research.

Summary updated

BibTeX
@article{rijsdijk2021reinforcement,
  author = {Jorai Rijsdijk and Lichao Wu and Guilherme Perin and Stjepan Picek},
  title = {{Reinforcement Learning-based Design of Side-channel Countermeasures}},
  journal = {IACR Cryptol. ePrint Arch.},
  year = {2021},
  url = {https://openreview.net/forum?id=G9ZrDAuqAX},
}

Loading related content...

Figure