Aligning Machine Learning Objectives with Cryptographic Key Recovery

28 concepts16 papers2 proposals

Imagine training a deep neural network to break an AES cryptographic implementation by analyzing power consumption traces. After hours of training, the validation metrics report a disheartening 0% classification accuracy. In a standard machine learning context, this model is a complete failure.

In hardware security, however, this model might be a devastatingly effective weapon.

This paradox highlights a fundamental metric misalignment between generic machine learning objectives and the physical reality of cryptographic attacks. Standard machine learning is built around pointwise accuracy and absolute probabilities: did the model predict the exact right class on this specific sample? But a side-channel attacker does not need the model to guess the exact key byte correctly from a single trace. Instead, the attacker aggregates information across thousands of independent physical observations to rank key candidates.

A model can get every single individual prediction wrong, yet as long as it consistently ranks the correct key candidate slightly higher than the alternatives on average, the attack will succeed. Over the last few years, researchers have recognized that forcing neural networks to optimize for standard classification metrics severely limits their performance, leading to a concerted effort to align machine learning objectives directly with domain-specific cryptographic goals.


The Anatomy of Metric Misalignment

To understand why standard metrics fail, we must look at how information is extracted during a side-channel attack. As explained by Stjepan Picek et al. in The Curse of Class Imbalance and Conflicting Metrics with Machine Learning for Side-channel Evaluations, traditional classification metrics focus exclusively on the top-1 prediction. This ignores the rich probabilistic information contained across the rest of the output distribution.

Because side-channel attacks rely on the cumulative probability distribution across multiple observations to rank candidates, a model's true utility is determined by its consistency across traces rather than its single-trace accuracy. As Daan van der Valk et al. demonstrate in Bias-variance Decomposition in Machine Learning-based Side-channel Analysis, an attack tool can maintain a successful key recovery rate even with zero classification accuracy, provided it keeps the correct key within a high-probability subset of candidates. Conversely, a model that achieves high accuracy by simply predicting the most frequent, imbalanced leakage state provides no statistical leverage to distinguish a secret key from candidates.

Standard ML Objective (Top-1 Accuracy):
[Trace] ──> [Neural Network] ──> Is Max Probability == Correct Class? (Yes/No)

Side-Channel Objective (Key Ranking):
[Trace 1] ──┐
[Trace 2] ──┼─> [Neural Network] ──> Aggregate Probabilities ──> Rank Candidates ──> Is Correct Key #1?
[Trace N] ──┘

This misalignment means that standard loss functions like categorical cross-entropy provide a poor optimization signal. In A Practical Tutorial on Deep Learning-based Side-channel Analysis, Sengim Karayalcin et al. reiterate that evaluation must instead center on domain-specific metrics—most notably, Guessing Entropy (GE). Structured comprehensively by Stjepan Picek et al. in SoK: Deep Learning-based Physical Side-channel Analysis, Guessing Entropy quantifies the average number of attempts required to identify a secret key after an attack. It is a security-centric metric that directly measures an attacker's remaining workload.

This core principle—that optimization must match the domain-specific threat model—extends across security disciplines:


Shifting from Probabilities to Relative Ranks

If the ultimate goal of an attack is to rank the correct key candidate above all others, why do we train neural networks using loss functions that optimize for absolute pointwise probabilities?

Gabriel Zaid et al. address this core contradiction in Ranking Loss: Maximizing the Success Rate in Deep Learning Side-Channel Analysis. Standard classification losses force the network to minimize the distance between its prediction and a one-hot encoded vector. This introduces severe approximation errors because the network wastes capacity trying to drive the target class probability to exactly $1.0$ and alternative classes to $0.0$.

The authors propose a custom Ranking Loss function that alters the training objective entirely. Instead of optimizing absolute values, the ranking loss penalizes the network based on the relative ordering of the target class against alternatives. By aligning the mathematical objective of the gradient descent loop with the practical goal of the attacker, this approach minimizes approximation errors and yields significantly stronger cryptographic distinguishers.


Domain-Aligned Optimization Frameworks

Because standard validation accuracy does not reliably predict side-channel performance, manual or automated hyperparameter tuning guided by generic metrics often fails to find effective architectures. To bypass this, researchers have integrated cryptographic metrics directly into automated optimization loops.

Reinforcement Learning and Bayesian Optimization

In Reinforcement Learning for Hyperparameter Tuning in Deep Learning-based Side-channel Analysis, Jorai Rijsdijk et al. utilize Q-learning to automatically design Convolutional Neural Networks (CNNs). Instead of using validation accuracy as a reward, their framework uses custom reward functions based on the average rank of the correct key and the data volume required for success. This domain-aligned signal allowed the agent to discover "tiny" architectures that drastically outperform massive, human-designed models.

Similarly, Lichao Wu et al. introduced the AutoSCA framework in I Choose You: Automated Hyperparameter Tuning for Deep Learning-based Side-channel Analysis (and its extended work in I Choose You: Automated Hyperparameter Tuning for Deep Learning-Based Side-Channel Analysis), leveraging Bayesian Optimization guided by key rank and leakage metrics rather than generic validation scores to find high-performing architectures in high-noise environments.

Evolutionary Activation Functions

Rather than adjusting network macro-structures, Karlo Knezevic et al. target the internal mathematical operations of the network in NeuroSCA: Evolving Activation Functions for Side-Channel Analysis. They use Genetic Programming to evolve custom activation functions to replace standard choices like ReLU. By embedding Guessing Entropy directly into the evolutionary loop as the fitness function, the optimization process naturally selects for non-linearities that maximize the extraction of physical secret key leakages.


Overcoming the Evaluation Bottleneck

While evaluating models using Guessing Entropy is conceptually ideal, it introduces severe practical and computational bottlenecks.

The Computational Cost: Fast Guessing Entropy (FGE)

To compute Guessing Entropy traditionally, a researcher must run a full attack simulation across thousands of validation traces, sorting and tracking key candidate probabilities. Doing this at the end of every training epoch to guide early stopping or hyperparameter selection is computationally prohibitive.

To solve this, Guilherme Perin et al. developed Fast Guessing Entropy (FGE) in The Need for Speed: A Fast Guessing Entropy Calculation for Deep Learning-based SCA. By drastically reducing the number of validation traces required to compute a stable ranking signal during the training phase, FGE accelerates validation by up to 16×. This provides a highly sensitive, computationally viable signal for early stopping and large-scale hyperparameter exploration.

The Data Scarcity Problem: Model Fitting Metrics

Another critical flaw of Guessing Entropy is its behavior when data is scarce. In practical scenarios where an attacker has access to only a small number of attack traces, Guessing Entropy may fail to converge, showing a flatline that implies the model has learned nothing.

However, as Lichao Wu et al. discover in Everything is Connected: From Model Learnability to Guessing Entropy, a model can successfully capture the underlying physical leakage patterns of a chip long before it has enough data to show convergence in standard rank-based metrics. To bridge this gap, the authors proposed the Profiling Model Fitting Metric ($L_m$). By measuring the mathematical correlation between predicted key ranks and the ideal leakage distributions, $L_m$ provides a reliable indicator of a model's generalization ability even when attack data is highly constrained.

When Data is Scarce:
Traditional Guessing Entropy ──> No Convergence (Looks like failure)
Model Fitting Metric (Lm)     ──> High Correlation (Proves physical learning)

The Statistical Trap of Legacy Metrics

Finally, researchers must be careful with how they aggregate these domain-specific metrics. In On the Evaluation of Deep Learning-based Side-channel Analysis, Lichao Wu et al. point out that the hardware security community frequently relies on the simple arithmetic mean of Guessing Entropy across multiple training runs.

However, neural network performance distributions are highly skewed; a few failed training runs can disproportionately inflate the average Guessing Entropy, leading to inaccurate conclusions about a model's true viability. Standardizing frameworks around robust, domain-aligned validation measures—such as those integrated into the open-source framework introduced by Guilherme Perin et al. in AISY - Deep Learning-based Framework for Side-channel Analysis—is essential for moving the field toward reproducible and realistic evaluations.

Key Foundational Work

The Curse of Class Imbalance and Conflicting Metrics with Machine Learning for Side-channel Evaluations (2018) — contributed 4 concept elements to this direction.
Bias-variance Decomposition in Machine Learning-based Side-channel Analysis (2019) — contributed 2 concept elements to this direction.
Quantitative Mitigation of Timing Side Channels (2019) — contributed 1 concept elements to this direction.
History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 15 papers
  2. Version 1 · 2026-10-04 · 22 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_metric_misalignment_f91b10a7,
  title = {metric misalignment},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/metric-misalignment/rdr_f91b10a79fe844808c8e5aab46ec0564},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 2, frozen text):

@misc{lacuna_metric_misalignment_f91b10a7_v2,
  title = {metric misalignment},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/metric-misalignment/rdr_f91b10a79fe844808c8e5aab46ec0564/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake