Building heterogeneous multistep workflows in automated microscopy: reward design and knowledge transfer for autonomous structure property discovery

Sergei V. KalininarXiv 2026 · arXiv

In the world of automated science, we’ve gotten quite good at teaching microscopes where to look. But a human expert does much more: they decide how to look, balancing the need for a quick overview against the detail of a slow scan, and knowing exactly when to stop and recalibrate a drifting instrument. A new paper by Sergei V. Kalinin at the University of Tennessee, Knoxville, introduces a framework to bring this high-level strategy to autonomous microscopy.

TL;DR

This paper moves autonomous microscopy from simple "point-picking" to complex, budget-aware workflows where a controller decides not just where to measure, but whether to image for context, perform spectroscopy for detail, or recalibrate to fix instrument drift. By treating time as a finite currency, the study reveals that the "best" machine learning model depends entirely on whether your goal is to find a single scientific "needle in a haystack" or to learn the entire material "map."

What Problem Does This Paper Address?

Most current autonomous experiments are "myopic." They use a single type of measurement and simply pick the next coordinate that maximizes a score. However, real-world microscopy is heterogeneous and constrained.

Researchers face constant trade-offs:

  1. Modality: Should I take a fast structural image to find interesting areas, or a slow spectroscopic measurement to get the "ground truth" physics?
  2. Fidelity: Do I scan quickly (low quality, low cost) or slowly (high quality, high cost)?
  3. Stability: Instruments drift over time. When is it worth spending part of my limited "time budget" to stop and recalibrate?

The paper addresses these by developing a non-myopic framework that uses Reinforcement Learning (RL) to manage a multi-step experimental campaign under a strict time budget.

Methods

The author developed a budget-constrained framework using an on-policy Actor-Critic RL controller. Unlike standard active learning, this controller manages a "bank" of different actions with varying costs and rewards.

Key methodological innovations include:

  • Heterogeneous Actions: The controller chooses between IMAGE (varying window size $L$ and scan time $\tau$), SPECTRUM (point measurement), and CALIBRATE (cross-correlation to fix drift).
  • Learner Maturity: A novel weighting function that prevents the controller from over-relying on model predictions early in the experiment when the model hasn't seen enough data to be "mature."
  • Pre-experiment Calibration: Before the "live" run, the system uses historical data to establish a Pareto frontier—identifying which models offer the best trade-off between prediction accuracy and computational speed (see Figure 2).
Figure 2: Pre-experiment calibration showing the Pareto frontier of accuracy vs. compute time.

The control loop is summarized in Figure 3, showing how the controller uses available information to select the next optimal instrument operation.

Figure 3: The information flow of the Actor-Critic controller.

Datasets

The study utilizes a real-world plasmonic image–EELS (Electron Energy-Loss Spectroscopy) dataset.

  • Structural Data: High-resolution images of nanoparticle arrangements.
  • Functional Data: EELS spectra that reveal localized surface plasmon resonances.
  • Simulation Environment: The pre-acquired data is used to build a "gym" where the RL agent can "buy" measurements. The environment simulates realistic effects like scan-speed-dependent blur and instrument drift.

Key Insights and Ideas

  • Objective-Dependent Success: There is no "one size fits all" model. Deep Kernel Learning (DKL) was the champion for "Exploitation" (finding the highest physical response), while Random Forests (RF) were often superior for "Exploration" (learning the global structure-property relationship) on a tight budget.
  • The Uncertainty Trap: The study found that model uncertainty isn't always a reliable signal. In some cases (especially with DCNNs), a model's confidence can increase even as its actual prediction error gets worse.
  • Simple Rules are Strong: Surprisingly, simple "fixed-alternation" rules (e.g., take one image, then one spectrum) formed very strong baselines that the complex RL controller only narrowly beat. This suggests that for many applications, just adding multi-modal capability is a huge win.
  • The Cost of Accuracy: In the drift-prone campaign, explicit registration (re-aligning the beam) significantly improved positional accuracy but reduced the total number of spectra that could be acquired. The framework allows researchers to quantify this "scientific cost" of instrument instability.

Key Results

The paper compares three model families: Random Forest (RF), Deep Convolutional Neural Networks (DCNN), and Deep Kernel Learning (DKL).

  • Discovery Performance: Under an "Exploitation" objective, DKL frequently found the absolute maximum response in the dataset (255.0 units), hitting the "exact maximum" in 8 out of 12 runs (see Figure 4).
  • Global Learning: For "Exploration," RF achieved the lowest global Root Mean Square Error (RMSE) on a common test panel, proving to be a robust and computationally efficient "workhorse."
  • Drift Correction: In campaigns with simulated drift, the use of a Kalman filter-based controller reduced target position error from ~7 pixels (uncontrolled) to ~1.2 pixels, though this required spending ~15% of the budget on calibration.
Figure 4: Comparison of discovery and prediction performance across different models and objectives.

Limitations and Future Work

While powerful, the author notes several boundaries:

  • Single Specimen: The results are demonstrated on one specific nanoparticle system; transferability to vastly different materials remains to be tested.
  • Simulated Laws: The blur and drift laws are mathematical models. Future work needs to calibrate these on "live" instruments.
  • A Call to Manufacturers: The paper concludes with a "wish list" for microscope manufacturers. For these non-myopic workflows to work in the real world, microscopes must provide:
  • Independently callable action interfaces.
  • Timestamped telemetry (to track drift).
  • Characterized detector responses.

By treating the microscope as a strategic agent rather than a simple camera, this work paves the way for "self-driving" labs that can truly navigate the complexities of materials discovery.

Summary updated

Abstract

Autonomous microscopy can turn fast acquisition into scientific progress when it decides not only where to measure, but also what information to acquire, at what scan rate, and when to recalibrate the instrument. We develop a budget constrained framework for such multistep structure-property workflows. It combines explicit scientific rewards, action costs, instrument observation models, transfer of pre experiment knowledge into the online campaign, and evaluation kept independent of the controller. The transferred knowledge includes predictor accuracy and computational cost, empirical learner maturity functions that weight model-derived rewards, and drift characteristics established before controller training. Using a real electron energy-loss spectroscopy dataset, we run two complementary campaigns. In the first, the controller chooses between imaging and spectroscopy, image size, and scan speed. In the second, the instrument drifts and the controller also decides when to recalibrate. The campaigns show that response discovery and structure-property learning favor different surrogate models and acquisition strategies. They also show that simple acquisition rules form strong, practical baselines. Independent error analysis reveals where model uncertainty is and is not a reliable learning signal. In the drift campaign, explicit registration greatly improves positional accuracy, and the framework measures what this costs in spectroscopy throughput. Together, these results provide a reproducible way to design, compare, and audit autonomous experiments, and they define what microscopes must provide for deployment: independently executable actions, characterized scan and detector responses, timestamped registration telemetry, and complete action records.
BibTeX
@misc{kalinin2026building,
  author = {Sergei V. Kalinin},
  title = {{Building heterogeneous multistep workflows in automated microscopy: reward design and knowledge transfer for autonomous structure property discovery}},
  year = {2026},
  eprint = {2611.11609},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.11609},
}
Figure