Live-cell fluorescence microscopy runs on a fixed photon budget. Every photon used to excite a probe also bleaches it and stresses the cell, so experimenters usually dim the light across the whole field and accept noisy measurements everywhere. This proposal spends that budget where the experimenter is actually looking. An eye tracker on the microscope monitor reports which cell the operator is attending to, a quick button press confirms the choice, and a programmable illuminator concentrates excitation light on the chosen cells while keeping the rest of the field dim.
The reason this has not been practical is that spatially uneven, constantly moving illumination corrupts ordinary single-channel fluorescence: brightness then reflects where the light went, not the chemistry. Ratiometric probes remove that problem. These are sensors that emit in two channels, only one of which responds to the quantity being measured, so dividing one channel by the other cancels excitation intensity, path length and dye concentration. A ratiometric probe therefore stays quantitatively valid under arbitrary illumination patterns. Patterned light leaves one confound that the ratio does not cancel: the two fluorophores bleach at different rates, and the dose each pixel has received now varies across space. The system controls the pattern, so it knows that dose exactly and can correct for it.
Core Claim
Gaze-confirmed, cell-snapped patterned excitation combined with a per-pixel dose ledger should give lower measurement error on attended cells than uniform illumination at the same total photon dose. It should also keep ratio drift in unattended regions no worse than under uniform illumination. Without the dose correction, gaze-steered illumination should introduce a measurable spatially patterned ratio artifact, which the correction should remove. The new element is neither gaze selection nor ratiometry alone. It is the observation that ratiometry is the property that makes attention-driven, non-uniform excitation safe, and that the residual error has a known physical form that can be modeled.
The Lacuna search found no paper joining the two directions. The nearest relatives work on the gaze side and stay in software. PureGaze removes lighting and identity from learned features with an adversarial reconstruction loss. De-confounded Gaze Estimation uses a causal correction (front-door adjustment) to separate gaze from nuisance features. Both treat confounds the way ratiometry does, as factors to be divided out, but neither touches a physical optical channel. This proposal reverses the direction: gaze steers the sensing, and the sensor's ratio structure is what makes the steering harmless.
Mechanism
The probe is the genetically encoded thermometer gTEMP. It pairs the temperature-quenched protein Sirius with the thermally stable reference mT-Sapphire, expressed 1:1 through a self-cleaving linker so that energy transfer between them does not occur. The paper reports about 2.6% ratio change per °C over 5–50 °C at 50 ms frame time. In this design, gTEMP supplies the measurement method; nothing about the probe is modified.
The operator task follows the coarse-to-fine pattern that gaze-interaction work has converged on: gaze proposes and a manual action confirms. In Gaze-Based Menu Navigation in VR, 77% of participants preferred gaze-to-point with button-to-click over dwell and controller-only selection, and that condition reached a System Usability Scale score near 80.39. Gaze trackers on desktop screens still err by a centimetre or more; GazeSwipe reports 1.5–2 cm residual error even after its autocalibration. The gaze point is therefore never used directly as the illumination mask. It instead selects among cell outlines produced by an automatic segmentation, and the mask snaps to the chosen cell plus a margin. This design matches Fast Explicit-Input Assistance, where the human gives an explicit coarse pointer and the machine resolves the precise geometry. That system cut failures to 1.13 per session, against 2.48 for implicit intent prediction. Its result constrains this proposal: the system will not guess intent from gaze alone.
Two equations govern the design. Under shot-noise-limited detection, the temperature error on a pixel is approximately
$$\sigma_T \approx \frac{1}{s}\sqrt{\frac{1}{N_1}+\frac{1}{N_2}},$$
where $N_1$ and $N_2$ are the photon counts in the responsive and reference channels and $s \approx 0.026$ per °C is gTEMP's ratio sensitivity. The form follows from Poisson statistics on each channel, propagated through the ratio. Concentrating a fixed budget onto a fraction $f$ of the field raises the counts there by about $1/f$, so error on attended cells should fall by about $\sqrt{f}$. This is the predicted gain the experiments test.
The residual confound is differential bleaching. If each fluorophore decays exponentially with dose, the observed ratio is
$$R_{\text{obs}}(x,t) = R_{\text{true}}(x,t)\, e^{-(k_1-k_2)D(x,t)},$$
where $D(x,t)$ is the cumulative excitation dose delivered to pixel $x$ by time $t$, and $k_1$ and $k_2$ are the two channels' bleaching constants per unit dose. Under uniform light this drift is the same everywhere and is often ignored. Under gaze-steered light, $D$ inherits the history of the operator's attention, and the drift becomes a spatial artifact that could be mistaken for biology. The illuminator logs $D$ exactly. The constants $k_1$ and $k_2$ are fitted on a fixed calibration sample, and the exponential factor is then divided out. If bleaching turns out to be non-exponential in practice, the exponential is replaced with a lookup table measured against dose.
Research Program
The build has four parts. The first is a widefield microscope with a digital micromirror device (a chip of tiny tilting mirrors that can project any binary light pattern) in the excitation path, run at the camera frame rate. The second is a screen-mounted infrared eye tracker with button confirmation. The third is a live cell segmenter that supplies the outlines gaze selects among. The fourth is the dose ledger and correction.
Two failure modes must be checked before any claim is made. First, the mask edges must not create artifacts in the ratio. Because gTEMP is excited at a single wavelength and read in two emission channels, both channels see the same pattern, so the ratio should cancel it. Residual edge error is measured on uniform dye slides. Second, gTEMP is a thermometer, so concentrated light could heat the very cells it reports on. Local temperature is measured with and without the boost on cells held in a temperature-controlled stage. Any heating found is reported and becomes an upper bound on how far light can be concentrated.
Baselines
The naive reference point is uniform illumination at the same total photon dose per experiment. This is what a careful microscopist does today. Beating it means lower temperature error on the cells the operator attended to, with no increase in drift or phototoxicity elsewhere. A second reference is uniform illumination at full power with no dose constraint. It gives the best error obtainable and the worst bleaching and cell stress, and it shows how much of the full-power accuracy the proposal recovers at a fraction of the dose.
The strongest competitor is automated region selection. In this baseline, the same segmenter plus a detector that flags cells whose ratio changes quickly decides where to concentrate light, with no human input. It gets the same dose budget, the same illuminator and the same segmentation. Its detection threshold and refresh rate are tuned on held-out sessions with the same tuning effort given to the gaze system's margin and confirmation settings. It is the strongest because it needs no operator and could plausibly match gaze whenever "interesting" can be defined from the signal. Gaze beats it only if, on tasks where the target is defined by the operator's goal rather than by signal change, it reaches equal or lower error on the cells the operator was instructed to monitor while missing fewer of them. The third competitor is mouse-click selection with the same snapping, which isolates the cost of gaze against a familiar pointing device.
Two ablations isolate the key ingredients. The first runs gaze-steered illumination without the dose ledger, which should expose the spatial drift artifact predicted by the bleaching equation. The second runs gaze steering with single-channel Sirius intensity instead of the ratio, which should show that ratiometry, not gaze, is what keeps patterned excitation quantitative.
For reference numbers, gTEMP reports 2.6% per °C sensitivity and 50 ms temporal resolution (gTEMP). Neural-network-aided thermometry reports ±0.05 °C cellular mapping precision, but it relies on fluorescence lifetime, which is slower to acquire. No reported result was found on Lacuna for ratio drift or temperature error under spatially patterned or attention-driven excitation, so the uniform-illumination baselines above must be measured in-house.
Why not just…?
Why not just lower the light everywhere? This is the matched-dose uniform baseline, and it is the default. The square-root relation predicts it loses to concentration by about $\sqrt{f}$ on attended cells whenever only a small fraction of cells matter at any moment. If the measured gain is much smaller, or if operators attend to most of the field, the idea fails, and the experiments report this directly by sweeping the attended fraction.
Why not just let an algorithm pick the cells? This is the real threat. When the target is "whatever changes", a ratio-change detector may match or beat gaze, and the proposal says so. The comparison is therefore split into two task types. In signal-defined tasks, the instruction is to catch any heating event. In goal-defined tasks, the instruction is to follow cells of a particular shape, or cells near a pipette. The prediction is that gaze helps mainly in the second type, and that a hybrid in which the algorithm proposes cells and gaze confirms them may dominate both. The hybrid is run as a fourth condition so the result is not decided by framing.
Why not just click with a mouse? For a static field, clicking may be just as good, and that would be a fair negative result. The case for gaze is speed when transient events must be followed across the field while the hands are busy with perfusion or a micromanipulator. The click baseline is run under exactly that dual-task load.
Why not use fluorescence lifetime, which is also unaffected by intensity? Lifetime readout is also robust to patterned light, and neural-network-aided thermometry shows it can also separate viscosity from temperature. Its acquisition is slower, however, and the steering mechanism here applies to lifetime as well. This proposal tests ratiometry because it allows 50 ms frames. Extending the method to lifetime is noted as future work, not presented as a competing method.
Proposed Experiments
The first experiment is a phantom with known ground truth: gTEMP or a matched dye pair in a hydrogel, with programmable infrared heating spots that produce temperature events at known places and times. The heating spots stand in for biological events, so error can be scored exactly. Eight to twelve operators monitor fields of about 50 regions under every condition. Metrics are temperature root-mean-square error during events on monitored regions, event detection latency, the fraction of instructed targets missed, ratio drift per hour in unattended regions, and total delivered dose. Operators also rate usability on the System Usability Scale, which allows comparison with the 80.39 reported for gaze-plus-button selection.
The second experiment uses gTEMP-expressing HeLa cells. Mitochondrial uncoupling with FCCP, for which gTEMP reports a 6–9 °C rise, serves as the biological event. Cell morphology and survival over hours of imaging are compared across conditions at matched dose. Because the file contains a conflict on this point, the analysis also checks whether higher-quality attended measurements change the size of the FCCP rise. The neural-network thermometry paper reports that mitochondria cap at 42–43 °C and that viscosity can masquerade as heat. Because gTEMP cannot separate viscosity, any discrepancy is reported as unresolved, not as a temperature claim.
The decisive checks are three. The first is whether attended-cell error at matched dose falls by roughly the predicted $\sqrt{f}$. The second is whether the dose-ledger ablation shows a gaze-shaped drift artifact that the correction removes to within uniform-illumination levels. The third is whether gaze beats automated selection on goal-defined tasks while at least matching it on signal-defined ones. If the first check fails, the photon-budget premise is wrong. If the third fails, the gaze component should be dropped in favor of automated steering, leaving a dose-corrected patterned ratiometry method as the contribution.
References
- PureGaze: Purifying Gaze Feature for Generalizable Gaze Estimation
- De-confounded Gaze Estimation
- Genetically encoded ratiometric fluorescent thermometer with wide range and rapid response
- Gaze-Based Menu Navigation in Virtual Reality: A Comparative Study of Layouts and Interaction Techniques
- GazeSwipe: Enhancing Mobile Touchscreen Reachability through Seamless Gaze and Finger-Swipe Integration
- Fast Explicit-Input Assistance for Teleoperation in Clutter
- Decoding Cellular Temperature via Neural Network-Aided Fluorescent Thermometry