This study, conducted by researchers at Google Health, investigates whether deep learning models can truly "see" current hemoglobin levels in retinal photographs or if they are simply relying on static demographic proxies like age and sex. By tracking volunteers through the physiological changes of a blood donation, the team demonstrated that the AI can indeed detect subacute, dynamic changes in blood chemistry, moving the technology one step closer to being a viable non-invasive diagnostic tool.
TL;DR
Researchers used a "natural experiment"—blood donation—to prove that AI models predicting hemoglobin from eye photos are measuring real-time physiological changes rather than just guessing based on a patient's age or sex. The model successfully detected a significant drop in hemoglobin concentration 2–3 days after donation, confirming its potential for tracking disease progression and recovery.
What Problem Does This Paper Address?
Recent breakthroughs have shown that deep learning models can predict blood hemoglobin concentration ([Hb])—a key marker for anemia—simply by looking at retinal fundus photographs (standard "back-of-the-eye" images). However, a skeptical question remained: is the AI actually measuring the blood, or is it just a very good "profiler"?
Because factors like age, sex, and ethnicity are both visible in retinal photos and correlated with hemoglobin levels, the model might just be calculating a statistical "prior" (a guess based on demographics). If the model only relies on these non-modifiable traits, it would be useless for tracking a patient’s recovery from anemia or their response to treatment. This paper seeks to determine if the model is a "thermometer" (measuring current state) or a "demographic chart" (predicting risk).
Methods
The researchers designed a prospective, pre-post observational study centered around a blood drive.
- The Intervention: 14 volunteers donated approximately 475mL of blood (~10% of total volume).
- The Timing: Retinal photos were taken at three critical points:
- Immediately before donation (baseline).
- 1–2 hours after donation.
- 2–3 days after donation.
- The Model: They used an ensemble of artificial neural networks based on the Inception-v3 architecture. This model was previously trained to predict complete blood count variables from retinal images.
- The Logic: Physiologically, [Hb] concentration doesn't drop immediately after blood loss because you lose both red cells and plasma at the same rate. The concentration only drops 2–3 days later when the body replenishes the plasma volume (hemodilution). If the model is accurate, it should only show a significant drop at the 2–3 day mark.
Datasets
- Training: The model was originally trained using the UK Biobank, a massive dataset containing fundus photographs paired with lab-verified blood measurements.
- Evaluation: A new, small-scale dataset of 14 volunteers (9 male, 5 female) recruited during a blood drive at Google's campus in collaboration with the Stanford Blood Center.
Key Insights and Ideas
- The "Self-Control" Advantage: By measuring the same individuals before and after an intervention, the researchers effectively "canceled out" static variables like genetics, eye anatomy, and age. Any change in the AI's prediction had to be due to the blood donation.
- Catching the Hemodilution Window: The study cleverly aligned its imaging schedule with the body's homeostatic response. The fact that the model's prediction followed the expected physiological curve (a delayed drop) is strong evidence that it is sensitive to actual blood concentration.
- Volume vs. Concentration: The model showed a slight (though not statistically significant) drop immediately after donation. This suggests the AI might be picking up on subtle changes in total blood volume or retinal vessel filling, not just the concentration of hemoglobin itself.
Key Results
- Significant Detection: The model detected a mean decrease of 0.51 ± 0.10 g/dL in predicted hemoglobin at the 2–3 day mark (p = 0.009).
- Immediate Post-Donation: The change immediately after donation was only -0.25 g/dL and was not statistically significant (p = 0.056).
- Accuracy: At baseline, the model's Mean Absolute Error (MAE) was 0.76 g/dL, which is consistent with its performance on much larger, general populations (0.67 g/dL in the UK Biobank).
- Theoretical Alignment: The detected drop of ~0.5 g/dL was comparable to the theoretical expected drop of ~1.0 g/dL, especially considering the model's known tendency to overestimate low hemoglobin levels.
Limitations and Future Work
- Sample Size: The study was very small (n=14), which limits the ability to generalize the findings.
- Lack of a Control Group: There was no "non-donor" group to account for potential variations in lighting or camera setup between the different days of photography.
- Model Bias: The authors noted a "proportional bias" where the model struggles to accurately capture the full magnitude of low hemoglobin levels. They suggest that better calibration could improve the sensitivity of the tool.
- Next Steps: The authors call for randomized controlled trials and studies involving patients with actual clinical anemia to see if the model can track real-world treatment responses.
Summary updated