PRIMED for the Future: Purposing Raw Intake for Machine Learning-Enabled Detection (Final Report)
Sarah H. Sandholtz, Camilo Valdes, Nisha Mulakken, Matthew P. Torres, Aram Avila-Herrera, Jeffrey A. Drocco, Jose Manuel Martí, Jonathan Allen, Crystal Jaing, Nicholas A. Be2023 · DOI
The COVID-19 pandemic underscored the critical need for flexible analytical modalities agnostic to the identity of biological material. Traditional biodefense relies on "Select Agent Lists" of known pathogens, a reactive approach that creates systemic weaknesses against novel or engineered threats. This report from Lawrence Livermore National Laboratory (LLNL) evaluates the potential for agent-agnostic biodetection using artificial intelligence (AI) and machine learning (ML). By applying deep learning (DL) directly to raw, minimally processed datasets—such as electrical signals from DNA sequencers or raw mass spectra—researchers can identify anomalies and functional characteristics of potential threats in real-time. This landscape assessment explores the transition from reference-dependent bioinformatics to resilient, multi-scale systems capable of identifying "off-normal" biological signatures without prior knowledge of the specific agent.
Introduction: Moving Beyond the "Naughty List"
Traditional biodefense is fundamentally reactive, focusing on a catalog of known "bad actors" like Anthrax or Ebola. If a threat is novel, such as a newly emerged virus or a synthetically engineered organism, it may bypass current detection systems entirely. Furthermore, standard bioinformatics pipelines are often slow, requiring raw instrument data to be converted into human-readable formats (like A-C-G-T base pairs) before analysis can begin.
The PRIMED effort proposes a shift toward "bioagent-agnostic signatures" (BASs). By leveraging the capacity of deep learning models to recognize complex patterns in high-dimensional data, we can move toward a biodefense posture that identifies threats based on their functional potential or anomalous behavior rather than their taxonomic identity.
Methodology and the MLTRL Framework
The study provides a comprehensive landscape assessment of AI/ML platforms for biodetection. To evaluate these technologies consistently, the researchers utilized a specialized Machine Learning Technology Readiness Level (MLTRL) framework, which tracks the progression of an ML model from basic research to full deployment.
The Machine Learning Technology Readiness Level (MLTRL) framework used to assess the maturity of agent-agnostic detection tools.
Direct Raw Signal Processing: Using Convolutional Neural Networks (CNNs) to classify DNA based on the raw electrical current ("squiggles") from Oxford Nanopore sequencers, bypassing the time-consuming base-calling step.
Deep Embeddings & Large Language Models (LLMs): Treating DNA and protein sequences as "languages" using models like BERT and GPT to predict function and taxonomy for sequences lacking close relatives in existing databases.
Transfer Learning: Repurposing DL models originally trained on natural images to analyze complex 2D representations of mass spectrometry data.
Host-Based Signatures: Analyzing the host's transcriptional response to infection as an indirect indicator of pathogens that may be present at concentrations too low for direct detection.
Key Insights and Results
The assessment revealed that while the field is promising, most current technologies reside in the early stages of the MLTRL scale (Levels 0–3). However, specific results demonstrate significant potential:
Speed and Efficiency: Squiggle-based classifiers can identify human versus bacterial DNA in as little as one second of sequencing data. Tools like SquiggleNet demonstrated 75–95% accuracy in broad biological classification.
Reference-Independent Classification: The BERTax tool can classify the "superkingdom" of a DNA fragment with approximately 90% precision, even for novel sequences where traditional database-reliant methods fail.
Functional Labeling:SeqScreen can assign up to 32 different "Functions of Sequences of Concern" (FunSoCs), allowing researchers to understand what a sequence does, even if they cannot identify who it belongs to.
The "Squiggle" Advantage: Raw nanopore signals preserve information often lost in standard processing, such as epigenetic markers like DNA methylation.
Challenges and Future Directions
The transition to agent-agnostic detection faces several significant hurdles. Data scarcity is a primary concern, particularly for image-based classification where training sets are often limited. Furthermore, most models are currently validated on "clean" clinical samples; real-world environmental samples (e.g., air or wastewater) are much noisier and contain complex mixtures that can confound current algorithms.
Hardware lock-in also remains a risk, as many raw-signal models are tied to specific proprietary instruments. The authors recommend an integrated approach, combining multiple platforms—such as pairing nanopore sequencing with host-response signatures—to create a "convergence of capabilities" for the next generation of biosurveillance.
Project Information
This research was conducted under the Department of Homeland Security (DHS) HAC-TC AI/ML Agent Agnostic Biodetection program.
Administrative and contact information for the PRIMED project at Lawrence Livermore National Laboratory.
The COVID-19 pandemic demonstrated how a novel, elusive, and diffuse biological threat can engender uncertainty and misinformation, and it underscored the need for flexible analytical modalities agnostic to the identity of biological material. Yet even before the pandemic, recognition of the limitations of the current, list-based approach, which focuses on known pathogens and biotoxins, and of the importance of agent-agnostic biodetection was growing within the biosecurity community. In a 2018 report on “Biodefense in the Age of Synthetic Biology,” for example, the National Academy of Sciences stated that “an overreliance on the Select Agent List is a systematic weakness affecting many aspects of the United States’ current biodefense mitigation capability”. More recently, a group of biodefense researchers proposed the identification and adoption of “bioagent-agnostic signatures (BASs)” as a way of detecting and characterizing not only existing agents but also novel ones, an approach they believe will “enable a more flexible and resilient biodefense posture”. Indeed, the future of biodetection requires us to begin developing novel analytics that can identify anomalies and/or characteristics that indicate a potential threat, whether known or unknown, without looking for a specific signature that has been identified previously. To assess potential threats more rapidly, it is critical to develop agnostic artificial intelligence (AI)/machine learning (ML) systems that can be employed for real-time assessment of the nature and source of a perturbation. Such systems should be multi scale and multi-dimensional, integrating sensor data from a range of biological, chemical, and physical application spaces. Emerging deep learning (DL) models demonstrate exceptional promise for identification of discriminatory features within multi-dimensional datasets. DL models have the capacity to recognize and encode highly complex patterns in a wide range of input data modalities, including images, text, and biological/chemical/physical spectra. As such, they can execute a wide range of assessments and determinations that have traditionally required a human operator. The promise of advances in DL is apparent in the realm of human health and medicine. DL models have been validated for evaluating a variety of clinical threats to human health in a range of contexts, including infection and cancer, and they demonstrated improved performance in predicting stroke relative to human neurologists in some categories of data. Continuously evolving advances in AI/ML are expected to support more efficient evaluation of raw sequence, spectroscopy, and spectrometry data. For instance, recent advances and deployment of large language models (LLM) such as Generative Pre training Transformer (GPT) and Bidirectional Encoder Representations from Transformers (BERT) have already motivated application of these models for biological function prediction. As frameworks such as LLMs become larger and more complex in their representations, their capacity to serve as pre-trained models that can be fine-tuned for biological/biodetection purposes will similarly be amplified. While existing and emerging AI/ML have found broad applicability and use cases in the clinical sciences, development for environmental evaluation and biodetection has been limited. Functionalizing such capabilities for this purpose requires an understanding of the existing technical landscape and how the respective tools and algorithms are currently being employed. This landscape awareness then allows an assessment of the current practical capabilities of existing models and the anticipated requirements and development efforts that will be needed to adapt available algorithms for biodetection applications relevant to DHS. Leveraging expertise in biodetection, ML, and operational biodetection, the effort described in this report is comprised of a systematic landscape assessment (Subtask 2.1), comparative evaluation (Subtask 2.2), and formulation of a value proposition (Subtask 2.3) for the prospect of ML-enabled, agnostic biodetection from raw, or minimally-processed, datasets.
BibTeX
@misc{sandholtz2023primed,
author = {Sarah H. Sandholtz and Camilo Valdes and Nisha Mulakken and Matthew P. Torres and Aram Avila-Herrera and Jeffrey A. Drocco and Jose Manuel Martí and Jonathan Allen and Crystal Jaing and Nicholas A. Be},
title = {{PRIMED for the Future: Purposing Raw Intake for Machine Learning-Enabled Detection (Final Report)}},
year = {2023},
doi = {10.2172/2204943},
}