Abstract
The burden of disease is fundamental to understanding, prioritizing, and monitoring public health interventions. Cause of death is required to calculate the burden of disease, but in many parts of the developing world, deaths are neither detected nor given a cause. Verbal autopsy (VA) is a feasible way of assigning a cause to a death based on an interview with those who cared for the person who died. As civil registration and vital statistics (CRVS) systems improve in the developing world, VA is playing an increasingly important role in providing information about cause of death and the burden of disease. This paper motivates the creation of a global Symptom-Cause Information (SCI) Archive containing reference deaths with both a verbal autopsy and a cause assigned through an independent mechanism. This archive could provide training and validation data for refining, developing, and testing machine-based algorithms to automate cause assignment from VA data, improving the comparability of machine-assigned causes and providing a means to fine-tune individual cause assignment within specific contexts.
Introduction
Accurate mortality data is the bedrock of public health, yet more than half of global deaths are never officially registered. In regions without robust medical infrastructure, most people die at home, leaving no clinical record. To fill this gap, researchers and health officials utilize Verbal Autopsy (VA)—a process where a trained interviewer asks the deceased's family about the symptoms and circumstances leading to death.
Historically, physicians reviewed these interviews to assign a cause (Physician-Coding or PCVA). However, PCVA is slow, expensive, and prone to individual bias. While several automated algorithms—such as InterVA, Tariff, and InSilicoVA—have been developed to replace physician review, they often produce conflicting results. This paper identifies a critical bottleneck: these algorithms perform inconsistently not because their underlying logic is flawed, but because the Symptom-Cause Information (SCI)—the reference data used to "teach" the algorithm how symptoms relate to specific diseases—is fragmented, outdated, or unrepresentative of different regions.
The Proposed Symptom-Cause Information (SCI) Archive
This paper proposes a technical policy shift: the creation of a centralized, global repository of "labeled" deaths. The SCI Archive would be a living benchmark that continuously accepts data from cases where both a Verbal Autopsy and a cause of death assigned through a reliable independent mechanism are available. Reliable mechanisms include medical certification, standard autopsies, or "de-biased" physician coding where multiple doctors reach a consensus.
By centralizing this data, the global health community can move beyond static, foundational datasets like the PHMRC "Gold Standard" Dataset (which contains deaths from the early 2000s across six sites). An evolving archive would allow for the development of algorithms that are responsive to changing epidemiological landscapes and diverse geographic settings.
Methodological Framework
The VA process can be broken down into three distinct components:
- The Questionnaire: Standardized sets of questions, such as the WHO 2016 standard.
- The Interview: The actual data collection process in the field.
- The Analysis: The algorithm that maps symptoms to causes.
The SCI Archive focuses on the third component, providing the necessary data to refine the mapping of symptoms to causes. Most successful VA algorithms rely on Bayesian logic to calculate the probability of a cause given a set of symptoms ($Pr(c|s)$). To do this accurately, algorithms require precise conditional probabilities ($Pr(s|c)$)—the likelihood that a specific symptom appears given a specific cause. The Archive would provide the empirical basis for these probabilities.
Key Insights and Algorithmic Performance
Research into VA automation has yielded several critical insights regarding the relationship between data and logic:
- Data Quality vs. Algorithmic Logic: A primary finding is that the quality and relevance of the training data (the SCI) generally outweigh the differences in algorithm logic. A standard algorithm supplied with high-quality, context-specific data will likely outperform a more complex algorithm trained on generic or unrepresentative data.
- The Importance of Context: Algorithms often lose accuracy when trained on data from one region (e.g., a hospital in India) and applied to another (e.g., rural South Africa). The SCI Archive would facilitate "fine-tuning" algorithms to specific local contexts, accounting for regional variations in disease prevalence and symptom reporting.
- De-biasing Physician Data: To utilize physician-coded data for machine learning, the authors suggest a protocol where each death is coded by multiple physicians. This allows researchers to quantify and eliminate individual idiosyncratic biases, creating a cleaner training set for automated tools.
Comparative studies demonstrate the potential of these automated tools. For instance, when controlling for the SCI used, the InSilicoVA algorithm has shown strong performance relative to its peers. Furthermore, in evaluations using the PHMRC dataset, the "Tariff" algorithm has been shown to outperform human physicians in certain contexts, suggesting that automation can serve as a quality improvement measure rather than just a cost-saving one.
Limitations and Future Directions
The primary challenge to the SCI Archive is not technical, but organizational. Its success depends on the willingness of the global health community to share sensitive, de-identified data in a centralized repository.
Future work must focus on:
- Establishing a standard design for rigorous algorithm comparison.
- Developing a universal set of performance metrics to ensure researchers are evaluating tools on equal footing.
- Implementing the practical infrastructure required to accept, protect, and disseminate data continuously.
By treating symptom-cause information as a shared global resource, the public health community can significantly improve the accuracy of mortality data, leading to better-informed interventions and a clearer understanding of the global burden of disease.
Summary updated