TL;DR
This paper introduces a novel "report node" mechanism to improve Bayesian expert systems by accounting for human reporting biases in open-ended diagnostic questions. By modeling the tendency to report present symptoms over absent ones, and more severe symptoms before less severe ones, these systems can more accurately infer the likelihood of unobserved conditions, leading to more focused questioning and better diagnoses.
What Problem Does This Paper Address?
Traditional Bayesian expert systems, when faced with missing information in a diagnostic process, often assume that unobserved variables provide no useful information. This is problematic because human diagnosticians, and by extension patients interacting with expert systems, don't behave this way. They exhibit reporting biases: they are more likely to mention symptoms that are present than those that are absent, and more likely to report significant or urgent symptoms first. This paper addresses the gap in current expert systems by proposing a way to model these "reporting biases" and "reportability" to make diagnostic inferences more robust, especially in the early stages of interaction where open-ended questions are common.
Methods
The core contribution is the introduction of a "report node" mechanism. This mechanism models the event that a symptom is reported in response to an open-ended question. The authors propose two key assumptions for this model:
- Veridicality: A reported symptom is assumed to be present with probability 1.0. This simplifies the model by directly linking reported symptoms to their actual presence.
- Likelihood Independence: For symptoms that are not mentioned, their likelihoods are treated as mutually independent.
This report node mechanism is then integrated into a belief network. The authors further refine this by introducing global parameters for "reportability" (the probability a symptom is reported given it's present) and "reporting bias" (the ratio of reporting a present symptom versus an absent one). They also explore how to learn these parameters from user interactions and how to model the interaction between reportability and symptom severity.
Datasets
Not discussed in the paper. The authors use a nerve compression disorder knowledge base (Carpal Tunnel Syndrome - CTS) for testing their mechanism, which was developed by a hand surgeon. This is a domain-specific knowledge base rather than a general dataset.
Key Insights and Ideas
- The "Silent Symptom" is Informative: The absence of a reported symptom in response to an open-ended question is not necessarily a lack of information. Instead, it can be a strong indicator that the symptom is not present, especially if it's a symptom that people are generally likely to report.
- Reporting Biases Focus Diagnosis: Explicitly modeling reporting biases (e.g., present > absent, severe > mild) can significantly narrow down the differential diagnosis early on. This leads to more efficient and targeted questioning.
- Linguistic Parallels: The authors draw strong parallels between their reporting biases and concepts in linguistics, specifically Grice's Maxims of Conversation (Quality and Quantity) and scalar implicature. This suggests that modeling human communication patterns can lead to more intelligent AI systems.
- Learning User Preferences: The proposed mechanism can be extended to learn individual user biases and reportability preferences, making the expert system more personalized and adaptive.
Key Results
The paper demonstrates through examples and figures that incorporating the report node mechanism leads to:
- More Focused Differentials: In the cubital tunnel syndrome example, increasing the reporting bias significantly reduced the probability of competing disorders, focusing the diagnosis on more relevant conditions.
- Improved Inference for Unobserved Symptoms: The model allows for the inference of likelihood functions for unobserved symptoms, which in turn influences the probabilities of underlying diseases.
- Effective Modeling of Severity: The model for differing symptom severities showed that reporting a minor symptom can significantly decrease the likelihood of a major symptom being present, while the converse is not as strongly implied.
Limitations and Future Work
- Relaxing Independence: The authors acknowledge that the assumption of likelihood independence for unobserved symptoms might be too strong and suggest relaxing it in future work.
- Learning Global Parameters: While they propose a method for learning global reportability and bias, they note that using an optimization routine to find the optimal parameters might be an alternative approach.
- Impact on Final Diagnosis: The authors question whether the observation order (influenced by the open probe model) significantly impacts the final diagnosis, suggesting further experimentation is needed.
- Exploiting Linguistic Research: They believe further exploitation of contemporary linguistic research on implicature could further improve the model of open probe questions.
Summary updated