Introduction
Collecting a patient’s family medical history is a cornerstone of preventative medicine, particularly for identifying risks associated with hereditary diseases such as cancer. However, the current reliance on in-person genetic counseling creates a significant bottleneck due to time-intensive appointments and long waitlists. While automated systems like chatbots offer a potential solution, they face a fundamental trade-off: rigid, structured questionnaires are tedious for patients, whereas natural, narrative responses are notoriously difficult for computational systems to parse.
This research addresses this gap by introducing the first large-scale, publicly available dataset of natural language interactions specifically annotated for medical family history, alongside a robust baseline NLP pipeline for relation extraction.
The Family History Dataset
The study introduces a new, expert-annotated dataset comprising 228 family histories. These histories were gathered from two primary sources: 156 collected via Amazon Mechanical Turk (AMT) and 72 derived from actual genetic counseling session (GCS) test cases.
The dataset is characterized by its rich annotations, which go beyond simple entity recognition to include attributes such as age at diagnosis, cause of death, and complex relationships extending to the third degree (e.g., great-grandparents and cousins). In total, the corpus contains 4,304 sentences and 20,774 annotated relations.
Methodology: A Two-Stage Pipeline
To extract structured data from narrative text, the authors developed a two-stage pipeline focused on Relation Extraction.
1. Entity Identification
The first stage employs a Conditional Random Field (CRF) classifier using IOB (Inside, Outside, Beginning) tagging. This system identifies key entities within a sentence, such as family members (FT), names, ages, genders, and specific illnesses.
2. Relation Classification
Once entities are identified, the system must determine the semantic links between them—for instance, associating a specific age of diagnosis with the correct relative. This is achieved using a bi-directional LSTM (Bi-LSTM) classifier. For every pair of entities identified in a sentence, the model predicts the existence of a valid medical relation based on GloVe word embeddings, part-of-speech tags, and the relative positions of the entities.
Key Insights and Results
The baseline system demonstrated strong performance, achieving an overall F-score of 0.87 on complex sentences (those containing multiple relatives). When the system was evaluated solely on its ability to link pre-identified entities, the F-score rose to 0.933.
Several key insights emerged from the analysis:
- Narrative Elicitation: Choosing to elicit narrative responses rather than structured ones proved more patient-friendly and resulted in more comprehensive data collection.
- Data Density: AMT-sourced data was significantly denser than clinical GCS data, averaging 14.2 relations per complex sentence compared to 6.1.
- Cross-Domain Robustness: Models trained on crowdsourced data performed effectively when tested on clinical data, suggesting that crowdsourcing is a viable method for developing real-world medical NLP applications.
Conclusion
This work provides a foundational resource for the medical NLP community. By releasing a high-quality "gold standard" dataset (with an inter-annotator agreement Cohen’s Kappa of 0.8908), the authors enable future research into automated family history collection. Future work will focus on expanding the scope beyond cancer history and developing active "Dialog Agents" capable of real-time, intelligent follow-up questions.
The dataset and baselines are available at http://lit.eecs.umich.edu/downloads.html.
Summary updated