The importance of sharing patient-generated clinical speech and language data

Kathleen C. Fraser, Nicklas Linz, Hali Lindsay, Alexandra KönigNAACL 2019

TL;DR

This paper argues that the field of computational linguistics for clinical applications is severely hampered by the lack of large, clinically-validated datasets of patient-generated speech and language. The authors propose that fostering data sharing across research groups and creating collaborative platforms are crucial steps to overcome this limitation and advance the field.

What Problem Does This Paper Address?

The paper tackles a critical bottleneck in the advancement of Natural Language Processing (NLP) for clinical applications: the scarcity of sufficient, high-quality data. While NLP has seen tremendous progress fueled by large datasets, studies involving clinically validated patient speech and language data are often limited by very small sample sizes. This is primarily due to the inherent difficulty and expense of collecting such data. The consequences of these small datasets are significant: they lead to a lack of diversity and representativeness, models that struggle to converge, poor generalizability to new data, and ultimately, limited clinical utility. The paper highlights that even attempts to use scraped data from social media, while addressing data volume, introduce new problems like diagnostic uncertainty and a lack of clinical validation.

Methods

This paper is primarily a position paper and a call to action, rather than a report on a specific experimental method. The "methods" discussed are proposals for how to address the data sharing problem. These include:

  • Promoting Data Sharing: The core "method" is advocating for increased collaboration and data sharing between clinical researchers (who collect data) and NLP researchers (who analyze it).
  • Collaborative Data Sharing Platform: A key proposal is the creation of a dedicated platform where NLP researchers can take a more active role in data curation, transcription, and annotation, thereby alleviating some of the burden on clinical researchers.
  • Best Practices for Data Collection: The paper outlines recommended practices for researchers collecting data, such as developing long-term data management plans, obtaining open and transparent consent, reviewing archival consent forms, collecting anonymizable data where possible, and using standardized file formats and transcription protocols.
  • Incentivizing Data Sharing: For researchers using shared data, the paper suggests making contributions to repositories (e.g., providing digitized data, transcriptions, scripts), incentivizing sharing through citations and collaborations, creating resources to lower technical barriers, and communicating openly with data owners.

Datasets

The paper does not introduce new datasets. Instead, it discusses the need for larger, clinically-validated datasets and references existing efforts and challenges in data collection and sharing. Examples of existing data repositories and projects mentioned include:

  • Linguistic Data Consortium (LDC): A well-established repository for speech and language corpora, which is beginning to include clinical databases for autism and neurodegenerative disorders.
  • TalkBank Project (AphasiaBank, DementiaBank): Initiatives that facilitate data sharing and aggregation for clinical speech research, with a focus on standardized protocols and data formats.

The paper emphasizes that the current datasets are often too small for robust machine learning analysis.

Key Insights and Ideas

  • The "Data Divide" in Clinical NLP: There's a fundamental disconnect between the data needs of NLP for clinical applications (large, validated datasets) and the practical realities of clinical data collection (expensive, time-consuming, privacy-sensitive).
  • Shared Responsibility is Key: Simply asking clinical researchers to share data isn't enough. A collaborative model where NLP researchers actively contribute to data preparation (transcription, annotation) is essential to make data sharing sustainable and beneficial for all.
  • Beyond "Solved Problems": While NLP has made strides in areas like machine translation, applying these to clinical contexts requires a different approach to data and validation, moving beyond easily accessible, non-clinical data.
  • Ethical and Practical Hurdles are Intertwined: Privacy concerns, legal regulations (like HIPAA), and professional ethics are significant barriers, but practical issues like data formatting and resource limitations also play a crucial role.
  • Data Sharing as a Scientific Imperative: Just as in other scientific fields (astronomy, climatology), aggregating data is necessary to build comprehensive pictures and achieve robust scientific conclusions.

Key Results

As this is a position paper, it does not present quantitative experimental results. The "results" are the arguments and recommendations made by the authors. The paper's key contribution is the articulation of the problem and a proposed path forward, which includes:

  • Argument for Necessity: Large, clinically-validated datasets are imperative for the field's progress.
  • Proposed Solutions: A collaborative data sharing platform and a multi-disciplinary working group are suggested as concrete next steps.
  • Best Practices: A set of actionable recommendations for researchers involved in data collection and usage.

Limitations and Future Work

Limitations:

  • Incomplete Recommendations: The authors acknowledge that their set of recommendations is preliminary and likely incomplete.
  • Burden on Clinical Researchers: While proposing a collaborative model, the paper notes that current contributions to projects like TalkBank are still largely from clinical researchers, implying the burden hasn't been fully shifted.
  • Practical Implementation Challenges: The paper doesn't delve into the specific technical architecture or funding models for the proposed collaborative platform, leaving these as areas for future work.

Future Work:

  • Creation of a Multi-Disciplinary Working Group: The authors advocate for forming such a group to review recommendations, gauge interest, define platform requirements, and plan for prototyping and sustainability.
  • Development of a Collaborative Data Sharing Platform: The core future direction is the establishment of a platform that facilitates continuous data aggregation and allows for specialized contributions from clinicians and NLP researchers.
  • Addressing Ethical and Legal Nuances: While acknowledged as challenges, the paper implies further work is needed to navigate the complex ethical and legal landscape of clinical data sharing.

Summary updated

BibTeX
@inproceedings{fraser2019importance,
  author = {Kathleen C. Fraser and Nicklas Linz and Hali Lindsay and Alexandra König},
  title = {{The importance of sharing patient-generated clinical speech and language data}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2019},
}
Figure