Automated Coding of Under-Studied Medical Concept Domains: Linking Physical Activity Reports to the International Classification of Functioning, Disability, and Health

Denis Newman-Griffis, Eric Fosler-Lussier

Abstract

Linking clinical narratives to standardized vocabularies is essential for unlocking medical data, yet many domains—such as physical mobility—lack the mature terminologies required for effective automated coding. This paper presents a framework for mapping narrative "activity reports" to the International Classification of Functioning, Disability, and Health (ICF). By evaluating classification and candidate selection paradigms using domain-specific word embeddings and deep learning, the authors achieved a macro F-1 score of 84%. The study demonstrates that high-performing medical NLP is achievable in under-studied domains through the strategic combination of small annotated datasets, expert definitions, and representative text corpora.

The Challenge of Functional Status Information

While medical Natural Language Processing (NLP) has successfully automated the identification of diseases and medications, Functional Status Information (FSI)—data describing how patients move and interact with their environment—remains a significant challenge. FSI is vital for rehabilitation, disability assessment, and geriatric care, but it is notoriously difficult to extract from clinical notes.

The primary hurdle is the "semantic gap" between clinical practice and formal standards. The ICF is a classification hierarchy rather than a clinical terminology; it does not reflect the shorthand or specific phrasing used by clinicians. Consequently, mapping a therapist's observation to a formal code requires bridging a divide between natural language and rigid conceptual structures.

Anatomy of an activity report and its mapping to an ICF code (d450).

Methodological Framework

The researchers investigated two primary paradigms to automate this mapping process:

  1. Classification: This approach treats ICF codes as fixed categories. The team tested established models, including k-Nearest Neighbors (KNN), Support Vector Machines (SVM), and Deep Neural Networks (DNN), utilizing both unigram features and word embeddings.
  2. Candidate Selection: A more flexible approach where the model compares the mathematical representation of a clinical note to the definition of an ICF code. The authors introduced a context-dependent projection model, where a DNN learns to project code definitions into a vector space that specifically matches the context of the activity report being analyzed.

To isolate the difficulty of the task, they also tested an "Action Oracle" setting. In this scenario, the model is explicitly told which word represents the physical action (e.g., "walking"), allowing researchers to measure how much performance is lost due to the difficulty of distinguishing the core action from surrounding context (like assistive devices or locations).

Data and Embedding Strategies

The study utilized 400 Physical Therapy records from the NIH Clinical Center, comprising 4,527 annotated activity reports mapped to 12 distinct ICF mobility codes. To bridge the semantic gap, the authors experimented with various pre-training corpora for word embeddings, ranging from general web text to highly specialized clinical notes.

Summary of pre-training corpora used for word embeddings, highlighting the trade-off between size and representativeness.

Key Insights and Results

Representativeness Trumps Size

A critical finding was that the representativeness of a corpus is more important than its sheer volume. The domain-specific PT-OT corpus, despite containing only 1.2 million words, consistently outperformed the massive Google News corpus (100 billion words). The specialized jargon and shorthand captured in the PT-OT data proved more valuable for the model than the broad linguistic knowledge of a general-purpose dataset.

The Importance of Action Extraction

Access to the "Action Oracle" boosted performance by 12–20%, with the best system (an SVM using domain-specific embeddings) reaching a macro-averaged F1 score of 84%. Without the oracle, performance dropped to approximately 64–65%. This indicates that identifying the core action within a complex sentence is the most significant bottleneck in automated FSI coding.

Model Complexity vs. Data Size

Interestingly, standard SVMs with domain-specific static embeddings often matched or outperformed complex BERT-based models. This is likely due to the relatively small size of the annotated dataset, which may not provide enough signal for the massive parameter space of contextualized models like BioBERT to fully excel.

Conclusion and Future Directions

This research highlights that automated coding in specialized medical domains is feasible without massive datasets, provided that domain-specific corpora are used for feature representation. While classification models currently offer the highest accuracy, the authors suggest that candidate selection is the more scalable path forward, as it allows for the inclusion of new codes based on their definitions without requiring model retraining.

Future work will focus on "joint modeling"—integrating the extraction of activity reports and their subsequent coding into a single, cohesive step to improve real-world clinical utility.

Summary updated

Abstract

Linking clinical narratives to standardized vocabularies and coding systems is a key component of unlocking the information in medical text for analysis. However, many domains of medical concepts lack well-developed terminologies that can support effective coding of medical text. We present a framework for developing natural language processing (NLP) technologies for automated coding of under-studied types of medical information, and demonstrate its applicability via a case study on physical mobility function. Mobility is a component of many health measures, from post-acute care and surgical outcomes to chronic frailty and disability, and is coded in the International Classification of Functioning, Disability, and Health (ICF). However, mobility and other types of functional activity remain under-studied in medical informatics, and neither the ICF nor commonly-used medical terminologies capture functional status terminology in practice. We investigated two data-driven paradigms, classification and candidate selection, to link narrative observations of mobility to standardized ICF codes, using a dataset of clinical narratives from physical therapy encounters. Recent advances in language modeling and word embedding were used as features for established machine learning models and a novel deep learning approach, achieving a macro F-1 score of 84% on linking mobility activity reports to ICF codes. Both classification and candidate selection approaches present distinct strengths for automated coding in under-studied domains, and we highlight that the combination of (i) a small annotated data set; (ii) expert definitions of codes of interest; and (iii) a representative text corpus is sufficient to produce high-performing automated coding systems. This study has implications for the ongoing growth of NLP tools for a variety of specialized applications in clinical care and research.
BibTeX
@inproceedings{newmangriffis2021automated,
  author = {Denis Newman-Griffis and Eric Fosler-Lussier},
  title = {{Automated Coding of Under-Studied Medical Concept Domains: Linking Physical Activity Reports to the International Classification of Functioning, Disability, and Health}},
  booktitle = {Frontiers Digit. Health},
  year = {2021},
  url = {https://openreview.net/forum?id=7NsfIMtZO2},
}
Figure