Noise Pollution in Hospital Readmission Prediction: Long Document Classification with Reinforcement Learning

Liyan Xu, Julien Hogan, Rachel E. Patzer, Jinho D. Choi

Abstract

Predicting 30-day hospital readmissions after specialized surgeries like kidney transplants is a critical task for improving patient outcomes. However, clinical notes in Electronic Health Records (EHRs) present significant challenges: they are exceptionally long, often exceeding 10,000 tokens; datasets are small due to the specialized nature of the procedures; and the text is saturated with "noise pollution" such as HTML tags and irrelevant tabular data. This paper evaluates various text encoders and proposes a Reinforcement Learning (RL) framework to automatically prune non-informative text segments. Our findings reveal that simple Bag-of-Words (BoW) models outperform complex deep learning architectures in these low-data, high-noise environments. Additionally, the RL pruning agent successfully discarded approximately 25% of document content while maintaining or slightly improving predictive performance.

Introduction

In the clinical domain, predicting patient readmission is essential for resource allocation and quality of care. While structured EHR data (vitals, labs) is commonly used, unstructured clinical notes contain nuanced information that is often overlooked. However, standard Natural Language Processing (NLP) models struggle with clinical text. The Emory Kidney Transplant Dataset (EKTD) exemplifies these hurdles: it contains fewer than 2,000 patients, yet individual clinical records are massive and cluttered with repetitive headers, lab "fishbones," and medication lists that do not necessarily correlate with readmission risk. This study addresses whether automated "noise extraction" can improve model performance and efficiency.

Methodology

Encoder Benchmarking

The researchers first established baselines by testing four distinct document representation methods:

  1. Bag-of-Words (BoW): Utilizing TF-IDF scores to represent token importance.
  2. Averaged Word Embeddings (AWE): Using fastText to capture semantic meaning through vector averaging.
  3. ClinicalBERT: A transformer-based model pre-trained on large-scale clinical corpora.
  4. Weight-dropped LSTM: A recurrent neural network optimized for long sequences through aggressive regularization.

Reinforcement Learning for Document Pruning

To handle the high noise floor, the authors implemented an RL agent designed to act as a selective filter. The problem is framed as a sequential decision process where the agent observes a text segment (the state) and decides whether to Keep or Prune it (the action).

The Reinforcement Learning framework for document pruning, illustrating the interaction between the policy network, the environment (text segments), and the downstream classifier.

The reward function balances two objectives: maximizing the accuracy of the downstream classifier and a "pruning reward" that penalizes excessive verbosity. The policy network was trained using the REINFORCE algorithm with entropy regularization to encourage exploration of different pruning strategies.

Experimental Setup

The study utilized the Emory Kidney Transplant Dataset (EKTD), comprising 2,060 patients. The unstructured data includes eight specific clinical note types, such as Discharge Summaries (DS), Progress Notes (PG), and Surgical Reports (OP). The dataset is characterized by a 3:7 class imbalance (readmitted vs. not readmitted) and extreme document lengths, making it a challenging benchmark for modern NLP.

Results and Discussion

The Superiority of Simple Models

Contrary to trends in general NLP, deep learning models like ClinicalBERT and LSTMs performed significantly worse than the BoW baseline. In small-scale clinical datasets, complex models tend to overfit by memorizing noise rather than learning generalizable clinical indicators. The BoW model achieved the highest baseline Area Under the Curve (AUC) of 62.3% on Discharge Summaries.

Comparison of AUC performance across different encoders and clinical note types, showing the consistent lead of Bag-of-Words models.

Impact of RL-Based Pruning

Integrating the RL pruning layer led to a modest but consistent improvement in AUC (approximately 1%) across most note types. More importantly, the agent learned to identify and remove "task-specific noise." It frequently pruned high-frequency tokens like "chloride" or "hemoglobin" when they appeared in tabular formats, recognizing that while these values are clinically relevant in a vacuum, their presence in massive tables often obscures the predictive signal for 30-day readmission. The agent successfully pruned between 19% and 26% of text in several note categories, effectively streamlining the data without losing critical information.

Conclusion

This research highlights the "Deep Learning Trap" in specialized clinical tasks where data is scarce and noise is high. By using Reinforcement Learning as an automated data cleaner, researchers can strip away the "dead weight" of clinical documentation. Future work will focus on multimodal integration, combining these pruned text features with structured EHR data to further enhance the precision of hospital readmission forecasts.

Summary updated

Abstract

This paper presents a reinforcement learning approach to extract noise in long clinical documents for the task of readmission prediction after kidney transplant. We face the challenges of developing robust models on a small dataset where each document may consist of over 10K tokens with full of noise including tabular text and task-irrelevant sentences. We first experiment four types of encoders to empirically decide the best document representation, and then apply reinforcement learning to remove noisy text from the long documents, which models the noise extraction process as a sequential decision problem. Our results show that the old bag-of-words encoder outperforms deep learning-based encoders on this task, and reinforcement learning is able to improve upon baseline while pruning out 25% text segments. Our analysis depicts that reinforcement learning is able to identify both typical noisy tokens and task-specific noisy text.
BibTeX
@inproceedings{xu2020noise,
  author = {Liyan Xu and Julien Hogan and Rachel E. Patzer and Jinho D. Choi},
  title = {{Noise Pollution in Hospital Readmission Prediction: Long Document Classification with Reinforcement Learning}},
  booktitle = {BioNLP},
  year = {2020},
  url = {https://openreview.net/forum?id=hjmd9N3gQY},
}
Figure