Set to Ordered Text: Generating Discharge Instructions from Medical Billing Codes

Litton J. Kurisinkel, Nancy F. Chen

Abstract

Writing discharge instructions is a critical yet labor-intensive task in clinical settings. This paper introduces the "set to ordered text" task, which aims to automatically generate structured discharge instructions from a set of medical billing (ICD) codes. We propose a novel neural architecture that jointly learns to group relevant codes into subsets, determine an optimal sequence for instructions, and generate coherent natural language. By incorporating a redundancy penalization mechanism, our model significantly outperforms standard sequence-to-sequence baselines in both content generation and logical ordering.

Introduction

In modern healthcare, medical billing codes (ICD codes) serve as a standardized method for documenting patient diagnoses and procedures. However, these codes exist as an unordered "bag of entities" lacking narrative structure. Converting this data into clear, actionable discharge instructions for patients is a complex Natural Language Generation (NLG) challenge.

Unlike traditional summarization or data-to-text tasks, generating instructions from ICD codes requires significant inference. The output is not a direct description of the input; rather, the model must infer the necessary clinical advice associated with specific code combinations. Furthermore, the model must learn an unsupervised ordering for these instructions, as the input set provides no sequential cues.

Overview of the Set-to-Ordered Text task, mapping input ICD sets to ordered discharge instructions via subsets.

Proposed Architecture

To address the transition from an unordered set to a structured sequence of sentences, we developed a specialized neural architecture. The model moves beyond simple sequence-to-sequence mapping by explicitly modeling the grouping and sequencing of information.

Content and Subset Selection

The model utilizes attention-based gates—specifically Content Selection and Set Selection gates—to identify relevant features from individual ICD codes. These gates allow the model to group codes into meaningful subsets, acknowledging that each instruction typically corresponds to a specific clinical theme (e.g., wound care or medication management).

Hierarchical RNN Structure

The generation process is managed by two distinct Recurrent Neural Networks (RNNs):

  1. Instruction Ordering RNN: A GRU-based controller that maintains the overall narrative state. It decides which subset of information should be processed at each step of the discharge report.
  2. Instruction Decoder RNN: A separate GRU that receives the selected subset and generates a single, coherent instruction sentence.
The detailed neural architecture featuring self-attention, selection gates, and hierarchical RNNs for instruction ordering and decoding.

Redundancy Penalization

A common failure mode in long-form text generation is "looping," where the model repeats information. To mitigate this, we modified the beam search algorithm to include a penalization factor based on Jaccard Similarity. This discourages the decoder from repeating words or concepts that have already appeared in previous instructions within the same report.

Experimental Setup

The study utilized the MIMIC-III (Medical Information Mart for Intensive Care) dataset. The authors extracted 18,900 training pairs of ICD codes and discharge instructions. To ensure data quality, a logistic regression classifier was employed to filter out non-instructional boilerplate text (e.g., signatures and headers) from the raw reports. All sensitive information was anonymized using standard placeholder tags.

Results and Discussion

The proposed model, Set2MultipleSeq+Opt, demonstrated substantial improvements over standard baselines across multiple metrics:

  • Content Generation: The model achieved a BLEU score of 12.6, nearly doubling the performance of a standard Seq2Seq baseline (6.3).
  • Logical Ordering: Using a specialized F-measure for content ordering, the model scored 0.22, a significant improvement over the baseline's 0.07. This indicates a much stronger grasp of how clinical instructions should be sequenced.
  • Human Evaluation: In blind qualitative assessments, human evaluators preferred the proposed model's output as more grammatical 77% of the time and more informative 61% of the time compared to simpler variants.

The results highlight that treating the output as a sequence of discrete instructions (Set2MultipleSeq) is far more effective than generating a single continuous string. This "set-to-multiple-sequence" approach better mirrors the cognitive process of clinical documentation.

Conclusion

This work establishes a robust framework for generating structured text from unordered entity sets. By jointly modeling subset selection and instruction ordering, the proposed architecture captures the underlying logic of medical documentation. Future research will explore incorporating broader clinical contexts beyond billing codes to further enhance the specificity and personalization of generated patient instructions.

Summary updated

Abstract

We present set to ordered text, a natural language generation task applied to automatically generating discharge instructions from admission ICD (International Classification of Diseases) codes. This task differs from other natural language generation tasks in the following ways: (1) The input is a set of identifiable entities (ICD codes) where the relations between individual entity are not explicitly specified. (2) The output text is not a narrative description (e.g. news articles) composed from the input. Rather, inferences are made from the input (symptoms specified in ICD codes) to generate the output (instructions). (3) There is an optimal order in which each sentence (instruction) should appear in the output. Unlike most other tasks, neither the input (ICD codes) nor their corresponding symptoms appear in the output, so the ordering of the output instructions needs to be learned in an unsupervised fashion. Based on clinical intuition, we hypothesize that each instruction in the output is mapped to a subset of ICD codes specified in the input. We propose a neural architecture that jointly models (a) subset selection: choosing relevant subsets from a set of input entities; (b) content ordering: learning the order of instructions; and (c) text generation: representing the instructions corresponding to the selected subsets in natural language. In addition, we penalize redundancy during beam search to improve tractability for long text generation. Our model outperforms baseline models in BLEU scores and human evaluation. We plan to extend this work to other tasks such as recipe generation from ingredients.
BibTeX
@inproceedings{kurisinkel2019set,
  author = {Litton J. Kurisinkel and Nancy F. Chen},
  title = {{Set to Ordered Text: Generating Discharge Instructions from Medical Billing Codes}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2019},
  doi = {10.18653/v1/d19-1638},
}
Figure