Abstract
DreamNLP is a memory-efficient natural language processing (NLP) system designed to extract metadata from Electronic Health Records (EHRs). By treating clinical reports as a continuous data stream and employing the Count Sketch algorithm, the system identifies frequent terms ("heavy hitters") and evaluates their relevance through Shannon entropy. This approach facilitates the rapid extraction of diagnostic features, such as BI-RADS scores from breast imaging reports, without extensive prior knowledge of dataset structure or high computational costs.
Introduction
Extracting structured information from unstructured Electronic Health Records (EHRs) presents significant challenges. Traditional NLP tools often demand substantial computational memory and detailed prior knowledge of report structures. As EHR data volumes grow, there is an increasing need for fast, memory-efficient methods to discover extractable data within large, unfamiliar text corpora. DreamNLP addresses this by providing a way to identify critical features across thousands of documents without the overhead of naive counting approaches.
Methodology
DreamNLP processes text documents as a continuous data stream. The core of the system is the Count Sketch Algorithm (CSA), which avoids storing every unique word by using a $t \times b$ matrix of counters updated via MurmurHash functions. This significantly reduces space complexity while tracking "heavy hitters."
Following term identification, the system calculates Shannon entropy to measure term distribution across reports. High entropy indicates terms common across the corpus (useful for broad features), while low entropy identifies terms specific to certain reports. The system also features a multidimensional histogram GUI for visualizing term-report relationships and extracting neighboring words for contextual understanding.
Data and Results
The system was tested on a private dataset of breast imaging reports (mammography, ultrasound, and MRI) following the BI-RADS format. Out of 61 reports, 46 were analyzed after automatic filtering of non-BI-RADS reports using low-entropy keywords.
DreamNLP demonstrated high efficiency, processing 24,487 terms to extract the top 100 in approximately 20 milliseconds. With matrix parameters $t=20$ and $b=10,000$, the Count Sketch estimates closely aligned with exact word frequencies.
The system successfully isolated critical diagnostic features without prior labeling. Extracted high-entropy terms included lesion size ("cm"), type ("mass"), location ("o'clock", "quadrant"), and BI-RADS assessment scores.
Insights and Future Directions
Applying data streaming algorithms like Count Sketch to clinical text effectively bypasses memory bottlenecks. Shannon entropy serves as a robust filtering mechanism, distinguishing universally useful metadata from highly specific terms. For instance, the unsupervised extraction of "upper outer" aligns with clinical reality, as this quadrant is the most frequent site for breast carcinoma.
Future work includes integrating the NegEx algorithm to handle negated expressions and utilizing extracted features for downstream machine learning and quality assurance. The methodology is expected to generalize well to other domains, such as legal document analysis.
Summary updated