Introduction
In the rapidly evolving biomedical field, the ability to automatically summarize dense research papers is critical for both professionals and the public. However, standard sequence-to-sequence models like BART or T5 often suffer from "lexical superficiality." While these models are proficient at predicting word sequences based on surface patterns, they frequently fail to grasp the underlying factual connections between entities—such as specific drug-protein interactions. This failure manifests as hallucinations, redundancy, and a general inability to navigate complex medical jargon.
To address these challenges, the COGITOERGOSUMM framework introduces a neuro-symbolic approach that infuses large language models with structured semantic knowledge, ensuring that summaries are not just fluent, but factually grounded.
The COGITOERGOSUMM Framework
COGITOERGOSUMM extends the BART architecture by integrating two distinct types of semantic parsing graphs to capture the "meaning" of a document before generating a summary.
Dual Semantic Parsing Graphs
Unlike traditional models that rely solely on raw text, COGITOERGOSUMM extracts and encodes two complementary graph structures:
- Event Extraction (EE): A task-driven approach that identifies specific n-ary interactions (e.g., Binding, Regulation) between biological entities. This captures domain-specific biological facts.
- Abstract Meaning Representation (AMR): A linguistically grounded approach that captures high-level semantic relations (who did what to whom). This provides the general linguistic "glue" necessary for coherence.
As shown in the figure above, while simple binary relation extraction might fail to find a connection or oversimplify it, EE and AMR provide a rich, structured representation of complex sentences, such as those involving conditional triggers ("If over-expressed...") or multi-location movements ("...from cytoplasm to nucleus"). These sentence-level graphs are merged into a document-level structure using artificial "sentence nodes" and "master nodes" to maintain global context during the summarization process.
Consistency-Guided Reinforcement Learning
To further ensure factual accuracy, the researchers implemented a second stage of training using Proximal Policy Optimization (PPO). Standard models are typically trained to minimize word-prediction error (Cross-Entropy), which does not necessarily correlate with truthfulness.
In COGITOERGOSUMM, the reinforcement learning "reward" is based on the Smatch score—a metric that measures the overlap between the semantic graphs of the source document and the generated summary. This forces the model to prioritize preserving the core meaning of the text over simply matching keywords or phrases.
Evaluation and Key Insights
The model was evaluated on the Cochrane Database of Systematic Reviews (CDSR) dataset, which requires transforming professional medical abstracts into plain-language summaries.
Key Results:
- Efficiency: COGITOERGOSUMM (181M parameters) achieved results competitive with BART-large (406M parameters), demonstrating that structured knowledge is more efficient than raw model scale.
- Performance: The framework outperformed previous graph-augmented models like EASumm by 6 points in ROUGE-1/L scores.
- Human Evaluation: Blind tests with human experts revealed a 12.46% improvement in factualness and a 6.69% improvement in informativeness compared to standard BART models pre-trained on PubMed.
- Readability: The model successfully simplified complex terminology, producing summaries with lower Flesch-Kincaid grade levels without sacrificing semantic consistency.
Conclusion
COGITOERGOSUMM demonstrates that for a model to effectively summarize medical literature, it must first "understand" the semantic structure of the information. By combining domain-specific event extraction with general linguistic representation and optimizing for semantic consistency, the framework sets a new standard for faithful and succinct biomedical summarization. Future work will look toward deep metric learning to further refine the alignment between text and graph representations.
Summary updated