Abstract
Decision making theories such as Fuzzy-Trace Theory (FTT) suggest that individuals tend to rely on gist, or bottom-line meaning, in the text when making decisions. In this work, we delineate the process of developing GisPy, an open-source tool in Python for measuring the Gist Inference Score (GIS) in text. Evaluation of GisPy on documents in three benchmarks from the news and scientific text domains demonstrates that scores generated by our tool significantly distinguish low vs. high gist documents. Our tool is publicly available to use at: https://github.com/phosseini/GisPy.
1. Introduction
In cognitive psychology, Fuzzy-Trace Theory (FTT) posits that human memory and reasoning rely on two distinct mental representations: verbatim and gist. Verbatim representations capture exact surface details and words, which tend to fade quickly. In contrast, gist representations capture the essential, bottom-line meaning of a message. Gist is more robust over time and serves as the primary driver for human decision-making.
Despite the critical role of gist in communication, Natural Language Processing (NLP) research has historically focused on surface-level metrics like readability or structural cohesion. While tools such as Coh-Metrix exist, they are often proprietary or lack the scalability required for modern large-scale document processing. GisPy was developed to address this gap, providing a transparent, Python-based framework to quantify the "gist-friendliness" of a text.
2. Methodology
2.1 The GisPy Pipeline
GisPy processes text through a multi-stage pipeline leveraging state-of-the-art NLP libraries, including spaCy for tokenization, Stanza for coreference resolution, and Sentence Transformers for semantic embeddings. The architecture is designed to move from raw text to high-level semantic features.
2.2 Linguistic Indices and GIS Calculation
The tool extracts features across five primary dimensions to calculate the final Gist Inference Score (GIS):
- Referential Cohesion: Measures the overlap of ideas across sentences using cosine similarity of embeddings.
- Deep Cohesion: Identifies causal and intentional connectives (e.g., "because," "therefore") that help readers link events.
- Verb Overlap: Utilizes WordNet and language models to determine semantic relationships between verbs, facilitating the construction of a mental "situation model."
- Concreteness & Imageability: Quantifies the tangibility of the language used.
- Hypernymy: Measures the specificity of nouns and verbs within a semantic hierarchy.
The final GIS is a linear combination of these normalized indices. As shown in the formula below, the score increases with cohesion and verb overlap while decreasing with high levels of specificity (hypernymy) and concreteness, which are markers of verbatim-heavy text.
3. Evaluation and Benchmarking
The authors validated GisPy across three distinct datasets to ensure cross-domain robustness:
- News Reports vs. Editorials: Reports (low gist) focus on factual details, while editorials (high gist) focus on interpretation.
- Scientific Methods vs. Discussion: "Methods" sections are detail-oriented (low gist), whereas "Discussion" sections synthesize findings (high gist).
- Disneyland Measles Outbreak Data: A standardized dataset of 191 articles manually annotated for gist by human workers.
4. Key Results and Insights
GisPy successfully distinguished between low and high gist documents across all benchmarks with statistical significance ($p < 0.05$).
- Performance Benchmarking: On the Disneyland dataset, GisPy achieved a mean score difference ("distance") of 2.418, significantly outperforming the Coh-Metrix baseline of 0.998.
- Domain Versatility: In the scientific domain, GisPy achieved a t-statistic of 7.188 when separating Methods from Discussion sections, proving its efficacy in technical contexts.
- Structural Importance: A key finding was that indices calculated at the paragraph level performed better than those ignoring paragraph boundaries. This suggests that the way humans organize thoughts into paragraphs is central to how gist is communicated and extracted.
5. Conclusion and Future Work
GisPy provides a theoretically grounded, scalable tool for measuring gist in text. While effective, future iterations aim to incorporate discourse parsers to identify implicit causality and explore more sophisticated coreference chain analysis. The tool is open-source and available for the research community to facilitate further studies into cognitive-based text analysis.
Summary updated