An Informational Space based Semantic Analysis for Scientific Texts

Neslihan Süzen, Alexander N. Gorban, Jeremy Levesley, Evgeny M. Mirkes

Introduction

Natural Language Processing (NLP) has long grappled with the inherent ambiguity of human language. In the context of scientific literature, the primary function of language is representation—the precise transfer of complex concepts from the author to the reader. However, traditional computational methods often struggle to distinguish between generic "service words" (e.g., analysis, using, study) and the domain-specific terms that carry the weight of scientific meaning. This paper proposes a novel framework for "Informational Semantics," which moves beyond simple word frequency to represent meaning through a high-dimensional geometric space based on information theory.

Methodology: From Frequency to Information

The researchers transition from traditional bag-of-words models to a system based on Relative Information Gain (RIG). This approach is rooted in the Wittgensteinian philosophy that "meaning is use." By analyzing how words are distributed across different scientific disciplines, the authors quantify how much information a specific word provides about the context of a text.

The Meaning Space Pipeline

The methodology follows a rigorous three-step pipeline: Corpus → Meaning Space → Geometric Representation.

  1. Relative Information Gain (RIG): Instead of counting occurrences, the system calculates the RIG of a word relative to specific scientific categories. A word like "polymer" possesses high RIG because its presence significantly increases the probability that a text belongs to the field of Chemistry.
  2. Vector Construction: Each word is mapped as a 252-dimensional vector. Each dimension corresponds to one of the 252 Web of Science (WoS) subject categories, with the value representing the word's RIG for that specific field.
  3. Dimensionality Reduction: To identify the underlying structure of scientific discourse, the authors applied Principal Component Analysis (PCA). This process compresses the 252-dimensional space into a set of "Principal Components of Meaning," identifying which scientific categories are semantically linked.

Data Resources

The study leverages three primary resources developed by the authors to facilitate this analysis:

  • Leicester Scientific Corpus (LSC): A dataset comprising 1,673,350 scientific abstracts.
  • Leicester Scientific Dictionary-Core (LScDC): A lexicon of 103,998 unique words extracted from the LSC.
  • Leicester Scientific Thesaurus (LScT): A curated list of the 5,000 most informative words in science, ranked by their average RIG across all categories.

Key Insights and Results

Frequency vs. Informativeness

A critical finding of the study is that word frequency is often a distraction in semantic analysis. Word clouds generated by frequency are typically dominated by generic terms that appear across all disciplines. In contrast, word clouds generated by RIG immediately surface topic-specific terminology (e.g., "protein" and "enzyme" for biological sciences), proving that RIG is a superior proxy for semantic content.

The Dimensions of Scientific Meaning

The PCA revealed that the vast landscape of scientific discourse is highly structured and redundant. While the original space contained 252 dimensions, the researchers found that as few as 13 to 16 dimensions (determined by the Broken Stick rule and PCA-CN) could capture the essential variation of the entire space.

Through qualitative analysis, the authors labeled the first five "Dimensions of Meaning":

  • PC1 (Magnitude): Represents general informativeness across all scientific categories.
  • PC2 (Nature of Science): Distinguishes between social sciences/humanities and the natural/engineering sciences.
  • PC3 (Human Scale): Contrasts microscopic or technical research (e.g., microscopy, coding) with population-scale or human-centric research.
  • PC4 (Human Condition): Separates experimental studies of human biology/behavior from critical discourse and literary criticism.
  • PC5 (Inner vs. Outer World): Contrasts the study of the physical world with the study of intelligence, including both psychology and artificial intelligence.

Conclusion and Future Work

This research demonstrates that the "meaning" of scientific text can be quantified and mapped into a manageable geometric space. By identifying the core dimensions of scientific discourse, the authors provide a roadmap for more sophisticated NLP tools that understand context rather than just counting tokens.

Future work will focus on applying this "Meaning Space" to full-text representations and downstream tasks. While the first five dimensions offer clear qualitative interpretations, further study is required to articulate the philosophical significance of the remaining higher-order dimensions. Additionally, the researchers aim to explore how these semantic spaces evolve as new interdisciplinary fields emerge and redefine the boundaries of scientific knowledge.

Summary updated

Abstract

One major problem in Natural Language Processing is the automatic analysis and representation of human language. Human language is ambiguous and deeper understanding of semantics and creating human-to-machine interaction have required an effort in creating the schemes for act of communication and building common-sense knowledge bases for the ‘meaning’ in texts. This paper introduces computational methods for semantic analysis and the quantifying the meaning of short scientific texts. Computational methods extracting semantic feature are used to analyse the relations between texts of messages and ‘representations of situations’ for a newly created large collection of scientific texts, Leicester Scientific Corpus. The representation of scientific-specific meaning is standardised by replacing the situation representations, rather than psychological properties, with the vectors of some attributes: a list of scientific subject categories that the text belongs to. First, this paper introduces ‘Meaning Space’ in which the informational representation of the meaning is extracted from the occurrence of the word in texts across the scientific categories, i.e., the meaning of a word is represented by a vector of Relative Information Gain about the subject categories. Then, the meaning space is statistically analysed for Leicester Scientific Dictionary-Core and we investigate ‘Principal Components of the Meaning’ to describe the adequate dimensions of the meaning. The research in this paper conducts the base for the geometric representation of the meaning of texts.
BibTeX
@misc{suzen2022informational,
  author = {Neslihan Süzen and Alexander N. Gorban and Jeremy Levesley and Evgeny M. Mirkes},
  title = {{An Informational Space based Semantic Analysis for Scientific Texts}},
  year = {2022},
  eprint = {2205.15696},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2205.15696},
}
Figure