TL;DR
This paper presents a computer-based system for automatically generating thesauri, which are crucial for organizing and retrieving information in various domains. The system leverages linguistic theories and subject analysis techniques to create hierarchical, associative, and synonymous relationships between terms, enabling thesaurus generation in multiple languages.
What Problem Does This Paper Address?
The paper addresses the challenge of efficiently organizing and retrieving vast amounts of information, particularly in the context of libraries and information systems. Traditional methods of thesaurus construction are often manual, time-consuming, and can be inconsistent. The authors highlight the parallelism between library classification and the search for "universal forms" and "deep structure" in language, suggesting that linguistic theories can inform automated information organization. They aim to develop a system that can automatically generate thesauri, capturing complex relationships between terms, and can be adapted to different languages.
Methods
The core of the system is built upon Ranganathan and Bhattacharyya's subject analysis techniques, specifically POPSI (Postulate-based Permuted Subject Indexing). The system takes "subject-propositions" (natural language statements describing the subject of a document or query) as input. These propositions are then processed through several stages:
- Subject Analysis: Identifying elementary categories (Discipline, Entity, Action, Property) and modifiers within the subject-propositions.
- Modulation: Augmenting the subject-propositions with superordinates and synonymous terms.
- Relation Mapping: Representing relationships (NT/BT, RT, SYN) between terms using codes.
- Computer Coding: Serializing unique terms and replacing them with their serial numbers in the subject-propositions.
- Thesaurus Generation: Using programs like
CODER,GENTHES,GENCORD, andTRANSLATto process coded subject-propositions, generate term-pairs, create coordinate entries, and translate them back into natural language. - Multilingual Capability: The system can produce thesauri in different languages by simply creating a table of equivalent terms in the required language.
Datasets
The paper mentions that the programs were used to generate a thesaurus of Leather Technology terms using test data of about 1500 subject-propositions. No specific external datasets are detailed, and no new benchmarks are introduced.
Key Insights and Ideas
- Linguistic Parallelism: The strong analogy between library classification/indexing and linguistic deep structures is a foundational insight, suggesting that linguistic theories can be powerful tools for information organization.
- POPSI as a Foundation: The POPSI language, with its structured approach to subject representation, provides a robust framework for automated thesaurus construction.
- Beyond Keyword Lists: The paper emphasizes that a true thesaurus is more than just a list of grouped keywords; it requires capturing nuanced relationships (hierarchical, associative, synonymous) to be effective for retrieval.
- Multilingual Potential: The system's design allows for relatively straightforward translation into other languages, addressing a significant practical need in global information sharing.
- Role Indicators: The use of role indicators (e.g., "agent used," "type of") is crucial for disambiguating and categorizing the numerous associative relationships that can arise.
Key Results
The paper reports on the successful generation of a thesaurus for Leather Technology terms. The system processed approximately 1500 subject-propositions, resulting in 1851 unique terms and 13,717 entries. The generation process took about 3 months for input preparation and 10 minutes and 26.73 seconds of CPU time on an IBM System/370-155. The output thesaurus includes hierarchical (NT/BT), associative (RT), and synonymous (SYN) relationships.
Limitations and Future Work
- Manual Input: The initial preparation of subject-propositions, while automated in processing, still requires manual formulation.
- Expert Consensus: The paper acknowledges that establishing associative relationships often relies on expert consensus, and the validity of this consensus compared to literature analysis is a point of discussion.
- Computational Cost: While the system automates generation, the initial input preparation and the computational resources required for processing are significant.
- Future Work: The authors suggest that further research into the fundamentals of subject indexing languages and the development of more efficient algorithms for term relationship identification and modulation could lead to even more robust and versatile thesaurus generation systems. The potential for automatic generation of coordinate entries from NTs is also highlighted as an area of ongoing development.
Summary updated