Abstract
Classifying sentences in industrial, technical, or scientific reports can enhance text mining and information retrieval tasks with useful machine-readable metadata. This paper describes a search engine that employs sentence classification to search for abstracts from scholarly papers in Petroleum Engineering. The sentences were classified into four classes based on the popular IMRAD categories. We produced a dataset containing more than 2,200 manually labeled sentences from 278 scholarly articles in the field of Petroleum Engineering to be used as training and testing data. The classifier with the best results was logistic regression, with an accuracy of 86.4%. The information retrieval system built on top of the classification system yielded a mAP of 0.80.
Introduction
The rapid expansion of digital text repositories in technical fields makes locating specific information increasingly difficult. While domains like biomedicine benefit from advanced tools like PubMed that utilize metadata for filtering, the Petroleum Engineering sector has lacked specialized semantic search capabilities. Standard search engines often treat documents as a "bag of words," failing to distinguish between a term mentioned in a background "Introduction" versus a core "Result."
Furthermore, keyword-based searches often fail due to the "zero-frequency" problem, where a query misses a relevant document because it uses a synonym (e.g., "expenditure elimination" vs. "reducing costs"). This research aims to bridge these gaps by creating a system that understands the structural role of every sentence in an abstract and utilizes semantic similarity for retrieval.
Methodology
The researchers proposed a two-stage pipeline integrating Natural Language Processing (NLP) with Information Retrieval (IR).
1. Sentence Classification
The system adapts the standard IMRAD (Introduction, Methods, Results, and Discussion) framework to categorize sentences into four classes: Introduction, Objectives, Methods, and Results/Conclusion. Three supervised learning algorithms were evaluated: Naive Bayes, Support Vector Machines (SVM), and Logistic Regression. A key feature used for classification was sentence position (scaled 0 to 1), which significantly improved accuracy by leveraging the predictable structure of scientific abstracts.
2. Information Retrieval with Semantic Smoothing
To move beyond simple string matching, the team built a probabilistic language model integrated with word embeddings (specifically a pre-trained GloVe model). This allows the system to calculate the probability of relevance based on semantic similarity, effectively handling synonyms and technical variations.
Data and Evaluation
The study introduced a new resource for the field: the Petroleum Engineering Dataset, consisting of 2,263 sentences manually labeled by experts from 278 scholarly articles. Human annotators achieved a strong Kappa score of 0.897, validating the dataset's reliability. For IR testing, a separate collection of 320 scientific papers was used to evaluate performance against 50 user-generated queries.
Results and Analysis
Experimental results demonstrated that Logistic Regression was the most effective classifier, reaching a peak accuracy of 86.4%. It proved more robust than Naive Bayes in handling the high-dimensional, sparse feature space characteristic of technical vocabulary.
The IR system yielded a mean Average Precision (mAP) of 0.80, indicating that 80% of the top results were deemed relevant by human judges. Qualitatively, the system successfully retrieved documents for queries like "Reduce Costs" by identifying semantically related phrases such as "cost reduction" and "eliminate expenditures."
One identified challenge was the "Objectives" class, which had the lowest recall (0.76). These sentences are critical for researchers but are often difficult for models to distinguish from the "Introduction" without more extensive training data.
Conclusion
By combining structural sentence classification with semantic word embeddings, this research provides a more nuanced and effective search tool for Petroleum Engineering. Future work will focus on expanding the labeled dataset to improve "Objectives" detection and testing the pipeline on less structured industrial reports and technical guides.
Summary updated