Extracting Professional Preferences of Users from Natural Language Essays

Cigdem Toprak, Christof Müller, Iryna GurevychTUbilio 2015

Abstract: This paper presents an unsupervised sentiment analysis approach for extracting professional preferences of users from natural language essays in a career recommendation scenario. Our system first extracts terms facilitating career recommendation such as objects, activities, hobbies, and places from the essays. Then, it applies a lexicon-based sentiment analysis approach to assign polarities representing user preferences.

The Challenge of Natural Language in Career Guidance

Most career recommendation systems rely on structured data, requiring users to select keywords from predefined lists or fill out rigid forms. However, individuals describe their career aspirations most naturally through prose. The primary challenge is that simple keyword searches in personal essays are often misleading.

For instance, if a user writes, "I like working with animals, but I cannot stand the sight of blood," a traditional search engine might identify the word "blood" and incorrectly suggest a career in medicine or veterinary surgery. This research addresses the need for fine-grained sentiment analysis that can link specific "targets" (objects, activities, or professions) to the user's actual feelings, effectively filtering out "dispreferred" items from the recommendation process.

A Two-Stage Unsupervised Pipeline

The researchers propose a two-stage unsupervised pipeline to process these essays:

  1. Target Extraction: The system identifies potential career-related terms across four categories: objects, activities, places, and professions.
  2. Lexicon Generation: Using 57 seed terms, the team recursively queried GermaNet (a German version of WordNet) for synonyms and hyponyms, creating a "target constituent lexicon" of approximately 9,000 terms.
  3. Pattern Matching: Four manual linguistic patterns (e.g., base noun phrases, infinite verb constructions like "to program") were applied to extract multi-word targets from the text.
  4. Sentiment Analysis: To avoid the need for massive labeled datasets, the system employs a lexicon-based approach.
  5. Parsing: The BitPar parser breaks sentences into individual clauses.
  6. Polarity Assignment: The system identifies "sentiment cues" (e.g., "enjoy" or "dislike") within each clause. If a clause contains a negative cue or a negation, the polarity of the target within that clause is flipped. This allows the system to handle complex sentences where a user expresses contrasting preferences.
Examples of extracted professional preferences, their categories, and assigned polarities.

Key Insights: Granularity and Concrete Targets

The study highlights that clause-level granularity is essential. Because users often contrast likes and dislikes within the same sentence, analyzing sentiment at the sentence level is too broad. By looking at individual clauses, the system can correctly pair sentiment with the intended target.

Furthermore, the researchers focused on concrete vs. abstract targets. They intentionally ignored meta-characteristics (e.g., "I want a challenging job") in favor of concrete targets (e.g., "I want to work with computers"). While abstract goals are human-centric, concrete targets are significantly more useful for existing Information Retrieval (IR) systems used in career guidance.

Performance and Evaluation

The system was evaluated using a specialized corpus of 45 short essays written by students, manually annotated to create a gold standard.

Precision (P) and Recall (R) metrics for sentiment polarity across Development and Test corpora.
  • Target Extraction: The system achieved a recall of 0.55 to 0.66, catching the majority of relevant terms. However, precision was lower (~0.40) because the automatically generated lexicon was "noisy," occasionally flagging irrelevant words.
  • Sentiment Accuracy: The system proved highly accurate at identifying positive preferences, with a precision between 0.85 and 0.95.
  • The "Neutral" Trap: Precision was lower for neutral and negative assignments. The system often struggled when a user mentioned a target without an explicit sentiment cue, or when the sentiment was expressed in a separate clause from the target.

Limitations and Future Work

While promising, the system faces several hurdles. The automatically generated lexicon occasionally missed specific terms (e.g., "nutritional science") due to GermaNet's coverage limits. Additionally, the system currently lacks coreference resolution; it cannot yet track that a pronoun like "them" refers back to a previously mentioned target like "animals."

Future work will focus on integrating coreference resolution and improving parsing accuracy, as the system's performance is heavily dependent on the BitPar parser's ability to correctly identify clause boundaries. Addressing long-distance negations that span across subordinate clauses remains a key priority for refining the model's accuracy in complex natural language environments.

Summary updated

Abstract

This paper presents an unsupervised sentiment analysis approach for extracting professional preferences of users from natural language essays in a career recommendation scenario. Our system first extracts terms facilitating career recommendation such as objects, activities, hobbies, and places from the essays. Then, it applies a lexicon-based sentiment analysis approach to assign polarities representing user preferences.
BibTeX
@inproceedings{toprak2015extracting,
  author = {Cigdem Toprak and Christof Müller and Iryna Gurevych},
  title = {{Extracting Professional Preferences of Users from Natural Language Essays}},
  booktitle = {TUbilio},
  year = {2015},
}
Figure