Tell Me What You Do and I’ll Tell You What You Are: Learning Occupation-Related Activities for Biographies

Elena Filatova, John PragerEMNLP 2005

TL;DR

This paper introduces a novel method for automatically identifying an individual's occupation by analyzing their biographical text. By extracting and generalizing "atomic events" (relationships between entities and verbs), the authors create occupation-specific activity profiles that serve as features for an SVM classifier, achieving classification performance comparable to state-of-the-art methods.

What Problem Does This Paper Address?

Creating comprehensive biographies often requires understanding the important events in a person's life. While some events like birth and death are universal, many others are highly specific to an individual's occupation. The challenge lies in automatically identifying these occupation-specific activities from text to enrich biographical generation and to infer an individual's occupation itself. Existing approaches for biography generation, particularly in tasks like question answering (e.g., "Who is X?"), often rely on manually curated information or struggle to generalize effectively across diverse occupations. This paper aims to bridge this gap by developing an unsupervised approach to learn occupation-specific activities and use them for occupation classification.

Methods

The core of this work lies in the concept of generalized atomic events. The authors first extract "atomic events" from text, which are triplets consisting of two named entities and a verb connecting them (e.g., (Christopher Columbus, sailed, India)). To make these events more generalizable, they are "generalized" by replacing specific named entities with their types (e.g., (NAME/PERSON, sailed, PLACE)).

These generalized atomic events are then aggregated for individuals within the same occupation. To identify activities that are truly indicative of an occupation, the authors employ a bipartite graph approach. They construct graphs where one set of nodes represents named entity types and the other represents verbs (activities). By analyzing the connections within these graphs and calculating steady-state eigenvectors of transition matrices derived from these graphs, they can rank the importance of activities for each occupation.

Finally, these ranked occupation-specific activities are used as features for a multi-class Support Vector Machine (SVM) classifier to predict an individual's occupation. The authors experiment with two feature sets: one using only the ranked verbs (activities) and another using the full generalized atomic events.

Datasets

The authors created their own dataset due to the lack of sufficiently diverse existing benchmarks. They collected biographies for individuals across ten occupations: artists, athletes, composers, dancers, explorers, mathematicians, physicists, politicians, singers, and writers. For each occupation, they aimed for approximately 20 practitioners, resulting in a final dataset of 189 individuals after document retrieval and filtering. Documents were retrieved from four corpora: AQUAINT, TREC, part of World Book, and part of a general web corpus.

Key Insights and Ideas

  • Generalization is Key: Simply extracting raw atomic events is insufficient. Generalizing them by replacing specific entities with their types (e.g., PERSON, PLACE, DATE) is crucial for capturing common activity patterns across individuals within an occupation.
  • Occupation-Specific Activities as Powerful Features: The paper demonstrates that occupation-specific activities, identified through the generalized atomic event analysis, are highly effective features for classifying individuals into their respective occupations.
  • Bipartite Graphs for Activity Ranking: The use of bipartite graphs and Markov chain-like steady-state analysis provides a principled way to rank the importance of activities for a given occupation, moving beyond simple frequency counts.
  • Value of Structured Information: The study highlights that structured information captured by generalized atomic events is more reliable for classification than just using individual verbs, suggesting that the context (entity types) surrounding an activity is important.

Key Results

The paper reports classification accuracy using leave-one-out cross-validation on their custom dataset of 189 individuals across 10 occupations.

  • SVM with Verbs Only: Achieved a classification accuracy of approximately 48% (average ratio across occupations).
  • SVM with Generalized Atomic Events: Significantly outperformed the verb-only approach, achieving approximately 66.3% accuracy. This is a substantial improvement, demonstrating the value of the structured atomic event features.
  • Comparison to Probing: The SVM classification using atomic events achieved performance comparable to a "probing" classification method, which is considered state-of-the-art for similar tasks. The SVM outperformed probing in six out of ten occupations.
  • Misclassification Analysis: The authors note that some misclassifications could be considered valid, as individuals might excel in multiple occupations (e.g., Paul McCartney being classified as a singer but also a composer).

Limitations and Future Work

  • Dataset Size and Diversity: The dataset, while novel, is limited to 10 occupations and 189 individuals. The authors acknowledge that this is a pilot study and a more diverse set of occupations would be beneficial.
  • Document Availability: The performance for certain occupations (e.g., mathematicians, physicists) was impacted by the limited availability of relevant documents in the chosen corpora.
  • Person-Specific Activities: The paper suggests exploring the identification of "person-specific" activities, which are highly scored for an individual but not common across their occupation. This could lead to more nuanced biographical generation.
  • Biography Generation: The authors envision using these generalized atomic events not just for classification but also for automatic biography generation, creating richer and more accurate life stories.

Summary updated

BibTeX
@inproceedings{filatova2005tell,
  author = {Elena Filatova and John Prager},
  title = {{Tell Me What You Do and I’ll Tell You What You Are: Learning Occupation-Related Activities for Biographies}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2005},
}
Figure