TL;DR
This paper introduces iTAG, a novel system for automatically annotating textual resources with human intentions, going beyond traditional topic-based tagging. By mapping indicative actions to a taxonomy of human goals, iTAG demonstrates the feasibility of inferring what users want to achieve from their text, opening up new avenues for resource navigation.
What Problem Does This Paper Address?
The vastness of the "social web" is often navigated through annotations, but current methods primarily focus on what a resource is about (e.g., "sports," "politics"). This leaves a significant gap: understanding the intentions or goals behind the content. People rarely explicitly state their ultimate goals, making direct annotation difficult. For instance, someone might write about daily actions related to "Achieve Salvation" without ever using that exact phrase. This paper tackles the challenge of automatically inferring these underlying human intentions from textual content.
Methods
The iTAG approach has two main steps:
- Enriching a Taxonomy of Human Goals: The authors start with a pre-existing, psychologically-grounded taxonomy of 135 human intent categories (e.g., "A good marriage," "Getting an education"). They then semantically enrich these categories by identifying descriptive phrases associated with each. This enrichment process was iterative and involved expert evaluation.
- Automatic Annotation:
- Knowledge Base Construction: The system builds a knowledge base by searching the web for sentences that link descriptive phrases (from step 1) with indicative actions using causal relation phrases like "in order to" or "for the purpose of." This creates a mapping between actions and intentions.
- Text Scanning: Target texts are segmented into sentences. Each sentence is then queried against the knowledge base to find the most similar entry containing indicative actions. The intent category associated with that entry is assigned as an annotation to the sentence. Document-level annotations are aggregated from sentence-level ones.
Datasets
- Human Goal Taxonomy: A pre-existing taxonomy of 135 human intent categories, based on a social-psychological framework.
- Web Data: Used to build the knowledge base by searching for indicative actions and descriptive phrases.
- Political Speeches: Transcripts of 44 speeches (23 by B. Obama, 21 by J. McCain) from the 2008 US presidential campaign were used for evaluation.
- Human Subject Annotations: A dataset of 6399 sentences from Obama's and McCain's speeches were annotated by human subjects to serve as "ground truth" for evaluation.
Key Insights and Ideas
- Intent is Implicit: The core insight is that while people rarely state their ultimate goals directly, they are prolific in describing actions that indirectly contribute to those goals. This provides a proxy for inferring intent.
- Indicative Actions as Proxies: The paper proposes using "indicative actions" (verbs and phrases describing activities) as a bridge to infer higher-level human intentions.
- Novel Annotation Dimension: Intent annotation offers a new layer of understanding beyond topic or sentiment, focusing on the "why" behind content creation and consumption.
- Feasibility Demonstrated: The study shows that automatically inferring these intentions is not only possible but can outperform random guessing, even in a simplified setting.
Key Results
- Intent Tag Clouds: For the political speeches, iTAG generated intent tag clouds that highlighted dominant goals for each candidate. For example, McCain's top intents were "Helping Others" and "Being better than others," while Obama's were "Pursuing ideals" and "Helping Others." This offers a distinct perspective compared to traditional topic-based tag clouds.
- Knowledge Base Quality: A human evaluation of 674 knowledge base entries found that 57% contained plausible action/category mappings.
- iTAG Performance: In an evaluation against a random baseline using human-annotated political speeches, iTAG achieved precision/recall curves that outperformed the baseline for recall levels up to 70%.
Limitations and Future Work
- Simplified Setting: The current iTAG prototype was evaluated on a simplified dataset (political speeches) and a limited taxonomy. The authors acknowledge that real-world web data is more complex and diverse.
- "Winner Takes It All" Approach: Sentence-level intent annotations are assigned based on the single most similar entry in the knowledge base, which might oversimplify complex sentences.
- Future Work: The authors suggest that iTAG could be improved by expanding the taxonomy, refining the knowledge base construction, and exploring more sophisticated sentence-to-intent mapping techniques. They also envision iTAG helping to open up new intentional dimensions for navigating and browsing web resources.
Summary updated