Automating Creation of Hierarchical Faceted Metadata Structures

Emilia Stoica, Marti Hearst, Megan RichardsonNAACL 2007

TL;DR

This paper introduces Castanet, an algorithm that automatically generates hierarchical faceted metadata from textual descriptions. This is crucial for improving navigation and exploration in large digital collections, as manually creating such structures is time-consuming and doesn't scale. Castanet shows promising results, with information architects finding it significantly more useful than other automated methods.

What Problem Does This Paper Address?

Navigating large digital collections, like online catalogs or digital libraries, can be overwhelming with a simple search box and a list of results. Users often prefer interfaces that organize and group retrieved items, making exploration more intuitive. Hierarchical faceted metadata (HFC) is a popular approach for this, offering a structured way to browse collections by different "facets" or dimensions (e.g., dish type, ingredients for recipes). However, the biggest hurdle to adopting HFC is the manual effort required to create these metadata hierarchies and assign items to them. This process is labor-intensive, making it difficult to keep up with the ever-growing volume of digital content. This paper aims to automate this creation process.

Methods

The core of Castanet's approach is to leverage existing lexical databases, specifically WordNet, to build domain-specific metadata structures. The algorithm works in several steps:

  1. Select Target Terms: It identifies key terms from the collection's text descriptions based on their distribution, filtering out common words.
  2. Build the Core Tree: It uses the "IS-A" (hypernym) relationships from WordNet to construct an initial tree. For unambiguous terms, their entire IS-A path is added. For ambiguous terms, it uses WordNet Domains (a cross-categorization mechanism) to help disambiguate senses and select the most relevant path for the collection.
  3. Augment and Compress the Tree: The algorithm merges paths from ambiguous terms into the Core Tree, biasing towards paths that are more represented in the collection. It then compresses the tree to reduce depth, making it more user-friendly.
  4. Prune Top-Level Categories: Finally, it prunes the top levels of the tree to create a set of manageable facet sub-hierarchies, aiming for moderate depth and breadth.

The novelty lies in its ability to carve out a domain-specific structure from a general lexical database like WordNet, using a combination of term distribution, WordNet's domain information, and tree compression techniques.

Datasets

The primary dataset used for evaluation was a collection of 13,000 recipes from Southwestcooking.com. The algorithm itself relies on WordNet (a large lexical database of English) and WordNet Domains for its core functionality.

Key Insights and Ideas

  • Leveraging General Lexicons for Specific Domains: The paper demonstrates that a general-purpose lexical resource like WordNet can be effectively adapted to create useful domain-specific metadata structures with surprising accuracy.
  • The Power of Ambiguity Resolution: Effectively disambiguating terms using external resources like WordNet Domains is crucial for building accurate and relevant hierarchies.
  • User Interface Matters: The paper highlights that the presentation of the generated metadata is as important as its creation. The goal is to create structures that enhance discovery without overwhelming users.
  • Information Architects as Evaluators: Involving the target users (information architects) in the evaluation process provides critical insights into the practical utility and usability of the generated metadata.

Key Results

A study involving 34 information architects compared Castanet to a baseline (frequent words) and two other automated methods (LDA and Subsumption).

  • High Likelihood of Use: 85% of participants indicated they would likely use Castanet's output, compared to 74% for the Baseline, 38% for Subsumption, and 0% for LDA.
  • Meaningfulness and Systematicity: Castanet scored significantly higher than LDA and Subsumption on measures of category meaningfulness, systematicity, and concept capture.
  • Usability: Participants found Castanet's generated categories to be more meaningful and systematic than those produced by LDA and Subsumption.

Limitations and Future Work

The authors acknowledge several limitations and suggest future directions:

  • Spelling and Morphological Variations: The current algorithm doesn't fully handle variations like "aging" vs. "ageing" or different verb/adjective forms.
  • Term Ambiguity in Context: The assumption that a term in a document uses its assigned sense in the facet hierarchy is often incorrect. Future work should focus on disambiguating terms within the document context.
  • Coverage of Unknown Terms: WordNet is not exhaustive. Mechanisms to improve coverage for terms not present in WordNet are needed.
  • Multiple Senses/Facets: For some terms, allowing them to occur with multiple senses or in multiple facets (even with the same sense) could improve accuracy, as seen in the "brain" example.
  • Manual Refinement: While the algorithm is automated, a manual editing pass over the output is still advisable for optimal results.

Summary updated

BibTeX
@inproceedings{stoica2007automating,
  author = {Emilia Stoica and Marti Hearst and Megan Richardson},
  title = {{Automating Creation of Hierarchical Faceted Metadata Structures}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2007},
}
Figure