Towards a Framework for Semantic Exploration of Frequent Patterns

Behrooz Omidvar-Tehrani, Sihem Amer-Yahia, Alexandre Termier, Aurélie Bertaux, Éric Gaussier, Marie-Christine RoussetIMMoA 2013 · OpenReview

Abstract

Frequent pattern mining is a powerful tool for discovering correlations in large datasets, yet it often produces an overwhelming number of results that are difficult to interpret. This paper proposes a framework utilizing two semantic primitives—abstraction and refinement—to navigate the pattern space. By leveraging domain taxonomies and user demographics, these techniques reduce pattern redundancy and highlight significant user sub-populations, facilitating more effective data exploration.

Introduction

The "pattern explosion" problem is a significant hurdle in frequent pattern mining. Traditional algorithms often return millions of patterns, typically ranked only by support. This raw metric fails to explain the semantic context or the specific user groups behind the behaviors. To address this, we introduce a framework that uses external metadata to group patterns semantically and automatically identify interesting demographic traits within the user base.

Methods

Our approach centers on two data-driven primitives designed to simplify and enrich pattern analysis.

Abstraction: Shrinking the Pattern Space

Abstraction utilizes hierarchical taxonomies to generalize specific items into broader categories. For instance, in a mobile usage dataset, specific tools like "Calculator" or "Calendar" can be abstracted into a broader category if their collective usage exceeds a defined threshold ($\rho$). This process collapses multiple similar patterns into a single, more readable generalized pattern, significantly reducing the cognitive load on analysts.

A hierarchical taxonomy of mobile applications used for pattern abstraction.

Refinement: Identifying Salient Attributes

Refinement helps analysts understand the "who" behind the "what." It calculates an Attribute Saliency score based on the distribution of demographic traits among the users of a pattern. If a trait—such as age or gender—shows a highly non-homogeneous distribution compared to the general population, it is flagged as "salient."

Visualizing attribute saliency: the left histogram shows a salient age distribution (28-33), while the right shows normalized usage patterns across different users.

This automatic flagging directs analysts toward unique sub-communities, such as a pattern predominantly exhibited by users in the 28-33 age bracket, as illustrated in the saliency analysis above. This prevents choice paralysis by narrowing down exploration paths to highly relevant demographic segments.

Evaluation and Results

The framework was evaluated using the Nokia smartphone usage dataset and the MovieLens 1M benchmark.

Key findings include:

  • Pattern Reduction: Applying abstraction with thresholds between 15% and 60% reduced the total pattern space by 20% to 30%.
  • Focused Exploration: In the MovieLens dataset, refinement narrowed exploration paths to an average of just 1–3 highly relevant choices at realistic support levels.
  • User Validation: In user studies, 83% of participants found the salient histograms generated by the refinement primitive to be highly informative for understanding user behavior, and 69% preferred them over random distributions.

Conclusion

By integrating domain-specific taxonomies and demographic analysis, the proposed framework transforms frequent pattern mining from a raw data dump into a semantic exploration tool. Future work will focus on alternative saliency measures like entropy and scaling the framework for real-time, interactive use.

Summary updated

BibTeX
@inproceedings{omidvartehrani2013towards,
  author = {Behrooz Omidvar-Tehrani and Sihem Amer-Yahia and Alexandre Termier and Aurélie Bertaux and Éric Gaussier and Marie-Christine Rousset},
  title = {{Towards a Framework for Semantic Exploration of Frequent Patterns}},
  booktitle = {IMMoA},
  year = {2013},
  url = {https://openreview.net/forum?id=oJ6wXQCYMg},
}
Figure