Identifying Cohorts that Differ in their Behaviour: Tool Support

Sander J. J. Leemans, Shiva Shabaninejad, Kanika Goel, Hassan Khosravi, Shazia Sadiq, Moe Thandar WynnER Forum/Posters/Demos 2020 · OpenReview

Abstract

Process mining often involves navigating complex event logs to find meaningful patterns. This paper introduces tool support for Cohort Identification, an automated technique that identifies and recommends groups of cases (cohorts) exhibiting behavior significantly different from the general population. By integrating this method into ProM, visual Miner, and Course Insights, the authors provide analysts with actionable insights into process deviations without the need for exhaustive manual filtering.

The Challenge of Complex Event Logs

In process mining, event logs are used to understand real-world business processes, such as insurance claims or student enrollments. However, these processes are often messy and complex. Analysts typically attempt to simplify their analysis by "drilling down"—filtering data by specific attributes like customer type or geographic region.

The primary challenge lies in selection: with hundreds of possible attribute combinations, manually checking every subset to see if "Gold Customers in the West" behave differently than "Silver Customers in the East" is both tedious and prone to error. This research addresses the need for automated guidance to identify the most "interesting" and behaviorally distinct subsets of data.

Automated Cohort Identification Methodology

The authors implement a technique called Cohort Identification, which operates through three main steps:

  1. Defining Cohorts: The system uses trace attributes (e.g., "claim amount," "gender," "major") to split the log into a "cohort" (cases possessing the attribute) and an "anti-cohort" (all other cases).
  2. Measuring Distance: A distance measure between 0 and 1 is calculated. A score of 1 indicates the two groups follow entirely different paths, while 0 suggests their behavior is indistinguishable from a random split.
  3. Ranking: The system provides a ranked list of attribute-value pairs, allowing analysts to focus on groups with the highest behavioral variance.

Tool Support and Implementation

The cohort identification method has been integrated into three distinct environments to cater to different user needs:

  • ProM: A stand-alone plug-in designed for researchers and power users, providing a detailed table of ranked cohorts.
  • visual Miner (vM): An interactive tool that allows users to click a recommended cohort to instantly filter and update the visual process map. Cohort analysis interface in visual Miner showing ranked behavioral deviations. Source, p. 3
  • Course Insights: A learning analytics dashboard designed for educators to identify student groups (e.g., by enrollment status or program) who interact with course materials in unique ways. Course Insights dashboard recommending cohorts based on student attributes. Source, p. 4
Cohort analysis interface in visual Miner showing ranked behavioral deviations.
Course Insights dashboard recommending cohorts based on student attributes.

Key Insights and Results

The integration of automated cohort identification offers several advantages:

  • Automated "Drill-Down": Instead of analysts manually testing hypotheses, the tool proactively identifies where significant behavioral differences exist.
  • Explainability: Results are provided in plain English (e.g., "paymentAmount < 36.0"), making insights immediately actionable for business users or teachers, unlike black-box clustering algorithms.
  • Quantifiable Differences: The distance metric provides an objective ranking. In practice, cohorts showing a distance of ~0.69 indicate strong behavioral deviations that warrant immediate investigation.
  • Cross-Domain Utility: The research demonstrates that the same underlying algorithm is effective for both industrial process optimization and educational learning analytics, as evidenced by a study involving 875 diverse students in Course Insights.

Limitations and Future Work

While the current implementation is robust, the authors identify several areas for future development. For the Course Insights tool, work continues on making the dashboard "fully process aware" to help instructors better visualize specific student journeys. Additionally, the authors plan to develop techniques that explain how cohorts differ (e.g., identifying specific steps skipped or repeated). Finally, while the current implementation is multithreaded, scaling the exhaustive search of attribute combinations for massive logs remains a computational challenge.

Summary updated

BibTeX
@inproceedings{leemans2020identifying,
  author = {Sander J. J. Leemans and Shiva Shabaninejad and Kanika Goel and Hassan Khosravi and Shazia Sadiq and Moe Thandar Wynn},
  title = {{Identifying Cohorts that Differ in their Behaviour: Tool Support}},
  booktitle = {ER Forum/Posters/Demos},
  year = {2020},
  url = {https://openreview.net/forum?id=mUXPdFNtuK},
}
Figure