ProcK: Machine Learning for Knowledge-Intensive Processes

Tobias Jacobs, Jingyi Yu, Julia Gastinger, Timo Sztyler

TL;DR

This paper introduces ProcK, a novel machine learning framework that combines event logs with knowledge graphs to build more powerful predictive models for business processes. By integrating structured knowledge, ProcK significantly enhances the accuracy of process monitoring tasks, outperforming existing methods.

What Problem Does This Paper Address?

Predictive process monitoring aims to forecast future events, outcomes, or timings within ongoing business processes. Traditional approaches primarily rely on event logs, which capture the sequence of activities. However, these logs often miss crucial contextual information embedded in an organization's broader data landscape, such as relationships between entities or static attributes that aren't direct event properties. This limits the predictive power of models. The paper addresses this gap by proposing a method that leverages both sequential event data and structured knowledge from knowledge graphs to create richer representations for more accurate predictions.

Methods

ProcK employs a hybrid neural network architecture that combines a Graph Neural Network (GNN) with a sequence model.

  • Knowledge Graph Integration: The GNN processes a knowledge graph, learning embeddings for its nodes (representing entities like customers, products, or locations) and relations.
  • Event Representation: Event attributes from the event log are mapped to nodes in the knowledge graph. The GNN's node embeddings are then aggregated to form an initial representation for each event.
  • Sequential Modeling: A timestamp embedding is added to the event representation, and these enriched event embeddings are fed into a sequence model (implemented as a feedforward network in their experiments) to capture temporal dependencies and make predictions for the entire process case.

This end-to-end trainable architecture allows ProcK to learn how to effectively combine information from both data sources.

Datasets

The authors evaluated ProcK on four datasets:

  • OULAD (Open University Learning Analytics): A dataset structured as a relational database dump, from which they extracted event logs and a knowledge graph representing student interactions with courses.
  • PKDD99: A financial domain dataset with multiple interconnected tables, used to predict loan status.
  • BPI12: A well-known process mining benchmark dataset containing only an event log. For this dataset, they treated event attributes as isolated nodes in a graph without edges to simulate a scenario where no explicit knowledge graph is available.
  • DBLP: A dataset of academic publications, used to predict future citations.

Key Insights and Ideas

  • Knowledge Graphs as a Richer Context: The core idea is that static and relational information, often residing in databases and best represented by knowledge graphs, can significantly enrich sequential event data for predictive tasks.
  • Bridging the Gap: ProcK effectively bridges the gap between structured, relational data (knowledge graphs) and sequential, event-based data (event logs), which are common in organizational systems.
  • End-to-End Learning: The ability to train the GNN and sequence model jointly in an end-to-end fashion allows the model to learn optimal ways to integrate knowledge graph information into event representations.
  • Flexibility: The approach is flexible, allowing for scenarios where a knowledge graph is available and also providing a baseline when only event logs are present.

Key Results

ProcK consistently demonstrated strong performance across various prediction tasks:

  • Classification Tasks: ProcK, when utilizing a knowledge graph, generally achieved improved Area Under the Curve (AUC) scores compared to ProcK without a knowledge graph and a baseline LSTM model. For instance, on OULAD (dropout, late), ProcK achieved 0.93 AUC, while ProcK (no KG) and LSTM achieved 0.93 and 0.92 respectively.
  • Regression Tasks: For predicting exam scores (OULAD) and future citations (DBLP), ProcK showed a small but consistent advantage in terms of Root Mean Square Error (RMSE). For OULAD (score, late), ProcK achieved an RMSE of 18.93, compared to 18.95 for ProcK (no KG) and 20.35 for LSTM.
  • State-of-the-Art Performance: The paper claims ProcK achieves state-of-the-art performance on several tasks, particularly when a knowledge graph is available.

Limitations and Future Work

  • Accuracy Metric Nuances: The paper notes that while AUC shows clear benefits from knowledge graphs, the advantage in the accuracy metric is not always as pronounced. Further investigation into the accuracy metric is suggested.
  • Automatic Extraction: A significant future direction is developing automatic and configuration-free methods to extract both knowledge graphs and event logs from source databases.
  • Process Recommendations: While ProcK is applied to prediction tasks, the authors suggest extending it to make process recommendations (e.g., suggesting actions to positively influence case outcomes) as an important next step.
  • Numerical Attributes: The current implementation assumes attribute values are nodes in the knowledge graph. Extending ProcK to directly incorporate numerical attributes without discretization is mentioned as future work.

Summary updated

Abstract

We present a novel methodology to build powerful predictive process models. Our method, denoted ProcK (Process & Knowledge), relies not only on sequential input data in the form of event logs, but can learn to use a knowledge graph to incorporate information about the attribute values of the events and their mutual relationships. The idea is realized by mapping event attributes to nodes of a knowledge graph and training a sequence model alongside a graph neural network in an end-to-end fashion. This hybrid approach enhances the flexibility and applicability of predictive process monitoring, as both the static and dynamic information residing in the databases of organizations can be taken as input data. We demonstrate the potential of ProcK by applying it to a number of predictive process monitoring tasks, including tasks with knowledge graphs available as well as an existing process monitoring benchmark where no such graph is given. The experiments provide evidence that our methodology achieves state-of-the-art performance and improves predictive power when a knowledge graph is available.
BibTeX
@inproceedings{jacobs2022prock,
  author = {Tobias Jacobs and Jingyi Yu and Julia Gastinger and Timo Sztyler},
  title = {{ProcK: Machine Learning for Knowledge-Intensive Processes}},
  booktitle = {ECML/PKDD},
  year = {2022},
  url = {https://openreview.net/forum?id=36NcJIhF27N},
}

Loading related content...

Figure