A systematic approach for dynamic targeted monitoring of KPIs

Alejandro Maté, Kostas Zoumpatianos, Themis Palpanas, Juan Trujillo, John Mylopoulos, Elvis Koci

Introduction

Monitoring Key Performance Indicators (KPIs) is a cornerstone of modern Business Intelligence (BI). However, traditional dashboards often provide only aggregate views, leaving managers to manually "drill down" through vast multi-dimensional data cubes to identify the root causes of performance deviations. This manual process is labor-intensive, error-prone, and frequently misses subtle but significant sub-segment issues. This paper proposes a semi-automatic, goal-aware approach that leverages information theory to navigate complex data structures and pinpoint the exact drivers of KPI failure.

System Architecture

The proposed framework bridges the gap between high-level strategic goals and granular data storage through a four-part architecture designed for automated exploration.

The system architecture integrates business modeling with an automated monitoring engine and query generator to bridge the gap between strategy and data.
  1. Business Strategy Model: Utilizes the Business Intelligence Model (BIM) to define KPIs within the context of strategic goals and specific performance targets.
  2. Monitoring Engine: Acts as the central processing unit, executing a Top-K search algorithm to explore the multi-dimensional data space.
  3. Query Generator: Automatically translates search logic into MDX and SQL queries, facilitating seamless interaction with the data warehouse (DW).
  4. Result Reporter: Presents findings through an interactive drill-down tree, allowing users to visualize the "path of failure."

Information-Theoretic Search Strategy

The core innovation of this approach is the application of information theory to automate data exploration. The system employs two primary metrics to guide the search:

  • Information Gain (IG): This metric determines which dimension (e.g., Time, Geography, Product) is most likely to contain the root cause of a deviation. High IG indicates that the performance gap is concentrated within specific members of that dimension, making it a high-priority path for exploration.
  • Information Loss (IL): Serving as a stopping criterion, IL measures the proportion of the total problem ignored as the search narrows. This prevents the algorithm from descending into statistically insignificant details, ensuring that the final report remains actionable for human decision-makers.

Case Study: Europe 2020

The methodology was implemented in the HERMES CASE tool and evaluated using real-world data from Eurostat regarding the Europe 2020 strategic planning framework. The dataset included indicators such as employment rates and GDP across various European regions, structured in a multi-dimensional snowflake schema.

The algorithm identifies the geographic dimension as the primary driver of employment failure, specifically highlighting Greece and its sub-regions using IG values.

In the evaluation, the algorithm successfully bypassed the "Time" dimension, which showed stable trends (low IG), and prioritized the "Geographic" dimension (high IG). As illustrated in the drill-down tree, the system pinpointed Greece and subsequently specific sub-regions like Central Greece as the primary contributors to the KPI deviation. This automated navigation significantly reduces the search space from an exhaustive exploration to an optimized complexity of $\Omega(d * l)$, where $d$ is the number of dimensions and $l$ is the depth of the hierarchies.

Conclusion and Future Work

By combining goal-aware monitoring with information-theoretic search, this approach provides a systematic way to identify the "why" behind KPI performance. While the current system relies on well-defined business models and has primarily been tested on historical data, it offers a robust foundation for automated root-cause analysis. Future work aims to integrate forecasting for proactive monitoring and incorporate SWOT analysis to suggest strategic responses to identified issues.

Summary updated

Abstract

There has been growing interest for more than a decade in Business Analytics as a means for improving business performance. One of the most popular Business Analytics technique involves monitoring performance by means of Key Performance Indicators (KPIs). A KPI is a powerful tool that relates enterprise data to business goals, thereby enabling managers to guide the analytic process and identify deviations in their strategic plan. Nevertheless, monitoring KPIs requires that they are evaluated at multiple levels of detail, in order to identify potential problems earlier instead of being noted after the fact. Unfortunately, there are obstacles to the generation and enactment of such monitoring processes. In particular, there is no systematic, tool-supported process for defining what is to be monitored given a strategic plan, nor are there tools for automatically generating monitoring queries. As a result, monitoring consists of a manual process whereby queries are generated for high level indicators across a few scorecards and dashboards. In this paper we present a systematic semi-automatic approach that covers the entire monitoring process. Our approach performs a partial search guided by the KPIs of the company, generating queries required during the monitoring process. Thanks to our approach, users become aware of the existence of problems and where they are located, without requiring a priori information about the nature of the problem being searched. Moreover, we have implemented our approach in our CASE tool HERMES and evaluated the results on a case study using real data.
BibTeX
@inproceedings{mate2014systematic,
  author = {Alejandro Maté and Kostas Zoumpatianos and Themis Palpanas and Juan Trujillo and John Mylopoulos and Elvis Koci},
  title = {{A systematic approach for dynamic targeted monitoring of KPIs}},
  booktitle = {CASCON},
  year = {2014},
}
Figure