Optimising Rule-Based Classification in Temporal Data

Polla Fattah, Uwe Aickelin, Christian Wagner

TL;DR

This paper introduces a novel method for optimizing rule-based classification of temporal data by refining expert-defined rules. By leveraging temporal behavior, their approach significantly improves classification accuracy in dynamic scenarios, as demonstrated in a public goods game experiment.

What Problem Does This Paper Address?

Classifying objects whose behavior changes over time is a significant challenge. Traditional classification methods often treat time-series data as static snapshots or struggle to incorporate evolving patterns. This leaves a gap in understanding dynamic systems, such as tracking stock price stability, student learning trajectories, or, as explored in this paper, player strategies in repeated economic games. The authors aim to bridge this gap by developing a method that optimizes existing rule-based classifiers to better capture temporal dynamics and human interpretability.

Methods

The proposed method consists of two main stages:

  1. Rule Generation: This stage begins with expert-defined, general rules for classifying objects. These rules are based on aggregated attributes derived from temporal data. For instance, in the student grade example, teachers provide general definitions for "bad," "good," and "excellent" students based on average marks.
  2. Rule Optimization: This is the core of the paper's contribution. The system takes the initial, potentially overlapping, rule ranges and optimizes them to find the "best" single value that defines class boundaries at each time point. This optimization is achieved through a brute-force search, evaluating each potential classifier using a cost function that measures class compactness (e.g., using standard deviation). The goal is to create the most compact and well-separated classes at every time point.

The authors also employ Support Vector Machines (SVM) for evaluating their proposed classifier's performance against traditional methods.

Datasets

The primary dataset used for evaluation is from a public goods game experiment conducted by Fischbacher et al. (2012). This dataset comprises data from 140 players over 10 rounds, including attributes like player beliefs, contributions, and other players' contributions. This dataset is temporal in nature and was used to compare the proposed classification method against the economists' original classification. No new benchmarks were introduced; rather, the existing dataset was re-analyzed with the novel temporal classification approach.

Key Insights and Ideas

  • Temporal Dynamics Matter: The paper strongly emphasizes that ignoring the temporal evolution of an object's behavior leads to suboptimal classification, especially in dynamic environments.
  • Optimizing Expert Rules: Instead of discarding expert knowledge, the method refines it. This hybrid approach leverages human interpretability while enhancing it with data-driven optimization.
  • Class Compactness as a Metric: The core idea of optimizing for "compact classes" at each time point is crucial. This ensures that within a given time slice, members of a class are as similar as possible, leading to more robust classifications.
  • Public Goods Game as a Testbed: The public goods game is an excellent case study because player behavior is inherently dynamic, influenced by learning and interactions over multiple rounds.

Key Results

The proposed method demonstrated superior performance compared to the original economists' classification.

  • SVM Evaluation: When using SVM for classification, the labels generated by the proposed method achieved higher Area Under the Curve (AUC) scores than those from the original classification, particularly when using only temporal attributes like "Belief & Contrib."
  • Correlation with Behavior: The new class labels showed a stronger correlation with players' actual contribution amounts over time compared to the original labels.
  • Improved Temporal Representation: The derived attributes (e.g., average contribution, initial deviation) significantly improved the classification performance of the new method, highlighting the benefit of temporal data.

Limitations and Future Work

  • Brute-Force Optimization: The current optimization process uses brute-force, which can be computationally expensive for larger datasets or more complex rule sets. The authors suggest that heuristic methods like Differential Evolution could be explored as alternatives.
  • Data Requirements: The method requires temporal data with equal and discrete time points.
  • Generalizability: While demonstrated on a public goods game, further validation on other temporal data domains (e.g., stock prices, student performance) would be beneficial.
  • Future Work: The authors suggest that categorizing players based on their dynamic behavior opens new avenues for studying behavioral changes over time in game play.

Summary updated

Abstract

This study optimises manually derived rule-based expert system classification of objects according to changes in their properties over time. One of the key challenges that this study tries to address is how to classify objects that exhibit changes in their behaviour over time, for example how to classify companies' share price stability over a period of time or how to classify students' preferences for subjects while they are progressing through school. A specific case the paper considers is the strategy of players in public goods games (as common in economics) across multiple consecutive games. Initial classification starts from expert definitions specifying class allocation for players based on aggregated attributes of the temporal data. Based on these initial classifications, the optimisation process tries to find an improved classifier which produces the best possible compact classes of objects (players) for every time point in the temporal data. The compactness of the classes is measured by a cost function based on internal cluster indices like the Dunn Index, distance measures like Euclidean distance or statistically derived measures like standard deviation. The paper discusses the approach in the context of incorporating changing player strategies in the aforementioned public good games, where common classification approaches so far do not consider such changes in behaviour resulting from learning or in-game experience. By using the proposed process for classifying temporal data and the actual players' contribution during the games, we aim to produce a more refined classification which in turn may inform the interpretation of public goods game data.
BibTeX
@inproceedings{fattah2016optimising,
  author = {Polla Fattah and Uwe Aickelin and Christian Wagner},
  title = {{Optimising Rule-Based Classification in Temporal Data}},
  booktitle = {Repository@Nottingham (University of Nottingham)},
  year = {2016},
  doi = {10.48550/arxiv.1607.05913},
}
Figure