Closed-Form Expressions for Global and Local Interpretation of Tsetlin Machines with Applications to Explaining High-Dimensional Data

Christian D. Blakely, Ole-Christoffer Granmo

Introduction

Tsetlin Machines (TMs) represent a significant advancement in interpretable machine learning, utilizing simple boolean logic to form complex decision boundaries. However, as these models scale to handle high-dimensional data, they often require thousands of conjunctive clauses. This complexity renders manual inspection impractical, creating a gap between theoretical interpretability and practical utility. This paper addresses this challenge by introducing closed-form mathematical expressions that allow for the real-time extraction of global and local feature importance directly from the model's architecture.

Methodology: Extracting Logic-Based Insights

The core contribution of this work is a set of equations that aggregate the internal states of a TM—specifically the inclusion and exclusion of features within its learned clauses. By binarizing continuous features into threshold-based bits (e.g., Income > 50k), the model calculates:

  • Global Feature Strength: An aggregate measure of how frequently a feature influences the voting process across the entire dataset for a specific class.
  • Local Feature Strength: A sample-specific score derived from the subset of clauses actively triggered by a particular input, explaining individual predictions.
  • Feature Range Strength: A mapping of boolean thresholds back to continuous values to identify the specific ranges (e.g., Age between 25 and 40) that most strongly influence the model.

Additionally, the authors introduce a novel clustering algorithm that groups data based on "reasoning" similarity—how the model classifies them—rather than raw geometric proximity.

Performance and Interpretability Parity

A critical concern for interpretable models is the "accuracy-interpretability trade-off." The authors demonstrate that their Integer Weighted Tsetlin Machine (IWTM) maintains competitive performance against state-of-the-art black-box models like XGBoost across standard tabular datasets.

Table 1: Performance comparison between IWTM and XGBoost, showing competitive accuracy and AUC across various metrics.

Beyond raw accuracy, the proposed expressions provide insights that align with established post-hoc methods. On the Wisconsin Breast Cancer dataset, the global feature strength calculated by the TM identifies the same primary indicators of malignancy as SHAP values applied to an XGBoost model, such as perimeter_worst and concave points.

Figure 3: Global feature strength for the Breast Cancer dataset calculated via the proposed closed-form expressions, identifying key diagnostic features.

Conclusion

By providing a mathematical bridge between raw logic clauses and human-readable importance scores, this research enables Tsetlin Machines to offer SHAP-level insights in real-time. This framework not only enhances trust in logic-based models but also introduces novel applications like clustering by model reasoning, paving the way for more transparent and accountable high-stakes decision-making in fields like medicine and criminal justice.

Summary updated

BibTeX
@inproceedings{blakely2021closed,
  author = {Christian D. Blakely and Ole-Christoffer Granmo},
  title = {{Closed-Form Expressions for Global and Local Interpretation of Tsetlin Machines with Applications to Explaining High-Dimensional Data}},
  booktitle = {Iea/aie},
  year = {2021},
  doi = {10.1007/978-3-030-79457-6\_14},
}
Figure