Hybrid interpretation of induction results

Xindong WuOpenalex Unknown 1996 · DOI · OpenAlex

Abstract

When applying rules produced by induction from training examples to a test example, there are three possible cases that demand different actions: (1) no match; (2) single match; and (3) multiple match. Existing techniques for dealing with the first and third cases are exclusively based on probability estimation. However, when there are continuous attributes in the example space, and if these attributes have been discretized into intervals before induction, fuzzy interpretation of the discretized intervals at deduction time could be very valuable. This paper describes the fuzzy matching techniques implemented in the HCV (Version 2.0) software and presents a hybrid interpretation mechanism which combines fuzzy matching with probability estimation. Experiment results of the HCV (Version 2.0) software with different interpretation techniques are provided on a number of data sets from the University of California at Irvine Repository of Machine Learning Databases.

Introduction

Rule induction is a fundamental process in machine learning, aimed at extracting interpretable "if-then" rules from structured data. A significant challenge in this domain, particularly during the mid-1990s, was the effective handling of continuous (real-valued) attributes. Most induction algorithms addressed this by discretizing continuous variables into fixed intervals defined by "sharp" borders. For instance, an attribute like 'Age' might be split into 'Age < 50' and 'Age >= 50'.

While computationally efficient, these rigid boundaries are inherently brittle. A test instance with an age of 49.9 might fail to trigger a rule designed for the 'Age >= 50' interval, despite being virtually indistinguishable from a 50.1-year-old instance. This leads to the "no match" problem, where a system fails to classify a valid data point simply because it falls into a gap between induced rules.

The Three Scenarios of Deduction

When applying induced rules to new data, three primary scenarios occur:

  1. No Match: The data point does not satisfy the conditions of any existing rule.
  2. Single Match: Exactly one rule is triggered, providing a clear classification.
  3. Multiple Match: Several rules apply, but they may predict conflicting classes.

Historically, researchers relied on probability estimation—such as the Laplacian Law of Succession—to resolve conflicts in multiple-match cases. However, these methods did not address the underlying issue of rigid discretization in no-match scenarios.

Methodology: The HCV (Version 2.0) Hybrid Mechanism

The HCV (Version 2.0) software introduces a hybrid interpretation mechanism that integrates statistical probability with fuzzy logic to enhance classification performance. The core innovation is the application of fuzzy membership functions to the boundaries of discretized intervals.

Illustration of a fuzzy membership function $I_i$ superimposed on sharp interval borders.

As illustrated in Figure 1, the system replaces sharp borders (dashed lines) with a fuzzy membership function (solid line). This allows a continuous value to belong to an interval with a "degree of membership" ranging from 0 to 1. The software supports various transition curves, including linear, 3rd-degree polynomial, and arctangent functions. The "spread" of this fuzzy border is controlled by a parameter $s$, which determines how far the fuzzy border extends beyond the sharp interval limits.

The hybrid logic functions as follows:

  • Single Match: The system accepts the rule's prediction directly.
  • Multiple Match: The system utilizes the Estimate of Probability (EP) method. By applying the Laplacian Law of Succession, it weighs the statistical reliability of each matching rule based on the training data to resolve the conflict.
  • No Match: This is where the fuzzy logic is most effective. Instead of returning a failure, the system calculates the membership degree of the instance across relevant rules. It then assigns the class of the rule to which the instance is "closest" in fuzzy terms.

Experimental Results

The hybrid approach was evaluated using 17 datasets from the University of California at Irvine (UCI) Repository. These datasets were selected for their real-world complexity, containing both noise and continuous attributes. Key datasets included medical diagnostics (Heart Disease, Pima Indians Diabetes), chemical analysis (Glass, Wine), and business metrics (Credit Screening).

The hybrid HCV system was benchmarked against contemporary standards, including the C4.5 decision tree algorithm and NewiD. The results were compelling:

  • Broad Success: HCV (hybrid) achieved the highest accuracy in 9 out of the 17 experimental problems.
  • Performance Gains: In the "Anneal" dataset, the hybrid method reached 98.0% accuracy, significantly higher than C4.5's 93.0%. Similarly, in the "Glass" dataset, it achieved 72.3% compared to C4.5's 64.6%.
  • The Hybrid Advantage: Comparisons showed that the hybrid version (combining probability and fuzziness) performed better than a "pure fuzzy" version, which only outperformed C4.5 on 4 datasets.

Conclusion

The research demonstrates that fuzzy logic is most powerful when used as a "safety net" for instances that fall between the cracks of rigid rules. While statistical probability remains the superior tool for resolving rule conflicts (multiple matches), fuzzy interpretation of continuous boundaries is significantly more effective for handling gaps in the rule space (no matches). The hybrid mechanism in HCV 2.0 successfully bridges these two paradigms, offering a more resilient and accurate approach to rule-based machine learning. Future developments will focus on providing users with more granular control over membership functions to better capture domain-specific nuances.

Summary updated

Abstract

When applying rules produced by induction from training examples to a test example, there are three possible cases that demand different actions: (1) no match; (2) single match; and (3) multiple match. Existing techniques for dealing with the first and third cases are exclusively based on probability estimation. However, when there are continuous attributes in the example space, and if these attributes have been discretized into intervals before induction, fuzzy interpretation of the discretized intervals at deduction time could be very valuable. This paper describes the fuzzy matching techniques implemented in the HCV (Version 2.0) software, and presents a hybrid interpretation mechanism which combines fuzzy matching with probability estimation. Experiment results of the HCV (Version 2.0) software with different interpretation techniques are provided on a number of data sets from the University of California at Irvine Repository of Machine Learning Databases.
BibTeX
@inproceedings{wu1996hybrid,
  author = {Xindong Wu},
  title = {{Hybrid interpretation of induction results}},
  booktitle = {Openalex Unknown},
  year = {1996},
  doi = {10.1007/978-0-387-34979-4\_55},
}
Adjusting Membership Functions in Fuzzy Rule-Based Classification SystemsTransactions · 1997
Heuristic Rule Weight Specification for Fuzzy Rule-Based Classification SystemsJournal of Japan Society for Fuzzy Theory and Intelligent Informatics · 2004
Hybrid Interval Type-2 Mamdani-TSK Fuzzy System for Regression AnalysisarXiv · 2025
Construction of Fuzzy Classification Systems Using Genetic AlgorithmsJournal of Japan Society for Fuzzy Theory and Systems · 1995
Fuzzy Classifier SystemsJournal of Japan Society for Fuzzy Theory and Systems · 1998
Pattern Classification by Fuzzy RulesJournal of Japan Society for Fuzzy Theory and Systems · 1993
Figure