Automatic Construction of Fuzzy Rule Bases: a further Investigation into two Alternative Inductive Approaches

Estevam Hruschka, Marcos Evandro Cintra, Heloisa de Arruda Camargo, Estevam R. Hruschka Jr., Maria do Carmo NicolettiJ. Univers. Comput. sci 2008 · DOI

TL;DR

This paper explores two distinct methods for automatically generating fuzzy rule bases (FRBs) from data: a genetic algorithm-based approach (DoC-based) and a Bayesian classifier-based approach (BayesFuzzy). Both methods aim to create interpretable fuzzy systems, with the DoC-based method focusing on rule selection and pruning, and BayesFuzzy translating a Bayesian classifier into an FRB. The study compares their performance on several benchmark datasets, offering insights into their strengths and weaknesses.

What Problem Does This Paper Address?

Designing effective fuzzy systems often hinges on constructing a robust Fuzzy Rule Base (FRB). This task is notoriously challenging, especially when dealing with complex, high-dimensional datasets. Manually crafting these rules requires significant domain expertise and can be a time-consuming process. This paper addresses the need for automated methods to generate FRBs, aiming to bridge the gap between raw data and interpretable fuzzy logic systems. Specifically, it investigates two hybrid approaches that leverage machine learning techniques to automate this process, seeking to improve both the accuracy and interpretability of the resulting fuzzy systems.

Methods

The paper presents and compares two distinct methods for automatic FRB construction:

  • DoC-based Method: This approach falls under "Genetic Construction" methods. It begins by defining fuzzy partitions for attribute domains. Then, it uses a heuristic called "Degree of Coverage" (DoC) to pre-select candidate rules, significantly reducing the search space for a genetic algorithm (GA). The GA then builds the final FRB from this reduced set of candidate rules. The DoC helps discard rules with low coverage, aiming to simplify the chromosome representation and speed up the learning process. Two criteria for rule pre-selection are explored: one based on the Wang & Mendel (WM) method and another selecting rules with non-null DoC. The fitness function for the GA balances Correct Classification Rate (CCR) with the number of rules to promote parsimonious and accurate FRBs.
  • BayesFuzzy Method: This method translates a Bayesian Classifier (BC) into an FRB. It first fuzzifies the dataset and then induces a BC using an algorithm like K2. The core idea is to leverage the structure of the BC to extract classification rules. Specifically, it utilizes the concept of a Markov Blanket (MB) of the class variable to identify relevant attributes, thereby reducing the number and complexity of the rules generated. The rule extraction process is guided by a Maximum A Posteriori (MAP) approach, aiming to find the most probable explanation for a given evidence. A post-processing step removes irrelevant rules.

Datasets

The experiments were conducted on four publicly available datasets from the UCI Machine Learning repository:

  • Diabetes: Numerical-valued attributes.
  • MPEG: Numerical-valued attributes.
  • Iris: Numerical-valued attributes.
  • Machine: Numerical-valued attributes.

For each dataset, three distinct fuzzy partitions were defined (three, five, and seven fuzzy sets per input attribute), resulting in 12 experiment setups. Only four randomly selected attributes were used for each domain to manage computational time.

Key Insights and Ideas

  • The power of pre-selection: The DoC-based method's pre-selection of candidate rules significantly reduces the search space for the GA, making the FRB generation process more efficient and potentially leading to better results.
  • Balancing accuracy and complexity: The fitness function in the DoC-based method explicitly penalizes a large number of rules, encouraging the generation of more concise FRBs without sacrificing accuracy.
  • Leveraging Bayesian structure for interpretability: BayesFuzzy demonstrates that the inherent structure of a Bayesian classifier, particularly the Markov Blanket concept, can be effectively used to derive interpretable fuzzy rules.
  • Attribute relevance is key: BayesFuzzy's reliance on the Markov Blanket for attribute selection is particularly beneficial in domains with many irrelevant attributes, leading to simpler and more efficient rule sets.
  • Fuzzy partition influence: The study suggests that the choice of fuzzy partition can impact the performance of the generated FRBs, with smaller numbers of fuzzy sets sometimes yielding better results in terms of CCR and rule count.

Key Results

  • DoC-based GA (GA I) often excels: In most experiment setups, the GA with pre-selection of the best rules (GA I) produced FRBs with higher CCRs and a smaller number of rules compared to other methods.
  • Diabetes domain performance: For the Diabetes domain, GA I achieved the best combination of high CCR and a small number of rules.
  • MPEG domain performance: GA I also produced the best CCR for the MPEG domain, with a favorable balance of CCR and rule count.
  • Iris domain performance: GA I generated the best CCR for the Iris domain, with a rule count comparable to BayesFuzzy.
  • Statistical significance: A Friedman test indicated statistically significant differences in performance among the four methods, rejecting the null hypothesis that they are equivalent.
  • BayesFuzzy's efficiency with multiple classes: The paper notes that BayesFuzzy's ability to build a single BC for multiple class attributes makes it computationally efficient for domains with many classes.

Limitations and Future Work

  • Dependency on initial fuzzy partition: The DoC-based GA is dependent on the initial fuzzy partition of attribute domains, which can influence the search space and final results.
  • Attribute selection in BayesFuzzy: While BayesFuzzy's attribute selection strategy (Markov Blanket) is effective, it doesn't guarantee a minimal rule set, suggesting the potential need for further pruning.
  • Computational cost of BayesFuzzy: The rule extraction process in BayesFuzzy can be computationally expensive, especially for datasets with many variables.
  • Future work for BayesFuzzy: The authors suggest exploring domains with multiple class attributes to further leverage BayesFuzzy's computational advantage in such scenarios. They also propose investigating data domains that allow for a better understanding of this characteristic.
  • Further exploration of DoC-based method: The paper suggests that the DoC-based method's pre-selection strategy could be further explored as an efficient tool to address the dimensionality problem in machine learning.

Summary updated

BibTeX
@article{hruschka2008automatic,
  author = {Estevam Hruschka and Marcos Evandro Cintra and Heloisa de Arruda Camargo and Estevam R. Hruschka Jr. and Maria do Carmo Nicoletti},
  title = {{Automatic Construction of Fuzzy Rule Bases: a further Investigation into two Alternative Inductive Approaches}},
  journal = {J. Univers. Comput. sci},
  year = {2008},
  doi = {10.3217/jucs-014-15-2456},
}
Figure