On the Functional Equivalence of TSK Fuzzy Systems to Neural Networks, Mixture of Experts, CART, and Stacking Ensemble Regression

Dongrui Wu, Chin-Teng Lin, Jian Huang, Zhigang Zeng

Abstract

Takagi-Sugeno-Kang (TSK) fuzzy systems are renowned for their interpretability but often struggle with the "curse of dimensionality" and optimization efficiency. This paper establishes formal mathematical equivalences between TSK fuzzy systems and four mainstream machine learning paradigms: Neural Networks, Mixture of Experts (MoE), Classification and Regression Trees (CART), and Stacking Ensemble Regression. By unifying these fields, the authors provide a framework for cross-pollinating techniques—such as deep learning regularization and tree-based initialization—to enhance the scalability and performance of fuzzy models while maintaining their inherent transparency.

Introduction

Rule-based fuzzy systems, particularly the TSK variant, offer a unique blend of logic-based interpretability and mathematical rigor. However, they face significant hurdles in high-dimensional spaces where the rule base can explode exponentially. Conversely, modern machine learning models like deep neural networks and ensemble methods excel in scalability and optimization but often operate as "black boxes."

This work bridges the gap by proving that TSK fuzzy systems are functionally equivalent to several popular ML models under specific constraints. This unification allows researchers to leverage the strengths of both worlds: applying the robust optimization and regularization of mainstream ML to fuzzy systems, and using fuzzy logic to provide linguistic explanations for complex ML models.

Functional Equivalences

1. Neural Networks (RBFNs)

The authors demonstrate that a Radial Basis Function Network (RBFN) is functionally equivalent to a TSK fuzzy system when specific conditions are met: the RBFN must use the same number of receptive fields as fuzzy rules, employ Gaussian response functions (equivalent to Gaussian Membership Functions), and use a product t-norm for firing levels.

The layered architecture of a TSK fuzzy system demonstrates its structural equivalence to neural networks.

2. Mixture of Experts (MoE)

A TSK fuzzy system is conceptually identical to a Mixture of Experts model. In this mapping, the rule consequents (output functions) serve as "local experts," while the rule antecedents (IF conditions) function as the "gating network" that determines the contribution of each expert based on the input.

The Mixture of Experts (MoE) architecture, where a gating network weights the outputs of local experts.

3. Classification and Regression Trees (CART)

While standard decision trees use "crisp" splits that lead to discontinuous outputs, the paths of a CART tree can be translated directly into TSK fuzzy rules. "Fuzzy CART" models smooth these boundaries, and using CART to initialize fuzzy rules provides a powerful defense against the curse of dimensionality by avoiding combinatorial rule grids.

4. Stacking Ensemble Regression

TSK fuzzy systems can be reframed as an adaptive stacking model. Unlike traditional stacking, which uses static weights for base models, the TSK equivalence reveals a system where base models (rule consequents) are aggregated using weights (rule antecedents) that change dynamically based on the input's location in the feature space.

Key Insights for Hybrid Design

The theoretical unification presented in this paper opens several avenues for algorithmic innovation:

  • Regularization (DropRule): Inspired by "Dropout" in deep learning, "DropRule" involves randomly dropping fuzzy rules during training to prevent overfitting.
  • Cooperative vs. Competitive Learning: Blending the competitive training of MoE (where experts work independently) with the cooperative training of TSK (where rules minimize global error) can lead to more efficient and accurate rule bases.
  • High-Dimensional Scalability: Initializing fuzzy systems via CART allows for manageable rule bases in high-dimensional data, as the tree structure naturally selects relevant feature combinations.

Conclusion and Future Directions

This paper provides a qualitative roadmap for the formal unification of Fuzzy Logic, Neural Networks, MoE, Trees, and Ensembles. While primarily conceptual, it sets the stage for future empirical work, such as developing efficient training for Interval Type-2 fuzzy systems and using fuzzy rules to "open" the black box of deep learning. Future research will likely focus on designing non-linear adaptive stacking strategies and linguistic explanation modules for complex neural architectures.

Summary updated

BibTeX
@article{wu2020functional,
  author = {Dongrui Wu and Chin-Teng Lin and Jian Huang and Zhigang Zeng},
  title = {{On the Functional Equivalence of TSK Fuzzy Systems to Neural Networks, Mixture of Experts, CART, and Stacking Ensemble Regression}},
  journal = {IEEE Trans. Fuzzy Systems},
  year = {2020},
  doi = {10.1109/tfuzz.2019.2941697},
}
Figure