Abstract
Transfer learning leverages labeled data from a source domain to address tasks in an unlabeled target domain. A central challenge is learning a shared feature space where data distributions from both domains match, a process known as Transfer Representation Learning (TRL). Conventional TRL methods often rely on kernel-based nonlinear mappings, which lack interpretability and are difficult to select. This paper proposes TRL-TSK-FS, a framework that integrates the Takagi-Sugeno-Kang fuzzy system (TSK-FS) into transfer learning. By using the antecedent part of the TSK-FS for nonlinear mapping and the consequent part for linear transformation, the model creates an intuitive, interpretable feature space. Experimental results on text and image datasets demonstrate that TRL-TSK-FS significantly outperforms traditional kernel-based methods.
The Interpretability Gap in Transfer Learning
In transfer learning, finding a shared feature space is critical for bridging the gap between a labeled "source domain" and an unlabeled "target domain." Traditionally, researchers have relied on kernel methods to perform the nonlinear mappings required to align these distributions. However, kernels are often treated as mathematical "black boxes," offering little insight into the transformation process. Furthermore, selecting the optimal kernel (e.g., RBF vs. Linear) typically requires exhaustive trial and error.
The TRL-TSK-FS Framework
To address these limitations, the proposed TRL-TSK-FS replaces opaque kernels with interpretable fuzzy logic. The architecture decomposes the TSK fuzzy system into two distinct functional roles:
- Nonlinear Mapping (Antecedent Part): The "IF-THEN" rules of the fuzzy system map raw data into a high-dimensional fuzzy feature space. Unlike standard fuzzy clustering, which can be unstable, this method uses a deterministic algorithm called Var-Part to ensure consistent parameter initialization.
- Dimensionality Reduction (Consequent Part): This component acts as a linear projection, mapping the high-dimensional fuzzy space down to a low-dimensional shared subspace where the source and target distributions are aligned.
Optimization Strategy
The model is trained by minimizing a multi-objective function that balances three key goals:
- Distribution Matching: Utilizing Maximum Mean Discrepancy (MMD) to ensure the source and target data are statistically similar in the new space.
- Discriminant Information: Incorporating Linear Discriminant Analysis (LDA) to maintain class separability.
- Geometric Preservation: Using Principal Component Analysis (PCA) to retain the underlying structure of the data.
Experimental Evaluation
The model was evaluated on two major benchmarks to test its versatility across different data types:
- Office-Caltech (Image): This dataset includes images from four distinct domains: Amazon, Webcam, DSLR, and Caltech-256. The authors conducted 12 transfer tasks to evaluate how well the model could adapt features across different photographic environments.
- NG20 (Text): A subset of the 20 Newsgroups dataset, focusing on high-dimensional text vectors across categories like technology, recreation, science, and politics.
Key Results and Insights
TRL-TSK-FS consistently outperformed established transfer learning algorithms such as TCA, JDA, and GFK:
- Superior Accuracy: On the Office-Caltech dataset, the method achieved an average accuracy of 54.29%, a significant improvement over the 46.33% achieved by the next best competitor (TJM). On the NG20 text dataset, it reached 89.47% accuracy.
- Efficiency: The model demonstrated rapid convergence, typically reaching optimal performance within just 5 iterations of the joint distribution adaptation process.
- Flexibility: Because TRL-TSK-FS is a representation learning method, the resulting "cleaned" features can be used with any standard classifier, such as 1-Nearest Neighbor.
Conclusion and Future Work
By treating fuzzy rules as "interpretable kernels," TRL-TSK-FS provides a transparent alternative to traditional black-box models. While highly effective for homogeneous transfer learning, future research will explore extending the framework to heterogeneous tasks (e.g., mapping text to images) and incorporating Locality Preserving Projections (LPP) to better maintain local data relationships during transformation.
Summary updated