FaFCNN: A General Disease Classification Framework Based on Feature Fusion Neural Networks

Menglin Kong, Shaojie Zhao, Juan Cheng, Xingquan Li, Ri Su, Muzhou Hou, Cong CaoarXiv 2023 · DOI · arXiv

Introduction

Deep learning has achieved remarkable success in computer vision and natural language processing. However, its application in the medical domain is often hindered by the inherent characteristics of clinical data. Medical datasets are typically small-scale, contain numerous missing values, and consist of heterogeneous features such as demographics, lab results, and clinical metrics. These factors make it difficult for deep neural networks to generalize effectively without overfitting.

To address these challenges, this paper proposes the Feature-aware Fusion Correlation Neural Network (FaFCNN). FaFCNN is a hybrid framework that combines the structural advantages of decision trees with the representational power of neural networks, offering a robust solution for disease classification under low-quality data conditions.

Problem Statement

The authors identify two primary obstacles in medical AI:

  1. Data Scarcity and Quality: Small sample sizes and high rates of missing data prevent deep models from learning robust representations.
  2. Feature Fusion Complexity: Simply concatenating diverse clinical features often leads to information redundancy and fails to capture the underlying correlations between different data types.

While previous attempts have used Random Forests (RF) to model patient similarity, such methods often provide coarse results. Graph-based approaches, on the other hand, are frequently sensitive to manually defined thresholds.

Methodology

The FaFCNN framework is structured into three main components: correlation feature construction, feature-aware interaction, and adversarial feature alignment.

1. Correlation Feature Construction

The first step involves generating "correlation features" that capture similarities between samples. Instead of relying solely on raw clinical data, the framework trains a Gradient Boosting Decision Tree (GBDT) ensemble. For each patient, the model identifies the specific leaf nodes reached across all trees in the ensemble.

Figure 1: Correlation Feature Construction mapping patient data to sparse features via GBDT leaf nodes.

As shown in Figure 1, these paths are converted into a high-dimensional sparse vector. If two patients consistently land in the same leaf nodes, their resulting vectors will be highly similar, effectively encoding complex, non-linear sample correlations that raw data might miss.

2. Feature-aware Interaction Module (FaIM)

The correlation features are then processed by the FaIM, which models second-order interactions between them. This module employs an Attention Net to weigh the importance of different feature combinations.

Figure 2: The FaFCNN Framework (a), Legend (b), and Attention Net architecture (c).

As illustrated in Figure 2, the Attention Net (Part c) uses a linear layer followed by ReLU and Softmax to produce interaction weights ($a_{i,j}$). A key feature of this module is the inclusion of L1-norm sparse regularization, which forces the model to focus on the most informative interactions while filtering out noise.

3. Feature Alignment Module (FAM)

To integrate the original clinical features with the GBDT-generated correlation features, the authors propose a Feature Alignment Module based on Domain Adversarial Learning. Because these two feature sets come from different distributions, simple concatenation can be problematic. The FAM uses a "discriminator" to distinguish between the feature sets and a "generator" (the main network) to map them into a shared high-dimensional space where they are indistinguishable, ensuring seamless and stable fusion.

Experimental Evaluation

The framework was evaluated on four public medical datasets from the UCI repository: Wisconsin Breast Cancer, Pima Indians Diabetes, Hepatitis, and Heart-Statlog. To rigorously test robustness, the authors simulated "low-quality" environments by introducing random perturbations and missing values at ratios ($\delta$) ranging from 0.5 to 0.9.

Key Results and Insights

FaFCNN demonstrated superior performance across all datasets and metrics:

  • Robustness to Noise: On the Wisconsin Breast Cancer dataset, the model maintained over 93% accuracy even when 90% of the data was perturbed.
  • Handling Imbalance: For the Pima Indians Diabetes dataset, FaFCNN achieved a sensitivity of 91.5%, significantly outperforming standard FCNN (78.1%) and Random Forest (49%).
  • Ablation Studies: The results confirmed that GBDT-based features provide more granular similarity information than RF-based alternatives. Furthermore, the adversarial alignment was crucial for maintaining stability during feature fusion.

Conclusion

FaFCNN provides a powerful methodology for medical AI, particularly when dealing with limited or noisy datasets. By leveraging the structural wisdom of decision trees within a neural network framework and employing adversarial alignment, it bridges the gap between traditional machine learning and deep learning for clinical applications. Future work will explore the scalability of this approach to larger, unstructured datasets such as medical imaging.

Summary updated

Abstract

There are two fundamental problems in applying deep learning/machine learning methods to disease classification tasks, one is the insufficient number and poor quality of training samples; another one is how to effectively fuse multiple source features and thus train robust classification models. To address these problems, inspired by the process of human learning knowledge, we propose the Feature-aware Fusion Correlation Neural Network (FaFCNN), which introduces a feature-aware interaction module and a feature alignment module based on domain adversarial learning. This is a general framework for disease classification, and FaFCNN improves the way existing methods obtain sample correlation features. The experimental results show that training using augmented features obtained by pre-training gradient boosting decision tree yields more performance gains than random-forest based methods. On the low-quality dataset with a large amount of missing data in our setup, FaFCNN obtains a consistently optimal performance compared to competitive baselines. In addition, extensive experiments demonstrate the robustness of the proposed method and the effectiveness of each component of the model\footnote{Accepted in IEEE SMC2023}.
BibTeX
@misc{kong2023fafcnn,
  author = {Menglin Kong and Shaojie Zhao and Juan Cheng and Xingquan Li and Ri Su and Muzhou Hou and Cong Cao},
  title = {{FaFCNN: A General Disease Classification Framework Based on Feature Fusion Neural Networks}},
  year = {2023},
  eprint = {2307.12518},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2307.12518},
}
Figure