A Novel Feature Selection Evolutionary Computation Framework for Privacy Preservation

Duan-Ting Duan, Bing Sun, Jian-Yu Li, Qiang Yang, Xiaofang LiuResearch Square 2024 · DOI

Abstract

The application of feature selection to identify important features in data processing is critical for enhancing the efficiency of learning models, reducing training duration, and improving interpretability. Evolutionary Computation (EC) is a powerful method for identifying representative feature subsets. However, increasing privacy concerns pose challenges to existing methods. This paper introduces a novel feature selection framework for EC that balances data privacy and computational efficiency. The framework employs a rank-based cryptographic function to return encrypted fitness values, ensuring privacy by preventing access to actual data. Additionally, an elite population learning strategy is introduced to accelerate convergence. Experiments on public datasets demonstrate that the proposed algorithms preserve privacy while enhancing feature selection efficiency.

Introduction: The Privacy-Efficiency Trade-off in Feature Selection

Feature selection is essential in machine learning for selecting relevant variables, making models faster and more interpretable. Evolutionary Computation (EC) algorithms, such as Differential Evolution (DE) and Particle Swarm Optimization (PSO), are highly effective at searching for optimal feature subsets. However, these methods typically require direct access to raw datasets and precise performance metrics (fitness values), which creates significant privacy risks in sensitive domains like healthcare or finance.

The core challenge addressed in this research is: How can an evolutionary search be guided toward optimal features without exposing sensitive data or exact model performance scores?

The Proposed Privacy-Preserving Framework

To bridge the gap between privacy and optimization, the authors propose a framework that decouples the optimization process from the raw data using two primary strategies:

  1. Rank-Based Cryptographic Function: Instead of returning a precise accuracy score (e.g., 88.42%), the data owner uses a cryptographic function that only returns the rank of a solution relative to others in the current population. By providing ordinal data (ranking) rather than cardinal data (exact scores), the actual performance metrics remain obscured from the optimization algorithm.
  2. Elite Population Learning Strategy: Because rank-based feedback provides less granular information than raw scores, convergence can be slower. To mitigate this, the authors introduced an "elite" memory strategy. The algorithm tracks historically successful solutions to guide mutation and velocity steps, preventing the search from becoming trapped in local optima.
The proposed privacy-preserving feature selection framework illustrating the interaction between the Learning Algorithm (LA), cryptographic function, and the EC optimizer.

Implementation and Process Flow

The framework was integrated into two new algorithms: PP-EDE (Privacy-Preserving Elite Differential Evolution) and PP-EPSO (Privacy-Preserving Elite Particle Swarm Optimization). The interaction between the data source, the learning model, and the cryptographic function ensures that sensitive information never leaves the secure environment. A local cache is also implemented to store previously evaluated feature subsets, reducing the computational burden of repeated model training.

Detailed process flow of the FS-EDE algorithm (a) and its interaction with the data source (b), learning model (c), and cryptographic components.

Experimental Evaluation

The framework was evaluated using five public datasets from the UCI Machine Learning Repository: Dry Bean, Image Segmentation, German Credit Data, Dermatology, and Ionosphere.

Key Results

  • Accuracy Improvements: In most cases, the privacy-preserving algorithms outperformed models using all features. For the Dry Bean dataset, accuracy increased from 0.685 to 0.865.
  • Feature Reduction: The algorithms achieved significant data compression. In the Dry Bean set, the feature count was reduced from 16 to just 2 while improving accuracy.
  • Performance Parity: PP-EDE and PP-EPSO performed as well as, or better than, standard non-private versions, proving that the elite learning strategy effectively compensates for the obscured fitness values.

Conclusion and Future Directions

This research demonstrates that evolutionary algorithms do not require exact performance values to succeed; relative ranking is sufficient. By utilizing a rank-based approach to preserve privacy, the framework offers a low-overhead solution for secure data analysis. Future work will focus on integrating more complex models like deep neural networks and scaling the framework for extremely high-dimensional "big data" environments.

Summary updated

Abstract

<title>Abstract</title> The application of feature selection to identify important features in data processing is critical for enhancing the efficiency of learning models, which not only reduces the training duration of the models but also enhances their interpretability. Evolutionary Computation (EC) is one of the most powerful and efficient feature selection methods, capable of identifying representative subsets of features through evolutionary processes in a reasonable computational time. However, with the increasing social and legal focus on privacy, the requirement to perform effective data analysis without revealing sensitive personal information has become a growing concern, posing new challenges to existing feature selection methods. To efficiently select features from privacy-sensitive data, this paper introduces a novel feature selection framework for EC to balance data privacy and computational efficiency. Specifically, a privacy-preserving framework is proposed that employs a rank-based cryptographic function to return encrypted fitness values rather than specific values, thereby ensuring privacy by preventing access to actual data information. Then, an elite population learning strategy is introduced to speed up EC algorithm convergence and prevent trapping in local optima. Finally, by integrating the privacy-preserving framework with enhanced EC algorithms, this paper introduces new methods for feature selection in privacy-sensitive data, and experiments on public datasets demonstrate that our algorithms not only preserve privacy but also enhance feature selection efficiency.
BibTeX
@inproceedings{duan2024novel,
  author = {Duan-Ting Duan and Bing Sun and Jian-Yu Li and Qiang Yang and Xiaofang Liu},
  title = {{A Novel Feature Selection Evolutionary Computation Framework for Privacy Preservation}},
  booktitle = {Research Square},
  year = {2024},
  doi = {10.21203/rs.3.rs-4472728/v1},
}
Figure