IBDA: Improved Binary Dragonfly Algorithm With Evolutionary Population Dynamics and Adaptive Crossover for Feature Selection

Jiahui Li, Hui Kang, Geng Sun, Tie Feng, Wenqi Li, Wěi Zhāng, Bai JiIEEE Access 2020 · DOI

Abstract

Feature selection is an effective method to eliminate irrelevant, redundant, and noisy features, which improves the performance of classification and reduces the computational burden in machine learning. In this paper, an improved binary dragonfly algorithm (IBDA) which extends from the conventional dragonfly algorithm (DA) is proposed as a search strategy to design a wrapper-based feature selection method. First, a novel evolutionary population dynamics (EPD) strategy is introduced in IBDA to enhance the exploitation ability while ensuring population diversity of the algorithm. Second, IBDA proposes a novel crossover operator which establishes connections between the crossover rates and iterations so that making the algorithm can adjust the crossover rates of solutions dynamically, thereby balancing the exploitation and exploration of the algorithm. Finally, a binary mechanism is proposed to make the algorithm suitable for the binary feature selection problems. Simulations are conducted on 27 classical datasets from the UC Irvine Machine Learning Repository, and the results demonstrate that the proposed IBDA has better performance than some other comparison algorithms.

Introduction

In the era of big data, machine learning models often face datasets with hundreds or thousands of variables (features). However, many of these are irrelevant, redundant, or simply noise. This "curse of dimensionality" leads to two major issues: decreased classification accuracy and massive computational waste.

The authors focus on wrapper-based feature selection. Unlike "filter" methods that look at statistical properties of data in isolation, wrapper methods use a specific classifier (like K-Nearest Neighbors) to evaluate subsets of features. While more accurate, wrapper methods are computationally expensive and require a highly efficient search strategy to navigate the astronomical number of possible feature combinations. The standard Dragonfly Algorithm (DA)—a popular swarm intelligence method—often struggles with "exploitation" (fine-tuning a good solution) and can get stuck in local optima when applied to discrete, binary problems like feature selection.

Methodology: The IBDA Framework

The authors transform the continuous Dragonfly Algorithm into a robust binary optimizer through three primary innovations:

1. Evolutionary Population Dynamics (EPD)

To prevent the swarm from wasting time on poor solutions, IBDA incorporates a "survival of the fittest" step. In each iteration, the bottom 50% of the population (the "worst" dragonflies) are removed. They are replaced by new individuals generated from the top 50% using Linear Ranking Selection (LRS). This ensures the search stays focused on promising areas of the feature space and maintains high population quality.

2. Adaptive Crossover (AC)

Metaheuristics must balance exploration (searching new areas) and exploitation (refining known good areas). IBDA uses a dynamic crossover operator where the probability of retaining a solution changes over time. Early in the search, the algorithm favors exploration to find diverse feature subsets; as iterations progress, it shifts to favor exploitation to polish the best-found subset. This is achieved by linking the crossover rate to the iteration count via a sigmoid-like curve.

3. V-Shaped Binary Mapping

Since feature selection is a binary choice (a feature is either "in" or "out"), the authors use a V-shaped transfer function to map the continuous movement of the dragonflies into a probability of flipping a bit (0 to 1 or vice versa). This makes the algorithm suitable for discrete optimization tasks.

Experimental Evaluation

The algorithm was evaluated on 27 classical datasets from the UC Irvine (UCI) Machine Learning Repository. These datasets vary significantly in scale, ranging from small sets like Diabetes (8 features) to high-dimensional sets like Arrhythmia (278 features) and Semeion (265 features). The Lung-Cancer dataset was specifically used for the initial parameter tuning of the algorithm.

Key Insights and Results

  • Superior Accuracy: IBDA outperformed five major competitors (including Binary Particle Swarm Optimization and the original Binary Dragonfly Algorithm) in classification accuracy on 19 out of 27 datasets.
  • Feature Reduction: In many cases, IBDA didn't just match the accuracy of other models; it did so with a significantly smaller feature subset. For example, on the Breastcancer dataset, it reduced the feature count while maintaining near-perfect accuracy.
  • Convergence Speed: Convergence curves demonstrated that IBDA typically finds a high-quality solution in fewer iterations than its predecessors, thanks to the EPD mechanism pushing the population toward the global optimum faster.
  • Stability vs. Performance: While IBDA is powerful, it can have higher variance (standard deviation) in its results compared to simpler algorithms. This suggests that the added complexity of EPD and AC makes the search more "aggressive," which can occasionally lead to less stability in exchange for higher peak performance.

Conclusion

The Improved Binary Dragonfly Algorithm (IBDA) represents a significant advancement in wrapper-based feature selection. By integrating evolutionary population dynamics and adaptive crossover, the algorithm effectively navigates high-dimensional search spaces to find optimal feature subsets. Future work will explore different EPD strategies and potentially hybridization with other swarm intelligence algorithms to further improve stability and search efficiency.

Summary updated

Abstract

Feature selection is an effective method to eliminate irrelevant, redundant and noisy features, which improves the performance of classification and reduces the computational burden in machine learning. In this paper, an improved binary dragonfly algorithm (IBDA) which extends from the conventional dragonfly algorithm (DA) is proposed as a search strategy to design a wrapper-based feature selection method. First, a novel evolutionary population dynamics (EPD) strategy is introduced in IBDA to enhance the exploitation ability while ensuring population diversity of the algorithm. Second, IBDA proposes a novel crossover operator which establishes connections between the crossover rates and iterations so that making the algorithm can adjust the crossover rates of solutions dynamically, thereby balancing the exploitation and exploration of the algorithm. Finally, a binary mechanism is proposed to make the algorithm suitable for the binary feature selection problems. Simulations are conducted on 27 classical datasets from the UC Irvine Machine Learning Repository, and the results demonstrate that the proposed IBDA has better performance than some other comparison algorithms. Moreover, the effectiveness and performance of the proposed improved factors are evaluated by tests.
BibTeX
@inproceedings{li2020ibda,
  author = {Jiahui Li and Hui Kang and Geng Sun and Tie Feng and Wenqi Li and Wěi Zhāng and Bai Ji},
  title = {{IBDA: Improved Binary Dragonfly Algorithm With Evolutionary Population Dynamics and Adaptive Crossover for Feature Selection}},
  booktitle = {IEEE Access},
  year = {2020},
  doi = {10.1109/access.2020.3001204},
}
Figure