IKOAfs: Multi-Objective KNN Classifier for Feature Selection Based on Improved Kepler Optimization Algorithm

Yi Zhang, Jiqiang TangResearch Square 2025 · DOI

TL;DR

The paper introduces IKOAfs, an enhanced version of the Kepler Optimization Algorithm designed to solve the "curse of dimensionality" in machine learning. By simultaneously selecting the most relevant features and tuning K-Nearest Neighbors (KNN) parameters, IKOAfs achieves superior classification accuracy and significantly higher feature reduction rates compared to traditional swarm intelligence metaheuristics.

What Problem Does This Paper Address?

In modern data science, we are often drowning in features—many of which are redundant or outright noisy. This "curse of dimensionality" degrades classifier performance and spikes computational costs.

The K-Nearest Neighbors (KNN) algorithm is a popular choice for classification due to its simplicity, but it has a "Goldilocks" problem: it only works well if the number of neighbors ($k$), the distance metric (e.g., Euclidean vs. Manhattan), and the feature subset are just right. Most researchers tune these components in isolation, but they are deeply interdependent. The authors tackle the massive, combinatorial challenge of searching for the optimal combination of features and hyperparameters simultaneously.

Methods

The authors build upon the Kepler Optimization Algorithm (KOA), a metaheuristic inspired by the laws of planetary motion. To turn this into a robust feature selection tool, they introduced three key "Improved" (the 'I' in IKOAfs) mechanisms:

  1. Tent Mapping Initialization: Instead of starting the search with random "planets," they use chaotic tent mapping to ensure the initial population is spread evenly across the search space, preventing the algorithm from starting in a "dead zone."
  2. Lévy Flight Strategy: To prevent the algorithm from getting stuck in local optima (valleys that look like the best solution but aren't), they incorporate Lévy flights—stochastic jumps that allow the search agents to "leap" to distant regions of the search space.
  3. Dynamic Opposition-Based Learning (DOBL): This strategy evaluates a solution and its "opposite" simultaneously, effectively doubling the search efficiency and accelerating convergence toward the global best.

The Multi-Objective Fitness Function: The algorithm doesn't just chase accuracy; it uses a weighted fitness function that penalizes large feature sets. This forces the model to find the "leanest" possible subset of features that still maintains high predictive power.

Datasets

The researchers put IKOAfs through a two-phase gauntlet:

  • CEC2022 Benchmarks: 12 complex mathematical functions used to test the raw optimization power of the algorithm against other swarm intelligences.
  • UCI Machine Learning Repository: 8 diverse real-world datasets (including Ionosphere, Breast Cancer, WDBC, Glass, Balance, Wine, Zoo, and Sonar) covering both binary and multi-class classification tasks.

Key Insights and Ideas

  • The Power of Joint Optimization: The most significant "aha moment" is that $k$, the distance metric, and feature weights shouldn't be treated as separate problems. By encoding them into a single "candidate solution vector," the algorithm discovers synergies that manual tuning would miss.
  • Physics as a Search Heuristic: Using gravitational pull (exploitation) and orbital velocity (exploration) provides a natural, balanced framework for searching high-dimensional spaces.
  • Stability Matters: Unlike many metaheuristics that produce wildly different results on different runs, the inclusion of DOBL and Tent Mapping gave IKOAfs remarkably low standard deviations, making it a "reliable" tool for production environments.

Key Results

IKOAfs consistently outperformed six state-of-the-art algorithms (like Whale Optimization and Grey Wolf Optimizer):

  • Accuracy: It achieved 100% accuracy on the Glass dataset and over 95% on most others, consistently beating the original KOA and other competitors.
  • Feature Reduction: On the Sonar dataset, IKOAfs slashed the feature count from 60 down to 11 while improving accuracy—a massive win for model interpretability and speed.
  • Optimization Efficiency: In the CEC2022 tests, IKOAfs secured the lowest average rank (1.583), proving it is a more robust general-purpose optimizer than its predecessors.

Limitations and Future Work

While powerful, the authors acknowledge that IKOAfs is computationally demanding. Because it uses K-fold cross-validation within every iteration of the optimization loop, the time complexity scales linearly with the number of samples and iterations.

Future Directions: * Hybridization: Integrating "attention mechanisms" to further refine feature weighting. * Algorithm Expansion: Testing the IKOA framework with other classifiers like Support Vector Machines (SVM) or Random Forests. * Real-world Scaling: Applying the model to high-stakes fields like medical image lesion detection and traffic flow prediction.

Summary updated

Abstract

<title>Abstract</title> To improve classification performance, feature selection is usually employed to reduce data dimensionality, eliminate irrelevant features, and decrease computational cost. Among various classifiers, the K-Nearest Neighbors (KNN) algorithm is widely used because of its simplicity and robustness. However, KNN performance is highly sensitive to interdependent parameters. Simultaneously optimizing these parameters is challenging due to the high-dimensional search space. A multi-objective feature selection method is proposed to address these challenges, which is based on an improved Kepler Optimization Algorithm (IKOAfs). The proposed algorithm employs tent mapping, Lévy flight, and self-perturbation to improve search performance, and uses a multi-objective fitness function to jointly optimize classification accuracy, feature selection, and KNN parameters. The reliability of IKOAfs was evaluated in CEC2022 and eight representative public benchmark datasets. The results demonstrate that IKOAfs achieves superior performance compared to six well-known swarm intelligence algorithms in terms of classification accuracy, feature reduction rate, and generalization stability, particularly on high-dimensional datasets.
BibTeX
@inproceedings{zhang2025ikoafs,
  author = {Yi Zhang and Jiqiang Tang},
  title = {{IKOAfs: Multi-Objective KNN Classifier for Feature Selection Based on Improved Kepler Optimization Algorithm}},
  booktitle = {Research Square},
  year = {2025},
  doi = {10.21203/rs.3.rs-7556230/v1},
}
Figure