Impact Learning: A Learning Method from Features Impact and Competition

Nusrat Jahan Prottasha, Saydul Akbar Murad, Abu Jafar Md Muzahid, Masud Rana, Md. Kowsher, Apurba Adhikary, Sujit Biswas, Anupam Kumar BairagiarXiv 2022 · DOI · OpenReview

Abstract

Impact Learning is a novel supervised learning framework that borrows principles from ecology—specifically population dynamics—to model how features "compete" to influence a target variable. By treating feature interactions through the lens of the Intrinsic Rate of Natural Increase (RNI) and carrying capacity, the algorithm offers a robust alternative for both classification and regression tasks, often outperforming traditional models like SVMs and KNN.

Introduction

In traditional machine learning, features are typically viewed as static inputs contributing to a prediction through weights or branching logic. However, real-world data often exhibits "competitive" characteristics where features don't exist in isolation; they exert "back forces" on the target variable and on each other. Standard algorithms like Logistic Regression or Support Vector Machines (SVM) may struggle to capture these dynamic, ecosystem-like interactions. Impact Learning addresses this by using mathematical structures originally designed to describe how populations grow and compete for limited resources in nature.

Methodology

The core innovation of Impact Learning is the adaptation of the Malthusian logistic growth model to a machine learning context.

Bio-Inspired Framework

The model utilizes two key ecological concepts:

  1. Intrinsic Rate of Natural Increase (RNI or $r$): The rate at which a population grows.
  2. Carrying Capacity ($k$): The maximum population an environment can sustain.

Feature Competition and Back Impact

The authors introduce the concept of "Back Impact on Target" (BIT). They conceptualize the dataset as a competitive ecosystem where the target variable ($y$) is influenced by the "impact" of autonomous features ($x$).

Conceptual model of feature competition and back impact where multiple features interact with a target variable.

The Mapping Function

Instead of a standard linear combination, Impact Learning derives a specific mapping function representing the equilibrium state of a competitive system: $$y = \frac{k(w^T \cdot x)}{r - w_y k} + b$$ Here, $w$ represents weights optimized during training, while $r$ and $k$ are calculated based on the distribution (min/max) of the target variable in the training set.

Algorithmic Flow

The training process involves calculating $r$ and $k$ as priors before optimizing weights through a gradient-descent-like process.

Flowchart illustrating the algorithmic steps of Impact Learning, from feature input to the final output calculation.

Experimental Evaluation

The model was validated using three real-world datasets:

  • Asthma Prediction: 500 samples with 23 characteristics.
  • Diabetes Prediction: 469 samples with 11 attributes.
  • Heart Disease Identification: A Kaggle dataset with 15 features for 10-year risk forecasting.

Key Results

Impact Learning demonstrated high competitiveness against "gold standard" algorithms:

  • Asthma Dataset: Achieved the highest accuracy at 80.6%, outperforming SVM (77.5%), KNN (75.7%), and Random Forest (75.2%).
  • Diabetes Dataset: Achieved 86% accuracy, outperforming Bernoulli Naive Bayes and SVM, and trailing slightly behind Random Forest (89%).
  • Heart Disease (Regression): Showed significantly lower error rates (MAE, MSE, and RMSE) compared to standard Linear and Logistic Regression models, maintaining Mean Absolute Error (MAE) below 40%.

Discussion

Features as "Back Forces"

By treating a dataset as a competitive ecosystem, the model accounts for how one feature's predictive gain might lead to another's decline, mimicking natural niche competition.

Intrinsic Growth as a Prior

Calculating $r$ and $k$ from the data's intrinsic properties before training provides the model with a "biological" understanding of the target's potential range and growth rate.

Conclusion and Future Work

Impact Learning offers a robust, bio-inspired alternative to traditional supervised learning. While currently tested on datasets with moderate feature counts, future work will explore its scalability to high-dimensional "big data" and its integration with deep learning architectures and Natural Language Processing (NLP) tasks.

Summary updated

Abstract

Machine learning is the study of computer algorithms that can automatically improve based on data and experience. Machine learning algorithms build a model from sample data, called training data, to make predictions or judgments without being explicitly programmed to do so. A variety of wellknown machine learning algorithms have been developed for use in the field of computer science to analyze data. This paper introduced a new machine learning algorithm called impact learning. Impact learning is a supervised learning algorithm that can be consolidated in both classification and regression problems. It can furthermore manifest its superiority in analyzing competitive data. This algorithm is remarkable for learning from the competitive situation and the competition comes from the effects of autonomous features. It is prepared by the impacts of the highlights from the intrinsic rate of natural increase (RNI). We, moreover, manifest the prevalence of the impact learning over the conventional machine learning algorithm.
BibTeX
@misc{prottasha2022impact,
  author = {Nusrat Jahan Prottasha and Saydul Akbar Murad and Abu Jafar Md Muzahid and Masud Rana and Md. Kowsher and Apurba Adhikary and Sujit Biswas and Anupam Kumar Bairagi},
  title = {{Impact Learning: A Learning Method from Features Impact and Competition}},
  year = {2022},
  eprint = {2211.02263},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2211.02263},
}
Figure