Abstract
Methods based on basis functions and similarity measures are widely used in machine learning and related fields. These methods often take for granted that data is fully observed and are not equipped to handle incomplete data in an organic manner. This assumption is often flawed, as incomplete data is a fact in various domains such as medical diagnosis and sensor analytics. Therefore, one might find it useful to be able to estimate the value of these functions in the presence of partially observed data. In this work, we present methodologies to estimate the Gaussian Kernel, the Euclidean Distance, the Epanechnikov kernel and arbitrary basis functions in the presence of possibly incomplete feature vectors.
Introduction
Most machine learning algorithms are designed for complete datasets, assuming every feature is present. However, real-world data is frequently incomplete due to sensor failures, skipped medical tests, or data entry errors. Standard industry responses, such as row deletion or imputation (filling gaps with mean or predicted values), are often fundamentally flawed. Imputation treats a "guess" as a "fact," ignoring the inherent uncertainty of missing values. This leads to biased similarity measures, which ultimately degrade the performance of distance-based models like K-Nearest Neighbors (k-NN), Radial Basis Function (RBF) networks, and Support Vector Machines (SVM).
Methodology
This research proposes a shift from simple imputation to a probabilistic framework that accounts for missing data uncertainty. By treating missing values as random variables rather than fixed numbers, we derive methodologies to estimate the expected value of common basis functions and similarity measures directly from incomplete data.
Key technical approaches include:
- Gaussian Kernel Estimation: Modeling the squared distance between incomplete vectors as a sum of gamma-distributed random variables to derive a closed-form expression for the expected Gaussian kernel.
- Expected Euclidean Distance (EED): Utilizing the Nakagami distribution to model Euclidean distance in the presence of missing data, providing a more accurate metric for algorithms like the Minimal Learning Machine (MLM).
- Unscented Transform (UT): For complex basis functions without closed-form solutions, the UT allows for estimating the expected value of non-linear transformations with $O(1)$ complexity, regardless of the number of missing features.
These methods generally assume that data is Missing-at-Random (MAR) and that the underlying distribution can be modeled, for instance, via Gaussian Mixture Models (GMM).
Experimental Validation and Impact
The methodologies were validated across synthetic and real-world datasets, including software defect prediction and textile industry data (predicting cotton yarn strength). Results consistently showed that "Expected" versions of kernels and distances outperformed traditional imputation-then-calculation methods. For instance, the Expected Euclidean Distance (EED) significantly improved the performance of the Minimal Learning Machine (MLM) on incomplete datasets.
The academic rigor and relevance of this body of work are evidenced by its publication record across high-tier venues. As shown in Figure 1, the research resulted in four journal articles in top-tier QUALIS categories (A1 and A2) and several conference papers.
Conclusion
This work demonstrates that handling incomplete data organically through expected value estimation is superior to traditional imputation. By incorporating the variance of missing components, models achieve higher accuracy and robustness. Future work will explore relaxing the MAR assumption and applying these building blocks to more complex neural architectures.
Summary updated