Abstract
Projection algorithms learn a transformation function to project data from input space to feature space, aiming to increase inter-class distance. However, this can adversely affect intra-class distance. Maintaining optimal inter-class separation without distorting intra-class distribution is challenging. Inspired by Coulomb's law of Electrostatics, we propose an algorithm to compute an "equilibrium space" where class separation is optimal. The algorithm learns a transformation between input and equilibrium spaces for classification. Evaluated on four datasets at multiple resolutions, the proposed method demonstrates significant performance gains, particularly for low-resolution images.
Introduction
In machine learning, projecting data into a feature space is a standard technique to facilitate classification. The primary objective is to maximize inter-class distance while preserving intra-class structure. These goals often conflict; aggressive separation can distort internal distributions. Moreover, traditional projection techniques like manifold learning often suffer from local optima or over-dependence on training distributions. This research explores whether an optimal projection space exists independently of classifier complexity by leveraging physical principles.
Methodology: A Physical Approach to Data Projection
The proposed framework treats data classes as physical entities governed by the laws of electrostatics. The process involves four key stages:
- Charge Representation: Each class is represented as a single point charge. The charge's position is the class mean, and its magnitude is proportional to the class variance. This ensures that classes with higher spread exert more repulsive force, securing sufficient space in the final projection.
- Computing Equilibrium: Using Coulomb’s Law ($F = k \frac{Q_1 Q_2}{R^2}$), the algorithm simulates repulsive forces between class charges. Charges move until the net force on each class approaches zero, defining the "Equilibrium Space."
- Learning the Transformation: A Convolutional Neural Network (CNN) is trained to map data from the input space to this calculated equilibrium space.
- Coulomb Classification: Test points are projected into the equilibrium space and assigned a negative charge. Classification is determined by the class (positive charge) exerting the maximum attractive force.
To mitigate mapping errors from the CNN, the authors implement Error Removal using Correlation (ERC), which employs Spearman’s rank correlation to refine projected coordinates.
Experimental Results
The method was evaluated using CIFAR-10, MNIST, Fashion-MNIST (FMNIST), and NORB datasets at various resolutions (8x8, 16x16, and original).
Performance in Low-Resolution Scenarios
The proposed algorithm excels in low-resolution environments where standard deep learning models typically lose accuracy due to missing fine-grained details. For instance, on 8x8 CIFAR-10 images, the "Proposed + ERC" method achieved 96.26% accuracy, significantly outperforming DenseNet (77.64%) and MobileNetV2 (77.72%).
As shown in the visualization above, the equilibrium space (b) provides much clearer class boundaries compared to the original input distribution (a). On FMNIST (8x8), the ERC refinement proved crucial, boosting accuracy from 77.25% to 87.30%. Furthermore, the equilibrium space computation is highly efficient, taking approximately 3 seconds per dataset.
Conclusion
By using physics as a geometric prior, this research provides a robust method for finding ideal class separation. The use of variance as charge magnitude ensures balanced representation for diverse class distributions. Future work will focus on automating the tuning of the electrostatic constant $k$ and exploring the method's efficacy in cross-domain tasks like sketch-to-photo matching.
Summary updated