Class Equilibrium using Coulomb's Law

Saheb Chhabra, Puspita Majumdar, Mayank Vatsa, Richa Singh

Abstract

Projection algorithms learn a transformation function to project data from input space to feature space, aiming to increase inter-class distance. However, this can adversely affect intra-class distance. Maintaining optimal inter-class separation without distorting intra-class distribution is challenging. Inspired by Coulomb's law of Electrostatics, we propose an algorithm to compute an "equilibrium space" where class separation is optimal. The algorithm learns a transformation between input and equilibrium spaces for classification. Evaluated on four datasets at multiple resolutions, the proposed method demonstrates significant performance gains, particularly for low-resolution images.

Introduction

In machine learning, projecting data into a feature space is a standard technique to facilitate classification. The primary objective is to maximize inter-class distance while preserving intra-class structure. These goals often conflict; aggressive separation can distort internal distributions. Moreover, traditional projection techniques like manifold learning often suffer from local optima or over-dependence on training distributions. This research explores whether an optimal projection space exists independently of classifier complexity by leveraging physical principles.

Methodology: A Physical Approach to Data Projection

The proposed framework treats data classes as physical entities governed by the laws of electrostatics. The process involves four key stages:

  1. Charge Representation: Each class is represented as a single point charge. The charge's position is the class mean, and its magnitude is proportional to the class variance. This ensures that classes with higher spread exert more repulsive force, securing sufficient space in the final projection.
  2. Computing Equilibrium: Using Coulomb’s Law ($F = k \frac{Q_1 Q_2}{R^2}$), the algorithm simulates repulsive forces between class charges. Charges move until the net force on each class approaches zero, defining the "Equilibrium Space."
  3. Learning the Transformation: A Convolutional Neural Network (CNN) is trained to map data from the input space to this calculated equilibrium space.
  4. Coulomb Classification: Test points are projected into the equilibrium space and assigned a negative charge. Classification is determined by the class (positive charge) exerting the maximum attractive force.
The proposed framework for computing equilibrium space using Coulomb's Law.

To mitigate mapping errors from the CNN, the authors implement Error Removal using Correlation (ERC), which employs Spearman’s rank correlation to refine projected coordinates.

Experimental Results

The method was evaluated using CIFAR-10, MNIST, Fashion-MNIST (FMNIST), and NORB datasets at various resolutions (8x8, 16x16, and original).

Performance in Low-Resolution Scenarios

The proposed algorithm excels in low-resolution environments where standard deep learning models typically lose accuracy due to missing fine-grained details. For instance, on 8x8 CIFAR-10 images, the "Proposed + ERC" method achieved 96.26% accuracy, significantly outperforming DenseNet (77.64%) and MobileNetV2 (77.72%).

Visualization of class separation in (a) input space and (b) equilibrium space.

As shown in the visualization above, the equilibrium space (b) provides much clearer class boundaries compared to the original input distribution (a). On FMNIST (8x8), the ERC refinement proved crucial, boosting accuracy from 77.25% to 87.30%. Furthermore, the equilibrium space computation is highly efficient, taking approximately 3 seconds per dataset.

Conclusion

By using physics as a geometric prior, this research provides a robust method for finding ideal class separation. The use of variance as charge magnitude ensures balanced representation for diverse class distributions. Future work will focus on automating the tuning of the electrostatic constant $k$ and exploring the method's efficacy in cross-domain tasks like sketch-to-photo matching.

Summary updated

Abstract

Projection algorithms learn a transformation function to project the data from input space to the feature space, with the objective of increasing the inter-class distance. However, increasing the inter-class distance can affect the intra-class distance. Maintaining an optimal inter-class separation among the classes without affecting the intra-class distance of the data distribution is a challenging task. In this paper, inspired by the Coulomb's law of Electrostatics, we propose a new algorithm to compute the equilibrium space of any data distribution where the separation among the classes is optimal. The algorithm further learns the transformation between the input space and equilibrium space to perform classification in the equilibrium space. The performance of the proposed algorithm is evaluated on four publicly available datasets at three different resolutions. It is observed that the proposed algorithm performs well for low-resolution images.
BibTeX
@inproceedings{chhabra2021class,
  author = {Saheb Chhabra and Puspita Majumdar and Mayank Vatsa and Richa Singh},
  title = {{Class Equilibrium using Coulomb's Law}},
  booktitle = {IJCNN},
  year = {2021},
  url = {https://openreview.net/forum?id=5VJrNJ4EGhJ},
}
Figure