Class Equilibrium using Coulomb's Law

Saheb Chhabra, Puspita Majumdar, Mayank Vatsa, Richa Singh

TL;DR

This paper introduces a novel data projection method inspired by electrostatics: treating data classes as point charges that repel each other according to Coulomb’s Law. By finding a physical "equilibrium space" where inter-class forces are balanced, the authors create a feature space that significantly boosts classification accuracy, particularly for challenging low-resolution images.

What Problem Does This Paper Address?

In machine learning, we often project data into a "feature space" to make it easier to classify. The goal is usually to maximize inter-class distance (pushing different classes apart) while maintaining intra-class distance (keeping the internal structure of a class intact).

However, these two goals often conflict—pushing classes apart can inadvertently distort their internal distributions. Furthermore, many existing projection techniques (like manifold or metric learning) are prone to getting stuck in local optima or are overly dependent on the specific training distribution. The authors ask a fundamental question: Is there an "optimal" projection space for a given dataset that exists independently of the classifier's complexity?

Methods

The authors propose a three-step framework based on the physics of charged particles:

  1. Charge Representation: Each class in the dataset is represented as a single point charge.
  2. The position of the charge is the mean of the class data points.
  3. The magnitude of the charge is determined by the variance (spread) of the class. A class with a wider spread "pushes" harder to ensure it has enough room in the equilibrium space.
  4. Computing Equilibrium: Using Coulomb’s Law ($F = k \frac{Q_1 Q_2}{R^2}$), the algorithm simulates the repulsive forces between these class charges. The charges move until the net force on every class is close to zero. This final configuration defines the "Equilibrium Space," where classes are optimally separated.
  5. Learning the Transformation: A Convolutional Neural Network (CNN) is trained to map data from the messy input space to this clean equilibrium space.
  6. Coulomb Classification: To classify a new test point, it is projected into the equilibrium space and assigned a negative charge. The class (positive charge) that exerts the maximum attractive force on the test point is the winner.

To handle mapping errors from the CNN, they also introduce Error Removal using Correlation (ERC), which uses Spearman’s rank correlation to refine the final coordinates of projected points.

Datasets

The researchers evaluated their approach on four standard benchmarks, specifically testing how the method handles decreasing image quality:

  • CIFAR-10: 60,000 color images across 10 classes.
  • MNIST: 70,000 handwritten digits.
  • Fashion-MNIST (FMNIST): 70,000 images of clothing/accessories.
  • NORB: Grayscale images of toys in five categories.

Each dataset was tested at three resolutions: 8x8, 16x16, and the original resolution.

Key Insights and Ideas

  • Physics as a Geometric Prior: Instead of relying solely on gradient descent to find a good separation, the authors use a physical law to define what "ideal" separation looks like. This provides a more "generalizable" target for the neural network to learn.
  • Variance as "Influence": One of the most intuitive leaps is using class variance as the charge magnitude. This ensures that "large" classes (geometrically speaking) aren't squeezed out by "small," dense classes.
  • Feature Enhancement in Equilibrium: Interestingly, when images are projected into the equilibrium space, they develop high-intensity regions that correspond to the most discriminative features of that class (e.g., the specific stroke of an '8' or the sleeves of a shirt). This suggests the equilibrium space naturally highlights what makes a class unique.

Key Results

The proposed method shines brightest in low-resolution scenarios, where traditional deep learning models often struggle due to the loss of fine-grained detail.

  • CIFAR-10 (8x8): The "Proposed + ERC" method achieved 96.26% accuracy. For comparison, standard models like DenseNet and MobileNetV2 achieved only 77.64% and 77.72%, respectively, on the same data.
  • FMNIST (8x8): Accuracy jumped from 77.25% (before ERC) to 87.30% (after ERC), showing the importance of the correlation-based refinement.
  • Computational Efficiency: Finding the equilibrium space is remarkably fast. For 8x8 images, the computation takes roughly 2.6 to 3.1 seconds across all datasets.

Limitations and Future Work

While the results are impressive, the paper notes a few areas for further exploration:

  • Hyperparameter Sensitivity: The constant $k$ in the force equation is currently determined experimentally for each dataset and resolution. A more automated way to tune this "medium constant" would be beneficial.
  • Mapping Complexity: The success of the method relies on the CNN's ability to learn the transformation. In very high-dimensional or complex data, this mapping might become a bottleneck.
  • Cross-Domain Application: The authors suggest that this "equilibrium" approach could be powerful for cross-domain tasks, such as matching low-resolution surveillance faces to high-resolution gallery images or matching sketches to photos.

Summary updated

Abstract

Projection algorithms learn a transformation function to project the data from input space to the feature space, with the objective of increasing the inter-class distance. However, increasing the inter-class distance can affect the intra-class distance. Maintaining an optimal inter-class separation among the classes without affecting the intra-class distance of the data distribution is a challenging task. In this paper, inspired by the Coulomb's law of Electrostatics, we propose a new algorithm to compute the equilibrium space of any data distribution where the separation among the classes is optimal. The algorithm further learns the transformation between the input space and equilibrium space to perform classification in the equilibrium space. The performance of the proposed algorithm is evaluated on four publicly available datasets at three different resolutions. It is observed that the proposed algorithm performs well for lowresolution images.
BibTeX
@inproceedings{chhabra2021class,
  author = {Saheb Chhabra and Puspita Majumdar and Mayank Vatsa and Richa Singh},
  title = {{Class Equilibrium using Coulomb's Law}},
  booktitle = {IJCNN},
  year = {2021},
  url = {https://openreview.net/forum?id=5VJrNJ4EGhJ},
}

Loading related content...

Figure