Minimizing Intra-Class Variance with Center Loss

324 concepts251 papers3 proposals

When training a deep neural network for classification, our go-to tool is typically the softmax classifier paired with cross-entropy loss. This combination is highly effective at finding a decision boundary that separates different categories. However, cross-entropy loss has a fundamental geometric limitation: it only cares about separability, not compactness.

As long as a feature vector falls on the correct side of a decision boundary, the cross-entropy loss is satisfied. This creates a problem in representation learning. Features belonging to the same class can remain highly scattered throughout the embedding space, leading to large intra-class variance. While this might suffice for simple closed-set classification, it fails in fine-grained verification, open-set recognition, and clustering, where we need features of the same identity to be tightly grouped together.

To solve this, researchers developed center loss—a regularizer that explicitly minimizes intra-class variance by pulling features toward dynamically learned class centroids.

The Geometry of the Pull-Push Dynamic

To understand why center loss works, it helps to visualize the embedding space as a physical system governed by two opposing forces: a push force and a pull force.

Standard classification loss acts as the push force. It maximizes inter-class distance by driving decision boundaries between different categories. However, it lacks the mechanism to organize the internal structure of each class.

Center loss introduces the crucial pull force. It defines a learnable prototype—a "center"—for each class in the feature space. During training, the network minimizes the Euclidean distance between every individual sample's feature vector and its corresponding class center.

As Hongchang Ku demonstrates in Face Recognition Based on MTCNN and Convolutional Neural Network, combining standard classification loss with this center-minimizing objective forces the network to optimize for both objectives simultaneously. The cross-entropy loss pushes the clusters apart, while the center loss pulls each cluster into a tight, compact hypersphere. This dual-objective optimization drastically reduces intra-class variance while maintaining clear inter-class separation.

Refining the Objective: Contrastive and Joint Centers

While the baseline center loss simply minimizes the distance to a sample's own class center, subsequent formulations have made this mechanism more discriminative by considering all class centers simultaneously.

For instance, Ce Qi et al. introduced an architectural variation in Contrastive-center loss for deep neural networks. Instead of penalizing absolute distance, their approach optimizes a ratio: the numerator represents the squared distance between a sample and its assigned class center, while the denominator sums the distances to all other class centers. This formulation explicitly forces the network to minimize intra-class variance while actively maximizing the distance between different class clusters.

This concept of anchor-based regularization also extends to unsupervised and self-supervised tasks. In deep clustering, where ground-truth labels are absent, the challenge is avoiding messy global structures. Hanyang Li et al. address this in You Can Trust Your Clustering Model: A Parameter-free Self-Boosting Plug-in for Deep Clustering by adaptively selecting high-confidence samples as anchors. Their dual-objective discriminative loss minimizes the distance between high-confidence samples of the same cluster while pushing different cluster prototypes apart, significantly boosting cluster separability.

Why Compactness Enables Open-Set Generalization

The true power of minimizing intra-class variance becomes apparent when a model encounters unseen categories or domains.

In open-set verification tasks (such as face or speaker recognition), the model must determine whether two novel samples belong to the same identity, even if that identity was never present in the training set. If a network is trained only with cross-entropy, the feature space for unseen classes can be highly overlapping and disorganized.

By enforcing intra-class compactness during training, the network learns a highly structured embedding space. As Shuai Wang et al. show in Deep Discriminant Analysis for i-vector Based Robust Speaker Recognition, this joint optimization produces compact feature clusters that generalize effectively to unseen classes. Because the trained classes are tightly packed, the remaining space is open and well-distributed, allowing novel classes to form their own distinct, non-overlapping clusters at test time.

This structural integrity is equally vital for domain adaptation and out-of-distribution detection. When transferring knowledge from a source domain to a target domain, unconstrained feature spaces often suffer from "negative transfer," where different classes get mixed up during alignment.

To prevent this, Renjun Xu demonstrates in Reliable Weighted Optimal Transport for Unsupervised Domain Adaptation that enforcing structural constraints via prototype alignment ensures the target domain maintains a clean, separable structure. Similarly, in open-set domain adaptation, Renjun Xu et al. utilize class prototypes in Joint Partial Optimal Transport for Open Set Domain Adaptation to maintain tight intra-class clustering, allowing the model to cleanly isolate unknown target samples from established categories.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_intra_class_variance_minimization_7c4b8433,
  title = {intra-class variance minimization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/intra-class-variance-minimization/rdr_7c4b843360e84d27ad61c2b7769c8d81},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_intra_class_variance_minimization_7c4b8433_v1,
  title = {intra-class variance minimization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/intra-class-variance-minimization/rdr_7c4b843360e84d27ad61c2b7769c8d81/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake