Abstract
This paper considers classification problems with hierarchically organized classes. We force the classifier (hyperplane) of each class to belong to a sphere manifold, whose center is the classifier of its super-class. Then, individual sphere manifolds are connected based on their hierarchical relations. Our technique replaces the last layer of a neural network by combining a spherical fully-connected layer with a hierarchical layer. This regularization is shown to improve the performance of widely used deep neural network architectures (ResNet and DenseNet) on publicly available datasets (CIFAR100, CUB200, Stanford dogs, Stanford cars, and Tiny-ImageNet).
Introduction
Standard deep learning classification typically treats labels as a "flat" set of independent categories. However, real-world data often exhibits a natural hierarchy—for instance, a "Golden Retriever" is a sub-category of "Dog," which in turn is an "Animal." Ignoring this structure in the final fully-connected layer of a neural network can lead to embeddings where classes are uniformly distributed to maximize separation, failing to capture the shared features between related categories. This paper introduces a hierarchical regularization technique that injects these structural priors directly into the network's architecture.
Methods: Hierarchically Connected Sphere Manifolds
The core contribution is a "hierarchically connected layer" that replaces the standard final layer. This layer utilizes three main principles:
- Recursive Parameterization: Instead of independent weight vectors, a class's classifier $w_{child}$ is defined as the sum of its parent's classifier $w_{parent}$ plus a displacement vector $\delta_{child}$.
- Spherical Constraints: To prioritize directional alignment between parent and child classifiers, each displacement vector $\delta$ is constrained to lie on a sphere manifold centered at the parent's location.
- Riemannian Optimization: To maintain stability on these manifolds, the authors employ Riemannian Stochastic Gradient Descent (RSGD), projecting gradients onto the tangent space of the spheres rather than simple post-hoc normalization.
Additionally, a radius decay hyperparameter $R_p = R_0 \gamma^{|p|}$ is introduced, ensuring that sub-classes are more tightly grouped than broader super-classes as the tree depth increases.
Experimental Results
The method was evaluated on fine-grained classification datasets including CUB200, Stanford Dogs, Stanford Cars, CIFAR100, and Tiny-ImageNet. Across architectures like ResNet and DenseNet, the proposed hierarchical approach consistently outperformed both standard "Plain" training and "Multitask" baselines.
For example, on the CUB200 bird species dataset, ResNet-18 accuracy improved from 54.88% to 60.98% when using the Riemannian spherical approach. The regularization also proved effective at mitigating performance degradation in deeper models, allowing a ResNet-18 with hierarchical priors to outperform a standard ResNet-50 on Tiny-ImageNet.
Qualitative analysis via t-SNE visualizations reveals that the proposed method creates significantly tighter and more distinct clusters for super-classes compared to standard training, where classes appear more scattered and less structured.
Conclusion
By forcing classifiers to respect known class hierarchies through nested sphere manifolds, this regularization technique provides a powerful inductive bias for fine-grained classification. While currently dependent on pre-defined hierarchies, future work may explore self-supervised discovery of these structures. The approach is computationally efficient and offers a robust alternative to hyperbolic embeddings for structured data in Euclidean networks.
Summary updated