Introduction
In unsupervised learning, autoencoders are typically optimized for reconstruction accuracy, ensuring the latent space preserves enough information to recreate the input. However, this objective often ignores the topological structure of the latent representations. For specialized tasks like one-class learning (anomaly detection), the spatial arrangement and connectivity of points in the latent space are paramount. This paper introduces a novel "connectivity loss" ($L_\eta$) based on persistent homology, a tool from algebraic topology, to actively shape the latent space during training.
Methodology
The core innovation is a differentiable topological loss function that controls how points in the latent space connect to one another.
Persistent Homology and Connectivity Loss
The authors utilize persistent homology to track the evolution of connected components in a mini-batch of latent representations. By conceptualizing a ball of radius $\epsilon$ around each point, they monitor the specific radii at which separate groups of points merge.
The Connectivity Loss ($L_\eta$) penalizes deviations from a target merge distance $\eta$. This forces the network to arrange latent representations such that they link up at a predictable, optimized scale. Crucially, the authors demonstrate that this topological information is differentiable, allowing for standard backpropagation.
Architectural Enhancements
To mitigate the "curse of dimensionality," the latent space is partitioned into multiple independent "branches." Each branch is independently optimized with its own connectivity loss. For final one-class scoring, a non-parametric Kernel Density Estimator (KDE) is applied to these structured representations.
Experimental Setup and Datasets
The method was evaluated on several computer vision benchmarks, including CIFAR-10, CIFAR-100, Tiny-ImageNet, and the full ImageNet (ILSVRC 2012). A significant focus was placed on transfer learning: autoencoders were trained on auxiliary unlabeled data (e.g., CIFAR-100) and then used for one-class tasks on different datasets without further fine-tuning.
Key Results
The proposed approach demonstrates significant advantages in two main areas:
- Low-Sample Performance: In scenarios with limited training data (e.g., 120 samples per class), the connectivity loss significantly outperforms existing state-of-the-art methods. On CIFAR-10, it achieved an AUC of 0.76, compared to 0.69 for the nearest competitor.
- Scalability to ImageNet: Unlike many competing methods that require training a separate model for every "normal" class, this approach uses a single, reusable encoder. This avoids the massive computational overhead typically associated with large-scale one-class learning.
Discussion and Insights
The transition from "passive" topological analysis to "active" topological shaping represents a shift in representation learning. By imposing a connectivity prior, the model achieves a "densification" effect, making the latent space more predictable for density-based classifiers and reducing dependence on large datasets. While the method introduces new hyperparameters like the target distance $\eta$, the resulting structural consistency across datasets suggests that these topological properties are highly transferable. Future work may explore higher-order topological features, such as loops or voids, to further refine latent space organization.
Summary updated