Detecting Anomalies in the Residual Subspace of Neural Features

This direction is merged into another direction: near-out-of-distribution detection. Its last version stays readable and citable here.

Version 1 of 18 papers15 conceptsfrozen 2026-10-04

Deep neural networks are remarkably good at distinguishing between a cat and a dog, but they are notoriously bad at admitting when they are looking at something else entirely—like a piece of toast or a galaxy. This "overconfidence" occurs because the network is trained to project high-dimensional data onto a small set of class logits. In this process, the model learns to focus only on the features that help it discriminate between known categories, while discarding everything else as irrelevant noise.

However, recent research suggests that the key to identifying Out-of-Distribution (OOD) samples lies exactly in that discarded "noise." By looking at the residual subspace—the dimensions of the feature space that the classifier usually ignores—we can find a much clearer signal for anomaly detection.

The Intuition of the Quiet Room

Imagine a crowded rock concert. If someone shouts your name, you might not hear it because the background noise (the "variance" of the music) is so high. Now imagine a perfectly silent library. In that environment, even a pin drop is immediately noticeable.

In a neural network, the "principal" dimensions are like the rock concert. These are the high-variance directions in the feature space that the model uses to tell classes apart. Because these dimensions are designed to capture variation, an OOD sample can easily "blend in" with the high variance of the training data.

The "residual" dimensions, however, are like the silent library. For the training data, these dimensions have near-zero variance; the data is tightly clustered or entirely absent from this space. When an OOD sample appears, it often possesses features that don't fit the training manifold. These features manifest as sudden "noise" in these normally quiet dimensions, making the anomaly stick out like a sore thumb.

Finding the Null Space

To exploit this, researchers use techniques like Principal Component Analysis (PCA) on the feature embeddings of the training set. As Haoqi Wang and colleagues demonstrate in ViM: Out-Of-Distribution with Virtual-logit Matching, we can decompose the feature space into two parts:

  1. The Principal Subspace: The dimensions that capture the most variance and are used to calculate class logits.
  2. The Residual Subspace (or Null Space): The remaining dimensions that are typically discarded during the projection to logits.

By calculating the "residual" (the part of the input vector that cannot be explained by the principal subspace), we get a direct measure of how much the input deviates from the known training distribution.

Why the Mahalanobis Distance Works

This perspective helps explain a long-standing mystery in the field: why the Mahalanobis distance is so effective for anomaly detection. The Mahalanobis distance measures how many standard deviations a point is away from the mean of a distribution, accounting for the covariance of the data.

Ryo Kamoi and Kei Kobayashi argue in Why is the Mahalanobis Distance Effective for Anomaly Detection? that the method's success isn't due to its modeling of the "cat-ness" or "dog-ness" of the data. Instead, it works because it effectively weights the low-variance dimensions more heavily. In these "quiet" dimensions, even a small absolute deviation represents a massive statistical outlier, providing a high signal-to-noise ratio for identifying anomalies.

Applications in Zero-Shot Learning

The power of the residual subspace isn't limited to simple image classification. In the realm of Audio-Visual Generalized Zero-Shot Learning (AV-GZSL), the challenge is to distinguish between "seen" classes (encountered during training) and "unseen" classes (only described by attributes).

In Extremely Simple Out-of-distribution Detection for Audio-visual Generalized Zero-shot Learning, Yang Liu and co-authors show that a simple residual subspace projection can outperform complex generative models. By performing eigen-decomposition on the features of seen classes, they can calculate the norm of a new sample's projection onto the residual space. If the norm is high, the sample is likely "unseen," allowing the model to route it to a specialized zero-shot classifier rather than forcing it into a "seen" category.

Summary

The core insight across these works is that classification and detection are two different tasks. Classification requires looking at where the data is (the principal components), while anomaly detection requires looking at where the data should not be (the residual subspace). By shifting our focus to the "null space" of the model's feature projection, we can build detectors that are far more sensitive to the subtle signatures of out-of-distribution data.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_detecting_anomalies_in_the_residual_subs_02d7238e,
  title = {Detecting Anomalies in the Residual Subspace of Neural Features},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/detecting-anomalies-in-the-residual-subspace-of-neural-features/rdr_02d7238e6eeb4da192d580faf1343827},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_detecting_anomalies_in_the_residual_subs_02d7238e_v1,
  title = {Detecting Anomalies in the Residual Subspace of Neural Features},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/detecting-anomalies-in-the-residual-subspace-of-neural-features/rdr_02d7238e6eeb4da192d580faf1343827/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}