When a deep neural network encounters an input that looks completely different from its training data—such as an out-of-distribution (OOD) sample or a malicious adversarial attack—it should ideally flag it as an anomaly. Traditionally, detection mechanisms look at the very end of the pipeline: the final hidden layer or the output logits. The intuition seems straightforward: the final layer contains the most abstract, semantic representation of the data, which should be best suited for high-level decision-making.
However, this final-layer focus creates a critical blind spot. As an input propagates through a network, each layer compresses and transforms the data, discarding information that is irrelevant to the specific classification task but vital for identifying anomalies. To build a robust safety net, a growing body of research argues that we must look inside the model, aggregating features across multiple intermediate layers to construct a holistic "fingerprint" of the input.
The Hierarchical Nature of Anomalies
Why does a single layer fail? Because anomalies are not uniform; they manifest at different levels of abstraction.
Deep neural networks naturally form a feature hierarchy. Early layers capture low-level structural, textural, and sensory features (such as edges, background patterns, or noise), while deeper layers specialize in high-level semantic concepts. As Gerhard Krumpl et al. observe in ATS: Adaptive Temperature Scaling for Enhancing Out-of-Distribution Detection Methods, shallow layers excel at identifying coarse, "far" out-of-distribution samples like random noise, whereas deeper layers are required to distinguish "near" out-of-distribution samples that share semantic similarities with the training set.
Similarly, Fangzhen Zhao et al. demonstrate in A Uniform Framework for Anomaly Detection in Deep Neural Networks that early convolutional layers are highly effective at catching low-level structural anomalies like noise, while final logit layers are better suited for detecting subtle adversarial perturbations.
If we only monitor the final layer, we risk missing low-level textural or background anomalies that the network has already "washed out" during its forward pass. Conversely, if we only monitor early layers, we miss high-level conceptual deviations. As Harry Anthony et al. show in On the use of Mahalanobis distance for out-of-distribution detection with neural networks for medical imaging, there is no single optimal layer for identifying all types of anomalies. Restricting feature analysis to the end of the network inevitably discards discriminative information that is vital for comprehensive safety monitoring.
Unmasking Adversarial and Poisoned Inputs
The multi-layer perspective is particularly powerful against malicious inputs, such as adversarial attacks or backdoors. Adversarial examples are often explicitly optimized to mimic the feature distribution of a target class at the final layer, successfully fooling the classifier into a confident but incorrect prediction.
However, these perturbations leave suspicious signatures in the intermediate processing stages. As Ambra Demontis et al. explain in Deep neural rejection against adversarial examples, monitoring internal representations across multiple network depths reveals adversarial perturbations that are masked by the time data reaches the final output layer. This multi-level inspection catches deviations from the training data distribution while they are still being processed, before they can be transformed into legitimate-looking target-class features.
This concept also applies to data poisoning and backdoor defenses. In Defending Against Backdoor Attacks by Layer-wise Feature Analysis (Extended Abstract), Najeeb Jebreel et al. demonstrate that the critical layer for detecting poisoned samples often occurs much earlier than the final classification layer. Inspecting a sequence of intermediate layers acts as a firewall, capturing anomalies that would otherwise blend seamlessly into the benign distribution at the output layer.
Mechanisms of Multi-Layer Aggregation
To exploit these distributed signals, researchers have developed several frameworks to extract and fuse layer-wise information.
A foundational approach was introduced by Kimin Lee et al. in A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks. The authors compute layer-wise confidence scores based on Mahalanobis distances and combine them using a simple linear model, leveraging both low-level and high-level feature characteristics simultaneously.
In the context of modern transformer models for text, Maxime Darrin demonstrates in Unsupervised Layer-Wise Score Aggregation for Textual OOD Detection that out-of-distribution signals are scattered across various hidden layers rather than being concentrated in the final layer. Aggregating anomaly scores across all layers provides superior performance compared to selecting any single layer, as different depths capture complementary syntactic and semantic information.
Rather than looking only at static activations, some methods analyze how the network responds to an input dynamically. Jayaram Raghuram proposes an unsupervised framework in A General Framework For Detecting Anomalous Inputs to DNN Classifiers that performs joint statistical testing across all layers simultaneously. This approach catches inputs that might look normal in isolation at one specific layer but exhibit highly improbable trajectories when their behavior is analyzed across the entire network hierarchy.
Taking this dynamic analysis a step further, Gukyeong Kwon et al. introduce Backpropagated Gradient Representations for Anomaly Detection, which measures the backpropagated gradients across multiple layers. Instead of asking what the model sees, it measures how much the model's weights would have to adapt to accommodate the input, revealing anomalies across multiple levels of abstraction by monitoring how different layers respond to the data.
Conclusion
By treating a deep neural network not as a black box with a single output, but as a rich, multi-tiered feature hierarchy, multi-layer feature aggregation transforms how we detect anomalies. It acknowledges that an anomaly can hide from the final classifier, but it cannot easily hide from the entire network pipeline. Fusing these distributed signals provides a more robust, comprehensive, and adaptive safety mechanism for deep learning systems.