Improving Compositionality of Neural Networks by Decoding Representations to Inputs

Mike Wu, Noah Goodman, Stefano Ermon

TL;DR

Researchers from Stanford have introduced Decodable Neural Networks (DecNN), a framework that jointly trains a classifier with a generative model to map internal activations back to the input space. This "invertibility" allows neural networks to be composed like traditional software, enabling superior uncertainty estimation, out-of-distribution detection, and the ability to "scrub" protected features without sacrificing classification accuracy.

The Challenge of Compositionality in Deep Learning

In traditional software engineering, code is inherently modular. Developers can trace variables, write assertion statements to catch errors, and seamlessly compose different programs together. Deep learning, however, is notoriously a "black-box." An internal activation—the neural network equivalent of a "variable"—has no inherent semantic meaning. This opacity makes it nearly impossible to debug, interpret, or compose models without retraining them from scratch.

The authors of this paper aim to bring the engineering practicality of traditional software to deep learning. They tackle the fundamental challenge of compositionality: how can we make the internal states of a neural network "readable" and "usable" by other models, or even by the network itself?

Introducing Decodable Neural Networks (DecNN)

The core innovation presented is the Decodable Neural Network (DecNN). This architecture bridges the gap between abstract internal representations and human-interpretable inputs.

Decodable Neural Networks (DecNN) architecture
  1. Joint Training: The model consists of a standard classifier and a generative decoder. During training, the network is forced to minimize two objectives simultaneously: the standard classification error and a reconstruction error.
  2. Decoding Activations: Every hidden layer activation is passed through the decoder to reconstruct the original input. This effectively constrains the network's internal "logic" to remain grounded in the input domain, ensuring that intermediate representations retain semantic meaning.

Recursive Self-Composition (ReDecNN)

Because activations can be decoded back into the original input space (e.g., images), the authors propose a novel mechanism: feeding these "reconstructed" images back into the network as new inputs.

Ensembling through Self-Composition

By doing this recursively, they create a computational tree. By sampling different "paths" through this recursive tree, the model acts as an ensemble. If different paths yield different predictions, the model is deemed "uncertain." This provides a powerful, built-in mechanism for uncertainty estimation without needing multiple distinct models.

Constraint via Pretrained Models

To handle protected features (like gender or age in facial recognition), the authors demonstrate composing the DecNN with a pretrained "protected feature" classifier. They regularize the DecNN so that its decoded activations contain no information that the pretrained model can use to identify the protected attribute, effectively "scrubbing" the bias from the network's internal logic.

Key Insights

  • Activations as a Universal Language: By forcing activations to decode back to the input space, the authors create a "common language" for different models. An activation is no longer just a vector of numbers; it’s a representation that can be "seen" and processed by any other image-based model.
  • Uncertainty for "Free": Most methods for uncertainty (like Bayesian Neural Networks) come with a significant performance penalty. ReDecNN matches the accuracy of standard neural networks while providing uncertainty metrics that often outperform specialized baselines like MC-Dropout.
  • Semantic Regularization: Recursive composition acts as a form of learned data augmentation. The decoded images often "interpolate" between the input and the predicted class, helping the model ignore irrelevant features (like background color) and focus on task-relevant ones.

Experimental Results

The authors evaluated their approach across multiple modalities, including images (MNIST, FashionMNIST, CelebA) and speech (AudioMNIST, Fluent Speech Commands), testing against out-of-distribution (OOD) benchmarks like adversarial examples and cross-dataset shifts.

  • Accuracy Parity: DecNN and ReDecNN maintained parity with standard models. For example, on the complex CelebA dataset, both achieved ~90.8% accuracy.
  • Improved Calibration: ReDecNN significantly improved model "honesty." On MNIST, the Expected Calibration Error (ECE) dropped dramatically from 1.18 (Standard) to 0.33 (ReDecNN).
  • Robust OOD Detection: On CelebA, ReDecNN achieved a 0.60-0.64 ROC-AUC for detecting held-out attributes (like "wearing a hat"), while standard ensembles and Bayesian methods performed near random chance (~0.50).
  • Feature Scrubbing for Fairness: By regularizing against a "baldness" classifier, the authors successfully trained a "beardedness" classifier that was significantly more fair, reducing the performance gap between bald and non-bald individuals.

Limitations and Future Directions

The primary drawback of this approach is computational cost. Training a DecNN is roughly 6x slower than a standard network because it requires running a generative model for every layer and backpropagating through recursive loops.

Additionally, the system's success is heavily tethered to the quality of the generative model. If the decoder cannot produce high-quality reconstructions, the compositional benefits vanish. However, as generative modeling (such as diffusion models) continues to advance rapidly, DecNNs stand to become even more powerful. Future work may also explore applying this framework to more complex architectures, such as Transformers, further bridging the gap between deep learning and modular software engineering.

Summary updated

Abstract

In traditional software programs, it is easy to trace program logic from variables back to input, apply assertion statements to block erroneous behavior, and compose programs together. Although deep learning programs have demonstrated strong performance on novel applications, they sacrifice many of the functionalities of traditional software programs. With this as motivation, we take a modest first step towards improving deep learning programs by jointly training a generative model to constrain neural network activations to "decode" back to inputs. We call this design a Decodable Neural Network, or DecNN. Doing so enables a form of compositionality in neural networks, where one can recursively compose DecNN with itself to create an ensemble-like model with uncertainty. In our experiments, we demonstrate applications of this uncertainty to out-of-distribution detection, adversarial example detection, and calibration --- while matching standard neural networks in accuracy. We further explore this compositionality by combining DecNN with pretrained models, where we show promising results that neural networks can be regularized from using protected features.
BibTeX
@inproceedings{wu2021improving,
  author = {Mike Wu and Noah Goodman and Stefano Ermon},
  title = {{Improving Compositionality of Neural Networks by Decoding Representations to Inputs}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2021},
  url = {https://openreview.net/forum?id=ms1fOdxXhWH},
}
Figure