Sample-Specific Output Constraints for Neural Networks

Mathis Brosowsky, Olaf Dünkel, Daniel Slieter, Marius Zöllner

Abstract

Deep neural networks (DNNs) are increasingly deployed in safety-critical domains, yet they often lack formal guarantees regarding their output behavior. This paper introduces ConstraintNet, a novel architecture designed to ensure that network outputs remain within a user-defined, sample-specific "safe zone" during every forward pass. By integrating a non-learnable "Constraint Guard Layer" directly into the model's structure, the authors provide mathematical safety guarantees for applications such as autonomous driving and medical imaging without incurring significant computational overhead.

Introduction

While DNNs achieve state-of-the-art performance across various tasks, their "black box" nature remains a significant hurdle for deployment in fields where a single out-of-bounds prediction can be catastrophic. Current safety measures typically rely on post-hoc verification or reachability analysis, both of which are computationally intensive and difficult to scale to real-time systems. ConstraintNet addresses this by making the network provably constrained by design, ensuring outputs are always physically or anatomically valid based on external expert knowledge or rules.

Methodology: The ConstraintNet Architecture

The core innovation of ConstraintNet lies in its dual-input structure and the addition of a specialized output layer.

  1. Constraint Input ($s$): In addition to the standard data input ($x$), the network receives a parameter $s$ that defines the valid output space for that specific sample (e.g., coordinates of a bounding box or a safe acceleration range).
  2. Constraint Guard Layer ($\phi$): This final, non-learnable layer maps the network's internal hidden representation ($z$) onto the constrained space $C(s)$.
The high-level block diagram of ConstraintNet showing the integration of the constraint input $s$ and the Constraint Guard Layer $\phi$.

Mathematical Implementation

To maintain outputs within complex shapes like convex polytopes (e.g., triangles or pyramids), the layer utilizes a Softmax function. Since Softmax outputs are positive and sum to one, they serve as weights for a convex combination of the polytope's vertices. This ensures the final prediction $\hat{y}$ lies within the "convex hull" of the defined vertices. For radial constraints, Sigmoid functions are employed to map variables to valid radius and angle ranges.

Detailed architecture for facial landmark prediction, illustrating how the Constraint Guard Layer uses Softmax weights to combine vertices $v^{(i)}$ into a safe output $\hat{y}$.

Applications and Datasets

The authors evaluated ConstraintNet on two distinct safety-critical tasks:

  • Facial Landmark Prediction: Using image data to predict eye and nose locations, constraints were derived from face-detector bounding boxes and anatomical rules (e.g., "eyes must be above the nose").
  • Follow Object Controller (FOC): In an autonomous driving simulation, the model was trained via Reinforcement Learning (TD3 algorithm) to control vehicle acceleration. Constraints were derived from physical safety limits and ISO15622 standards to prevent collisions.

Key Results and Insights

  • Safety by Design: Unlike traditional verification, ConstraintNet makes it architecturally impossible to produce an invalid output. In the FOC task, this enabled "collision-free training," where the agent learned a control policy without ever violating safety limits.
  • Differentiability and Efficiency: Because the Constraint Guard Layer uses differentiable functions like Softmax and Sigmoid, the network remains end-to-end trainable via standard backpropagation with zero significant computational overhead.
  • Anatomical Consistency: In landmark prediction, the model successfully maintained relative positioning even when the underlying CNN features were ambiguous.

Limitations and Future Work

ConstraintNet's primary dependency is the external source providing the "safe zone." If the constraint-provider (e.g., a separate face detector) is inaccurate, the network will be constrained to an incorrect region. Furthermore, while convex polytopes cover many scenarios, future work is needed to handle highly complex, non-convex, or disjointed output spaces and to improve robustness against coordinate-invariant transformations like rotation.

Summary updated

Abstract

Neural networks reach state-of-the-art performance in a variety of learning tasks. However, a lack of understanding the decision making process yields to an appearance as black box. We address this and propose ConstraintNet, a neural network with the capability to constrain the output space in each forward pass via an additional input. The prediction of ConstraintNet is proven within the specified domain. This enables ConstraintNet to exclude unintended or even hazardous outputs explicitly whereas the final prediction is still learned from data. We focus on constraints in form of convex polytopes and show the generalization to further classes of constraints. ConstraintNet can be constructed easily by modifying existing neural network architectures. We highlight that ConstraintNet is end-to-end trainable with no overhead in the forward and backward pass. For illustration purposes, we model ConstraintNet by modifying a CNN and construct constraints for facial landmark prediction tasks. Furthermore, we demonstrate the application to a follow object controller for vehicles as a safety-critical application. We submitted an approach and system for the generation of safety-critical outputs of an entity based on ConstraintNet at the German Patent and Trademark Office with the official registration mark DE10 2019 119 739.
BibTeX
@inproceedings{brosowsky2021sample,
  author = {Mathis Brosowsky and Olaf Dünkel and Daniel Slieter and Marius Zöllner},
  title = {{Sample-Specific Output Constraints for Neural Networks}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  url = {https://openreview.net/forum?id=yvQ3k4bU3m},
}
Figure