Sample-Specific Output Constraints for Neural Networks

Mathis Brosowsky, Florian Keck, Olaf Dünkel, Marius Zöllner

Ensuring neural network outputs adhere to specific safety or physical constraints is critical for real-world deployment. Traditional methods rely on computationally expensive projection layers that solve optimization problems at runtime. This paper introduces ConstraintNet, an architecture that enforces hard, sample-specific constraints through a differentiable Constraint Guard Layer. By parameterizing the valid output space directly, ConstraintNet achieves zero computational overhead during inference and improves accuracy over standard projection-based methods.

Introduction

Deep neural networks are increasingly used in safety-critical applications, yet they often lack guarantees that their outputs will remain within valid bounds. For instance, facial landmarks must reside within bounding boxes, and autonomous vehicles must operate within safe acceleration limits. While "soft constraints" via penalty terms in the loss function are common, they do not provide hard guarantees. Hard constraints are typically enforced via projection layers, which map invalid outputs to the nearest valid point. However, these layers are slow and can hinder learning by mapping points primarily to the boundary of the safe region. ConstraintNet aims to solve these issues by making the constraint a fundamental part of the network's geometry.

Methods

ConstraintNet introduces two primary components to address these limitations: the Constraint Guard Layer and Constraint-Aware Features.

  1. Constraint Guard Layer ($\phi$): This final layer is a non-learnable mapping that transforms any raw network output $z$ into a valid output $\hat{y}$ within a constrained region $C(s)$. For convex polytopes, the layer treats raw outputs as weights for the vertices of the polytope. By passing these through a softmax function, the output is guaranteed to be a convex combination of the vertices, effectively "trapping" the prediction inside the shape. For other geometries like circle sectors, sigmoid functions map raw values into valid radial and angular ranges.
  2. Constraint-Aware Features: To help the network learn invariant features, constraint parameters $s$ (e.g., bounding box coordinates) are transformed into a tensor $g(s)$ and concatenated to an intermediate layer. This allows the hidden layers to "see" the boundaries they must respect.
ConstraintNet architecture overview with facial landmark example.

Experiments and Results

The authors evaluated ConstraintNet on facial landmark detection using the CelebA dataset and safe reinforcement learning for vehicle control.

Speed and Accuracy

In facial landmark tasks with triangle constraints, ConstraintNet demonstrated significant speedups, completing test set predictions in 34.2 seconds, compared to 295.5 seconds for projection-based methods—a nearly 9x improvement. Accuracy also improved; on CelebA with bounding box and relative constraints, ConstraintNet achieved a Mean Squared Error (MSE) of 9.70, significantly outperforming a standard unconstrained ResNet50 (17.42) and the projection-based method (10.44).

Comparison of MSE over training epochs for various models on facial landmark detection.

Safety in Reinforcement Learning

In the reinforcement learning task (Follow Object Controller), ConstraintNet achieved a 0.0% crash rate during both training and evaluation. In contrast, unconstrained models, even those utilizing simple output clipping, still resulted in collisions.

Conclusion

ConstraintNet provides a robust framework for enforcing hard constraints with minimal overhead. By parameterizing the valid space rather than projecting onto it, the model can cover the entire interior of the valid region more naturally while remaining end-to-end differentiable. Future work includes extending the framework to complex non-convex geometries and exploring its utility in multi-task learning and AI fairness.

Summary updated

Abstract

It is common practice to constrain the output space of a neural network with the final layer to a problem-specific value range. However, for many tasks it is desired to restrict the output space for each input independently to a different subdomain with a non-trivial geometry, e.g. in safety-critical applications, to exclude hazardous outputs sample-wise. We propose ConstraintNet—a scalable neural network architecture which constrains the output space in each forward pass independently. Contrary to prior approaches, which perform a projection in the final layer, ConstraintNet applies an input-dependent parametrization of the constrained output space. Thereby, the complete interior of the constrained region is covered and computational costs are reduced significantly. For constraints in form of convex polytopes, we leverage the vertex representation to specify the parametrization. The second modification consists of adding an auxiliary input in form of a tensor description of the constraint to enable the handling of multiple constraints for the same sample. Finally, ConstraintNet is end-to-end trainable with almost no overhead in the forward and backward pass. We demonstrate ConstraintNet on two regression tasks: First, we modify a CNN and construct several constraints for facial landmark detection tasks. Second, we demonstrate the application to a follow object controller for vehicles and accomplish safe reinforcement learning in this case. In both experiments, ConstraintNet improves performance and we conclude that our approach is promising for applying neural networks in safety-critical environments.
BibTeX
@inproceedings{brosowsky2021sample,
  author = {Mathis Brosowsky and Florian Keck and Olaf Dünkel and Marius Zöllner},
  title = {{Sample-Specific Output Constraints for Neural Networks}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  url = {https://openreview.net/forum?id=yvQ3k4bU3m},
}
Figure