Approximation of dilation-based spatial relations to add structural constraints in neural networks

Mateus Riva, Pietro Gori, Florian Yger, Roberto Marcondes Cesar Junior, Isabelle BlocharXiv 2021 · arXiv

Abstract

Deep learning models often require vast amounts of data to learn spatial relationships that are intuitive to humans. This paper introduces a method to incorporate "spatial common sense" into neural networks by using standard convolutions as a differentiable approximation for morphological dilation. This approach allows for the integration of structural constraints—such as relative positioning—directly into loss functions. The proposed convolutional approximation is hyperparameter-free, computationally efficient, and provides smooth gradients, making it a robust alternative to traditional, non-differentiable morphological operations.

Introduction

Neural networks are notoriously data-hungry, particularly in specialized fields like medical imaging where annotated datasets are scarce. One effective strategy to improve model generalization is the use of structural constraints based on prior knowledge (e.g., "the liver is to the right of the stomach"). In classical image processing, these relationships are modeled using mathematical morphology, specifically dilation. By dilating a reference object with a structuring element, one can create a map defining the valid spatial domain for another object.

However, standard dilation relies on a non-differentiable max operation, making it incompatible with gradient-based optimization in neural networks. While previous research has explored differentiable approximations like the Counter-Harmonic Mean (CHM), these methods often introduce computational overhead and sensitive hyperparameters.

Proposed Method: Convolutional Approximation

The authors propose using standard linear convolutions as a fast, differentiable, and hyperparameter-free proxy for morphological dilation. In this framework, the "structuring element" of a spatial relation is represented as a convolutional kernel.

Figure 1: Illustration of generating a relational map by applying a kernel to a source object.

Unlike traditional morphological approximations that require tuning parameters (like the power $p$ in Generalized Mean or CHM), the convolutional approach leverages highly optimized GPU kernels available in standard deep learning frameworks. This makes it significantly more scalable for high-resolution imagery.

Comparison of Spatial Relations

The study evaluated four primary spatial relationships: "To the right of," "Closeness," "Farness," and "Insideness." The convolutional approximation ($\Phi_*$) was compared against true dilation ($\Phi_\delta$) and other differentiable methods like CHM ($\Phi_{CHM}$).

Figure 4: Visual comparison of different dilation approximations across various spatial relations. The convolutional approach ($\Phi_*$) shows a characteristic 'feathering' effect.

Key Insights

  • The Feathering Effect: Convolution produces a smoothed or "feathered" version of the dilation map. This lack of sharp boundaries is actually an advantage for optimization, as it provides a smoother loss landscape for gradient descent.
  • Handling "Inside" Constraints: While true dilation is extensive (the result always includes the original object), convolution is more flexible. This allows researchers to penalize object overlap if desired, which is useful in tasks where objects should be adjacent but not overlapping.
  • Computational Efficiency: As image resolution increases, the time required for exponent-based approximations (CHM) grows exponentially. Convolution remains efficient, avoiding the numerical stability issues associated with high-power calculations.

Conclusion and Future Work

This work establishes the convolutional operator as a viable, efficient tool for embedding spatial logic into neural network training. By providing a differentiable path for morphological operations, it opens the door for "structural loss functions" that can guide models even when data is limited. Future research will focus on integrating this operator into full-scale medical segmentation tasks to quantify its impact on data efficiency and model robustness.

Summary updated

Abstract

Spatial relations between objects in an image have proved useful for structural object recognition. Structural constraints can act as regularization in neural network training, improving generalization capability with small datasets. Several relations can be modeled as a morphological dilation of a reference object with a structuring element representing the semantics of the relation, from which the degree of satisfaction of the relation between another object and the reference object can be derived. However, dilation is not differentiable, requiring an approximation to be used in the context of gradient-descent training of a network. We propose to approximate dilations using convolutions based on a kernel equal to the structuring element. We show that the proposed approximation, even if slightly less accurate than previous approximations, is definitely faster to compute and therefore more suitable for computationally intensive neural network applications.
BibTeX
@misc{riva2021approximation,
  author = {Mateus Riva and Pietro Gori and Florian Yger and Roberto Marcondes Cesar Junior and Isabelle Bloch},
  title = {{Approximation of dilation-based spatial relations to add structural constraints in neural networks}},
  year = {2021},
  eprint = {2102.10923},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2102.10923},
}
Figure