Learning Structural Priors with Auxiliary Geometric Tasks

This direction is merged into another direction: auxiliary geometric supervision. Its last version stays readable and citable here.

Version 1 of 177 papers89 conceptsfrozen 2026-10-04

Deep neural networks are notoriously "texture-biased," often prioritizing local pixel patterns over the global geometric structure of a scene. In tasks ranging from 3D object detection to medical image segmentation, this lack of spatial common sense leads to "blobby" boundaries, physically impossible poses, or a failure to handle noisy environments.

To solve this, researchers have converged on a powerful multi-task learning strategy: supervising a model on auxiliary geometric tasks alongside its primary objective. By forcing a shared backbone to predict structural properties—such as surface normals, distance maps, or skeletal edges—the network is compelled to internalize the underlying geometry of the world.

The Geometry as a Regularizer

The core intuition behind this approach is that geometry provides a "skeleton" for semantic information. When a model is trained solely on a primary task like semantic segmentation, it may find shortcuts in local color distributions. However, if that same model must simultaneously predict a distance map or surface normals, it cannot rely on shortcuts; it must understand the 3D shape and spatial extent of the object.

In medical imaging, Fernando Navarro and Suprosanna Shit demonstrated in Shape-Aware Complementary-Task Learning for Multi-organ Segmentation that regressing distance maps forces the encoder to learn global geometric properties. This "shape-awareness" significantly improves the segmentation of small, difficult-to-resolve structures like the trachea. Similarly, Theodore Yanagihara and colleagues used DSGnet to show that supervising with geometric distance information helps identify complex anatomical structures that traditional masks might miss.

Forcing Spatial Common Sense

In autonomous driving and robotics, the challenge is often mapping 2D observations into 3D space. Without explicit geometric constraints, models struggle with depth and occupancy.

  • Occupancy and Depth: Yan Wang 0051 and Bin Yang introduced PLUME, which uses 3D occupancy prediction as a secondary task to provide implicit depth supervision. This ensures the feature representation captures the scene's metric geometry without requiring high-resolution depth maps.
  • Navigation: For agents navigating indoor environments, Rui Liu and Wenguan Wang found that integrating auxiliary tasks like room layout estimation and object detection provides the agent with "spatial common sense," leading to more robust policies in Volumetric Environment Representation for Vision-Language Navigation.
  • Trajectory Prediction: Liang Xiao showed that predicting road segmentations and waypoints acts as structural guidance, preventing models from forecasting physically impossible paths in Trajectory Prediction for Autonomous Driving with Topometric Map.

Sharpening Boundaries with Structural Tasks

Standard segmentation often produces "soft" or "melted" edges. By adding auxiliary tasks that focus specifically on boundaries and orientations, researchers can "crispen" the model's output.

In remote sensing, Dmitriy Smirnov proposed predicting a "frame field"—a set of directions aligned with building walls—in Polygonal Building Segmentation by Frame Field Learning. This geometric objective forces the network to learn the underlying architectural regularity, resulting in sharp, polygonal boundaries rather than amorphous blobs. Similarly, Weijia Li and Lingxuan Meng used MTBR-Net to decompose 3D building reconstruction into auxiliary tasks like edge orientation and skeleton extraction, improving height estimation accuracy by 40%.

This principle extends to industrial applications. In Power Battery Detection, Xiaoqi Zhao and Lihe Zhang integrated auxiliary line prediction and object counting. These tasks provide structural constraints that prevent the model from hallucinating features in noisy X-ray images, ensuring the structural integrity of EV batteries.

Geometry in Generative Models

Even in generative AI, where the goal is to create new images, auxiliary geometry is becoming a standard tool for maintaining realism.

Jae-Bin Cha and Mengwei Ren demonstrated in Text2Relight that jointly training on shadow removal and light source localization helps a diffusion model understand 3D scene geometry. This leads to more physically plausible relighting. For facial generation, Xiao Guo and Xiaoming Liu used Dense-Face to show that predicting depth maps and facial landmarks during training regularizes the latent space, allowing for precise pose control while preserving identity.

Why It Works: The Multi-Task Synergy

The success of these methods suggests that geometric tasks are not just "extra work" for a model; they are informative priors. By sharing a backbone across a primary task (e.g., detection) and an auxiliary geometric task (e.g., surface normal estimation), the model learns a representation that is inherently aware of the physical world.

As Dan Xu and Xiaogang Wang explored in PAD-Net, these intermediate auxiliary tasks provide "deep supervision." They ensure that the features being passed to the final prediction head are already rich with structural cues, effectively bridging the gap between raw pixels and high-level semantic understanding.

Papers in this version

Showing 60 of 77 papers.

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_learning_structural_priors_with_auxiliar_b000990f,
  title = {Learning Structural Priors with Auxiliary Geometric Tasks},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/learning-structural-priors-with-auxiliary-geometric-tasks/rdr_b000990f66c04b52979ac8e321c3a0a2},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_learning_structural_priors_with_auxiliar_b000990f_v1,
  title = {Learning Structural Priors with Auxiliary Geometric Tasks},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/learning-structural-priors-with-auxiliary-geometric-tasks/rdr_b000990f66c04b52979ac8e321c3a0a2/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}