Structural and Geometric Stylization of 3D Scenes

This direction is split into other directions: geometric stylization. Its last version stays readable and citable here.

Version 1 of 191 papers126 conceptsfrozen 2026-10-04

Traditional style transfer has long been treated as a "skinning" problem: taking the colors and textures of a Van Gogh painting and "shrink-wrapping" them onto a photograph or a 3D mesh. However, true artistic style is rarely just surface-level. A Cubist portrait isn't just a face with a specific texture; it is a fundamental reorganization of geometry into jagged, overlapping planes.

Recent research has shifted from simple color manipulation toward structural stylization, where the underlying 3D geometry—whether represented as meshes, Neural Radiance Fields (NeRF), or 3D Gaussian Splats (3DGS)—is deformed to match the artistic intent.

The Limits of Texture-Only Stylization

Most early 3D stylization methods focused on modifying appearance while keeping the scene's density or mesh static. As noted by Feida Zhu and Jan Svoboda, this approach creates a "visual dissonance" where artistic textures are applied to physically incompatible, realistic shapes.

For example, in human avatar generation, Jianhui Yu and Hao Zhu point out that a system restricted to texture cannot turn a thin shirt into a heavy, bulky coat because it cannot add physical volume to the underlying mesh. To achieve high-fidelity artistic results, the model must be able to "break" the original geometry.

Deforming the Foundation

To move beyond surface-level changes, researchers have introduced mechanisms to displace or warp the 3D representation itself.

Local Displacement Fields

In Text2Mesh, Oscar Michel and colleagues proposed a "Neural Style Field" that predicts local geometric displacements. Instead of just painting a "stony" texture onto a sphere, the model actually pushes and pulls the vertices of the mesh to create the physical relief of stones. This ensures that as the camera moves, the shadows and highlights react to actual 3D bumps rather than flat pixels.

Volumetric and Gaussian Morphing

With the rise of point-based and volumetric rendering, stylization has moved into the latent space of these representations. Jamie Wynn and the team behind Morpheus demonstrate that by optimizing the positions and shapes of 3D Gaussian primitives, a scene can be structurally transformed—for instance, turning organic shapes into mechanical ones.

Similarly, Abhishek Saroha in GAS-NeRF shows that deforming the underlying structure to match a style's "vibe" (like making surfaces blocky for Cubism) produces results that are more physically plausible than those that only change color.

Stylization as Physical Reassembly

One of the most intuitive leaps in this field is treating stylization as a distribution matching problem of physical building blocks.

In WaSt-3D, Dmytro Kotovenko and Nikolaos Sarafianos move away from "painting" altogether. Instead, they treat the style scene as a collection of physical primitives (like pebbles or brushstrokes). They then "reassemble" the content scene using these style-specific building blocks. If the style is a gravel path, the system doesn't just make the target look like gravel; it builds the target out of gravel-shaped 3D Gaussians. This ensures the stylized scene maintains accurate surface normals and reacts realistically to lighting.

Constraining the Palette

A different approach to structural style involves restricting the "geometric palette" available to the model. Hao-Bin Duan and Miao Wang in Neural 3D Strokes use vectorized 3D primitives like splines and ellipsoids. By forcing the 3D scene to be represented only by these primitives, the system naturally adopts styles like brushwork or voxel art.

This is echoed in Cubic Stylization by Hsueh-Ti Derek Liu and Alec Jacobson, where they use $\ell_1$-regularization on surface normals to force a mesh into a "cubic" or polyhedral form. Here, the style isn't learned from an image but emerges from the mathematical constraints placed on the geometry itself.

Maintaining Consistency Across Views

The primary challenge in geometric stylization is maintaining multi-view consistency. If a 3D object is deformed to look "jagged" from the front, it must remain "jagged" in a spatially coherent way when viewed from the side.

Methods like DiffStyle3D by Yitong Yang solve this by integrating explicit geometric guidance into the self-attention mechanisms of diffusion models. By linking different viewpoints through the underlying 3D structure, these models avoid the "flickering" or "information leakage" that plagued earlier frame-by-frame video stylization techniques.

Ultimately, the goal of this research direction is to move 3D stylization from a 2D-inspired filter to a true 3D transformative process, where the computer "understands" that style is as much about the shape of the world as it is about the color of the light.

Papers in this version

Showing 60 of 91 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_structural_and_geometric_stylization_of__4928301c,
  title = {Structural and Geometric Stylization of 3D Scenes},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/structural-and-geometric-stylization-of-3d-scenes/rdr_4928301c7aea4d7f979eee9639527e74},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_structural_and_geometric_stylization_of__4928301c_v1,
  title = {Structural and Geometric Stylization of 3D Scenes},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/structural-and-geometric-stylization-of-3d-scenes/rdr_4928301c7aea4d7f979eee9639527e74/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}