Industrial Style Transfer with Large-scale Geometric Warping and Content Preservation

Jinchao Yang, Fei Guo, Shuo Chen, Jun Li, Jian Yang

Abstract

Industrial Style Transfer (InST) is a novel framework designed to automate visual product design by simultaneously transferring large-scale geometric shapes and artistic textures between semantically unrelated objects. Unlike traditional Neural Style Transfer (NST) methods that struggle with significant shape deformations and detail preservation, InST decouples the process into two specialized modules: Large-scale Geometric Warping (LGW) and Interest-Consistency Texture Transfer (ICTT). By operating on shape masks and utilizing interest-point regularization, InST achieves high-fidelity transformations that maintain the recognizable identity of the source product.

Introduction

Visual Product Design (VPD) is a critical yet labor-intensive process where designers iterate on aesthetics and functionality. While Neural Style Transfer has revolutionized artistic image manipulation, its application in industrial design is limited by two major hurdles. First, standard algorithms cannot handle the "large-scale" geometric transformations required to map one object's silhouette onto another (e.g., transforming a car into the shape of an airplane). Second, the stylization process often obscures fine product details, rendering the output unusable for professional reference.

Overview of InST results showing geometric warping and style transfer across diverse object categories, such as company logos and classical vases.

The InST framework addresses these challenges by focusing on silhouettes rather than raw pixels for geometric alignment and anchoring textures to high-interest local features to ensure content preservation.

Methodology

The InST pipeline is divided into two distinct stages to handle shape and texture separately.

1. Large-scale Geometric Warping (LGW)

To bridge the gap between semantically different objects, LGW operates on shape masks. It utilizes a modified RAFT (Recurrent All-Pairs Field Transforms) architecture, termed Mask RAFT, to predict a warping field between source and target silhouettes. The module is trained using an unsupervised approach with a shape-consistency loss and a novel mask smoothness regularization term. This ensures that as the object expands or compresses to fit a new target shape, its internal structure remains coherent and free of chaotic distortions.

2. Interest-Consistency Texture Transfer (ICTT)

Once the geometric warping is complete, the texture must be applied without losing the product's identity. InST introduces Interest Regularization (IR) using the SuperPoint network to extract key interest points and descriptors. By minimizing the perceptual difference between these points in the original and stylized images, the model preserves the "soul" of the product—its key edges and defining features—even under heavy artistic stylization.

The InST pipeline: Large-scale Geometric Warping (LGW) predicts the warping field $w$ to create an intermediate shape $N$, followed by Interest-Consistency Texture Transfer (ICTT) to produce the final output $O$.

Key Insights

  • Mask-Based Optimization: Using masks provides a cleaner signal for geometric optimization than RGB data, allowing the model to successfully warp unrelated objects like a snail into a logo.
  • SuperPoint for Stability: Standard VGG-based losses often prioritize global structure at the expense of local detail. SuperPoint anchors the stylization to specific local features, which is crucial for brand identity.
  • Iterative Refinement: The recurrent update strategy from RAFT allows for progressively more accurate warping fields, handling deformations that single-pass methods cannot.

Results and Evaluation

InST was validated against several state-of-the-art methods using diverse datasets including WikiArt, MS-COCO, and the Metropolitan Museum of Art Open Access API.

  • Geometric Accuracy: InST achieved a mean Intersection-over-Union (mIoU) of 0.9284, significantly outperforming previous methods like GTST (0.7285) and DST (0.6000).
  • Detail Preservation: Quantitative SSIM (Structural Similarity Index) tests confirmed that Interest Regularization preserves more content details than standard AdaIN or ArtFlow methods.
  • User Preference: In a study involving 114 participants, InST was preferred 88.2% of the time for its ability to balance geometry and texture effectively.

Conclusion and Future Work

InST represents a significant step forward in automated industrial design. While highly effective, the authors note a limitation in "semantic correspondence"—the model may occasionally map functionally unrelated parts (like an eye to a chin) if their silhouettes overlap. Future research will focus on incorporating semantic priors to ensure that functionally important regions, such as wheels or screens, are mapped to appropriate target areas.

Summary updated

Abstract

We propose a novel style transfer method to quickly create a new visual product with a nice appearance for industrial designers' reference. Given a source product, a target product, and an art style image, our method produces a neural warping field that warps the source shape to imitate the geometric style of the target and a neural texture transformation network that transfers the artistic style to the warped source product. Our model, Industrial Style Transfer (InST), consists of large-scale geometric warping (LGW) and interest-consistency texture transfer (ICTT). LGW aims to explore an unsupervised transformation between the shape masks of the source and target products for fitting large-scale shape warping. Furthermore, we introduce a mask smoothness regularization term to prevent the abrupt changes of the details of the source product. ICTT introduces an interest regularization term to maintain important contents of the warped product when it is stylized by using the art style image. Extensive experimental results demonstrate that InST achieves state-of-the-art performance on multiple visual product design tasks, e.g., companies' snail logos and classical bottles (please see Fig. 1). To the best of our knowledge, we are the first to extend the neural style transfer method to create industrial product appearances.
BibTeX
@inproceedings{yang2022industrial,
  author = {Jinchao Yang and Fei Guo and Shuo Chen and Jun Li and Jian Yang},
  title = {{Industrial Style Transfer with Large-scale Geometric Warping and Content Preservation}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2022},
  doi = {10.1109/cvpr52688.2022.00768},
}
Figure