Developing an Image-Based 3D Model Editing Method

Min Pang, Ligang He, Fengguang Xiong, Xiaowen Yang, Zhiying He, Xie HanIEEE Access 2020 · OpenReview

Abstract

As 3D technologies advance rapidly, generating a large number of 3D models effectively and efficiently has become a remarkable challenge. This paper proposes a novel editing method based on the feature lines of images (contour and principal axis) for generating new 3D models. By comparing the feature lines of a target image and an original 3D model, constraint conditions are established to control the editing process. The method utilizes as-rigid-as-possible (ARAP) mesh deformation guided by a new energy function to produce models that resemble the target image while maintaining structural integrity. Experimental results demonstrate that this method is more effective and efficient than existing techniques.

Introduction

Creating high-quality 3D models from scratch is a labor-intensive process requiring specialized skills. While "model reuse"—modifying an existing 3D model—is more efficient, current image-based editing methods often struggle with geometric distortion and high computational complexity. Existing algorithms frequently focus only on distances between points, leading to "collapsed" features or unnatural stretching (e.g., a bear’s nose flattening or a cup’s rim curling). The authors aim to bridge the gap between a designer’s 2D vision and a 3D asset, ensuring the final model looks like the target image while maintaining its structural integrity.

Methodology

The researchers built their method on the foundation of As-Rig-As-Possible (ARAP) mesh deformation, introducing two critical innovations:

Similar Triangle Correspondence

Instead of just mapping points based on their position along a contour (length proportion), the system uses the principal axis of the object to form triangles. By ensuring the triangles in the 3D projection are similar to those in the 2D sketch, the algorithm captures angular features. This prevents the "shearing" or "squashing" effects common in older methods.

Feature lines including contour and principal axis used for geometric constraints.

Global Energy Function ($E_{IARAP}$)

Standard ARAP focuses on local rigidity—keeping small neighborhoods of vertices consistent. However, this can lead to global distortion. The authors added a global energy term ($E_c$) that measures the stability of the entire region's structure. A balancing coefficient ($\gamma$) allows the system to weigh local details against global shape stability. Through empirical testing, a $\gamma$ value of 0.15 was found to provide the best balance.

The Pipeline

  1. Extract: Get the contour and principal axis from a 2D image/sketch.
  2. Project: Project the original 3D model onto a 2D plane from a user-selected viewpoint.
  3. Match: Establish vertex-to-pixel correspondence using the similar triangle rule.
  4. Deform: Solve a sparse linear equation set to minimize the combined energy function, resulting in the new 3D shape.

Results and Discussion

The method was evaluated using the SHREC12 and Princeton University datasets.

Accuracy and Fidelity

The researchers used a Laplacian distance metric ($E_a$) to measure similarity. Their method consistently achieved lower $E_a$ scores (0.015 to 0.12) compared to previous methods (0.08 to 0.24), indicating a much closer match to the target image.

Comparison showing the original 3D model (left), the target image (middle), and the resulting edited 3D model (right).

Computational Efficiency

The proposed method is roughly 2x to 4x faster than existing literature. For example, editing an airplane model took 6.42 seconds, compared to 20.35 seconds and 15.46 seconds for competing methods. This efficiency is achieved by comparing contours only once to establish control points, rather than using iterative cross-section comparisons.

Conclusion

This paper presents an automated image-based approach that allows users to generate high-quality 3D models without manual "point-pulling." While the method currently requires manual viewpoint selection, future work will focus on automating view selection and integrating deep learning to address depth ambiguity in 2D sketches. Despite these limitations, the method provides a significant advancement in making 3D model editing more accessible and efficient.

Summary updated

Abstract

As 3D technologies advance rapidly, 3D printing, 3D animation, and 3D Movie are springing out in different areas. It becomes a remarkable challenge to generate a large number of 3D models effectively and efficiently. This paper proposes a novel editing method based on the feature lines of images (i.e., image contour and principal axis) for generating new 3D models. Our method takes as input an existing 3D model (as the original model) and an image selected by the user (or a sketch hand-drawn by the users), and performs the model editing to generate a new 3D model. In particular, our method first takes as input an original 3D model and an image selected by the user. Second, the selected image is processed to produce the feature lines, i.e., contour and principal axis of the image. Third, the silhouette of the original model from a given view is acquired, and projected on a projection plane to produce a contour and principal axis of the model, which is the feature lines of the original model. Fourth, by comparing the feature lines of the image and the original model, the constraint conditions are established to control the editing of the 3D model. Finally, 3D model editing is conducted through the as-rigid-as-possible mesh deformation to produce a new 3D model with the appearance resembling the selected image. Furthermore, this paper proposes an energy function to guide the detailed model editing, and measure the similarity between the generated 3D model and the corresponding image. We have conducted extensive experiments to evaluate the proposed method. The results show that comparing with the existing editing methods in literature, the proposed model editing method is able to construct various types of 3D models more effectively and more efficiently.
BibTeX
@inproceedings{pang2020developing,
  author = {Min Pang and Ligang He and Fengguang Xiong and Xiaowen Yang and Zhiying He and Xie Han},
  title = {{Developing an Image-Based 3D Model Editing Method}},
  booktitle = {IEEE Access},
  year = {2020},
  url = {https://openreview.net/forum?id=rHLkyJQdUQ},
}
Figure