ICE: Interactive 3D Game Character Facial Editing via Dialogue

Haoqian Wu, Minda Zhao, Zhipeng Hu, Changjie Fan, Lincheng Li, Weijie Chen, Rui Zhao, Xin YuIEEE Trans. Multimedia 2025 · DOI

Abstract

Interactive Character Editing (ICE) is a novel framework designed to streamline the creation and refinement of 3D game characters through natural language dialogue. By integrating Large Language Models (LLMs) with a semantic-guided parameter solver, ICE translates high-level user requests into precise adjustments of hundreds of in-game technical parameters. This approach enables an iterative, collaborative design process that significantly reduces the manual labor traditionally required for character customization.

Introduction

Modern Role-Playing Games (RPGs) offer extensive character customization, often involving hundreds of parameters ranging from facial bone structures to makeup details. However, manually adjusting these parameters is a time-consuming and labor-intensive task. While existing AI-driven "auto-creation" tools attempt to automate this process, they typically follow a single-round approach that lacks a refinement phase. If a generated character requires specific minor adjustments, these systems often necessitate a full regeneration, losing the progress made. Furthermore, direct parameter optimization frequently leads to anatomically impossible or "uncanny valley" results. ICE addresses these challenges by providing a multi-round dialogue interface for fine-grained, iterative character editing.

Methodology

The ICE framework bridges the gap between natural language and game engine parameters through two core modules:

Instruction Parsing Module (IPM)

The IPM utilizes an LLM (e.g., GPT-4) to manage the conversational flow. To maintain consistency across multiple rounds of editing, the module incorporates a Character Attribute Memory Bank that tracks the state of previously modified attributes. The IPM processes user input to generate a clean text prompt, determine editing strength, and provide a natural language response to the user.

Semantic-guided Low-dimension Parameter Solver (SLPS)

To translate text prompts into character adjustments, the SLPS employs three key components:

  1. Character Control Parameter Localizer: A RoBERTa-based model identifies the specific parameters relevant to a user's request, ensuring that localized edits (e.g., "pointy nose") do not affect unrelated facial features.
  2. Low-dimension Space (PCA): To maintain anatomical plausibility and improve optimization speed, bone parameters are projected into a 60-dimensional latent space using Principal Component Analysis (PCA).
  3. Neural Imitator: Since game engines are non-differentiable, a neural network is trained to mimic the game's renderer. This allows the system to use gradient descent to align character parameters with the CLIP embedding of the user's prompt.
Figure 1: An example of the multi-round dialogue process in ICE, showing the evolution of a character from a 'secret agent' to a 'powerful warrior' through iterative feedback.

Experiments and Results

The framework was evaluated using models from Justice Online Mobile and Naraka: Bladepoint, utilizing a 450-dimensional parameter space. A PCA-based prior was trained on 10,000 character sets derived from the CelebA dataset, while the Parameter Localizer was trained on a custom dataset of 10,000 instruction strings.

Performance Metrics

  • Efficiency: ICE achieves a response time of approximately 9 seconds per round, significantly faster than existing methods like TADA, which can take hours.
  • Alignment: ICE outperformed competing methods (DreamFace, TADA, T2P) with a higher CLIP score of 0.2699, indicating superior alignment between user descriptions and the resulting 3D models.
  • User Satisfaction: In a study involving 100 participants, ICE was preferred 77.1% of the time, largely due to its ability to maintain semantic consistency during fine-grained edits.

Discussion and Future Work

A key innovation of ICE is the use of semantic masking, which prevents unintended changes to unrelated facial features. Additionally, by constraining optimization to a low-dimensional space derived from real human faces, the system ensures realistic results even with extreme prompts.

Despite its advantages, ICE faces limitations in optimization latency for real-time applications and a potential lack of diversity for highly stylized or non-human characters due to its reliance on human-face priors. Future research will focus on further reducing latency and expanding the system's generative capabilities to include a wider variety of artistic styles.

Summary updated

BibTeX
@article{wu2025ice,
  author = {Haoqian Wu and Minda Zhao and Zhipeng Hu and Changjie Fan and Lincheng Li and Weijie Chen and Rui Zhao and Xin Yu},
  title = {{ICE: Interactive 3D Game Character Facial Editing via Dialogue}},
  journal = {IEEE Trans. Multimedia},
  year = {2025},
  doi = {10.1109/tmm.2025.3557611},
}

Loading related content...

Figure