TDEdit: A Unified Diffusion Framework for Text-Drag Guided Image Manipulation

Qihang Wang, Yaxiong Wang, Lechao Cheng, Zhun Zhong

Introduction

Generative image editing has traditionally been divided into two distinct paradigms: text-driven editing and drag-driven editing. Text-driven methods excel at semantic changes, such as altering textures or object identities, but often lack precise spatial control. Conversely, drag-driven methods provide intuitive point-and-drag interactions for geometric manipulation but can be "semantically blind," often leading to blurry textures or unrealistic distortions when the model fails to understand the object it is manipulating.

TDEdit supports joint text-drag control, as well as standalone drag and text editing.

TDEdit addresses this gap by introducing a unified diffusion framework that allows for simultaneous text and drag instructions. By integrating these two modalities, TDEdit enables complex edits—such as changing an object's species while repositioning its limbs—within a single, efficient process. This "best of both worlds" approach ensures that geometric changes are semantically grounded and that semantic changes respect the desired spatial layout.

Methodology

The core of TDEdit lies in two primary technical innovations designed to bridge the gap between 2D pixels and 3D-aware semantic editing: Point-Cloud Deterministic Drag (PCDD) and Drag-Text Guided Denoising (DTGD).

Point-Cloud Deterministic Drag (PCDD)

To overcome the limitations of 2D pixel warping, which often results in "mushy" textures, TDEdit leverages depth estimation (using Depth Anything v2) to lift 2D images into a 3D point cloud. This 3D representation allows for more realistic geometric transformations. The model performs global rigid movements (like rotating an entire object) followed by local non-rigid deformations using Radial Basis Functions (RBF). The resulting 3D points are then projected back into the 2D latent space. To handle the "holes" or voids left behind when an object is moved, the model employs Bidirectional Nearest Neighbor Interpolation (BNNI) to fill in gaps with contextually relevant features.

Drag-Text Guided Denoising (DTGD)

To balance semantic and geometric instructions without them conflicting, TDEdit employs a three-branch denoising architecture:

  1. Source Branch: Preserves the original image's layout and structural anchors.
  2. Reference Branch: Focuses on the target text prompt while maintaining the original global structure to ensure semantic consistency.
  3. Target Branch: The final output branch that integrates the dragged layout from the PCDD module with the semantic details provided by the reference branch.
Joint control example: transforming a wolf into a dog with pose adjustments.

A Hybrid Attention Control mechanism ensures that the final image inherits the correct appearance from the text prompt and the correct geometry from the drag operation. Furthermore, the researchers implemented a dynamic guidance factor ($\eta$) that shifts the balance during the generation process: it prioritizes preserving drag details in the early stages of denoising and shifts toward ensuring smooth, high-quality textures in the final stages.

Experimental Results

TDEdit was evaluated against specialized baselines on PIE-Bench (for text) and DragBench (for drag), as well as through a comprehensive user study.

  • Efficiency: TDEdit is remarkably fast. It completes an edit in approximately 2.52 seconds on an NVIDIA RTX 4090, significantly outperforming competitors like GoodDrag (~45s) and DragDiffusion (~21s).
  • Quantitative Performance: On DragBench, TDEdit achieved a Mean Distance (MD) of 26.91, outperforming mainstream methods like DragDiffusion (33.70). It also maintained the highest Image Fidelity (IF) score (0.92) among all tested baselines, indicating that its edits are both precise and visually natural.
  • User Preference: In head-to-head comparisons, users preferred TDEdit’s results 63% of the time over InfEdit for text tasks and 40.6% of the time for drag tasks, beating out specialized tools like FastDrag.

Conclusion and Future Work

TDEdit demonstrates that a unified approach to image editing is not only possible but often superior to specialized models. By treating images as 3D objects and employing a sophisticated three-branch denoising strategy, it provides users with unprecedented control over both the identity and the positioning of objects.

While the framework is robust, the authors note that it remains sensitive to certain hyperparameters in the PCDD module, particularly when dealing with objects of complex depths. Additionally, extremely large-scale displacements can still occasionally result in semantic artifacts. Future work will likely focus on more sophisticated feature fusion to further enhance pixel-level fidelity and automate parameter tuning for diverse image types.

Summary updated

Abstract

This paper explores image editing under the joint control of text and drag interactions. While recent advances in text-driven and drag-driven editing have achieved remarkable progress, they suffer from complementary limitations: text-driven methods excel in texture manipulation but lack precise spatial control, whereas drag-driven approaches primarily modify shape and structure without fine-grained texture guidance. To address these limitations, we propose a unified diffusion-based framework for joint drag-text image editing, integrating the strengths of both paradigms. Our framework introduces two key innovations: (1) Point-Cloud Deterministic Drag, which enhances latent-space layout control through 3D feature mapping, and (2) Drag-Text Guided Denoising, dynamically balancing the influence of drag and text conditions during denoising. Notably, our model supports flexible editing modes - operating with text-only, drag-only, or combined conditions - while maintaining strong performance in each setting. Extensive quantitative and qualitative experiments demonstrate that our method not only achieves high-fidelity joint editing but also matches or surpasses the performance of specialized text-only or drag-only approaches, establishing a versatile and generalizable solution for controllable image manipulation. Code will be made publicly available to reproduce all results presented in this work.
BibTeX
@misc{wang2025tdedit,
  author = {Qihang Wang and Yaxiong Wang and Lechao Cheng and Zhun Zhong},
  title = {{TDEdit: A Unified Diffusion Framework for Text-Drag Guided Image Manipulation}},
  year = {2025},
  eprint = {2509.21905},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.21905},
}

Loading related content...

Figure