Jianming Zhang

Jianming Zhang

Adobe Systems (United States)
3 previous affiliations
121Machine learning papers tracked by Lacuna

1 other author named Jianming Zhang: Jianming Zhang (Changsha University of Science and Technology · 3 papers)

Is this you?

Research Overview

Jianming Zhang focuses on computer vision, generative image editing, and 3D scene reconstruction. His trajectory spans early work in visual saliency and robust tracking to recent advancements in diffusion-based image manipulation and high-resolution depth estimation. Currently, he investigates layered image decomposition, lighting harmonization, and efficient architectures for high-fidelity visual generation.

Research Trajectory

Zhang’s early research concentrated on foundational computer vision tasks, particularly visual saliency and object tracking. In Saliency Detection: A Boolean Map Approach (2013), he introduced a model based on Gestalt principles of figure-ground segregation, generating saliency maps by analyzing the topological structure of randomly thresholded color channels. This was followed by MEEM: Robust Tracking via Multiple Experts Using Entropy Minimization (2014), which addressed model drift in online tracking through a multi-expert restoration scheme designed to correct undesirable model updates. He also contributed to the development of neural attention mechanisms for visual recognition in Top-Down Neural Attention by Excitation Backprop (2016).

Transitioning toward image manipulation and 3D understanding, Zhang began exploring how natural language and geometric priors could guide visual tasks. Open-Edit: Open-Domain Image Manipulation with Open-Vocabulary Instructions (2020) presented a framework leveraging a universal visual-semantic embedding space to perform text-guided edits with minimal supervision, advancing the research direction of Spatial Grounding for Localized Instruction Guided Image Editing. In the 3D domain, Learning To Recover 3D Scene Shape From a Single Image (2021) tackled geometric distortion caused by unknown depth shifts and focal lengths. By combining a 2D depth predictor with a 3D point cloud refinement module, this work contributed to the broader area of Geometric Invariance for Sim-to-Real Domain Adaptation.

His focus on depth estimation extended into video, where temporal consistency remains a primary challenge. Less is More: Consistent Video Depth Estimation with Masked Frames Modeling (2022) proposed a spatial-temporal transformer network that predicts the depth of masked frames from neighboring ones, learning intrinsic inter-frame correlations to improve consistency. This approach aligns with his contributions to the area of Temporal Consistency in Video Depth Estimation. Concurrently, Zhang advanced generative modeling with Text-to-Image Generation via Implicit Visual Guidance and Hypernetwork (2022), which utilized a hypernetwork to modulate the encoding process, allowing the model to incorporate structural and stylistic cues from retrieved images.

In 2023, Zhang’s work increasingly intersected with complex 3D environments and lighting control. GAIT: Generating Aesthetic Indoor Tours with Deep Reinforcement Learning (2023) introduced a framework that automatically generates cinematic camera tours through 3D indoor spaces by combining a neural aesthetics estimator with smoothness constraints. Addressing object-level illumination, Lasagna: Layered Score Distillation for Disentangled Object Relighting (2023) combined Score Distillation Sampling with a specialized lighting prior to alter an image's illumination while preserving identity and texture.

Subsequent research emphasized high-fidelity scene analysis and harmonization. Self-Distilled Depth Refinement with Noisy Poisson Fusion (2024) modeled depth refinement as a noisy Poisson fusion problem, using a coarse-to-fine self-distillation process to generate high-quality edge labels. This significantly advanced the research direction of Sharpening Depth Discontinuities for Accurate 3D Reconstruction. In amodal perception, Amodal Scene Analysis via Holistic Occlusion Relation Inference and Generative Mask Completion (2024) decoupled occlusion reasoning from mask completion to handle complex mutual occlusions. Zhang also developed unified frameworks for specific domains, such as UniHuman: A Unified Model For Editing Human Images in the Wild (2024), which integrated reposing, virtual try-on, and text-based manipulation into a single diffusion model. For compositing, Relightful Harmonization: Lighting-Aware Portrait Background Replacement (2024) inferred sophisticated lighting cues directly from a single background image to seamlessly composite portraits without requiring high-dynamic-range environment maps.

Most recently, Zhang has focused on the structural decomposition and efficiency of generative models. Generative Image Layer Decomposition with Visual Effects (2025) proposed a framework to decompose images into plausible layers while retaining transparent visual effects like shadows and reflections. To address computational bottlenecks in high-resolution generation, ZipIR: Latent Pyramid Diffusion Transformer for High-Resolution Image Restoration (2025) employed a Latent Pyramid VAE and a Diffusion Transformer to restore degraded images efficiently. Further optimizing latent spaces, collaborative work in Aligning Visual Foundation Encoders to Tokenizers for Diffusion Models (2026) repurposed pretrained foundation models as encoders for latent diffusion, utilizing a semantic preservation loss to accelerate training and improve generation quality.

Current Directions

The most recent papers in the evidence address the precise control and computational efficiency of diffusion-based generative models. Recent work investigates the decomposition of complex 2D images into editable, layered representations that accurately preserve transparent visual effects such as shadows and reflections. Additionally, there is a strong focus on scaling high-resolution image restoration through novel architectures, such as latent pyramid structures combined with diffusion transformers, to bypass traditional computational bottlenecks. This is complemented by efforts to align pretrained visual foundation models with diffusion tokenizers, creating optimized latent spaces that accelerate training and enhance the semantic fidelity of text-to-image generation.

Profile updated

See something wrong? Report a mistake