Recent advances in data generation and its applications in computer vision

Yuzhuo Ma, Yongfei Zhang, Wei Jia, Jiaying Liu, Tian Gan, Wenhan Yang, Junbao Zhuo, Wu Liu, Huimin MaJournal of Image and Graphics 2025 · DOI

Abstract

Large-scale image and video datasets are fundamental to the advancement of computer vision (CV) algorithms. However, the complexity and cost of real-world data collection necessitate alternative approaches. This paper reviews the state-of-the-art in data generation, categorizing methods into traditional augmentation, 3D rendering (including NeRF), and deep generative models (GANs and Diffusion). We analyze the application of these techniques across various domains—such as autonomous driving, biometric recognition, and video generation—and discuss the critical challenges of the "Sim-to-Real" gap and the need for standardized evaluation metrics.

Introduction: The Data Bottleneck

In the era of deep learning, data is the primary fuel for progress. The "Scaling Law" suggests that as models grow, their demand for high-quality, labeled data becomes insatiable. Relying solely on real-world collection faces three major hurdles: high labor costs for labeling, the difficulty of capturing "long-tail" corner cases (e.g., rare traffic accidents), and significant privacy concerns regarding biometric data.

Figure 1: The rapid growth of research publications in the field of data generation from 2020 to 2024.

As shown in Figure 1, the volume of research in this domain has seen a steady increase, reflecting its critical role in the next generation of AI development.

Evolutionary Stages of Data Generation

The research landscape is currently divided among three primary methodologies, each offering different levels of complexity and realism.

Figure 2: Distribution of literature across the three main technology types: Deep Generative Models, Traditional Data Generation, and 3D Rendering.
  1. Traditional Augmentation: This "old school" approach uses geometric transforms (flipping, cropping) and color jittering. While computationally efficient, these methods cannot create entirely new content.
  2. 3D Rendering & Simulation: By leveraging game engines (Unreal Engine, Unity) and newer techniques like Neural Radiance Fields (NeRF), researchers can create virtual environments with "perfect" ground-truth labels (depth, semantics) that are impossible to capture in reality.
  3. Deep Generative Models (AIGC): This is the current frontier, including GANs and the state-of-the-art Diffusion Models (e.g., Sora). These models generate high-fidelity, diverse content that can effectively mimic real-world distributions.

According to the literature distribution shown in Figure 2, Deep Generative Models now represent the largest share of research (37.9%), followed by 3D rendering (32.1%) and traditional methods (30.0%).

Applications and Impact

Synthetic data has transitioned from a mere supplement to a primary training tool in several high-stakes fields:

  • Autonomous Driving: Simulated versions of datasets like nuScenes allow for testing safety in extreme weather and rare edge cases without physical risk.
  • Human and Crowd Analysis: Datasets like MOTSynth and GCC (Grand Central Crowd) use game engines to simulate thousands of interacting agents. Models trained on MOTSynth have shown mean Average Precision (mAP) improvements of up to 16.3% over those trained only on real data.
  • Biometrics: Synthetic fingerprints and irises allow for training robust recognition systems while completely bypassing the legal and ethical hurdles of collecting private biometric data.

Challenges and Future Directions

Despite significant progress, several "unsolved" areas remain for researchers:

  • The Sim-to-Real Gap: Ensuring that features learned by an AI in simulation translate effectively to the messy, noisy real world remains the primary technical hurdle.
  • Evaluation Metrics: Current metrics like FID (Fréchet Inception Distance) measure visual quality but fail to capture "training utility"—a measure of how much a synthetic dataset actually improves a model's downstream performance.
  • World Models: The field is shifting toward models that understand physics and temporal consistency, which is essential for the development of autonomous systems and embodied robotics.

Conclusion

Controllable, large-scale data generation is the only viable path to satisfy the data hunger of modern AI. By bridging the gap between simulation and reality, synthetic data will continue to drive the evolution of computer vision toward more robust, safe, and ethical applications.

Summary updated

Abstract

大规模图像和视频数据集是驱动计算机视觉算法发展的核心要素。面向计算机视觉任务,构建大规模图像和视频数据集是一项重要但复杂的任务。基于生成对抗网络和扩散模型等数据生成方法可以可控地生成大规模、多样性的图像和视频数据,有效替代或弥补真实图像和视频数据集,为计算机视觉技术领域的发展提供了新的动力。本文在对面向计算机视觉的图像和视频数据生成与应用背景简介的基础上,首先,从以几何变换等为代表的传统数据增广和生成、以虚拟引擎和神经辐射场等为代表的基于三维渲染的数据生成方法、以生成对抗网络和扩散模型等为代表的基于深度生成模型的生成方法等3方面系统调研典型的图像和视频数据生成技术与模型;其次,梳理了典型的图像和视频数据生成技术与模型在图像增强,目标检测跟踪与姿态动作识别等个体分析,基于图像和视频的生物特征识别、人员计数与人群行为分析等群体行为分析、自动驾驶、视频生成、具身智能等典型计算机视觉相关任务中的应用;最后,分析面向计算机视觉的数据生成与应用中存在的问题,并展望未来发展趋势,以期促进图像和视频数据生成及计算机视觉技术的发展。
BibTeX
@article{ma2025recent,
  author = {Yuzhuo Ma and Yongfei Zhang and Wei Jia and Jiaying Liu and Tian Gan and Wenhan Yang and Junbao Zhuo and Wu Liu and Huimin Ma},
  title = {{Recent advances in data generation and its applications in computer vision}},
  journal = {Journal of Image and Graphics},
  year = {2025},
  doi = {10.11834/jig.250085},
}
Figure