Traditionally, computer graphics and computer vision were treated as inverse problems. Graphics focused on synthesis—taking a mathematical model and rendering it into pixels—while vision focused on analysis—extracting models or meaning from pixels. However, as visual data has exploded in complexity and volume, these silos have become a hindrance.
Researchers like Shi-Min Hu and Chang Wen Chen argue for a holistic approach where synthesis and analysis are interdependent. This convergence, often labeled "Computational Visual Media," treats images, videos, and 3D geometry not as static files, but as a dynamic medium requiring a unified computational framework.
The Synthesis-Analysis Loop
The core intuition behind this research direction is that you cannot effectively generate what you do not understand, and you cannot fully understand what you cannot reconstruct.
In Preface (J. Comput. Sci. Technol. 2023), Amit H. Bermano and Shi-Min Hu highlight that structural analysis of a scene’s geometry is now a prerequisite for high-quality generative synthesis. For example, to synthesize realistic cloth wrinkles, a model must first analyze the underlying physical constraints and geometric deformations of the fabric.
By integrating these processes, researchers create a feedback loop:
- Analysis provides the structural "scaffolding" (geometry, semantics, lighting).
- Synthesis uses this scaffolding to generate or manipulate media.
- The results of synthesis are fed back into the analysis engine to refine the structural understanding.
Machine Learning as the Common Language
The catalyst for this unification is machine learning. In the past, graphics relied on physics-based simulation while vision relied on hand-crafted geometric features. Today, deep learning serves as the shared engine for both.
As noted in PREFACE: TECHNICAL COMMISSION II (ISPRS 2022) by Jan Dirk Wegner and Fabio Remondino, the distinction between geometric 3D reconstruction and radiometric pattern analysis is blurring because machine learning handles both simultaneously. This allows for "deep geometry learning," where a system learns to represent 3D shapes in a way that is optimized for both recognition (vision) and rendering (graphics).
Bridging Low-Level Signals and High-Level Semantics
A unified framework allows researchers to bridge the gap between "low-level" tasks (like denoising or smoothing) and "high-level" tasks (like sketch-based modeling or autonomous driving).
In Message From the Editor-in-Chief (IEEE Trans. Multim. 2015), the authors emphasize that a holistic approach connects signal processing with semantic understanding. If a system understands that a group of pixels represents a "car" (high-level semantics), it can perform much more effective denoising or inpainting (low-level processing) by leveraging its knowledge of what cars look like.
This interdisciplinary synergy is not just theoretical. Shi-Min Hu and Piotr Didyk point out in Preface (J. Comput. Sci. Technol. 2025) that this integration is essential for processing complex 2D and 3D data in real-world applications, such as robotic perception and augmented reality.
Organizational Convergence
The shift toward unification is also reflected in how the research community organizes itself. The IEEE Visualization and Graphics Technical Committee (VGTC), as described by Klaus Mueller and Arie Kaufman, has expanded its mandate to cover visual analytics, VR, and HCI under one umbrella. This organizational strategy, echoed in VAST 2018 IEEE Visualization and Graphics Technical Committee (2018), ensures that specialized areas like temporal data visualization benefit from advancements in deep learning and geometric computing.
By dissolving the boundaries between synthesis and analysis, the field is moving toward a future where visual media is handled as a single, computationally fluid entity.