TL;DR
This paper introduces a novel system for creating "multiprojection images" and animations, where different parts of a 3D scene are rendered with independent camera perspectives. Their key contribution is an algorithm that automatically resolves visibility issues between these different views, enabling artists to achieve complex visual effects that are difficult or impossible with traditional single-projection rendering.
What Problem Does This Paper Address?
Computer graphics typically relies on a single, unified projection (like perspective) to render 3D scenes. While this creates realistic views, it can be restrictive for artistic expression. Artists have long used multiprojection techniques in traditional media to convey mood, improve scene comprehension, or visualize spatial relationships by showing different objects from their "best" viewpoints within a single image. However, creating such multiprojection images in computer graphics has been a labor-intensive process, often involving manual compositing and complex visibility resolution. This paper aims to bridge this gap by providing an automated, interactive system for generating multiprojection images and animations.
Methods
The core of the system is a multiprojection rendering algorithm with three stages:
- Rendering Image Layers: Each object or group of objects is rendered with its own "local camera." The user defines these camera groups.
- Visibility Resolution: This is the most challenging part. The paper proposes an algorithm that uses a "master camera" to establish a primary visibility ordering. It combines two techniques:
- 3D Depth-Based Compositing: Uses depth information from each rendered layer.
- Object-Level Occlusion Constraints: Automatically detects and applies constraints like "object A occludes object B" based on the master camera's view. Users can also manually specify these constraints.
- Compositing: The rendered image layers are merged into a final multiprojection image, using the resolved visibility information.
To make the system interactive and user-friendly, the authors also introduce several novel camera constraints. These constraints allow users to intuitively control camera parameters to achieve specific artistic effects, such as maintaining an object's perceived size while changing its perspective convergence (object-size constraint) or keeping an object's view fixed in the image plane as it moves in 3D (fixed-view constraint).
Datasets
Not discussed in the paper. The paper focuses on a novel rendering technique and its application to existing artistic examples and custom scenes.
Key Insights and Ideas
- Master Camera for Visibility: The insight that a single "master camera" can provide a consistent reference for resolving visibility across multiple, potentially conflicting, local camera views is crucial. This avoids the need for complex pairwise comparisons between all local cameras.
- Object-Level Occlusion Constraints: Automatically inferring and applying object-level occlusion relationships (e.g., "the vase occludes the table") is more robust than purely pixel-wise depth comparisons, especially when objects might appear to overlap in local views but not in the master view.
- Intuitive Camera Constraints: Developing constraints that map directly to artistic intentions (like maintaining object size or view) significantly simplifies the creation of complex multiprojection effects, moving beyond direct manipulation of abstract camera parameters.
- Interactive Rendering for Art: The emphasis on interactive rendering rates, achieved through hardware acceleration, is key to making multiprojection rendering a practical tool for artists, allowing immediate visual feedback.
Key Results
The paper demonstrates the system's effectiveness through several examples:
- Artistic Reconstructions: They successfully reconstructed famous multiprojection artworks like Giorgio de Chirico's "The Mystery and Melancholy of a Street" and Cézanne's "Still Life with Fruit Basket."
- Artificial Perspective: They generated images with an artificial sense of perspective using multiple oblique projections, which they argue offers advantages in visualizing object sizes and scene visibility compared to true perspective.
- Animations: The system was used to create animations, showcasing how the fixed-view constraint can lead to "cartoony" effects and improve scene composition, particularly in animated sequences.
- Performance: The system achieves interactive rendering rates (e.g., 3-5 frames per second for a still life with 10 camera groups at 333x256 resolution, and 10-12 fps for the de Chirico scene at 402x491) by leveraging hardware rendering.
Limitations and Future Work
- Animation Complexity: While animations are possible, achieving pleasing results can be challenging due to potential conflicts between motion parallax and perspective cues. The authors note that animating objects within a fixed background environment generally works best.
- Lighting and Shadows: Handling lighting effects like shadows and reflections in a multiprojection setting is an open problem. The current system assumes lights are global and each camera group is lit independently, which can lead to issues when casting shadows between different camera groups.
- Constraint Flexibility: The current camera constraints are designed for specific effects. The authors suggest incorporating a more flexible, constraint-based interface for cameras, similar to existing work, to allow for a wider range of animation and camera control.
- General Projections: While the visibility algorithms are designed to work with invertible projections, the current interactive rendering implementation is restricted to linear planar projections due to hardware limitations.
Summary updated