Abstract
This paper presents a novel client-driven 3DTV system capable of capturing, transmitting, and rendering dynamic 3D scenes in real time over standard IP networks. By integrating a "lengthened-B-field" coding scheme with a "Clarity Degree" rendering metric, the system enables seamless free-viewpoint navigation with bandwidth requirements under 1.2 Mbps per user. The architecture effectively addresses the challenges of massive data volume, view-switching latency, and rendering artifacts common in multi-view video systems.
Introduction
While high-definition 2D streaming is ubiquitous, interactive 3DTV—allowing viewers to change perspectives freely—remains technically challenging. Current systems struggle with three primary hurdles: the "data deluge" from dozens of cameras, the switching lag inherent in standard predictive video codecs, and visual artifacts like ghosting during view synthesis. This research proposes an end-to-end architecture that optimizes the entire pipeline based on the client's active viewpoint, significantly reducing both computational and network overhead.
System Architecture and Methods
The system utilizes an 8 × 8 camera array (64 cameras total) to capture the scene from multiple angles. These cameras are managed by 16 producer PCs that handle initial processing. The core of the system is a streaming server cluster that mediates between the producers and various client displays, including 2D, autostereoscopic, and multi-projector 3D displays.
Novel Coding and Transmission
To eliminate the lag typically associated with switching between camera views, the authors introduced a lengthened-B-field coding structure. Traditional codecs require waiting for a new "I-frame" to switch views. In this proposed setup, "I-fields" (spatial prediction across cameras) and "P-fields" (temporal refresh) provide the backbone, while optional "B-fields" (bi-predictive) allow the server to switch views instantaneously without waiting for a specific sequence completion.
Furthermore, the system employs Region of Interest (ROI) streaming. Instead of broadcasting full frames from all 64 cameras, the system identifies the specific pixel blocks required to render the user's current virtual perspective and transmits only that data, drastically lowering bandwidth consumption.
All-Focus Rendering via Clarity Degree
To avoid the computational cost of full 3D reconstruction, the system uses a plane-sweeping algorithm combined with a new metric called Clarity Degree (CD). Based on Mean Change Energy (MCE), the CD metric identifies the sharpest regions across different focal planes. By stitching these clear regions together, the system creates an all-focus 3D image that bypasses the need for perfect depth maps, effectively eliminating "ghosting" effects.
Experimental Results
The system was evaluated using synthetic scenes (teapot, grid) and real-world sequences captured by the 64-camera array. Key performance metrics include:
- Real-Time Performance: Achieved a rendering rate of 22 fps on standard commodity hardware.
- Bandwidth Efficiency: Transmission bitrates were maintained below 1.2 Mbps per user, suitable for broadband IP networks.
- Computational Savings: The proposed rendering scheme reduced computation time by approximately 32% compared to traditional all-focus methods by utilizing motion vectors to approximate depth changes.
Conclusion
The client-driven approach successfully demonstrates that high-quality, interactive 3DTV is viable over existing network infrastructures. By prioritizing data transmission based on the viewer's perspective and using sharpness as a proxy for depth, the system achieves a balance between visual quality and real-time performance. Future work will focus on adapting the system for non-grid, unstructured camera arrays and improving performance for moving camera setups.
Summary updated