TL;DR
This paper introduces an affordable system for synchronously capturing every frame from multiple video streams, a task previously limited by high costs and bandwidth constraints. By leveraging conventional VCRs, Vertical Interval Time Code (VITC), and off-line digitization, the system enables applications requiring precise temporal correlation of visual data across many cameras.
What Problem Does This Paper Address?
Capturing high-resolution video streams from multiple cameras simultaneously and synchronously is crucial for applications like psychological studies, surveillance, and advanced computer vision research (e.g., 3D reconstruction, motion tracking). However, the sheer bandwidth required for uncompressed video (around 26 MB/s per color stream) far exceeds the capabilities of most affordable storage and processing systems. Existing high-end solutions are prohibitively expensive, often costing tens of thousands of dollars per channel. Furthermore, achieving frame-accurate synchronization across multiple streams is challenging, especially when dealing with consumer-grade equipment that may not be frame-accurate in pause or freeze modes.
Methods
The core of the system relies on a clever combination of off-line recording and precise time-code interpretation.
- Synchronization and Recording:
- Multiple cameras are synchronized using a common sync signal.
- A Vertical Interval Time Code (VITC) is embedded into each video stream. VITC is a time code standard that is stored within the vertical blanking interval of the video signal, making it part of the image data itself.
- The synchronized video streams with VITC are recorded in real-time onto standard S-VHS tapes using conventional VCRs.
- Off-line Digitization:
- Tapes are played back on an editing VCR, controlled by a computer.
- A commercial frame grabber digitizes the video. Crucially, the digitizer is configured to capture the vertical blanking period, which contains the VITC data.
- The system interprets the VITC data (represented by black and white streaks) on-the-fly to identify specific frames.
- When a desired frame is identified, the VCR is instructed to freeze, and the frame is captured. This process is repeated, and tapes are rewound and replayed as needed to capture all requested frames.
- A computer-controlled VCR automates the process, but manual control is also an option.
- Process Verification:
- To ensure camera synchronization independent of VITC, cameras are pointed at a custom-built counter synchronized to the master sync signal.
- The system verifies that each video field records a single state of the counter.
- It also compares the counter's state across different camera streams and checks for a constant offset between the VITC and the counter display.
- Special attention is paid to handling "Frame mode" camera accumulation, which can reduce effective vertical resolution, by using multiple LEDs on the counter display for accurate field identification.
Datasets
Not discussed in the paper. The system is designed for capturing new data from real-world video streams rather than evaluating on pre-existing datasets.
Key Insights and Ideas
- VITC as a Game Changer: The paper highlights the power of VITC for frame-accurate identification and synchronization. By embedding the time code directly into the video signal and capturing it as part of the image, the system bypasses the limitations of VCRs' frame accuracy and allows for precise off-line retrieval.
- Affordability through Off-the-Shelf Components: The system's major contribution is its low cost ($500 per channel for recording, $5000 for the digitizing setup) by leveraging standard VCRs and digitizers, making multi-camera synchronous capture accessible to a much wider range of researchers.
- Scalability: The design is inherently scalable, allowing for an arbitrary number of cameras and long recording durations limited only by tape length.
- Overcoming VCR Limitations: The paper directly addresses the inaccuracies of consumer VCRs (frame-dropping, reduced resolution in pause mode) by relying on VITC interpretation during normal playback speed.
Key Results
- The system can synchronously capture every frame (or a user-specified subset) from multiple cameras at full resolution.
- It is fully scalable in the number of streams and duration of capture.
- The recording setup costs approximately $500 per channel.
- The digitizing setup costs approximately $5000 for the VCR and digitizer.
- The system has been successfully used to enable research projects like Virtualized Reality, which require dense sampling of synchronized video streams.
Limitations and Future Work
- Reduced Visual Quality: The paper acknowledges that storing video data on conventional tapes before digitizing can reduce visual quality compared to direct digital capture.
- Time-Consuming Digitization: Digitizing multiple streams independently can be time-consuming, especially for a large number of cameras.
- Future Work: The paper implies that further research could explore more efficient digitization methods or compression techniques that preserve temporal accuracy. The system's success in enabling complex research projects like Virtualized Reality suggests its continued relevance and potential for further development in related areas.
Summary updated