Design and Implementation of a Novel Compatible Encoding Scheme in the Time Domain for Image Sensor Communication

Trang Nguyen, Mohammad Arif Hossain, Yeong Min Jang

TL;DR

This paper introduces a novel time-domain encoding scheme for image sensor communication (ISC) that addresses the challenge of varying frame rates in rolling shutter cameras. By employing asynchronous bits and innovative data fusion techniques, the proposed system enables robust and compatible communication, paving the way for more reliable optical wireless communication (OWC) applications.

What Problem Does This Paper Address?

Optical Wireless Communication (OWC) is gaining traction as a green and cost-effective alternative to traditional wireless technologies, especially for the burgeoning Internet of Things (IoT). A key area within OWC is Image Sensor Communication (ISC), which utilizes image sensors as receivers. However, a significant hurdle for ISC, particularly with rolling shutter cameras (which are prevalent due to their lower cost and wider availability compared to global shutter cameras), is the inherent variability in their frame rates, shutter speeds, and sampling rates. This variability makes it difficult to establish a stable and compatible communication link, as the receiver's ability to capture data can fluctuate unpredictably. Existing solutions often struggle to accommodate these variations, leading to data loss or requiring complex synchronization mechanisms. This paper aims to bridge this gap by proposing a communication scheme that is inherently compatible with the diverse characteristics of rolling shutter image sensors.

Methods

The core of this paper lies in its novel encoding scheme designed for the time domain. The authors propose two main data frame structures and associated decoding strategies:

  • Oversampling Scheme: This scheme is designed for situations where the camera's frame rate is significantly higher than the transmitter's packet rate. It utilizes a data frame structure with a preamble, asynchronous bits (Ab), and a payload. The asynchronous bits play a crucial role in identifying data parts and enabling data fusion. A key feature is the data fusion algorithm, which merges incomplete data parts from multiple images (inter-frame fusion) or within a single image (intra-frame fusion) to reconstruct a complete data packet. This is particularly useful when the camera captures only portions of a data packet in a single frame. A majority voting scheme is also employed to correct errors that may arise during this process.
  • Undersampling Scheme with Error Detection: This scheme addresses the more challenging scenario where the camera's frame rate can drop below the transmitter's packet rate. It introduces a second data frame structure that incorporates two asynchronous bits (Ab1 and Ab2). These additional bits enable more robust detection of missed payloads. By analyzing the states of these two bits, the system can identify when entire packets are missed due to frame rate drops, allowing for error detection and subsequent correction.

Both schemes leverage On-Off-Keying (OOK) as the modulation method and Run Length Limited (RLL) coding for data transmission, aiming to maintain consistent LED brightness and minimize flickering.

Datasets

Not discussed in the paper. The authors conducted experiments using various cameras and modulated light sources, but no specific datasets were mentioned for training or evaluation.

Key Insights and Ideas

  • Asynchronous Bits as a Universal Compatibility Tool: The introduction of asynchronous bits (Ab) is a central innovation. These bits act as crucial timing and synchronization markers, allowing the receiver to identify data boundaries and facilitate the merging of incomplete data segments, thereby overcoming frame rate variations.
  • Data Fusion for Enhanced Range and Reliability: The data fusion algorithm is a clever approach to reconstruct complete data packets from partial captures. This not only improves the reliability of communication but also effectively extends the communication range, as partial data can be recovered from further distances.
  • Two-Stage Approach for Frame Rate Variation: The paper effectively tackles frame rate variation by proposing two distinct schemes: oversampling (for when the camera is faster) and undersampling with error detection (for when the camera is slower). This dual approach ensures compatibility across a wider range of camera performance.
  • Error Detection for Undersampling is Crucial: The realization that error detection is paramount when the camera's frame rate falls below the transmission rate is a key insight. The proposed two-asynchronous-bit scheme provides a practical mechanism for this.

Key Results

  • The proposed schemes demonstrate compatibility with rolling shutter cameras exhibiting frame rate variations from 20 fps to 35 fps.
  • The oversampling scheme, with data fusion and majority voting, achieved error-free communication under these conditions.
  • The undersampling scheme with error detection can detect up to three missed adjacent payloads when the frame rate is at least a quarter of the packet rate.
  • Experimental results showed achievable data rates of up to 1.9 kbps with Manchester coding and 1.5 kbps with 4B6B coding at specific modulation frequencies, demonstrating the potential for high-speed ISC.
  • Data fusion was shown to potentially double the communication distance or data throughput.

Limitations and Future Work

  • Limited Data Rate: While the paper achieves several kbps, this is still significantly lower than the data rates achievable with traditional Wi-Fi or even some VLC systems. The authors acknowledge that increasing the data rate is a challenging obstacle for ISC.
  • Error Correction Complexity: While the undersampling scheme focuses on error detection, the paper mentions that error correction is beyond the scope of discussion. Implementing robust error correction for detected errors would be a logical next step.
  • Physical Limitations: The modulation frequency is limited by the camera's shutter speed, which restricts the achievable bandwidth.
  • Future Work: The authors suggest exploring more advanced data fusion techniques (e.g., merging data from more than three images), incorporating zoom modules in receivers for extended range, and using larger LEDs on the transmitter side to boost transmission distance without sacrificing data rate. They also mention the possibility of generating inner codes within sub-packets for improved image merging.

Summary updated

Abstract

This paper presents a modulation scheme in the time domain based on On-Off-Keying and proposes various compatible supports for different types of image sensors. The content of this article is a sub-proposal to the IEEE 802.15.7r1 Task Group (TG7r1) aimed at Optical Wireless Communication (OWC) using an image sensor as the receiver. The compatibility support is indispensable for Image Sensor Communications (ISC) because the rolling shutter image sensors currently available have different frame rates, shutter speeds, sampling rates, and resolutions. However, focusing on unidirectional communications (i.e., data broadcasting, beacons), an asynchronous communication prototype is also discussed in the paper. Due to the physical limitations associated with typical image sensors (including low and varying frame rates, long exposures, and low shutter speeds), the link speed performance is critically considered. Based on the practical measurement of camera response to modulated light, an operating frequency range is suggested along with the similar system architecture, decoding procedure, and algorithms. A significant feature of our novel data frame structure is that it can support both typical frame rate cameras (in the oversampling mode) as well as very low frame rate cameras (in the error detection mode for a camera whose frame rate is lower than the transmission packet rate). A high frame rate camera, i.e., no less than 20 fps, is supported in an oversampling mode in which a majority voting scheme for decoding data is applied. A low frame rate camera, i.e., when the frame rate drops to less than 20 fps at some certain time, is supported by an error detection mode in which any missing data sub-packet is detected in decoding and later corrected by external code. Numerical results and valuable analysis are also included to indicate the capability of the proposed schemes.
BibTeX
@inproceedings{nguyen2016design,
  author = {Trang Nguyen and Mohammad Arif Hossain and Yeong Min Jang},
  title = {{Design and Implementation of a Novel Compatible Encoding Scheme in the Time Domain for Image Sensor Communication}},
  booktitle = {Sensors},
  year = {2016},
  url = {https://openreview.net/forum?id=iNaH2eS5ET},
}
Figure