This research addresses challenges in visible light communication (VLC) systems that use two orthogonally aligned rolling shutters (RS) image sensors as receivers and an LED array as a transmitter. The study focuses on overcoming burst signal loss caused by unsensed periods between frames in RS image sensors. To improve VLC performance, we propose and compare three different schemes. The first is a conventional approach that superimposes a Barker code synchronization signal on the transmission signal using pulse width modulation (PWM). The second introduces a new method of spatial synchronization by dedicating specific LEDs in an array for synchronization signals. The third scheme extends this spatial approach by incorporating a 4 -level PWM for information transmission to increase data rates. The study aims to evaluate and compare the error rate characteristics of these three schemes, assessing their effectiveness in mitigating burst errors and enhancing overall VLC system performance. This research has potential applications, including vehicle communication, Internet of Things devices, and smartphones.

To improve the operational efficiency of remotely operated vehicles (ROVs) used to investigate ocean resources, this paper focuses on underwater visible light communication using a camera as a receiver (i.e., underwater image sensor communication: UISC). UISC can realize wireless communication underwater, where wireless communication using radio waves is difficult. However, due to the limited area of the ROV, it is hard to increase the number of transmitter LEDs installed on the ROV, resulting in a lower communication speed. We have developed a propeller-type rotary LED transmitter (P-RTx) to solve this problem, assuming that the LEDs are attached to the ROV's thruster. P-RTx is a device that increases communication speed by rotating a blinking LED that transmits data and utilizing the receiver's ability to capture their blinking lights as multiple afterimages. As a result of experiments using a prototype P-RTx, we have achieved error-free communications up to a communication distance of 3.0 m at a communication speed of 3.24 kbps, 72 times faster than the conventional method.

Visible light communication based intelligent transportation systems (ITS-VLC) show great potential for future urban mobility. This study presents a performance evaluation of range estimation between vehicles and infrastructures in an ITS-VLC system. In the proposed ITS-VLC system, it is easy to simultaneously conduct communication and ranging using stereo cameras. However, the stereo camera calibration becomes a problem during simultaneous communication and ranging due to vehicle vibration. Using the data from LED transmitters and stereo cameras, it can obtain multiple measurements of distance. The monocular-stereo fusion algorithm is applied to visible light ranging in the proposed scheme using particle swarm optimization. We employed real data from the field trial experiment and achieved a ranging accuracy of 60 ± 1.0 m.

This paper presents an image sensor communication (ISC) using a digital micromirror device (DMD) projector as a transmitter. In particular, we focus on ISC for intelligent transport systems (ITSs) because DMD projectors are expected to be used in road traffic, such as vehicle headlights, street lights, and traffic signs. The DMD projector controls light patterns by switching tilt of micromirrors at high speed. Compared to the conventional LED array transmitter, DMD projector can design and alter the shape of the light pattern and data rate of transmission more easily. In the proposed system, the data rate can be easily increased by increasing the number of multiplexes. However, as the number of multiplexes is increased, the number of received pixels on the image sensor is decreased, and thus the performance of symbol detection deteriorates. Therefore, the purpose of this paper is to examine the relation between the number of received pixels per cell and communication performance for transmission pattern design. Hence, we experimentally clarify this relation using our prototype system via a DMD projector and a high-speed camera.

Visible light communication (VLC) is a wireless communication technology that employs visible light as a communication medium. Image-sensor-based VLC is suitable for multi-channel communication because multiple transmitters can be captured concurrently. In this paper, we propose a communication method for image-sensor-based VLC using multiple frames, which are intended to aid the simultaneous measurement of a multi-channel biological signal. The proposed method estimates the phase difference between the transmitter and the receiver using multiple frames shot at different timings compared to LED blinking, and communication is established through phase shift keying. Moreover, the proposed method is resistant to phase differences between the transmitter and the receiver, which is unavoidable in asynchronous communication. Through experiments, we confirmed that communication is possible using our proposed method.

This paper focuses on high-speed stereo camera receivers for visible light communication (VLC) in intelligent transport systems (ITSs), contrary to conventional studies where a single (high-speed) camera is used. An advantage of high-speed stereo cameras as VLC receivers is that both front-facing ranging and wireless communication can be achieved in one device. In this paper, utilizing features of VLC, we consider stereo ranging with high accuracy. Moreover, it is known that an LED transmitter for VLC has specific features, high-luminance and high- speed blinking. This paper proposes stereo ranging by applying LED transmitter detection (utilizing these features) to disparity estimation. Additionally, we introduce subpixel estimation using equiangular line fitting to obtain more accurate disparity. Finally, we experimentally demonstrate that the proposed stereo ranging achieves an accuracy of 60 m ± 0.1 m.

In-vehicle cameras can be used as receivers for visible light communication in addition to their primary functions: for viewing assistance and object recognition. Most studies with this design perform tests via a single high-speed camera that is used as the receiver, but this study employs a receiver that uses a high- speed stereo camera. The reception method developed in this paper effectively utilizes left and right stereo images to improve communication performance. Although the selective diversity reception method proposed in previous work improved communication performance, it cannot deal with cases wherein both the left and right cameras introduce errors. To further improve the communication performance of this system, maximal ratio combining (MRC) diversity is applied to the high-speed stereo camera in this study. Communication performance is then evaluated comparing to the selective diversity reception and conventional single-camera reception by experimental tests conducted under static and driving conditions.

Channel modeling is critical for the design and performance evaluation of visible light communication (VLC). Although a considerable amount of research has focused on indoor VLC systems using single-element photodiodes, there remains a need for channel modeling of VLC systems for outdoor mobile environments. In this paper, we describe and provide results for modeling image sensor based VLC for automotive applications. In particular, we examine the channel model for mobile movements in the image plane as well as channel decay according to the distance between the transmitter and the receiver. Optical flow measurements were conducted for three VLC situations for automotive use: infrastructure to vehicle VLC (I2V-VLC); vehicle to infrastructure VLC (V2I-VLC); and vehicle to vehicle VLC (V2V-VLC). We describe vehicle motion by optical flow with subpixel accuracy using phase-only correlation (POC) analysis and show that a single-pinhole camera model successfully describes these three VLC cases. In addition, the luminance of the central pixel from the projected LED area versus the distance between the LED and the camera was measured. Our key findings are twofold. First, a single-pinhole camera model can be applied to vehicle motion modeling of a I2V-VLC, V2I-VLC, and V2V-VLC. Second, the DC gain at a pixel remains constant as long as the projected image of the transmitter LED occupies several pixels. In other words, if we choose a pixel with highest luminance among the projected image of transmitter LED, the value remains constant, and the signal-to-noise ratio does not change according to the distance.

The present paper proposes a novel multiple-LED-arrays acquisition for an infrastructure-to-vehicle visible light communication (I2V-VLC) using LED arrays (transmitter) and an in-vehicle high-speed image sensor (receiver). In order to achieve a robust detection of LED arrays, we employ the block matching algorithm, which is a way of finding a corresponding position between two successive frames. The proposed method divides a captured image into a number of small domains (blocks) and determines if the LED array is present or absent using the block matching. We perform I2V-VLC experiments with multiple-LED arrays and evaluate the acquisition capability of the proposed method.

Free-viewpoint TV (FTV) is cutting the frontier of audiovisual communications. FTV is an innovative media that enables us to view 3-D space by freely changing our viewpoints. It also allows us to listen at any listening point in the 3-D space. Since FTV transmits all audiovisual information of the 3-D space, it can reconstruct an audiovisual replica of the 3-D space anywhere and anytime over distance and time. For video, FTV captures a part of rays in 3-D space by using many cameras, and the other rays that are not captured are obtained by interpolating the captured rays. We constructed real-time FTV systems including the complete chain of operation from image capture to display. We also carried out FTV on a laptop computer and a mobile player. For audio, two kinds of free listening-point systems are demonstrated. MPEG regarded FTV as the most challenging 3-D media and has been conducting its international standardization activities. The first phase of FTV was multiview video coding (MVC) and the second phase of FTV is 3-D video (3DV). MVC enables the efficient coding of multiple camera views and was completed in 2009. MVC has been adopted by Blu-ray 3-D. 3DV is a standard that targets serving a variety of 3-D displays and its call for proposals was issued in March 2011.

In this paper, we focus on a road-to-vehicle visible light communication (VLC) system using LED traffic lights. In this system, an LED traffic light consists of a 2-dimensional LED array (2D LED array), and cars are equipped with high-speed 2-dimensional cameras (2D image sensors). An important issue of this system is frame loss. Sometimes, 2D image sensor in a car fails to get a frame. So as to mitigate the influence of frame loss, we propose to apply erasure coding to the road-to-vehicle VLC system. In the proposed system, a data sequence is encoded by LDPC code whose length is much longer than the size of 2D LED array. In addition, code synchronization is required for the proposed system. We also propose a code synchronization scheme, which makes use of error detection capability of LDPC code. We evaluate the performance of our proposed system, and show that it can recover frame loss for high SNR when less than 8 frames among total 18 frames are lost. In addition, we show that our proposed code synchronization scheme can ignore its errors.

We have developed a 3DTV system that captures and shows 3D images that covers 360-degree of viewing zone horizontally. 3D images are captured and reproduced as gathering of dense light rays. This system consists of a cylinder-shaped 3D display that allows viewers to see 3D images from 360-degree, a ray capturing unit, a realtime image correction unit, and data transferring system. The capturing unit acquires multiview images from all horizontal directions around an object with narrow view interval, which the display needs as light ray data. The image correction unit can corrects rotation and distortion that caused by optics of ray capturing unit in real time.

Traffic sign recognition systems can be applied to assist drivers and improve the traffic safety. High resolution image of traffic sign can improve the recognition result, especially for some complex traffic signs. In this paper, a dual-focal active camera system is proposed to obtain a high resolution image of traffic sign. In the system an active telephoto camera is equipped as an assistant of a wide angle camera. To make the proposed system correctly capture a high resolution image of traffic sign, the shape, color features and the relationship between continuous frames are used together in the traffic sign detection. The experiment results demonstrate that the proposed method makes the dual-focal active camera system effectively work when the system is installed on an automobile and moves on the road.

View synthesis using depth maps is an important application in 3D image processing. In this paper, a novel method is proposed for the plausible view synthesis of Free-viewpoint TV (FTV), using two input images and their depth maps. The depth estimation based on stereo matching is known to be error-prone, leading to noticeable artifacts in the synthesized new views. To produce high-quality view synthesis, we introduce a probabilistic framework which constrains the reliability of each pixel of new view by Maximizing Likelihood (ML). The spatial adaptive reliability is provided by incorporating Gamma hyper-prior and the synthesis error approximation. Furthermore, we generate the virtual view by solving a Maximum a Posterior (MAP) problem using graph cuts. We compare the proposed method with other depth based view synthesis approaches on MPEG test sequences. The results show the outperformance of our method both at subjective artifacts reduction and objective PSNR improvement.

In this paper, we introduce a new vision system for traffic sign recognition. The new vision system is a hybrid camera system, including a wide angle camera, a narrow angle camera and control part. The traffic sign candidates are detected in the image of the wide angle camera, based on shape and local feature. The control part of the vision system is adjusted to the correct positions, which makes the narrow angle camera of this system to get a high resolution image of each detected candidate. This high resolution image is used for traffic sign classification. In the new vision system, the traffic sign detection and classification are processed in different resolution images separately. This advantage of the system makes traffic sign recognition at long distance possible.

As one of ITS technique, a new visible light road-to-vehicle communication system at intersections is proposed. In this system, the communication between a vehicle and an LED traffic light is conducted using an LED traffic light as a transmitter, and an on-vehicle high-speed camera as a receiver. The LEDs in the transmitter emit light in high frequency and those emitting LEDs are captured by the high-speed camera for making communication. Here, the luminance value of LEDs in the transmitter should be captured in consecutive frames to achieve effective communication. For this purpose, first the transmitter should be found, then it should be tracked in consecutive frames by processing the images from the high-speed camera. In this paper, we propose new effective algorithms for finding and tracking the transmitter, which result in a increased communication speed, compared to the previous methods. Experiments using appropriate images showed the effectiveness of the proposals.

The availability of multi-view images of a scene makes new and exciting applications possible, including Free-Viewpoint TV (FTV). FTV allows us to change viewpoint freely in a 3D world, where the virtual viewpoint images are synthesized by Image-Based Rendering (IBR). In this paper, we introduce a FTV depth estimation method for forward virtual viewpoints. Moreover, we introduce a view generation method by using a zoom camera in our camera setup to improve virtual viewpoint-ts' image quality. Simulation results confirm reduced error during depth estimation using our proposed method in comparison with conventional stereo matching scheme. We have demonstrated the improvement in image resolution of virtually moved forward camera using a zoom camera setup.

We propose a novel 3D space representation for multi-view video, using epipolar plane depth images (EPDI). Multi-view video plus depth (MVD) is used as common data format for FTV(Free-viewpoint TV), which enables synthesizing virtual view images. Due to large amount of data and complexity of the multi-view video coding (MVC), compression of MVD is a challenging issue. We address this problem and propose a new representation that is constructed from MVD using rayspace. MVD is converted into image and depth ray-spaces. The proposed representation is obtained by converting each of ray-spaces into a global depth map and a texture map using EPDI. Experiments demonstrate the analysis of this representation, and its efficiency.

In this paper, we propose a parallel processing method to generate free viewpoint image in realtime. It is impossible to arrange the cameras in a high density realistically though it is necessary to capture images of the scene from innumerable cameras to express the free viewpoint image. Therefore, it is necessary to interpolate the image of arbitrary viewpoint from limited captured images. However, this process has the relation of the trade-off between the image quality and the computing time. In proposed method, it aimed to generate the high-quality free viewpoint image in realtime by applying the parallel processing to time-consuming interpolation part.

3D scene information can be represented in several ways. In applications based on a (N-)view plus (N-)depth representation, both view and depth data is compressed. In this paper we present a depth compression method containing an depth up-sample filter which uses the color view as prior. Our method of depth down-/up-sampling is able to maintain clear object boundaries in the reconstructed depth maps. Our experimental results show that the proposed depth re-sampling filter, used in combination with a standard state-of-the art video encoder, can increase both the coding efficiency and rendering quality.

View synthesis using depth maps is a well-known technique for exploiting the redundancy between multi-view videos. In this paper, we deal with the bitrates of view synthesis at the decoder side of FTV that would use compressed depth maps and views. Both inherent depth estimation error and coding distortion would degrade synthesis quality. The focus is to reduce bitrates required for generating the high-quality virtual view. We employ a reliable view synthesis method which is compared with standard MPEG view synthesis software. The experimental results show that the bitrates required for synthesizing high-quality virtual view could be reduced by utilizing our enhanced view synthesis technique to improve the PSNR at medium bitrates.

In this paper, we propose a semi-automatic depth estimation algorithm whereby the user defines object depth boundaries and disparity initialization. Automatic depth estimation methods generally have difficulty to obtain good depth results around object edges and in areas with low texture. The goal of our method is to improve the depth in these areas and reduce view synthesis artifacts in Depth Image Based Rendering. Good view synthesis quality is very important in applications such as 3DTV and Free-viewpoint Television (FTV). In our proposed method, initial disparity values for smooth areas can be input through a so-called manual disparity map, and depth boundaries are defined by a manually created edge map which can be supplied for one or multiple frames. For evaluation we used MPEG multi-view videos and we demonstrate our algorithm can significantly improve the depth maps and reduce view synthesis artifacts.

In this paper, we consider visible light communication systems using LED array as a transmitter and high-speed camera as a receiver for Intelligent Transport System (ITS). Previously, we have proposed the hierarchical coding scheme which allocates data to spatial frequency components of the image depending on the priority. This scheme is possible to receive information of the high-priority even if communication distance is long. However, we need to distinguish multi-valued data from the received image by using a hierarchical coding. In this paper, we propose two improved decoding methods, and demonstrate to distinguish multi-valued data more correctly in the experiment.

Neighboring views must be highly correlated in multiview video systems. We should therefore use various neighboring views to efficiently compress videos. There are many approaches to doing this. However, most of these treat pictures of other views in the same way as they treat pictures of the current view, i.e., pictures of other views are used as reference pictures (inter-view prediction). We introduce two approaches to improving compression efficiency in this paper. The first is by synthesizing pictures at a given time and a given position by using view interpolation and using them as reference pictures (view-interpolation prediction). In other words, we tried to compensate for geometry to obtain precise predictions. The second approach is to correct the luminance and chrominance of other views by using lookup tables to compensate for photoelectric variations in individual cameras. We implemented these ideas in H.264/AVC with inter-view prediction and confirmed that they worked well. The experimental results revealed that these ideas can reduce the number of generated bits by approximately 15% without loss of PSNR.

We propose a new multi-view video coding method using adaptive selection of motion/disparity compensation based on H.264/AVC. One of the key points of the proposed method is the use of view interpolation as a tool for disparity compensation by assigning reference picture indices to interpolated images. Experimental results show that significant gains can be obtained compared to the conventional approach that was often used