Recent autostereoscopic 3-D (A3D) displays suffer from many limitations such as narrow viewing angle, low resolution, and shallow depth effects. As these limitations mainly originate from the insufficiency of pixel resources, it is not easy to obtain a feasible solution that can solve all the limitations simultaneously. In many cases, it will be better to find a good compromising design. Generally, the multiview display and the integral imaging display are the representative designs of A3D. However, as they are too canonical and lack flexibility in design, they tend to be a tradeoff. To address these design issues, we have analyzed the multiview display and the integral image display in a light-field coordinate and developed a 3-D display design framework in a light-field space. The developed framework does not use the “view” concept anymore. Instead, it considers the spatial distribution of rays of the 3-D display and provides more flexible and sophisticated design methods. In this paper, the developed design method is explained using a new pixel value assigning algorithm, called the light-field rendering, and vision-based parameter calibration methods for 3-D displays. We have also analyzed the blur effects caused by the depth and display characteristics. By implementing the proposed method, we have designed a 65-in 96-view display with a 4K panel. The developed prototype has showed almost seamless parallax with a high-resolution comparable to the conventional four to five views displays. This paper will be useful to readers interested in A3D displays, especially in the multiview and the integral imaging displays.

Stereo rectification removes vertical disparity between stereo image pair and it can improve visual quality of stereoscopic displays. This work addresses a practical issue when a stereo video input has been enhanced with post inserted objects (PIOs) such as captions and graphic objects. In such cases, previous stereo rectification methods would result in errors in the PIO regions due to PIOs' artificially edited epipolar geometry than those of the scene objects. To avoid such problem, this work proposes a PIO calibration method, which involves extracting PIOs, filling the PIO pixels, and re-inserting the PIOs into the rectified stereo video. The method constructs PIO models to extract PIOs in real stereo videos. Also, a modified stereo rectification procedure is proposed to work with the PIO calibration method, which reduces computational load and avoids video jitter artifacts. The method is tested on both real and simulated stereo videos to verify its validity.

This paper presents a bidirectional motion estimation (ME) method based on tracking feature trajectories and compensating for occlusion to enhance the temporal resolution of an input video sequence. First, we extract features and estimate their trajectories in the forward direction. We continue to track features only if the reliability of their trajectory is sufficiently high. Accordingly, if a feature can be continuously tracked through multiple frames, the proposed method assumes that its trajectory is a true motion vector (MV). Then, these forward feature trajectories are used as reference motion directions for backward block-based ME. If a block does not include any feature trajectories, we use the MVs of neighboring blocks as candidate MVs to propagate the predetermined true MV and to preserve the spatial correlation of MVs. Furthermore, the proposed method can detect occluded regions, where a continuously tracked feature does not have a corresponding point in the current frame. If an occluded region is detected, the intermediate frame is generated from either the previous frame or the current frame. In the non-occluded region, we generate the intermediate frames by taking into consideration the neighboring MVs to reduce blocking artifacts. Experimental results showed that the proposed temporal frame interpolation (TFI) method can improve the visual quality compared with conventional TFI methods, both objectively and subjectively.

The proposed method creates a high-resolution (HR) image on the basis of the frame registration of multiple low-resolution (LR) images. Not only does the super-resolution (SR) method based on using multiple LR images generally enhance the restored HR image quality compared to that based on using a single LR image, but it also increases the complexity and frame memory for hardware implementation. In order to generate an HR image, the multi-frame SR method has to estimate all motion vectors (MVs) between the target LR image and all the reference LR images. Additionally, the total frame memories used for storing LR images have to be preset according to the number of all the reference LR images. Therefore, the proposed multi-frame SR method focuses on a real-time and low frame memory system, thereby reducing the number of motion estimation (ME) operations and the total frame memory required, and preserving the image quality in an HR image restoration. First, we classify the input LR image into a feature and a uniform region in order to reduce the frame memory because the performance of SR algorithms is predominantly affected by restoring a feature region rather than a uniform region. Accordingly, we only save and use the feature region of the multiple LR images and not the uniform region for restoring an HR image. Next, the MV of each feature is estimated frame-wise to reduce the complexity of ME, and these MVs are accumulated as the feature trajectories through multiple LR frames. In the proposed method, the ME operation is conducted once between the reference LR image and the target LR image, and the estimated MVs are linked to the feature trajectories. These accumulated feature trajectories are used for generating an HR image. Experimental results show that the proposed multi-frame SR method can reduce the complexity and frame memory to one-third, while the quality of the restored HR image is equal to that obtained by using the conventional SR methods.

This paper presents a bidirectional motion estimation (ME) method based on tracking feature trajectories and compensating for occlusion to enhance the temporal resolution of an input video sequence. First, we extract features and estimate their trajectories in the forward direction. We continue to track features only if the reliability of their trajectory is sufficiently high. Accordingly, if a feature can be continuously tracked through multiple frames, the proposed method assumes that its trajectory is a true motion vector (MV). Then, these forward feature trajectories are used as reference motion directions for backward block-based ME. If a block does not include any feature trajectories, we use the MVs of neighboring blocks as candidate MVs to propagate the predetermined true MV and preserve the spatial correlation of MVs. Further, the proposed method can detect occlusion regions, where a continuously tracked feature does not have a corresponding point in the current frame. In the detected occlusion region, the intermediate frame is generated from either the previous frame or the current frame. Outside the occlusion region, we generate the intermediate frames by taking into consideration the neighboring MVs to reduce blocking artifacts. Experimental results show that the proposed temporal frame interpolation (TFI) method can improve visual quality over conventional TFI methods, both objectively and subjectively.

Color signal decomposition method is to decompose the conventional three-primary colors (RGB) into the multi-primary control values of multi-primary display (MPD) under the constraints of tri-stimulus matching. For the satisfaction of tri-stimulus matching between input and output display systems, the MPD color signals have to be estimated from a device-independent color space such as CIEXYZ or CIELAB. However, as the target of MPD is to display moving-picture data, it is necessary to simplify the color space conversion between input and output systems and decomposition of the multi-primary control values. In this paper, color signal decomposition method for MPD is proposed using 3-dimensional look-up-table (3D-LUT) in linearized LAB space. The linearized LAB space satisfies the linearity and additivity in color space conversion and easily constructs the 3-D LUT considering lightness, chroma, and hue. For the reproduction of moving-picture data in MPD, the proposed method is based on 3-D LUT structure to reduce the complexity of hardware and processing time. First, 3-D LUT containing gamut boundary data of MPD is created to derive the multi-primary control values. The input images are transformed to linearized LAB values. Then, the corresponding color signals on the gamut boundary points, which have the same lightness and hue as the input point, are calculated. Also, the color signal for a point on the gray axis is calculated. Based on the gamut boundary points and a point on the gray axis, the MPD color signals for the input values can be obtained by interpolating between the gamut boundary points and a point on the gray axis. In particular, the neighboring gamut boundary points are used in the region of occurring abrupt signal change for a smooth change of hue. As a result, the proposed method guarantees computational efficiency and color signal continuity and requires less memory rather than conventional color decomposition methods.

This paper proposes a method for estimating the illuminant chromaticity using the distributions of camera responses obtained by a CCD camera in a real-world scene. Illuminant estimation using a highlight method is based on the geometric relation between a body and its surface reflection. In general, the pixels in a highlight region are affected by an illuminant geometric difference, camera quantization errors and the nonuniformity of the CCD sensor. As such, this leads to inaccurate results if an illuminant is estimated using the pixels of a CCD camera without any preprocessing. Accordingly, to solve this problem, the proposed method analyzes the distribution of the CCD camera responses and selects pixels using the Mahalanobis distance in highlight regions. The use of the Mahalanobis distance based on the camera responses enables the adaptive selection of valid pixels among the pixels distributed in the highlight regions. Lines are then determined based on the selected pixels with r-g chromaticity coordinates using a principal component analysis (PCA). Thereafter, the illuminant chromaticity is estimated based on the intersection points of the lines. Experimental results using the proposed method demonstrated a reduced estimation error compared with the conventional method.