When evaluating extremely high-quality videos that reach maximum human perceptual limits, the conventional perceptual-level evaluation methods may seem insufficient. Hence, a paradigm shift in the evaluation methods is required. In this study, we propose a cognitive-level evaluation method for evaluating high-quality videos using impression strength as an index for the quality of experience (QoE) of an 8K ultrahigh-definition television (UHDTV). Based on the evaluation scheme, we quantitatively demonstrated the difference in the quality of high-quality images close to the perceptual limit of human vision, and the results validated the hypothesis of viewers experiencing strong psychological effects when watching 8K videos. In addition, the strength of higher-order impressions such as freshness and deliciousness change with changes in the signal bandwidth and luminance level, indicating that the QoE of a UHDTV can be modulated using signal processing techniques.

To help pave the way for Super Hi-Vision (SHV) broadcasting, we have developed a new codec system that can encode and decode SHV signals in real time. This is the third generation of SHV real-time hardware codec. This efficient compression system maintains high picture quality by using eight 1080/60p (60 frames/s) encoding units and a video format converter with signal compensation processing that takes the properties of the Dual Green format of SHV into account. The video format converter divides an SHV image spatially into eight 1920 × 1080 portions, each of which is fed to the encoding unit. In the previous SHV codec, the SHV image was divided into 16 portions (spatially eight and temporally two) and 16 1080/30p encoding units were used. Compared with the previous system, the new codec achieves a 50% bitrate saving and downsizes the codec by almost half. Furthermore, several new technologies were developed and installed in the codec. We conducted the world's first SHV international transmission over an advanced Internet connection using the codec at a TS rate of 260 Mb/s. The received picture quality was good enough to show any kind of SHV content on a large screen.

We have developed a motion-adaptive sub-Nyquist sampling technique to be used as a complement to super-resolution (SR) image reconstruction. These techniques are used together for pre- and post-processing with a conventional video codec (core codec). In this new video coding system, called “reconstructive video coding,” the sub-sampling process reduces the core codec's compression ratio, thereby reducing such nonlinear distortion as block noise. On the decoder side, the decimated pixels are replenished with ones from neighboring frames through a multi-frame SR technique that compensates for inter-frame local motion. As a preliminary design study, we developed a systematic procedure (a sub-sampler) for a spatiotemporal allocation of sampling points to cover the objects in a scene as uniformly as possible by taking into account their motions. Testing of this procedure in combination with a multi-frame SR technique demonstrated that the proposed sub-Nyquist sampling technique is more effective than simple non-adaptive sub-sampling.

Beyond HDTV is a very appealing notion for people involved in the development of video systems. Thinking outside the box, such a television system could be constructed in a completely original manner, but to design an actual working system in the present, the system would relate to current HDTV and follow in the footsteps of television systems that have come before it. This being the case, resolution, frame rate, scanning method, colorimetry, aspect ratio, and other parameters must be targeted for study. As for advanced 3DTV systems, a variety of systems can be considered, and by progressing in the order of multiview, integral, and holographic 3-D systems, we can envision the configuration of a "dream system" far removed from the traditional concept of 2-D systems. Depth information of the 3-D video systems, in general, is converted to 2-D images when recorded and transmitted. Therefore, a drastic improvement of resolution enables the system to carry enough depth information.

We propose a novel video coding paradigm with wavelet image size reduction and wavelet super resolution (SR) reconstruction for the pre- and post-processing of a conventional MPEG-4 advanced video coding (AVC) | H.264 codec. The proposed method features the wavelet SR reconstruction in the post-processing can ensure stability and precision by using SR reconstruction parameters which are adaptively optimized with the original image on local SR reconstruction in the pre-processing. Experimental results showed that our proposed method has a higher objectively measured peak signal-to-noise ratio and outputs images with a better subjectively evaluated appearance than the MPEG-4 AVC | H.264 codec without this paradigm.

We propose a novel super-resolution (SR) framework based on a sequential Monte Carlo (SMC) method, which is capable of robust optimization, for solving the inverse problem of degradation processes of imagery and sampling. The SR image is estimated from a set of multiple hypotheses, which are sequentially reorganized by evaluating their consistency with the input image. The concepts of norm regularization and motion registration in single/multi-frame SR are mapped into stochastic processes of an SMC's proposal distribution. The experiments showed that our framework is capable of seamlessly restoring both static and moving regions of degraded pictures.

To optimally design distortions in lossy video coding, we propose the use of a novel coding paradigm with adaptive nonlinear transforms as pre/post-processors of a conventional video codec. The preprocessor decimates less important pixels based on an image analysis. A conventional video encoder such as MPEG-4 AVC/H.264 further eliminates the redundancy of the decimated images. On the decoder side, the postprocessor restores small decoded images of the conventional decoder to the original resolution using an inverse mapping including a super-resolution technique that uses a priori knowledge on the decimation in the preprocessing. Results of the experiments showed the proposed coding scheme poses distortion that has a more straightforward appearance than that of a directly encoded/decoded image by a sole conventional H.264 codec.

We propose a novel video coding scheme targeting Super Hi-Vision (SHV) video sources. While it takes a conventional block-based MC + Transform hybrid coding approach that is suitable for hardware implementation of a SHV video codec, the proposed scheme achieved significant coding efficiency improvement by introducing several coding tools such as intra prediction and adaptive transform. According to our experimental analysis, the proposed scheme achieves significant bit-rate saving compared to the state-of-the-art AVC/H.264 high profile.

This paper presents a renovative priority queuing algorithm for jitter-sensitive MPEG-2 Transport Stream distribution on home network. Upon the dissemination of broadband access networks, there are many technical studies of IP-based video distribution (IPTV). In addition, it is becoming common for individual residences to use Ethernet home networks to interconnect PCs and home appliances. Under these circumstances, the home network will be a common traffic channel for them within residences. On the home network, it is difficult to comply with MPEG-2 Systems requirements in which the distribution delay jitter of Program Clock Reference in its stream should be suppressed to less than plusmn500 ns to synchronize system clocks between a transmitter and receivers. In this paper, we propose a Constant Delay Queuing (CDQ) scheduler to suppress the forwarding delay jitter, which is suitable for consumer Ethernet hubs to distribute the IPTV flow on home networks. By using an experimental hub which implements the CDQ scheduler, we demonstrate that the CDQ scheduler suppresses the jitter to less than plusmn500 ns. Furthermore, we discuss applications of CDQ to IPTV distribution on home networks and to improving synchronization stability of Network Time Protocol.

Object-based coding is a promising technique for next-generation coding. Arbitrarily shaped texture coding is the key technique for object-based coding. This paper proposes a new technique, called "region support DCT", for arbitrarily shaped texture coding. The method is based on conventional 2D DCT, and has a special feature that the decoding process is achieved by a conventional DCT process. The proposed technique outperforms other techniques and involves low complexity in the decoder.

Although it has been observed that motion-compensated frame differences increase toward block boundaries and overlapped block motion compensation (OBMC) has been shown to provide reduced blocking artifacts as well as improved prediction accuracy, there is almost no satisfactory theoretical basis that clearly interprets the space-dependent characteristics of motion-compensated frame differences, nor have the theoretical aspects of OBMC been investigated thoroughly. We first interpret the space-dependent characteristics of motion-compensated frame differences based on a novel statistical motion distribution model. We then apply the statistical motion distribution model to the analysis of prediction efficiency of OBMC. Through the analysis, we prove theoretically that OBMC can reduce and equalize the motion-compensated frame differences across a block. The analytical results are justified by empirical experiments with typical image sequences.

Overlapped block motion compensation (OBMC) has been shown to provide reduced prediction errors as well as reduced blocking artifacts compared with the conventional non-overlapped block motion compensation (NOBMC). However, there is no satisfactory theoretical basis that clearly interprets why OBMC can reduce prediction errors. We present a theoretical analysis of OBMC based on a novel statistical motion distribution model. Our analysis proves theoretically that prediction errors increase towards block boundaries and that OBMC has an error reduction and equalization property, with the errors being more reduced at block boundaries than at block centers. The analytical results are justified by empirical experiments with typical image sequences.

The depth-map of a scene conveys the fundamental information that is extremely useful for object segmentation, compact representation of videos, etc. We present an algorithm for robust depth-map estimation from image sequences with precise camera operation parameters. This algorithm exploits precise camera operation parameters to find correspondence points among multiple frames of an image sequence of a static scene. Our algorithm is able to deal with arbitrary camera movement. The experimental results show significant improvement in the accuracy of depth-maps compared with the conventional two-frame matching method.

In this paper we describe a high-precision camera operation parameter measurement system and apply it to image motion inferring. First, we outline the implemented system which is designed to provide camera operation parameters with a high precision required for image coding applications. Second, we calibrate the camera lens to determine its exact optical properties. A new technique is employed to improve the accuracy of internal camera parameters obtained from a commonly used algorithm. Last, we apply the pan, tilt and zoom operation parameters measured by the system to infer image motion. The experimental results show that the inferred motion coincides with the actual motion very closely, with an error of less than 0.5 pixel even for large motion up to 80 pixels.