A multi-format UHDTV video coding system with super resolution inter-layer prediction is described. Signaling optimal up-conversion parameters along with lower-resolution video, receivers are capable of decoding both higher- and lower-resolution videos without any help from inter-layer residuals. We developed real-time inter-layer prediction processors that calculate and signal optimal parameters on the transmission side and that up-convert low resolution video frames taking account of the signaled parameters. The system is equipped with a sophisticated optimization criterion that measures the structural similarity and the strength of ringing artifacts as well as error (i.e., PSNR) between the up-sampled and original images. The paper describes the concept, constituent functionalities and real-time implementation enabling 8K/4K simultaneous services.

In order to perform high compression coding of 8K UHDTV with a 12-bit depth, we propose a video coding method with spatio-gradational reduction before the HEVC/H.265 encoding procedure and its restoration after the decoding procedure. Original 8K UHDTV is reduced spatio-gradationally to a 4K image with an 8-bit depth by Lloyd-Max quantization with wavelet decomposition to minimize quantization errors and to prevent degradation for high compression coding. This spatio-gradationally reduced image is restored by wavelet image super-resolution with interpolation of gradation after the decoding procedure. In order to restore the original image faithfully, restoration parameters are selected so as to minimize the difference between the spatio-gradationally restored image and its original image on the encoder side and are then transmitted to the decoder side as side information. Experiments show that the proposed method has a better PSNR value and subjective quality than conventional HEVC/H.265 in cases of low bitrate. The proposed method delivers particularly high quality when an original image features high encoding difficulty and a biased histogram.

We have developed a prototype real-time UHDTV video coding system based on our proposed reconstructive video coding paradigm. In the paradigm, an image reduction process before a conventional lossy encoder suppresses non-linear distortions such as blocking artifacts alleviating the compression ratio at the encoder. On the receiver side, the super-resolution-based reconstruction process recovers the resolution of the decoded image by restoring and/or substituting the folded/omitted over-Nyquist components in the reduction. The paradigm is capable of transmitting side data that optimally controls the reconstruction process by trying the reconstruction in advance on the transmission side. The paper focuses on the hardware implementation of the reduction and super-resolution processes on FPGA-based 4K video processing PCI Express cards. We achieved real-time high-quality transmission of UHDTV video at a compression ratio of around 500:1.

We propose a novel ultra high-definition video coding method with bit-depth reduction before encoding procedure and bit-depth reconstruction after decoding procedure. The bit-depth reduction is performed by Lloyd-Max quantization; considering ultra high-definition video noise reduction for high coding efficiency and gradation conservation for pseudo-contour prevention. The bit-depth reconstruction is carried out accurately using side information which is determined by comparing a local-decoded bit-depth reconstructed image and an original image on encoder side. Experiments show that the proposed method has a pseudo-contour prevention effect and a better PSNR in comparison with conventional video coding methods.

We have developed a motion-adaptive sub-Nyquist sampling technique to be used as a complement to super-resolution (SR) image reconstruction. These techniques are used together for pre- and post-processing with a conventional video codec (core codec). In this new video coding system, called “reconstructive video coding,” the sub-sampling process reduces the core codec's compression ratio, thereby reducing such nonlinear distortion as block noise. On the decoder side, the decimated pixels are replenished with ones from neighboring frames through a multi-frame SR technique that compensates for inter-frame local motion. As a preliminary design study, we developed a systematic procedure (a sub-sampler) for a spatiotemporal allocation of sampling points to cover the objects in a scene as uniformly as possible by taking into account their motions. Testing of this procedure in combination with a multi-frame SR technique demonstrated that the proposed sub-Nyquist sampling technique is more effective than simple non-adaptive sub-sampling.

Event detection can be defined as the problem of detecting when a target event has occurred, from a given data sequence. Such an event detection problem can be found in many fields in science and engineering, such as signal processing, pattern recognition, and image processing. In recent years, many data sequences used in these fields, especially in video data analysis, tend to be high dimensional. In this paper, we propose a novel event detection method for high-dimensional data sequences in soccer video analysis. The proposed method assumes a Bayesian hidden Markov model with hyperparameter learning in addition to the parameter leaning. This is in an attempt to reduce undesired influences from ineffective components within the high-dimensional data. Implemention is performed by Markov Chain Monte Carlo. The proposed method was tested against an event detection problem with sequences of 40-dimensional feature values extracted from real professional soccer games. The algorithm appears functional.

We propose a novel video coding paradigm with wavelet image size reduction and wavelet super resolution (SR) reconstruction for the pre- and post-processing of a conventional MPEG-4 advanced video coding (AVC) | H.264 codec. The proposed method features the wavelet SR reconstruction in the post-processing can ensure stability and precision by using SR reconstruction parameters which are adaptively optimized with the original image on local SR reconstruction in the pre-processing. Experimental results showed that our proposed method has a higher objectively measured peak signal-to-noise ratio and outputs images with a better subjectively evaluated appearance than the MPEG-4 AVC | H.264 codec without this paradigm.

We propose a novel super-resolution (SR) framework based on a sequential Monte Carlo (SMC) method, which is capable of robust optimization, for solving the inverse problem of degradation processes of imagery and sampling. The SR image is estimated from a set of multiple hypotheses, which are sequentially reorganized by evaluating their consistency with the input image. The concepts of norm regularization and motion registration in single/multi-frame SR are mapped into stochastic processes of an SMC's proposal distribution. The experiments showed that our framework is capable of seamlessly restoring both static and moving regions of degraded pictures.

To optimally design distortions in lossy video coding, we propose the use of a novel coding paradigm with adaptive nonlinear transforms as pre/post-processors of a conventional video codec. The preprocessor decimates less important pixels based on an image analysis. A conventional video encoder such as MPEG-4 AVC/H.264 further eliminates the redundancy of the decimated images. On the decoder side, the postprocessor restores small decoded images of the conventional decoder to the original resolution using an inverse mapping including a super-resolution technique that uses a priori knowledge on the decimation in the preprocessing. Results of the experiments showed the proposed coding scheme poses distortion that has a more straightforward appearance than that of a directly encoded/decoded image by a sole conventional H.264 codec.

This paper proposes a distributed state estimation architecture for multi-sensor fusion. The system consists of networked subsystems that cooperatively estimate the state of a common target from their own observations. Each subsystem is equipped with a self-contained particle filter that can operate in stand-alone as well as in network mode with a particle exchange function. We applied this flexible architecture to 3D soccer-ball tracking by modeling the imaging processes related to the centroid, size, and motion-blur of a target, and by modeling the dynamics with ballistic motion, bounce, and rolling. To evaluate the precision and robustness of the system, we conducted experiments using multiocular images of a professional soccer match.

This paper proposes a direct geographic recognition scheme from a gray-scale image without shape reconstruction. Several geographic categories are expressed in 3 /spl times/ 3-arrays of normal-vector directions. Only by looking up a table, the observed pixel-values (intensities) in a window are transformed into indices of conformity whether the scene in the window is due to each geographic category or not. Geographical features are extracted by multi-resolution interpretation and fuzzy-logical operations. The feasibility and robustness of the proposed method is shown by some computer simulations.