Abstract Stereo matching is an important computer vision tool for three‐dimensional (3D) positioning, 3D visual representation, precision detection and recognition with 3D information. For 3D visual representation, for example, the depth image‐based rendering (DIBR) needs high accuracy depth maps to synthesise multiple views for 3D naked‐eyes displays. It is hard to predict high‐precision disparity for smooth regions and occlusion areas by traditional census‐based stereo matching methods. In this work, two new improved algorithms, called the improved quadruple sparse census transform and the adaptive multi‐shape aggregation, are proposed to achieve a precise stereo matching system. Instead of a binary threshold, the improved quadruple auxiliary census function with an adaptive threshold and the patch mean can raise the discrimination performance for better stereo matching. The selected sparse census patch is proposed to reduce the computation of the pixel‐matching cost. For effective aggregation of matching costs, multi‐shape windows and a texture‐aware algorithm is suggested to decide a suitable window. The proposed adaptive multi‐shape aggregation method performs better and solves the discontinuous depth problem occurring near the object boundaries. Experimental results show that the proposed stereo matching system with the proposed quadruple sparse census transform and multi‐shape cost aggregation achieves better raw depth maps than the existing algorithms for stereo matching.

Abstract In this paper, a fast interactive instance segmentation (IIS) system is proposed and it is composed of an effective heatmap generator, a multi‐resolution network (MRNet), and an adaptive threshold refiner to promptly and precisely predict the masks of the objects. The proposed heatmap generator after interaction clicks can help the MRNet to successfully learn the sensitive features for better prediction. Based on convolutional neural network models, the proposed MRNet backbone produces multiple features across multiple resolutions and can intrinsically predict the sharp contour of the object. After the probabilistic prediction achieved by the MRNet, the Otsu's threshold refiner is proposed to further remove some uncertain pixels in the predicted mask. Experimental results demonstrate that the proposed IIS system can promptly predict sharp masks of the targeted objects with mIoU of 89.1% in PASCAL VOC 2012 [1] validation set. Compared to other existing interactive methods, the proposed system can effectively predict the segmentation mask with higher accuracy and less interaction efforts.

To deliver three-dimension (3D) video services through the current two-dimension (2D) broadcasting systems, the frame compatible packing formats of views with depth maps will be an effective approach. With texture and depth information, the 3D videos with a depth image-based rendering engine can easily support all glasses and glasses-free 3D displays. In this paper, we propose a new YCbCr color depth packing method based on the centralized texture-depth packing (CTDP) formats to deliver effective 3D video services. Simulations show that the CTDP formats with YCbCr color depth packing method can achieve better objective and subjective texture and depth quality than the 2D-plus-depth packing (2DDP) formats. The benefits from using CTDP include light complexity and maturity and wide application of existing video codec. The packing scheme is compatible with any video codecs. Before the 3D-HEVC-based video broadcasting system, the proposed CTDP formats with YCbCr color depth packing method could help to deliver 3D videos in the current 2D broadcasting systems simply and efficiently.

A texture image plus its associated depth map is the simplest representation of a three‐dimensional image and video signals and can be further encoded for effective transmission. Since it contains fewer variations, a depth map can be coded with much lower resolution than a texture image. Furthermore, the resolution of depth capture devices is usually also lower. Thus, a low‐resolution depth map with possible noise requires appropriate interpolation to restore it to full resolution and remove noise. In this study, the authors propose potency guided upsampling and adaptive gradient fusion filters to enhance the erroneous depth maps. The proposed depth map enhancement system can successfully suppress noise, fill missing values, sharpen foreground objects, and smooth background regions simultaneously. Their experimental results show that the proposed methods perform better in terms of both visual and subjective metrics than the classic methods and achieve results that are visually comparable with those of some time‐consuming methods.

With advances in three‐dimension television (3DTV) technology, accurate depth information for 3DTV broadcasting has gained much attention recently. The depth map, either retrieved by stereo matching or captured by the RGB‐D camera, is mostly with lower resolution and often with noisy or missing values than the texture frame. How to effectively utilise high‐resolution texture image to enhance the corresponding depth map becomes an important and inevitable approach. In this study, the authors propose texture similarity‐based hole filling, texture similarity‐based depth enhancement and rotating counsel depth refinement to enhance the depth map. Thus, the proposed depth enhancement system could suppress the noise, fill the holes and sharpen the object edges simultaneously. Experimental results demonstrate that the proposed system provides a superior performance, especially around the object boundary comparing to the state‐of‐the‐art depth enhancement methods.

In this paper, a stereoview to multiview conversion system, which includes stereo matching and depth image-based rendering (DIBR) hardware designs, is proposed. To achieve an efficient architecture, the proposed stereo matching algorithm simply generates the raw matching costs and aggregates cost based on 1D iterative aggregation schemes. For the DIBR architecture, an inpainting-based method is used to find the most similar patch from the background, according to depth information. The simulation results show that the designed architecture achieves an averaged peak signal-to-noise ratio of 30.2 dB and structure similarity of 0.94 for the tested images. The hardware design for the proposed 2D to 3D conversion system operates at a maximum clock frequency of 160.2 MHz for outputting 1080p (1920 × 1080) video at 60 frames per second.

In this paper, a novel classifier, called superimposed sparse parameter (SSP) classifier is proposed for face recognition. SSP is motivated by two phase test sample sparse representation (TPTSSR) and linear regression classification (LRC), which can be treated as the extended of sparse representation classification (SRC). SRC uses all the train samples to produce the sparse representation vector for classification. The LRC, which can be interpreted as L2-norm sparse representation, uses the distances between the test sample and the class subspaces for classification. TPTSSR is also L2-norm sparse representation and uses two phase to compute the distance for classification. Instead of the distances, the SSP classifier employs the SSPs, which can be expressed as the sum of the linear regression parameters of each class in iterations, is used for face classification. Further, the fast SSP (FSSP) classifier is also suggested to reduce the computation cost. A mass of experiments on Georgia Tech face database, ORL face database, CVL face database, AR face database, and CASIA face database are used to evaluate the proposed algorithms. The experimental results demonstrate that the proposed methods achieve better recognition rate than the LRC, SRC, collaborative representation-based classification, regularized robust coding, relaxed collaborative representation, support vector machine, and TPTSSR for face recognition under various conditions.

Accuracy and speed of face recognition frameworks are two foremost concerns for practical applications in recent researches. Linear regression classification (LRC) is a very famous and powerful approach for face recognition; however, it cannot perform very well under occlusion situations. In this paper, the regression parameters of the module-LRC are analyzed when a query facial image is partially occluded. For removing contaminated modules, the weighted module linear regression classification (WMLRC) is proposed. In order to evaluate the effectiveness, AR face database is used to validate the proposed WMLRC as well as the well-known face recognition methods. Simulation results show that the proposed WMLRC method achieves the best performance for partially-occluded faces while keeping the advantage in speed over the SRC-based approaches.

In real‐world recognition applications, several poor situations such as varying environment, limited image information, and irregular status would lead performance degradation in recognition. To overcome the unexpected effects, the authors propose a generalised linear regression classification (GLRC) to fully use all the information of multiple components of input images. The proposed GLRC achieves the global adaptive weighted optimisation for linear regression classification (LCR), which can automatically use the distinction components for recognition. For colour identify recognition, the authors also suggest several similarity measures for the proposed GLRC to be tested in different colour spaces. Experiments are conducted on two object datasets and two face databases including Columbia Object Image Library‐100, SOIL‐47, SDUMLA‐HMT and FEI. For performance comparisons, the proposed GLRC approach is compared with the contemporary popular methods including colour principal component analysis, colour linear discriminant analysis, colour canonical correlation analysis, LRC, robust LRC (RLRC), sparse representation classification (SRC), colour LRC, colour RLRC, and colour SRC. The simulation results demonstrate that the proposed GLRC method achieves the best performance in multi‐component identity recognition.

Gradient phase, which is treated as an illumination insensitive measure, is an important feature for visual detection and recognition applications, especially under illumination variations. However, fewer statistical characteristics of the gradient phase have been reported till now. First, the statistical characteristics of the gradient phase against gradient signal‐to‐noise ratios (gradient SNRs) were investigated. The analysed results show that the confidence (or standard deviation) of gradient phases against gradient SNRs should never be linearly related, as is usually supposed. With the help of the statistical analyses of the gradient phase, the gradient‐based visual detection and recognition were improved by incorporating confidence information into the cost function. Moreover, inspired by the analysed characteristics of the gradient phase, an enhanced gradientface method is proposed to improve the performance of the gradient phase‐based face recognition. Intensive simulations and comparisons are performed to show its superior performance without the side effect of discrimination loss that existed in some illumination normalisation approaches.

The high‐efficiency video coding (HEVC) standard for three‐dimensional (3D) video, which is called 3D‐HEVC, improves the compression efficiency for high resolution 3D video sequences. In this study, three coding time reduction algorithms, namely fast coding unit partition, fast prediction unit (PU) partition and search range reduction, are proposed for inter‐prediction of 3D‐HEVC by referring to the depth edge information. The pixels in one object should have the same motion characteristics. By detecting depth edge, we can easily classify the regional pixels if they are in the same object. Thus, in the encoding stage, the sizes of coding and PUs can be determined by the proposed fast coding unit partition and fast PU partition methods, respectively. Furthermore, the range of motion search can be effectively reduced by the proposed search range reduction method. Experimental results show that the proposed depth edge‐based motion search algorithms can reduce the coding time of inter‐prediction by up to 62% on average with negligible quality degradation.

In this paper, two improved classifiers based on the nearest feature line (NFL) are proposed for image classification, where the neighborhood feature line segment-I (NFLS-I) classifier uses the neighborhood of prototypes to select the better-fitted feature lines (FLs) and the neighborhood feature line segment-II (NFLS-II) classifier utilizes the neighborhood of the query sample to choose more-likely FLs. With better selection of FLs, these two classifiers can both improve the recognition performance and the computation problem. A large number of experiments on Soil-100 object database, Yale face database, FEI face database, AR face database, and Jochen triesch static hand posture database are performed to evaluate these two proposed classifiers. The experimental results demonstrate that the proposed NFLS-I and NFLS-II classifiers outperform the original NFL and some other improved NFL classifiers for object recognition, hand posture recognition, and face recognition.

This paper presents an enhanced AMVP mechanism and an adaptive motion search range algorithm to reduce the coding complexity of the high efficiency video coding (HEVC) standard. The performance of motion estimation highly depended on the selection of advanced motion vector prediction (AMVP) mechanism. With this point of view, we firstly design a new selection algorithm to increase the accuracy of AMVP. After getting an accurate motion vector predictor (MVP), we analyze the relationship of motion vector difference (MD) between MVP and motion vector (MV) to design an adaptive motion search range algorithm. Experimental results show that the proposed algorithms can not only decrease the coding complexity but also keep the coding performance.

In the depth-image-based rendering (DIBR), hole-filling is one of the most important procedures for the quality of DIBR results, especially in modern multi-view 3D-TV systems. To achieve photorealistic performance, we propose a novel inpainting algorithm for hole-filling. Rather than fill the holes in the target image by using neighboring pixels, the proposed method makes use of a reference image which contains the occluded information of the input image. The reference image is first segmented to many planar segments by optical flow and graph cuts optimization, and a set of homography matrices between the reference and target images for all planar segments are estimated by SURF features. Then the segments are transformed by these homography matrices to fill the holes in target image. Finally, exposure compensation is performed to reduce the difference between exposures of the input and reference images. Experimental results reveal that the proposed method can yield satisfactory synthesis quality and outperforms other existing hole-filling methods.

The possible transmission format of future 3D Television broadcasting systems could be one monoscopic video with its associated depth information. In this case, a high-quality depth-image-based rendering (DIBR) technique is desired at the receiver side to generate multi-view contents. Due to lack of information in monoscopic video, there might induce some factitious artifacts in synthesized virtual views. To reduce the artifacts and enhance the quality of synthesized views, we propose a multi-view synthesis algorithm based on depth and texture consistency. Our algorithm first applies the trilateral filter for and use isolated pixels and contour noises eliminations in warping process. Afterward, hole regions are recovered by the proposed adaptive order-statistical filter and depth-based texture map refinement. Experimental results reveal that the proposed method can achieve better quality than other conventional methods, which will be a great help for audiences to perceive depth comfortably.

Owing to losing the detailed information, the low‐resolution problem in face recognition degrades the recognition performance dramatically. To overcome this problem, a novel face‐recognition system has been proposed, consisting of the extracted feature vectors from the multiple‐size discrete cosine transforms (mDCTs) and the recognition mechanism with selective Gaussian mixture models (sGMMs). The mDCT could extract enough visual features from low‐resolution face images while the sGMM could exclude unreliable observation features in recognition phase. Thus, the mDCT and the sGMM can greatly improve recognition rate at low‐resolution conditions. Experiments are carried out on George Tech and AR facial databases in 16 × 16 and 12 × 12 pixels resolution. The results show that the proposed system achieves better performance than the existing methods for low‐resolution face recognition.

Gradient‐based local descriptors have received more attention these years and have been successfully used in many applications such as human detection and face recognition. The advantages of the local descriptors are the resistance to the local geometric and photometric errors and the robustness to the expression variations. In this paper, the authors propose a new local descriptor called the histogram of gradient phases (HGP), which has some intriguing properties compared with the existing local descriptors such as the histogram of orientated gradients and DAISY for face recognition under the unconstrained conditions. In contrast with the histogram of the oriented gradient descriptor, the orientation histogram is computed from the estimated gradient phase distributions instead of weighting the votes of the gradient magnitudes. In this paper, the phase distributions are estimated by means of the gradient phases and the variances are decided by the estimated gradient signal‐to‐noise ratios of the pixels in a local region. The HGP descriptor which takes the confidence of the gradient phase into account is more discriminative and less sensitive to the normalisation process than most local descriptors, which significantly degrade without a proper normalisation. The simulation results show that the proposed HGP descriptor achieves a better performance and is more robust than the existing local descriptors.

In practical systems, face recognition under unconstrained conditions is a very challenging task, where their input images are first pre‐processed and initially aligned by a face detection algorithm. However, there are still some residual localisation errors after the initial alignment. If we do not take these errors into account, the recognition performance should be greatly degraded for most face recognition algorithms. Generally, when designing a practical face recognition system, we need to compromise the capability of residual error tolerance and the discriminating capability. Although it is feasible to apply an iterative alignment algorithm to fine‐tune alignment, it will increase the computation load significantly. In this study, we propose an adaptive two‐stage face recognition system consisting of two block‐based recognition stages with a relatively larger cell size (i.e. the size of local regions) in the first stage to provide sufficient tolerance for geometric errors followed by a smaller one in the second stage to accurately evaluate a most probable candidate subset, which is adaptively determined according to the proposed confidence measure. In addition, an iterative gradient‐based alignment algorithm is incorporated into the two‐stage system to refine the alignment such that the recognition performance can be improved and the computation load can be saved simultaneously.

An efficient lossless coding algorithm for encoding residual data based on H.264/AVC that uses the Hadamard transform to take advantage of data compaction is proposed. To reduce redundancies, the transformed coefficients are compacted using a compact truncation. The compacted coefficients are then compressed by an improved context‐based adaptive variable length coding coder. Simulation results show that the proposed lossless residual coding algorithm reduces the bit rate by >7.5% compared with those obtained using existing H.264/AVC lossless coding systems.

An efficient lossless coding technique is very important for storage and transmission applications of error sensitive information such as medical, seismic and digital artistic data. In this study, the authors proposed an H.264‐based advance video coding (H.264/AVC)‐based hierarchical lossless coding method, where the input video will be firstly encoded by H.264/AVC coder with a quantisation parameter (QP) selector in the base layer and the coded error is encoded by a QP‐adaptive Rice coder in the enhancement layer. To reduce encoding time, the QP selector can be simplified to select the nearly optimal QP. Simulation results show that the proposed hierarchical lossless coding architecture achieves better compression ratio than the traditional H.264/AVC‐based lossless coding systems. Since the proposed system could provide both lossy and lossless coding services at the same time, the proposed lossless video coding system has advantages of efficiency and flexibility for practical applications. Experimental results show that the proposed coding system can provide less coding bits and reduce coding complexity compare with H.264‐differential pulse code modulation.

To improve the discriminant nearest feature space analysis (DNFSA) methods [6], in this paper, we propose an improved DNFSA (IDNFSA) algorithm to increase the robustness for variable lighting face recognition. The IDNFSA removes the mean of each image and attempts to minimize the within-class feature space (FS) distance and maximize the between-class FS distance simultaneously. In the IDNFSA, the first n eigenvectors are dropped and a generalized whitening transformation is suggested. In the recognition phase, the projected coefficients are classified by the nearest feature space rule with the ridge regression classification algorithm. Furthermore, to achieve higher accuracy, the illumination compensation is used. Experiments on the Extended Yale B (EYB) and FERET face databases reveal that the proposed approach outperforms the state-of-the-art methods for variable lighting face recognition.

Face recognition for images acquired from uncontrollable environment and target positions is a challenging task. These input images are first pre-processed and initially aligned by the face detection algorithm. However, there are still some residual geometric errors after the initial alignment by the face detection algorithm. If we don't take these errors into account, the recognition performance should be unacceptable. Although some iterative optimization algorithms can be used to fine-tune alignment during recognition, it increases computation load significantly. A two-stage face recognition system is proposed which comprises a block-based recognition algorithm to provide sufficient tolerance for geometric errors and then followed by a pixel-based recognition algorithm which only needs to evaluate a candidate subset from the previous stage. From simulation results, we find that this proposed system can reduce the average computation complexity about 69% and achieve promising performance.

To improve the robustness of the linear regression classification (LRC) algorithm, in this paper, we propose a linear discriminant regression classification (LDRC) algorithm to boost the effectiveness of the LRC for face recognition. We embed the Fisher criterion into the LRC as a novel discriminant regression analysis method. The LDRC attempts to maximize the ratio of the between-class reconstruction error (BCRE) over the within-class reconstruction error (WCRE) to find an optimal projection matrix for the LRC such that the LRC on that subspace can achieve a high discrimination for classification. Then, the projected coefficients are executed by the LRC for face recognition. Extensive experiments carried out on the FERET and AR face databases show that the LDRC performs better than the related regression based algorithms and shows a promising ability for face recognition.

In this letter, we propose a unitary regression classification (URC) algorithm, which could achieve total minimum projection error, to improve the robustness of face recognition. Starting from linear regression classification, the goal of the proposed URC method is to minimize the total within-class projection error of all classes to seek the unitary projection for face classification. In the recognition phase, the recognition is determined by calculating the minimum projection error on the unitary rotation subspace. Experimental results carried out on FEI and FERET facial databases reveal that the proposed algorithm outperforms the state-of-the-art methods in face recognition.

With the possible advent of quantum computers, the most famous public key cryptosystem for network applications - RSA, is facing critical challenges in the near future. A replacement might be knapsack cryptosystems, which do not yield any benefits to quantum computation. Before making knapsack cryptosystems suitable for applications, it is necessary to guarantee their security. At present, the most significant challenge towards knapsack cryptosystems is lattice attacks. Public key density has historically been used to measure the security of the knapsack cryptosystems against these lattice attacks. However, an acceptably dense knapsack cryptosystem was broken using lattice attack implementation in our research. In order to study the security of knapsack cryptosystems against lattice attacks, we design a series of experiments to analyze these factors. We conclude it is not appropriate to assess knapsack cryptosystem security by simply considering density. From our results, we provide suggested directions for knapsack cryptosystem developers.

To improve the limitation of linear regression classification, a class specific kernel linear regression classification is proposed for low resolution face recognition under variable illumination. The nonlinear mapping function enhances the modeling capability for highly nonlinear data distribution. The explicit knowledge of the nonlinear mapping function can be avoided computationally by using the kernel trick. With kernel projection, the class label is also determined by calculating the minimum reconstruction error. Experiments carried out on Yale B facial database in size of 8×8 pixels reveal that the proposed algorithm outperforms the state-of-the-art methods and demonstrates promising abilities against severe illumination variation.

Since the 3D coding and transmission standard is not available nowadays, we need to transfer the 3D contents through the existing coding standards. To achieve this, the most common way is to combine both left and right eye frames into one single frame, which is the so-called stereo packing. So far, there are many packing methods and each of them has different characteristics and advantages. To obtain the better coding efficiency and reconstructed quality, we propose an adaptive checkerboard-based stereo coding system to retain the advantages of most different packing methods. The proposed system first computes the features of the stereo image pairs then finds the most suitable packing mode for efficient transmission. In the decoder side, the proposed de-packing mechanism is first adopted to find the correct packing mode. Afterward, an edge-dependent interpolation method is performed to attain better reconstructed quality. Experimental results reveal that the proposed method can achieve better results when comparing with other existing methods.

The uncontrollable illumination problem is a great challenge for face recognition. In this paper, we propose a novel face recognition framework, the improved principal component regression classification (IPCRC) algorithm, which could overcome the problem of multicollinearity in linear regression. The IPCRC approach first performs principal component analysis (PCA) process to project the face images onto the face space. The firstnprincipal components are intentionally dropped to boost the robustness against illumination changes. Then, the linear regression classification (LRC) is executed on the projected data and the identity is determined by the minimum reconstruction error. Experiments carried out on Yale B and FERET facial databases reveal that the proposed framework outperforms the state-of-the-art methods and demonstrates promising abilities against severe illumination variation.

Computational load of motion estimation in advanced video coding (AVC) standard is significantly high and even worse for HDTV and super-resolution sequences. In this article, a video processing algorithm is dynamically mapped onto a new parallel reconfigurable computing (PRC) architecture which consists of multiple dynamic reconfigurable computing (DRC) units. First, we construct a directed acyclic graph (DAG) to represent video coding algorithms in which motion estimation is the focus. A novel parallel partition approach is then proposed to map motion estimation DAG onto the multiple DRC units in a PRC system. This partitioning algorithm is capable of design optimization of parallel processing reconfigurable systems for a given number of processing elements in different search ranges. This speeds up the video processing with minimum sacrifice.

H.264/AVC is a standard developed for various low-complexity video applications and high-definition television. To improve coding performance, H.264/AVC may optionally adopt the rate-distortion optimization (RDO) method to find the best encoding mode among various inter and intra modes. However, the exhaustive RDO search among different modes increases the H.264/AVC encoder complexity and limits its application. In this paper, we propose an inter mode prediction algorithm for P slices based on spatial and temporal consistency analysis to reduce the complexity of the RDO computation. We apply the stochastic method to analyze the spatial consistency and use rate information for temporal consistency. The experimental results show a 0.03 peak signal-to-noise ratio loss, a 0.87% bit rate increase, and a 58.39% encoding time reduction on average.

For multiview video generation from 2-D video sequences, the most important stage is to efficiently synthesize the corresponding depth maps for all video frames. The depth map generation involving precise object segmentation and depth assignment requires intensive computation and human assistance which could be costly and possibly inconsistent. To achieve reliable depth map generation, an efficient depth map interpolation method from the existing pairs of keyframes and their corresponding depth maps at the both ends of video scene is presented. The proposed method contains forward/backward motion alignment, block-based updating, object-based updating, frame selection mechanism, and depth adjustment. Experimental results show that the proposed method can generate the depth maps successfully after objective and subjective evaluations. We believe that it could greatly help to achieve multiview video generation from 2-D video sequences effectively.

An efficient lossless video codec based on H.264/AVC that uses the Hadamard transform and adaptive truncation (HTAT) method is proposed. Most lossless video coding algorithms avoid transforms in their coding procedures. However, without this transformation, those lossless video coding algorithms cannot take advantage of data compaction. The HTAT method adopts the Hadamard transform to compact data, where the transformed coefficients are adaptively separated into the quotient parts and remainder parts. The quotient parts can be better encoded by entropy coders due to the energy compaction provided by the Hadamard transform. To ensure lossless coding, the remainders are encoded into compact form after exploring the redundancies caused by the Hadamard transform. Simulation results show that the proposed HTAT method has better coding performance than that of the original H.264/AVC lossless coding, with 6.2% and 3.7% reductions in bit rate for all I frame sequences and IPPP sequences, respectively.

In this paper, a novel face recognition system using partitioned hidden Markov models is introduced to deal with partial occlusion problems. The proposed subface based system divides the face into forehead, eyes, nose, mouth, and chin, five subregions, which are characterized by five separated subface HMMs such that we can reconfigure these subface HMMs to achieve partially occluded face recognition. Moreover, we also suggested a facial grammar network to manipulate these subface HMMs to form various composite face HMMs. The Viterbi algorithm is used to estimate the likelihood score to perform face recognition with maximum likelihood criteria. Experiments are carried out on George Tech (GT) and AR facial databases. Experimental results reveal that the proposed system outperforms the embedded HMM (EHMM) and demonstrates promising abilities against partial occlusions and robustness against different facial expressions and illumination variations.

With the recent 3D display technology improvements, people now can perceive 3D effects in 3D display systems without wearing 3D glasses. However, due to 3D contents are not sufficient enough, the 2D to 3D conversion technique becomes more and more important in the near future. For 2D to 3D video conversion, the most common way is to generate the correspondent depth video from the original 2D video first then synthesizing the desired video by utilizing generated depth videos. The main key point for the 2D to 3D conversion is how to generate the corresponding depth video precisely and efficiently. In this paper, we will propose an efficient depth maps interpolation method from existing pairs of key frames and depth maps. The proposed method contains forward/backward block-based motion estimation/compensation, block refinement, object refinement, and frame selection mechanism. Experimental results show that the proposed method can generate the depth maps successfully and effectively, which will be a great help for 3DTV systems.

Rate-distortion cost estimation is useful for many H.264/advanced video coding (AVC) applications including rate-distortion optimization (RDO) for mode-decision and rate-control. In this paper, we propose a new rate-prediction model and an adaptive algorithm to provide more accurate estimation of the number of coding bits for encoding intra and inter-blocks compared to previously proposed methods. The rate estimation is modeled by a linear combination of existing coding parameters, which are more accurately related to entropy coding and transform coefficients. Based on the proposed model, a cost estimation function is also proposed to give a more accurate rate-distortion cost estimation. Furthermore, we propose a block classification approach to further improve the effectiveness of the proposed scheme. The proposed schemes can achieve better results in estimating the bit-rate and rate-distortion cost compared to previously proposed approaches.

In this paper, an efficient mode decision algorithm and a high throughput hardware architecture with eight-pixel parallelism for improving the H.264/advanced video coding intra coding efficiency are proposed. Based on the inherent features of the discrete cosine transform, the input block is first transformed and then analyzed to determine its texture directional tendency. A few candidate modes are chosen for cost calculation, which adopts the error model in the sum of absolute integer-transformed differences (SAITD). Experimental results show that the proposed intra prediction algorithm has lower peak signal-to-noise ratio degradation and bit-rate increment compared to other recent designs. Using the SAITD technique, the proposed mode decision algorithm is effectively integrated into intra prediction rather than being a preprocessing unit. For hardware implementation, the proposed intra prediction algorithm in the macroblock level is implemented for prediction computation, mode decision, and reconstruction loop units. The synthesis results show that the proposed architecture can achieve a 100 MHz operation frequency, allowing it to easily support the real-time requirements for video resolutions of up to the 16 source input format.

H.264/scalable video coding (SVC) has spatial scalability which can provide various resolution sequences for a single encoded bit-stream. The various resolutions can be achieved by up-sampling a lower image resolution sequence. The up-sampling method employed in H.264/SVC is directly proportional to the video quality. In order to improve the video quality with an effective up-sampling method, we propose a classified multifilter up-sampling algorithm for H.264/SVC in spatial scalability which classifies an image region as edges and nonedges. An appropriate filter is then applied to up-sample the image. In the proposed scheme, we applied the Wiener filter for edges and conventional filter for nonedges within the defined window size to preserve the edges in the up-sampled image sequence. The experimental results show that the average peak signal-to-noise ratio improvement and bit-rate reduction are 0.5 dB and 9%, respectively, which confirm that the performance of the proposed method is better than that of the existing H.264/SVC standard.

The lossy video coding is a key technology to compress the video to become very low bit rate data and H.264/AVC standard performs very well in lossy coding. However, the lossless image and video coding still plays an important role, especially in medical applications. Hence, the lossless coding in use of H.264/AVC gains a lot of attentions. In this paper, we introduce simplified interpolations to replace the sample-by-sample DPCM concept, which uses the nearer pixel as the prediction for certain modes to achieve better coding efficiency. The proposed predictions are applied to all intra prediction modes suggested in H.264/AVC. Moreover, the luma/chroma dc blocks reduction can also reduce the syntax elements for entropy coding. With these proposed algorithms, the compression performance of the H.264/AVC lossless intra coding is greatly improved.

After availability of 3D LCD display systems, the stereo vision applications without glasses become more and more popular in our daily life. However, the stereo contents are still not rich enough for the public. In this paper, we propose a simple and novel generation procedure to produce stereoscopic video from monoscopic moving video. The proposed algorithm mainly focuses on exploring the effective methods for matching frame selection and other rectification steps such as frame shifting, transformation, and reshaping. Experimental results reveal that the proposed method can effectively pick up the matching frame to provide suitable horizontal parallax and adjust vertical parallax such that the generated video sequences can properly exhibit stereo scenes in stereoscopic LCD display systems.

In this paper, a partial cost intra prediction algorithm based on transform-domain property is proposed. After transform processing, the intensity of the whole block is collected into the first few pixels. The differences in the low frequency region are adopted to represent the total distortion of one 4times4 block. In comparison to the previous fast mode decision algorithms, which mainly reduce the number of candidates in the RDO process, the proposed algorithm simplifies the computation in the cost function. Moreover, the integer transform with cosine multiplication is applied in the proposed algorithm and gets better results in intensity collection. With the benefit of linear transform, the number of operations is further reduced approximately by 40% of the original cost calculation suggested in H.264 JM standard.

The stereo image pairs or two-view video sequences can be captured by two cameras at two horizontally different positions. If the stereo image pairs possess incompatible convergences of vertical parallax and horizontal parallax, human eyes generally cannot properly exhibit stereo visualisation. If the image pairs are taken from single camera or selected from a video sequence, perception of stereo visualisation will become even worse. The authors propose an effective calibration procedure to adjust the image pairs to achieve better stereo visualisation. The proposed calibration procedure contains six steps, including feature point extraction, bidirectional feature point matching, relative distance checking, image transformation, hole-filling and reshaping. Experimental results show that the proposed system can effectively adjust vertical and horizontal parallax such that the calibrated stereo image pairs can properly exhibit stereo scenes in stereoscopic display systems. Experimental results also reveal that the proposed method can achieve less vertical parallax than the existing rectification methods.

In this paper, a combined kernel architecture for efficiently decoding the residual data in the H.264/AVC baseline decoder is proposed. The kernel architecture in the H.264/AVC decoder consists of context-based adaptive variable length code (CAVLC) decoder, inverse quantization (IQ), and inverse transforms (IT) units. Since the decoding speeds of these kernel units vary with data, traditional methods require data buffers between these units. The first proposed architecture efficiently combines CAVLC decoding and IQ procedures. The multiple 2-D transforms architecture is applied to all inverse transforms, including the 4times4 inverse integer transform, the 4times4 inverse Hadamard transform and the 2times2 inverse Hadamard transform, to attain fewer gate counts than those of existing transform designs. Simulation results show that the total number of gates is 14.1 k and the maximum operating frequency is 130 MHz. For real-time requirements, in the worst case, the proposed architectures can achieve the operation speed of the H.264/AVC decoder up to 4VGA@30 frames/sec in 4:2:0 format.

In this paper, we propose two-layer coding algorithm to improve the performance of the H.264-based lossless (H.264-LS) image coding. From universal access point of view, the proposed method is based on the H.264 lossy image coding with other CABAC layer to compensate the lossy portion. Besides, the H.264-LS with DPCM (H.264-LS_DPCM) and H.264-LS achieve different coding performance in use of CABAC and CAVLC entropy coders without the DCT and quantization. We further suggest an adaptive entropy coding (AEC) algorithm to determine the best entropy coder by using the image content variations, which is calculated from the sum of absolute difference of intra prediction. Simulation results show that the proposed AEC method have good correct detection rates and improvement of compression rate for H.264-LS and H.264-LS_DPCM coders. The two-layer H.264-LS almost have the same compression rate than the H.264-LS_DPCM.

The interactive devices, which help humans to easily access the computer and video game player, become an important technology to provide interactive multimedia services and entertainments in the 21st century. Considering the cost and the effectiveness of the human machine interface (HMI) devices, in this paper, we design a low cost image-based auto-positioning brush (IA-Brush), which can automatically locate the position of the window and modify the affection of angle displacement of camera. The proposed IA-Brush based on a low cost video camera can be easily implemented for low cost liquid crystal display monitor by using image processing techniques figure with a designated background grids. Simulation results with exhibition of interactive painting games show that we can achieve a low cost and effective HMI by using a video camera only.

In this paper, a modified bit-rate estimation method is proposed to reduce the computation for 4×4 intra mode decision of H.264/AVC video encoder. The number of coded bits is modeled by a linear combination of existing coding parameters, which are highly related to the entropy coding of H.264/AVC. Furthermore, to improve the accuracy of the estimation, the proposed scheme is made adaptive to the information obtained from previously coded blocks. Comparing to the original rate distortion optimized (RDO) encoding process, which needs to calculate the actual encoded bits of H.264/AVC for each coding mode, the proposed adaptive rate estimation can save about 28% and 21% of the total encoding time for QCIF and VGA sequences, respectively. For the coding performance, the proposed method achieves nearly no loss in visual quality with only slight bit-rate increases.

In this paper, we propose an approximate square criterion for H.264/AVC intra mode decision. The sum of square difference (SSD) criterion achieves the best video quality but takes much high computation due to the square operation. A sum of approximate square difference (SASD) criterion is proposed to maintain the video quality and reduce the computation. By applying the characteristic of the SSD criterion, simulation results show the rate-distortion performance of the SASD criterion is close to that of the SSD method. For the hardware implementation, synthesized results show that the proposed approximate square operation respectively reduces 75% and 61% in the area cost and timing delay than that of the square function.

In H.264/AVC intra-frame coding, the rate-distortion optimization (RDO) is employed to select the optimal coding mode to achieve the minimum rate-distortion cost. Due to a large number of combinations of coding modes, the computational burden becomes extremely high in intra-prediction. In this paper, we propose two fast, efficient but reliable direction detection algorithms by computing subblock and pixel direction differences. Both proposed methods effectively estimate the edge direction inside the block to narrow down the predictive modes to reduce the RDO computation. Experimental results show that the proposed methods can reduce the encoding time by about 60% with negligible loss of coding performance. For hardware realization, a fast mode decision VLSI circuit for intra-prediction with the silicon core size of 0.12 times 0.12 mm2at 0.18-mum CMOS technology is implemented. The fast mode decision VLSI in three-stage pipelined architecture operated at 173 MHz can encode 30 fps real-time videos up to Level 5.

A low complexity fast mode decision algorithm for H.264/AVC intra prediction that uses discrete cross differences (DCD) to reduce the unlikely candidate modes in the RDO calculation is proposed. By using horizontal and vertical differences in different locations, the directions of the edges can be precisely detected. Experimental results show that the proposed fast mode decision algorithm reduces the encoding time by about 56%, with negligible loss of video quality. To realize the proposed algorithm, a VLSI design, which is comprised of a cross difference unit and direction detection unit, is implemented for the mode pre-selection stage of intra prediction. The design is synthesized using UMC 0.18 mum CMOS technology and simulated with Verilog-XL. The operating frequency of the synthesized core can exceed 50 MHz.

In this paper, an efficient algorithm is proposed to improve the decoding efficiency of the context-based adaptive variable length coding (CAVLC) procedure. Due to the data dependency among symbols in the decoding flow, the CAVLC decoder requires large computation time, which dominates the overall decoder system performance. To expedite its decoding speed, the critical path in the CAVLC decoder is first analyzed and then reduced by forwarding the adaptive detection for succeeding symbols. With a shortened critical path, the CAVLC architecture is further divided into two segments, which can be easily implemented by a pipeline structure. Consequently, the overall performance is effectively improved. In the hardware implementation, a low power combined LUT and single output buffer have been adopted to reduce the area as well as power consumption without affecting the decoding performance. Experimental results show that the proposed architecture surpassing other recent designs can approximately reduce power consumption by 40% and achieve three times decoding speed in comparison to the original decoding procedure suggested in the H.264 standard. The maximum frequency can be larger than 210 MHz, which can easily support the real-time requirement for resolutions higher than the HD1080 format.

In this paper, we design a novel architecture for computing all transforms required in H.264/AVC high profile decoder. This flexible architecture design can compute all transforms including 8 and 4-point integer transforms as well as 4 and 2-point Hardamard transforms such that we can reduce the implementation chip area dramatically. With 8 pixels/cycle throughput, this proposed design can complete the computation in 95 clock cycles with 8times8 inverse transform involved or 54 clock cycles without 8times8 inverse transform for one macroblock. Simulation results show that the implemented area is 18.5 k gate counts, and the maximum clock frequency is 125 MHz. For the real-time requirement, the architecture can deal with all existed frame sizes in 4:2:0 format. For example, if this architecture is operated at 106 MHz, it achieves 4096times2304@30 frames/sec.

In H.264/AVC intra frame coding, the rate-distortion optimization is employed to select the optimal coding mode to achieve the minimum rate-distortion cost. Due to a large number of combinations of coding modes, the computational burden becomes extremely high in intra prediction, hi this paper, we propose a fast, efficient but reliable vector based detection algorithm for H.264 intra frame coding system. The proposed algorithm effectively estimates the edge direction inside the block to narrow down the predictive modes to reduce the RDO calculations. Experimental results show that the proposed method can reduce the encoding time by about 60% with negligible coding loss.

This paper proposes combined decoding architecture and high-throughput flexible transform design to effectively decode the residual data for H.264/AVC decoders. The inverse quantization (IQ) procedure is combined with context-based adaptive variable length coding (CAVL) decoder to efficiently achieve the simplification. Besides, the flexible transform architecture is also proposed for effective computation of all transforms needed in H.264/AVC decoders. Since all the transforms are realized in the same architecture, the flexible transform design with the throughput of 8 pixels/sec needs fewer logic gate counts. Simulation results show that the implemented gate count is 18.6k and the maximum operating frequency is 125 MHz. For real-time requirements, this proposed design achieves 4VGA (1280times960)@30 frames/sec in the worst case.

In H.264 advanced video coding (AVC), the inter prediction with variable block sizes plays an important role to achieve compression efficiency. Although it actually raises the coding efficiency in H.264, it also increases the complexity due to the decision of the best inter mode. In this paper, we will propose a fast algorithm based on neighboring information and homogeneous condition to select the best mode effectively. For each macroblock, we first select the initial mode by using probability table and then use homogeneous condition analysis with motion vector relation to accelerate the inter mode decision process. Experimental results show that our proposed algorithm can save about 67~70% mode computation with negligible PSNR loss, which is very helpful for H.264 interframe coding.

This investigation describes a novel design and implementation of an interactive multimedia mirror system, called "Magic Mirror." The Magic Mirror can be easily implemented in existing personal computers or hand-held device with normal peripherals and regular reflective glass by integrating image/speech processing, Internet connectivity, and 3D and multimedia software. The integrated Magic Mirror, which includes speech recognition, speech synthesis, face detection/modified/recognition, 3D virtual genius, hidden LCD mirror, and camera, performs simple syndication to capture information about peripherals and network connections. The user can easily activate personal multimedia services using verbal commands. The Magic Mirror can function like a good friend who listens to the user's questions and automatically responds to these requests, providing relaxation and consolation. Moreover, the Magic Mirror can detect a user's feeling based on speech and image recognition features to select the appropriate music and speech to alter the user's mood.

The rate-distortion (R-D) optimization technique plays an important role in optimizing video encoders. Modeling the rate and distortion functions accurately in acceptable complexity helps to make the optimization more practical. In this paper, we propose a bit-rate estimation function and a distortion measure by modeling the transform coefficients with spatial-domain variance. Furthermore, with quantization-based thresholding to determine the number, the absolute sum, and the squared sum of transform coefficients, the simplified transform-domain R-D measurement is introduced. The proposed algorithms can reduce the computation complexity for the R-D optimized mode-decision by using the new cost function evolving from the simplified transform-domain R-D model. Based on the proposed estimations, a rate-control scheme in the macroblock layer is also proposed to improve the coding efficiency

In this paper, we present a fast mode decision algorithm and design its VLSI architecture for H.264 intra-prediction. A regular spatial domain filtering technique is proposed to compute the dominant edge strength (DES) to reduce the possible predictive modes. Experimental results revealed that the proposed fast intra-algorithm reduces 40% computation with slight peak signal-to-noise ratio (PSNR) degradation. The designed DES VLSI engine comprises a zigzag converter, a DES finite-state machine (FSM), and a DES core. The former two units handle memory allocation and control flow while the last performs pseudoblock computation, edge filtering, and dominant edge strength extraction. With semicustom design fabricated by 0.18 mum CMOS single-poly-six-metal technology, the realized die size is roughly 0.15 times 0.15 mm2and can be operated at 66 MHz.

This paper presents an intelligent guiding bulletin board system (iGBBS), which is based on vision-interactive and multiple key word-spotting technology. The system is aimed to provide different kinds of multimedia human-computer interaction (MMHCI) for users under different requirements. At first, a real-time front-view face detection using Harr-like features is used to decide when iGBBS should wake up and become interactive with the user. After system initialization, some feature points within the detected face area are going to be found. Then the orientation of user's head will be estimated via pyramidal Lucas-Kanade optical flow tracking. In addition, spotting the keyword from user's utterance with some related augmented reality responses would be provided as well. The performance of vision-interaction in iGBBS could be reached to 20 fps under Pentium IV 1G Hz PC. The error rate of multiple key word-spotting interaction in iGBBS is about 36.2% and people can get the right response in 2.76 times search averagely. With the comparison to the traditional guiding system, bulletin board, or other non-vision-based input devices system, such like gloves or markers, our system offers a simple, useful and economical solution for the realtime interaction between the user and computer

With reducing computational complexity, an approximated correlation matrix of the vocal impulse response is proposed in algebraic-code-excited linear prediction (ACELP) coders. By exploring statistical characteristics, we only need to calculate a small portion of correlation coefficients before ACELP search procedure. If we further combine a pulse position prediction algorithm, we can reduce the arithmetic complexity in pre-computing autocorrelation matrix and the number of pulse position combinations with imperceptible degradation in speech quality performance. The proposed scheme can be applied to all ACELP coders such as ITU G.729 and G.723.1

In this paper, an efficient cost function for mode decision in H.264/AVC is proposed. The proposed cost function is based on integer transform coefficients, where the rate and the distortion are jointly modeled by the number of nonzero quantized coefficients, the sum of absolute integer transformed differences (SAITD) and sum of squared integer transformed differences (SSITD). Comparing to the high-complexity cost function, which should be calculated from real bit-consumption and true reconstructed distortion for each coding mode, the proposed efficient cost function can achieve 79.93% and 22.61% time savings of computing rate-distortion cost and overall encoding, respectively, while introducing only slight degradation with 1.05% bit-rate increment and 0.049 dB PSNR drop

In H.264 advanced video coding (AVC), variable block size motion estimation plays an important role in compression of interframes. In this paper, we propose a fast inter prediction algorithm based on hierarchical homogeneous detection and cost analysis to select the best mode effectively. For each macroblock, we first detect that whether the macroblock is spatial homogeneous or not. For the non spatial homogeneous macroblock, we then perform the 16times16 motion estimation and examine if the 16times16 block is temporal homogeneous or not. Once the homogeneous macroblock is detected in the above process, the best mode will be chosen as 16times16 mode. For the non-homogeneous macroblock, we then execute 8times8 motion estimation and analyze the cost of 8times8 mode and 16times16 mode for deciding the best inter mode should be 16times16 mode or any other mode. The process for searching the best 8times8 block subtype is similar to the process for macroblocks. Finally, the best inter mode is decided by selecting the inter mode with least cost from the candidate modes. Experimental results show that our proposed algorithm can save about 32~54% computation time without introducing any noticeable performance degradation

This paper presents a real-time vision-interactive guiding system, which could be interactive with users based on the computer vision technology. A real-time front-view face detection using Harr-like features is used to decide when the guiding system should wake up and become interactive with the user. After system initialization, some feature points within the detected face area are going to be found. Then the orientation of user's head will be estimated via pyramidal Lucas-Kanade optical flow tracking. Compared to the traditional guiding system, our system has more flexibility. Guiding providers could choose a suitable display equipment according to the surround environment. With the comparison to other non-vision-based input devices system, such like gloves or markers, our system offers a simple, useful and economical solution for the real-time interaction between the user and computer.

In this paper, a low power architecture for realizing the CAVLC decoder is proposed. In traditional VLC decoding algorithms, we could search a level in Huffman coding tree per operation. Therefore, the throughput rate is limited by the searching level. The CAVLC algorithm takes the advantage of the trend among AC coefficients in each block to predict the next codeword. The prediction mechanism can significantly improve the decoding efficiency. Hence, we suggested two efficient approaches, table partitioning and prefix predecoding, to reduce the power consumption in decoding the VLC codes. The proposed low-power CAVLD architecture achieves the real-time requirement for 720p HD (1280/spl times/720) format, while the clock is operated at 125 MHz. In simulations, the proposed architecture can reduce about 25% of power consumption in comparison to its counterpart without low power design.

In this paper, an efficient in-loop deblocking filter architecture for H.264 video coding system is proposed. The deblocking filter usually need perform both vertical and horizontal directions. A novel data arrangement, called the group-of-pixel (GOP), is designed to efficiently arrange the pixel data stored in on-chip memory. With the proposed GOP arrangement, we do not need the transpose memory, which often occupies excessive chip area, to transpose the direction of the pixel data filtering. Furthermore, the number of total cycles required for GOP-based deblocking filter is reduced significantly. The proposed in-loop GOP-based deblocking filter architecture synthesized with UMC 0.18 /spl mu/m technology could process real-time video in 720p HD (1280/spl times/720) format operated at 100 MHz.

Based on statistical analyses of transform coefficients, we suggest estimations of bit-rate and distortion functions directly from spatial-domain statistics. Simulations show that the proposed rate and distortion estimators achieve similar results as the true rate-quantization (R-Q) and distortion-quantization (D-Q) curves. By exploiting the proposed statistical estimations, two major applications are suggested. One is the fast rate-distortion optimized (RDO) mode decision and the other is the improved rate control (RC) in video encoding. Simulation results show that the rate-distortion estimators have sufficient accuracy such that we could achieve considerably better performance in both applications. We achieved 51.69% R-D cost computation time reduction in average for the rate-distortion optimized mode decision, and a 0.20 dB average peak signal-to-noise ratio (PSNR) improvement for rate-control in H.264/AVC, respectively.

In this paper, we propose an enhanced rate-distortion cost function, which combines the sum of absolute integer-transformed differences (SAITD) and a rate predictor for H.264/AVC intra-4times4 mode decision. To reduce the computation of the SAITD, we further develop a fast computation algorithm, which successfully uses the property of linear transform and the fixed spatial relationship of predicted pixels in intra-modes. Simulation results show that the enhanced cost function with less computation achieves better coding performance than the cost function suggested in H.264 reference software JM 6.1d

In video coders, the optimal coding mode decision for each coding block can be achieved by exhaustively calculating the rate-distortion cost, which simultaneously considers the distortion performance and the coding bit consumption of all possible modes. The best mode is chosen from the one with the minimum Lagrange cost. To avoid the expensive computation of Lagrange costs, in this paper, we propose transform-domain bit-rate estimation and distortion measures, based on quantized and inverse quantized integer transform coefficients, for the inter-mode decision in H.264/AVC coders. With the proposed scheme, entropy coding, inverse transform, and pixel-reconstructions are not required in the process. With ignorable degradation in coding performance, simulations demonstrate that the proposed estimation method achieves about 40% reduced computation time of rate-distortion cost for the inter-mode decision and saves about 17% total encoding time while combining with fast motion estimation and fast mode decision algorithms.

In a video coder, the optimal coding mode decision for each coding block could be achieved by exhaustively calculating the Lagrange cost (which includes the coding distortion plus the Lagrange parameter times the coding bit consumption) of all possible modes. The best mode can then be chosen as the one with the minimum Lagrange cost. To speed up the computationally intensive Lagrange cost computation, in this paper, we propose transform-domain bit-rate estimation and distortion measures for the inter-mode decision in H.264/AVC coders. With the proposed scheme, entropy coding, inverse DCT, and pixel-reconstructions are not required in the process. Simulation results show that the proposed estimation method is accurate for the inter-mode decision and about 46.42% time reduction can be achieved.

This paper presents a discrete-cosine transform (DCT)-based adaptive thresholding algorithm to achieve an effective binarization of video image for binary motion estimation (BME). The BME through the full search mechanism requires extremely low hardware complexity. Experimental results show that the proposed adaptive BME attains performance close to the 8-bit resolution ME for low motion sequences with full search strategy. Compared to the well-known single-layer BME, the proposed adaptive algorithm shows better peak signal-to-noise ratio performance and more robustness to complex images. Moreover, the proposed algorithm has a lower computational complexity that is suitable for VLSI implementation with a smaller hardware overhead.

The parametric coders provide a good communication quality at low bit rate. Efficient encoding of variable dimension harmonic spectral envelope is an essential task in parametric speech coders. In this paper, we propose an efficient vector quantization (VQ) scheme with perception consideration to improve the performance of parametric speech coders. With the benefit of reduction in dimension, the computational complexity of spectral envelope VQ (SEVQ) has been reduced while the speech quality is retained. Experimental results show that the proposed perceptual SEVQ method significantly reduces the computational complexity of SEVQ by a factor of 9.

A merging procedure for variable-size block motion estimation is proposed to reduce the computation for block-size decision. A smaller block-size is initially used for motion estimation. An adaptive threshold, which depends on the information obtained from the motion estimation, quantization parameter, and rate distortion cost function is used to determine if the motion vectors of the neighboring blocks should be merged or not. We simulate the proposed merging procedure using an H.264 video encoder to determine if 16x16, 16x8, 8x16, or 8x8 blocksize should be used for each macroblock. Based on the simulations, performance of the proposed method is close to that of H.264 JM2.1 when 16x16, 16x8, 8x16 and 8x8 block modes are enabled and the exhaustive search method is used to determine the block-sizes. Computational complexity can be significantly reduced by the proposed algorithm since it only performs the motion search for one block-size.

In this paper, an adaptive postprocessor figured with discrete cosine transform (DCT)-based block classification to effectively remove the so-called blocky effect from compressed video sequences is proposed. The proposed DCT-based detection algorithms for both intraframes and interframes require much lower computation complexity than the spatial-domain approaches. In order to preserve the edge information, the adaptive postprocessor is also designed with a DCT-based edge detection mechanism such that a one-dimensional median filter can be adaptively adjusted to match with the edge orientation. Simulation results show that the proposed DCT-based detection algorithms accurately classifies smooth, edge, or nonsmooth blocks to help the adaptive postprocessor to effectively remove the blocky effect and sharply preserve the edge information.

In this paper, computation reduction algorithms of motion estimation are developed for low rate video coders. By jointly considering motion estimation, discrete cosine transform, and quantization, the all-zero and zero-motion detection algorithms are suggested to reduce the computation of motion estimation. Simulation results show that many unnecessary computations in motion estimation are greatly reduced. For the Akiyo sequence, the proposed algorithms reduce the average numbers of search points from 28.90 and 4.99 to 2.89 and 1.40 for three-step search and diamond-search algorithms, respectively.

The ACELP method makes use of multipulse structure to represent the excitation pulses of residual signal. With the purpose of computational complexity reduction, this paper provides the maximum-take-precedence ACELP (MTP-ACELP) search method under the acceptable degradation in performance. Because the maximum of target signal is preferentially compensated, the degradation of performance would be diminished. By predicting the locations of pulses, the computational complexity would be reduced. We not only reduce the possible pulse combinations in the search procedure but also avoid the computation of useless correlation functions before the search procedure. Furthermore, the proposed method is compatible to any ACELP type vocoder, e.g. the G.723.1, G.729, GSM- EFR standards.

In this paper, we propose a connected-index approach to construct efficient finite-length arithmetic codes by splitting the information of the last nonfitting symbol into the current and subsequent codewords. The proposed arithmetic codes, which limit the error propagation in about one block, require neither a post-appended end-of-block symbol, nor pre-affixed side-information, to characterize the number of encoded symbols. Hence, the proposed finite-length arithmetic codes can nearly achieve the coding efficiency attained by infinite-length arithmetic codes. With high coding efficiency, limited error-propagation, and the regular process, the proposed coding approach is suitable for information exchange with small packets in modern high-speed network systems.

In this paper, a novel voice-driven adaptive packet loss recovery algorithm is proposed to lessen the possible voice degradation and error propagation for analysis-by-synthesis speech coders in Internet applications. After voicing classification, we adaptively adopt random noise generation, multiresolution excitation generation, or pulse tracking procedure to recover the lost packets, By applying the algorithm to the G.723.1 coder, simulation results show that the proposed algorithm is superior to the recovery algorithm embedded in the G.723.1 standard through the subjective evaluation.

In this paper, we propose an eigen-based fuzzy C-means (FCM) method for color object segmentation. After sampling a few color samples, we can form the sampled covariance matrix and its related eigenvectors of the desired color space. Then, we transform the original color space into signal and noise planes of the desired color. Followed the transformation, the proposed eigen-based FCM algorithm is finally applied to the signal and noise subspaces individually. After few iterated classification processes, the desired color objects can be easily identified without using any threshold procedure. Inspecting the segmented results, the desired color objects without any pre- and post-processes can be extracted easily and robustly.

In this paper, we propose generalized layer winner-take-all (WTA) neural networks based on the suggested full WTA networks, which can be extended from any existing WTA structure with a simple weighted-and-sum neuron. With modular regularity and local connection, the layer WTA network in either hierarchical or recursive structure is suitable for a large number of competitors. The complexity and convergence performances of layer and direct WTA neural networks are analyzed. Simulation results and theoretical analyzes verify that the layer WTA neural networks with extendibility outperform their original direct WTA structures in aspects of low complexity and fast convergence.

In this paper, we propose transform-domain algorithms to effectively classify the characteristics of blocks and estimate the strength of the blocky effect. The transform-domain algorithms require much lower computational complexity and much less memory than the spatial ones. Along with the estimated blocky strength, we also propose an adaptive finite impulse response (FIR) filter to effectively remove the blocky effect. Simulation results show that the proposed algorithms sharply preserve the edge information and greatly improve the decoded image quality.

In this paper, a dynamic K-winners-take-all (KWTA) neural network, which can quickly identify the K-winning neurons whose activations are larger than the remaining ones, is proposed and analyzed. For N competitors, the proposed KWTA network is composed of N feedforward hardlimit neurons and three feedback neurons, which are used to determine the dynamic threshold. From theoretical analysis and simulation results, we found that the convergence of the proposed KWTA network, which requires Log(2)(N+1) iterations in average to complete a KWTA process, is independent of K, the number of the desired winners, and faster than that of the existing KWTA networks.

We first analyze the resolution performance of the Gerschgorin radii based source number detector proposed by Wu, Yang and Chen (see IEEE Trans. SP, vol.43, no.6, p.1325-33, 1995) for independent closely-spaced plane waves. Based upon the analysis, we can easily verify the resolution threshold of the Gerschgorin radii based algorithm. For improvement of the detection performance, we then further propose a weighted Gerschgorin radii algorithm. With closed-form expressions and simulations, we find that the analyzed performance in terms of the number of sensors, the relative angular separation of emitters, and the signal-to-noise ratios can properly characterize their behaviour for source number detection.

For cosine-modulated multirate subband filtering, we propose two implementation algorithms to reduce the computational complexity. First, we develop a recursive algorithm to further decrease the computation of polyphase subband filtering suggested by Nussbaumer and Vetterli (1984). Since the cosine modulation can be implemented by the discrete cosine transform (DCT), we combine both fast decimation-in-time (DIT) and decimation-in-frequency (DIF) DCT methods to further reduce the computation. Consequently, the recursive formula with the mixed fast-DCT method requires about 13.2% of the multiplications and 41.6% of the additions recommended by the ISO Std. ISO-IEC JTCI/SC2/WGI1.

The 16/9 widescreen TV is becoming popular in the marketplace; however, its aspect ratio is not compatible with the conventional 4/3 SDTV program. We propose a nonuniform conversion to efficiently display 4/3 SDTV programs on a 16/9 high-definition television (HDTV) monitor, for real time HDTV receivers, a parallel video converter is also proposed using nonuniform interpolation. To trade off complexity and performance, we only use one field-memory and one line-buffer for motion adaptation to interpolate the pixels, which can be finished by the parallel computation kernel in one clock cycle. In order to combine parallel computation, the memory-slice method is employed, which is efficiently controlled by a state machine for nonuniform interpolation. With parallel conversion, a high-speed video converter can be achieved for advanced HDTV receivers.

We propose the pipelined VLSI architecture and modular design to realize the coefficient-by-coefficient two-dimensional inverse discrete cosine transform (2-D IDCT) suggested by Yang, Bel, and Hisa (see ibid., vol.5, no.1, p.25-30, 1995). Based on parallel processing, the architecture of this chip is designed with a five-stage pipeline to meet the speed requirement for real-time applications. The key building modules of this chip include a generator of cosine angle index, a pipelined multiplier, and a matrix accumulator core. Satisfying the IEEE standard of 2-D IDCT in computational accuracy, this IDCT chip, which can work at a clock rate of higher than 50 MHz, is implemented by the CMOS technology in a reasonable die size. With modular and regular structures, the IDCT VLSI chip can be operated in a progressive transform mode. In a real video decoding system, the average pixel-rate of the proposed 2-D IDCT chip achieves over 150 MHz for decoding intraframes and up to 400 MHz for decoding interframes.>

In this paper, we propose an efficient two-dimensional inverse discrete cosine transform (2-D IDCT) algorithm based upon coefficient-by-coefficient implementations. The algorithm, which is developed in the use of the symmetrical properties of computational kernel matrices, can effectively combine with the run length code (RLC) decoder in progressive manners. The proposed method requires only N/2 multipliers in an 'N/spl times/N' 2-D IDCT processor. By evaluating the efficiency performance, the processor can complete 2N/M pixels per multiplier per system cycle where M is the number of non-zero quantized DCT coefficients. Due to its inherent parallel and coefficient-by-coefficient characteristics, the proposed efficient 2-D IDCT algorithm is suitable for the future digital HDTV receivers and recording systems.>

We propose a fast and low roundoff-noise implementation of alias-free quadrature mirror filters (QMFs) for tree-structured subband coding. When N-tap FIR QMFs are directly utilized, they require 2N multiplications and 2(N-1) additions to obtain a pair of high-band and low-band decimated outputs. Using recursive formulas and the polyphase concept, a QMF implementation is proposed that, for both analysis and synthesis stages, needs only (N/2+2) multiplications and (3N/2+8) additions to compute the same outputs. Furthermore, the proposed algorithm can reduce roundoff error when finite-wordlength processors are used to implement the encoder and decoder. Simulation examples for image coding are also given to demonstrate the differences between the proposed and previously-known algorithms.

The combination of singular value decomposition (SVD) and vector quantization (VQ) is proposed as a compression technique to achieve low bit rate and high quality image coding. Given a codebook consisting of singular vectors, two algorithms, which find the best-fit candidates without involving the complicated SVD computation, are described. Simulation results show that the proposed methods are better than the discrete cosine transform (DCT) in terms of energy compaction, data rate, image quality, and decoding complexity.

In this paper, a new iterative winner-take-all (WTA) neural network is developed and analyzed. The proposed WTA neural net with one-layer structure is established under the concept of the statistical mean. For three typical distributions of initial activations, the convergence behaviors of the existing and the proposed WTA neural nets are evaluated by theoretical analyses and Monte Carlo simulations. We found that the suggested WTA neural network on average requires fewer than Log(2)M iterations to complete a WTA process for the three distributed inputs, where M is the number of competitors. Furthermore, the fault tolerances of the iterative WTA nets are analyzed and simulated. From the view points of convergence speed, hardware complexity, and robustness to the errors, the proposed WTA is suitable for various applications.

The eigenstructure based estimator designed to be used with the aid of the Gerschgorin's disk theorem is proposed for source number detection. By introducing the unitary transformation of the covariance matrix, the Gerschgorin radii of the eigenstructure are exploited to determine the number of sources while overcoming a lack of data samples, noise model and data independency information. Unlike conventional methods such as Akaike information criterion (AIC) and minimum descriptive length criterion (MDL), which are based on the cluster analysis of the eigenvalues used in conjunction with statistical formulations, the proposed method called the Gerschgorin disk estimator (GDE), provide more accurate detection of the source number in situations of both simulated and measured experimental data.>

A parallel Rayleigh quotient iterative algorithm (PRQI) associated with the zeros extraction technique which assures that each processor converges to a different desired root is proposed. The suggested algorithm with arbitrary initialization can automatically converge to the desired roots which are close to the unit circle. The proposed algorithm has a computation complexity of O(N) for rooting spectrum polynomials. Simulations show that the suggested algorithm has a better tracking performance than the Gauss-Newton method and the gradient Newton algorithm.>

A robust and high-resolution adaptive eigensubspace algorithm which generalizes the inflation method is proposed. The convergence and initialization behavior of the algorithm and its parallelization surpass those of the Pisarenko harmonic retrieval (PHR) algorithm and other adaptive algorithms, if the number of sources is not known. The algorithm is suitable for the adaptive processing needed for tracking nonstationary signal parameter.>