In this paper, new motion estimation (ME) and motion vector refinement (MVR) methods for motion-compensated frame interpolation (MCFI) are proposed. To estimate motion vectors (MVs) with great reliability, triple-frame-based bi-directional ME (TFBME), which employs three sequential frames, is developed. In TFBME, the middle frame works as a reference to estimate MVs between two end frames, which leads to performance improvement especially when objects move linearly through three sequential frames. In addition, a modified bi-directional ME method, which employs only two successive frames, is combined with TFBME to handle cases in which objects move non-linearly through three sequential frames. In our proposed MVR, artifacts on interpolated frames are first detected by a convolutional neural network and then corrected by weighted vector median filtering. Experimental results show that the proposed MCFI method outperforms conventional methods in both subjective and objective evaluations.

This paper proposes a new metric to measure judder artifacts of video sequences. The judder artifacts appear as non-smooth motions in hold-type displays when the frame rate is low and object motion is fast. To analyze the judder artifacts in video sequences, the proposed judder metric is defined by analyzing the effects and cross-relations of judder features in the video sequences. The judder features include spatial features of image gradients and temporal features of motion vectors and the frame rate. In addition, sensitivity of the human visual system (HVS) is considered to determine the perceptual judder artifacts because it has special characteristics of sensitivity to image brightness and contrast. Therefore, a sensitivity map of the HVS is used to mask the judder artifacts. Then, the degree of perceptual judder artifacts is estimated using the judder features and a regression model. The experimental results demonstrate that the proposed judder metric is highly correlated with the subjective assessment results.

In this paper, we propose a new motion compensated frame interpolation (MCFI) method for occlusion and motion ambiguity regions of frame rate up-conversion (FRUC) assisted by depth information in color-plus-depth video. First, we estimate motion vectors in a similar way to conventional methods and refine them using depth information. In the intermediate frame, we try to solve challenging FRUC problems such as occlusion, boundary, and motion ambiguity using depth information. Finally, interpolation is performed to generate intermediate frames. Experiments were performed to compare the proposed method and conventional color and color-plus-depth FRUC methods. The experiment result shows that the proposed method using depth information can improve the FRUC performance.

The motion-compensated frame interpolation (MCFI) methods usually use block matching algorithms (BMAs) for motion estimation (ME). However, the conventional BMAs that are originally developed by minimizing the prediction errors often fail to project the object motion. In this paper, we present a new MCFI method that utilizes a convolutional neural network (CNN) to find the motion vector (MV) with reliability. The CNN model which is used to estimate MVs is trained to track the projected object motion as closely as possible. Experimental results using the standard test video sequences show that our proposed ME method acquired more reliable MVs than conventional ME methods. Furthermore, our proposed MCFI method improves the average peak signal-to-noise ratio (PSNR) of interpolated frames.

Recent advances on high frame-rate (HFR) hold-type display systems have resulted in the increased demands for HFR contents. Frame-rate up conversion (FRUC) technique could be used for creating HFR contents from the existing low frame-rate (LFR) contents. However, the judder artifacts caused by FRUC would degrade the quality of HFR. The aim of this work is to analyze the cause of judder in the reconstructed HFR video from FRUC, and to propose a new metric to evaluate judder. A judder measurement method using subjective assessments is presented. As the prior study defines judder as the differences between the natural motion and result from the sampled motion of video, the proposed metric quantifies the judder by measuring the differences in retinal space. Experimental results demonstrate that the proposed judder determination method is consistent with the subjective assessment results.

High Efficiency Video Coding (HEVC) has brought significant improvement to video compression in terms of coding performance and subjective quality. However, it is computationally more complex than H.264/Advanced Video Coding (AVC) and, hence, adaptation of HEVC to real-time video services has not been promising. In order to reduce the computational cost, we propose a fast mode decision method that is based on the adaptive ordering of modes. Our experimental results show that the proposed method reduces the total encoding time by 47.27%-60.73% and 47.65%-61.31%, with negligible degradation of coding performance, which is a Bjøntegaard delta rate increment of 0.35%-1.04% and 0.64%-1.80% under random-access configuration and low-delay configuration, respectively, compared with HEVC test model 12.1 with fast mode off.

Moving target detection is an important technique in visual surveillance systems. If a camera is freely moving, it becomes more difficult to detect a moving target, especially in the environment of a wide-range background. To compensate for the global motion of a wide-range background, a disparity-based adaptive multi-homography method is proposed. The proposed method comprises four steps: 1) feature point extraction; 2) generation of adaptive multi-homography matrices using motion grouping; 3) generation of background model and detection of background; and 4) target detection. Experimental results show that the proposed method can robustly detect moving targets in sequences taken by a freely moving camera.

A multiview video plus depth format was introduced to support 3D depth perception and arbitrary view generation. To achieve high coding performance in AVC-compatible 3D video coding (3D-AVC), we herein propose an efficient depth map coding method that includes synthesis-based depth residual coding and filtering. The synthesis-based depth residual coding method adaptively encodes a residual by predicting synthesis distortion from spatial complexity of the corresponding texture, in a nonnormative way. In order to compensate for possible error from the proposed coding method, synthesis-based depth filtering is additionally conducted on the reconstructed depth map. Experimental results demonstrate that the proposed method reduces the bit rates by 4.4% and 4.8%, compared with the original method for the decoded and synthesized PSNRs, respectively, while satisfying the compatibility with the current 3D-AVC.

Since thermal images represent only the temperature difference between objects and background and they have more blurred edges than color images, the segmented images from them have noisy object boundaries. Therefore, the well-known features developed for color images may not work well with thermal images. To overcome these limitations of thermal images, we propose a novel feature extraction method based on the target trait context (TTC). A robust keypoint detector is also proposed by analyzing the included angle moments of the object boundary points. At each keypoint, we define the boundary shape context and the normalized intensity context, which contain shape information and thermal distribution, respectively, of the object. These two contexts are combined to form a new feature set, the TTC. To validate our proposed feature extraction method, the keypoint repeatability test and the classification performance test were performed and compared with those from the previous methods. The experiment results show that the proposed feature works well for thermal video sequences and outperforms the previous methods in classification performance.

This brief proposes a new motion-compensated frame interpolation (MCFI) method, and it generates new intermediate frames from existing original frames. The proposed method partitions the original frames into arbitrary-shaped regions and estimates motions for the regions. The region shapes were initially determined by pixel intensity and iteratively refined on the basis of a variance-distortion curve. The experimental results show that the proposed method more accurately interpolates frames than previous MCFI methods.

The reconstruction of a dense depth map from sparsely sampled depth information has been studied to improve stereoscopic 3-D images. Among various sparse-to-dense depth map reconstruction methods, triangular-patch-based approaches have been widely used. However, the triangular-patch-based approaches suffer from a limitation in preserving depth discontinuities, because they assume that each triangular patch has a planar depth map. This paper presents a novel depth map rasterization method that considers the depth discontinuities and color consistencies in each triangular patch. The results of the experiment show that the proposed depth map rasterization method reconstructs dense depth maps with higher accuracy than the previous depth map upsampling methods for both regular and irregular depth map sampling structures.

The skip mode has been adopted to reduce the bitrate in hybrid video coding such as H.264/MPEG-4 Advanced Video Coding (AVC). To improve the coding performance by reducing the dc distortion in the skip mode, we propose a skip with offset method, which uses offset compensation with the estimated offset only in the skip mode. In the proposed method, no flag bit is used for offset because incorrectly estimated offsets are mostly filtered out through the rate distortion mode decision. Only a semantic change is needed at the skip mode in the proposed method. The existing skip mode can therefore be easily replaced with the proposed method. The proposed method shows better coding performance than previous methods, including the weighted prediction method, in H.264/AVC. Furthermore, the proposed method shows a significant coding gain of -7.60% and -5.81% in its averaged bitrate differences compared, respectively, with baseline and high profiles of H.264/AVC at 720p high definition sequences.

Various techniques for single-view to multi-view video conversion have been developed to visualize existing 2D content on 3D displays. Because the quality of the generated multi-view video is highly dependent on the accuracy of the applied depth information, obtaining reliable depth information from a single-view sequence is an important process. This paper presents an efficient method based on 3D affine geometry information to obtain depth information and to generate horizontal parallax images from a single-view sequence. In the proposed method, positions of the feature points in each horizontal parallax image plane are estimated using a computed infinite homography. The infinite homography is then updated to be applied to subsequent frames and to maintain consistent baseline distances between the original and the generated horizontal parallax images. Experiment results show that the proposed method has advantages over previous metric-structure-from-motion-based approaches that require 3D affine geometry information and additional internal camera parameters.

We introduce a new objective assessment method for visual discomfort of stereoscopic images that makes effective use of the human visual attention model. The proposed method takes into account visual importance regions that play an important role in determining the overall degree of visual discomfort of a stereoscopic image. After obtaining a saliency-based visual importance map for an image, perceptually significant disparity features are extracted to predict the overall degree of visual discomfort. Experimental results show that the proposed method can achieve significantly higher prediction accuracy than the state-of-the-art methods.

This paper proposes a new motion-compensated frame interpolation (MCFI) method. The performance of MCFI is highly dependent on the accuracy of the estimated motion vectors (MVs). The proposed iterative motion estimation (ME) method estimates reliable MVs using linearity checking and restricted search areas. An adaptive interpolation method is also proposed to obtain MVs for blocks that do not have reliable MVs in the iterative ME process. The experimental results show that the proposed method outperforms the conventional MCFI methods in terms of the quality of the interpolated frames. Furthermore, the MVs from the proposed method are more accurate than those from the conventional methods.

This paper proposes a new motion vector (MV) regularization method. We present an optimization method of the tradeoff between motion accuracy and regularity. The proposed method adopts Lagrangian multiplier and greedy algorithm for optimization. By smoothing MVs while preserving motion accuracy, the proposed method can more efficiently regularize MVs than previous regularization methods. The experimental results show that the proposed method outperforms previous regularization methods by generating objectively and subjectively better results.

This paper proposes a new frame rate up-conversion (FRUC) method. In the paper, we propose an overlap ratio between motion vectors, with which we determine adaptive block size for the motion estimation. In addition, the proposed method estimates the region of repeated patterns and treats them in a special manner. Conventional motion estimation methods usually produce wrong motion vectors for the repeated patterns. The experimental results show that the proposed method outperforms other FRUC methods in terms of generating better interpolated frames.

The H.264/AVC video coding standard can achieve higher compression performance than previous video coding standards, such as MPEG-2, MPEG-4, and H.263. In order to improve the coding performance, the H.264/AVC encoder employs various modes and the rate-distortion (RD) optimization method for selection of an optimum mode. Since the encoder computes RD costs of all possible coding modes to decide the optimum mode, it induces high computational complexity. In this paper, we introduce a fast mode decision method for inter picture macroblocks to reduce the computational complexity. The proposed method significantly reduces the number of candidate modes for the RD optimization process by detecting spatially and temporally homogeneous regions and analyzing motion costs for inter modes and intra prediction costs for intra modes. Experimental results demonstrate that the proposed method drastically reduces the encoding time and the number of the RD cost calculation process while maintaining the high coding efficiency.

This paper proposes a new ringing-artifact reduction method for image resizing in a block discrete cosine transform (DCT) domain. The proposed method reduces ringing artifacts without further blurring, whereas previous approaches must find a compromise between blurring and ringing artifacts. The proposed method consists of DCT-domain filtering and image-domain post-processing, which reduces ripples on smooth regions as well as overshoot near strong edges. By generating a mask map of the overshoot regions, we combine a ripple-reduced image and an overshoot-reduced image according to the mask map in the image domain to obtain a ringing-artifact reduced image. The experimental results show that the proposed method is computationally faster and produces visually finer images than previous ringing-artifact reduction approaches.

This paper proposes a new motion-compensated frame interpolation (MCFI) method. The proposed method utilizes a symmetric motion estimation (SME) method, which is a new pixel-wise motion estimation method for intermediate frame interpolation. By using an adaptive search range for the motion estimation, the proposed method can obtain a more reliable motion vector for each pixel than previous MCFI methods that use a conventional block matching algorithm (BMA). In addition, we propose a combined method of the SME and BMA to reduce the computation time of the pixel-wise motion estimation method. The experimental results show that the proposed method outperforms other MCFI methods in terms of generating objectively and subjectively better interpolated frames.

Recently, a new video coding technique, distributed video coding (DVC), is an emerging research area for low power video coding applications. In the DVC, the encoder is much simpler than the conventional video codec, whereas the decoder is very heavy. The DVC decoder exploits side information which is generated by motion compensated frame interpolation to reconstruct the Wyner-Ziv frame. This paper proposes an efficient side information generation algorithm using seed blocks for DVC. Seed blocks are firstly selected to be used for motion estimation of the other blocks. As the side information is close to the target image, the final reconstructed image in the DVC decoder has better quality and the compression ratio becomes high. The proposed method contributes to improve the DVC compression performance with reduced computing time. Experimental results show that accurate motion vectors are estimated by the proposed method and its computational complexity of motion estimation is significantly reduced in comparison with the previous methods.

The H.264/AVC achieves higher coding performance than the previous video coding standards, such as MPEG-2 and MPEG-4. In order to achieve this improved coding performance, H.264 adopts various advanced techniques including multiple reference frame motion estimation (MRF-ME), which requires very heavy encoding complexity in proportion to the number of reference frames. In this letter, we propose an efficient priority-based reference frame selection (PRFS) method to reduce the computational complexity. The PRFS method computes the priorities of all reference frames using spatial and temporal correlation of the reference frame index and motion vectors, and selects candidate reference frames to reduce the number of reference frames for each ME block. Experimental results show that the proposed method reduces the computational complexity of the MRF-ME and achieves similar coding performance to that of previous methods.

A new intra-frame coding method based on H.264/AVC is proposed. In the proposed method, every other pixel is sampled either in a horizontal or vertical direction without any anti-aliasing filtering, and two subframes of the even-pixel and odd-pixel are then generated. The even-pixel subframe is coded as a conventional intra frame whereas the other is coded by utilizing the reconstructed information of the even-pixel subframe. The experimental results show that our proposed scheme outperforms the conventional intra-frame coding in H.264/AVC and reduces the bit rate by 11.67% on average at the baseline profile, and by 4.53% at the high profile of H.264/AVC.

In this paper, we propose and object tracking algorithm to track an object on IR image sequences, which has appearance changes. In our framework based on particle filter, histograms of intensity and orientation on scale space are used as observation models. Histogram of intensity is robust to shape variation but lack of distinction between target and similar background due to loss of spatial information. Histogram of gradient orientation complements the spatial information. Because the histogram models are extracted from scale space, target model maintains consistency with initial model in different scales. We also propose an efficient method of model update and demonstrate the proposed method for several sequences in which a target changes largely in size and appearance.

The H.264/AVC achieves higher coding efficiency than the previous video coding standards such as MPEG-4 and H.263. H.264 adopted the multiple reference-frame motion estimation (MRF-ME). Although this new technique significantly improves the prediction accuracy, it requires very heavy encoding complexity in proportion to the number of reference frames. In this paper, we propose an adaptive reference-frame selection (ARFS) method using neighboring reference-frame information and Bayes theory. We show that the proposed algorithm effectively reduces the encoding complexity of MRF-ME and achieves similar coding gain, compared to the previous methods.

The H.264/AVC video coding standard can achieve higher coding efficiency than the existing video coding standards. In order to improve the coding performance, the encoder employs the rate-distortion (RD) optimization technique to select an optimum mode for each macroblock. Since it finds the mode having the minimum RD cost among seven possible modes, it requires exhaustive computations. In this paper, we introduce an efficient and fast mode decision method to reduce computational complexity. Experimental results show that the proposed method extremely reduces encoding time with only negligible coding loss.

A new intra-frame coding scheme is proposed for progressive video sequences in H.264/AVC. In the proposed method, every other pixel is sampled either in a horizontal or vertical direction without any anti-aliasing filtering, and two sub-frames of the even-pixel and odd-pixel are then generated. The even-pixel and odd-pixel sub-frames are coded by the intra-and inter-frame coding methods, respectively. Our proposed method can be expanded into a multi-level case in which the even-pixel sub-frame is again subdivided into an even-pixel sub-frame and an odd-pixel sub-frame, and so on to further improve the coding efficiency. The experimental results show that our proposed method reduces the bitrate by approximately 8.25% more than the conventional intra-frame coding in H.264/AVC.

This paper presents a high-resolution image reconstruction method from low-resolution image sequence. It is difficult to recognize details from a low-resolution image because of severe aliasing and poor image quality, hence recognition from the low-resolution image may result in false recognition decision. In order to improve the recognition performance, the proposed method performs a reconstruction-based super-resolution technique as a preprocessing. Then, we adopt a learning-based super-resolution technique to make high-resolution images. The proposed method also considers the illumination change between an input image and training images. To verify the accuracy and reliability of the proposed method, experiments and numerical analyses were performed with several video sequences of a moving car that simulate real surveillance systems.

In the incoming amendment of H.264/AVC for scalable video coding (SVC), down and up-samplings in a spatial domain are incorporated for spatial scalability. A new two-fold up-sampling method is proposed for SVC. The proposed up-sampling method is based on a combination of the forward and backward type-II discrete cosine transform (DCT). As the proposed up-sampling kernel has various symmetries for efficient computation, a fast algorithm of type-II DCT-based up-sampling method is also proposed. For further improvement of the up-sampling performance, an adaptive filtering method in the type-II DCT up-sampling is introduced, which applies different weighting parameters to DCT coefficients. Even as this adaptive method is applied, the up-sampling operation in the decoder has the same computational complexity as the fixed weighting method. The proposed adaptive up-sampling method shows a much improved PSNR in comparison with the recent H.264 SVC up-sampling method.

A Reduced Resolution Update (RRU) coding tool is first adopted in Annex Q of the H.263 standard [1]. This method is to perform down-sampling process of the prediction error signal and up-sampling process of the down-sampled error signal using spatial domain filters. In order to improve coding efficiency and to reduce computational complexity, we introduce a RRU coding tool on transform domain. The RRU coding tool on transform domain uses an integer 8x8 transform kernel in H.264 standard by zero-out in the encoder with quantization offset and zero-appending in the decoder. Our proposed RRU coding tool does not need additional computations for the up-sampling process in the decoder and has equal or better coding efficiency compared to the previous RRU using spatial domain filtering.

Nowadays, the slanted lenticular display becomes a representative one among the commercially introduced autostereoscopic displays. The paper presents a simple method to correct the lenticular alignment error by compensating the correction coefficients to the view number determination formula. Then, based on the corrected view numbers, the proposed algorithm simultaneously performs floating-point viewpoint generation and multiplexing on the scanline using the stereo image pairs and its depth information. Experimental results show that lenticular images, in which distortion and artifact due to lenticular alignment error are considerably reduced, are generated rapidly by using the proposed algorithm.

Wavelet-based resolution enhancement methods improve the image resolution by estimating the high-frequency band information. In this paper, we propose a new resolution enhancement method using inter-subband correlation in which the sampling phase in DWT is considered. Interpolation filters are designed by analyzing correlations between subbands having different sampling phases in the lower level, and applied to the correlated subbands in the higher level. The filters are estimated under the assumption that correlations between two subbands in the higher level are similar to that in the lower level in DWT. The experimental results show that our proposed method outperforms the conventional interpolation methods including the other wavelet-based methods with respect to peak signal-to-noise-ratio (PSNR) as well as the subjective quality.

This paper presents a new dense disparity estimation method using color segmentation and energy minimization. Segmentation-based methods have an advantage of clear representation of disparities on discontinuous region. This paper remodels the energy minimization method to fit with segmentation-based method. In the proposed method, the initial disparity map is obtained by variable block matching of the segmented plane and random sample consensus (RANSAC) fitting. Then energy minimization is performed on the small segmented planes and the overall segmented planes, respectively to refine the disparity map more accurately. At the end of the method, erroneous stripes that are not corrected by the segmentation-based approach are processed by a pixel-based method. Experimental results show that the error rate of the proposed method is comparable to other state-of-the-art matching methods on various stereo image pairs.

In this paper, we proposed a new scheme of forming quarter-pixel reference of LL band for in-band video coding with spatial scalability. The recently proposed leaky motion compensation and mode-based temporal filtering methods [5] have reduced the drifting error dramatically. However there still remains drifting error because of the use of high quality reference and the difference between high quality and low quality references [5]. In our method, a new reference is formed which gives PSNR higher than the low quality reference. The new reference is used instead of low quality reference in leaky and mode-based methods and experimental results show that improvement can be achieved in PSNR comparison with the existed ones.

In a scalable video coding (SVC), spatial up-sampling of video sequences is an important process for spatial scalability. We propose an up-sampling method based on the type-1 DCT. For further improvement of the up-sampling method, we introduce an adaptive filtering method, which applies different weighting parameters to DCT coefficients. The proposed adaptive up-sampling method shows much improved PSNR in comparison with the recent H.264 SVC up-sampling method.

Video sequences usually have many scene changes and contain rapid motion. The coding efficiency of such sequences is generally low, and the image quality fluctuates. If the rapid motion and scene changes are taken into consideration for video compression, the compression performance can be improved and the fluctuation of image quality can be reduced. An adaptive group of picture (GOP)-length structure is proposed here to consider the scene changes and rapid motion in video sequences. Our proposed algorithm for this adaptive GOP-length structure is composed of three steps. First, a correlation measure is proposed to estimate the scene changes. Second, a new intra-frame assignment method is proposed. Third, an intra-quantization parameter rule for avoiding buffer overflow and image quality fluctuation is also suggested. Our proposed algorithm has an initial delay of one-GOP length, which makes it difficult to apply for real-time communication, but this type of short initial delay is acceptable in a streaming service

We propose a fast arbitrary-ratio image resizing method for transcoding of the compressed images. The downsizing process in the discrete cosine transform (DCT) domain can be implemented by truncating high-frequency coefficients, whereas the upsizing process is implemented in the DCT domain by padding zero coefficients to the high-frequency part. The proposed method combines a fast inverse and forward DCT of composite length for arbitrary-ratio upsizing or downsizing. According to the resizing ratio, truncating the high-frequency coefficients and padding zeros are appropriately considered by combining the inverse DCT and forward DCT. The proposed method shows a good peak signal-to-noise ratio and less computational complexity compared with the spatial-domain and previous DCT-domain image resizing methods.

Down-up sampling of images is an essential process for spatial scalability of the video coding standard. We propose an efficient down-up sampling method in spatial domain using DCT kernel. The computational complexity of the proposed method is reduced by taking advantages of various symmetries of the combined DCT kernel. The proposed down-up sampling method is compared to that of MPEG-21 SVM 3.0 [ISO/IEC/JTCI 2004]. The proposed method outperforms the performance of MPEG-21 SVM 3.0 with slightly increased addition operation.

A new rate control algorithm is proposed, which is based on so-called /spl rho/-domain rate-distortion analysis . /spl rho/ is defined as the percentage of zero discrete cosine transform coefficients in a frame, and is known to be correlated with the compressed bit rate. Using a simple rate estimation method based on a variable length code table, the linear rate model of rate-/spl rho/ relation is constructed. The linear rate model is applied to the frame-level as well as the macroblock-level rate control. The proposed rate control algorithm provides a more accurate rate regulation, a better quality of picture, and lower computation amount than H.263 TMN8 rate control.

Many studies have performed transcoding of compressed video streams in the discrete cosine transform (DCT) domain to reduce computational complexity. On the other hand, most video compression standards recommend motion compensation in the spatial domain. Thus, compressed data that is motion-compensated in the spatial domain should be motion-compensated in the transform domain for the transform-domain transcoding. However, this combination of encoding in the spatial domain and decoding in the transform domain degrades image quality due to mismatches of the inverse DCT and motion compensation. There have been few investigations into this mismatch problem. In this paper, we establish that the rounding in calculation is the major cause of the mismatch. We investigate the mismatch problem in detail and propose a precision lifting method to reduce the effects of the mismatch problem. Experimental results show that our proposed method can effectively reduce the mismatch problem.

Image resizing of a compressed image can be performed in a spatial domain via decompression, resizing, and recompression, which is computationally inefficient. In order to reduce computation amount, efficient resizing approaches in the compressed domain have been developed. In this paper, we propose a novel approach to resize images that are compressed by 8 /spl times/ 8 block-discrete cosine transform (DCT)-based coding. The proposed image-resizing method is performed in the DCT domain and exploits the multiplication-convolution property of the DCT. Filter characteristics of the proposed image resizing method are analyzed by using a multirate filter bank representation. Since the characteristics of our resizing filters are better than those of others and the proposed image resizing method is computationally efficient, it can be widely used in a transcoder and shows visually fine results.

An automatic process of determining specified points in brain is presented, which is required to register brain MR images based on Talairach reference system. Generally, the 10 points that are needed to be determined for the registration are anterior commissure(AC), posterior commissure (PC), anterior point (AP), posterior point (PP), superiorpoint (SP), inferior point (IP), left point (LP), right point (RP) and two points for the midline of the brain. The proposed method automatically determines all the necessary points for registration except IP in a more stable manner than the manual selections. Projection information of the image intensity is used for the midline determination which is a necessary step for the midsagittal plane extraction. To find AC and PC in the midsagittal plane image, two-level shape matching of the corpus callosum followed by AC and PC shape matching is performed in the edge-enhanced midsagittal plane image. Remaining points are found by fitting the intensity curve of the cutview with the Gaussian model. After finding the necessary points, the brain MR images can be successfully registered.

Many studies have performed transcoding of compressed video streams in the discrete cosine transform (DCT) domain to reduce computation complexity. Most video compression standards recommend motion compensation in the spatial domain. On the other hand, compressed data that is motion-compensated in the spatial domain is motion-compensated in the transform domain for the transform-domain transcoding. However, this combination of encoding in the spatial domain and decoding in the transform domain degrades image quality due to mismatches of the inverse discrete cosine transform (IDCT) and motion compensation (MC). There have been few investigations into this mismatch problem. We investigate the mismatch problem and propose a precision lifting (PL) method to reduce the effects of the mismatch problem. Experimental results show that our proposed method effectively reduces the mismatch problem.

The paper proposes a fast view interpolation method based on image gradient and disparity triangulation. The image gradient is used to select the node points for triangulation and each node point is evaluated by its matching errors and cross correspondence of the disparity values. To model the abrupt changes of disparity on the object boundaries, new node points are added along the image gradient direction. In addition, some node points are removed by consideration of unreliable matching conditions. To construct the intermediate-view images, Delaunay triangulation and image warping are performed. The experimental results show that the proposed algorithm is fast and overcomes the drawbacks of previous methods.

The current international standard, the Joint Bilevel Image Experts Group (JBIG), is a representative of a bilevel image compression algorithm. It compresses bilevel images with high performance, but it shows relatively low performance in compressing error-diffused halftone images. This paper proposes a new bilevel image compression for error-diffused images, which is based on Bayes' theorem. The proposed coding procedure consists of two passes. It groups 2 x 2 dots into a cell, where each cell is represented by the number of black dots and the locations of the black dots in the cell. The number of black dots in the cell is encoded in the first pass, and their locations are encoded in the second pass. The first pass performs a near-lossless compression, which can be refined to be lossless by the second pass. Experimental results show a high compression performance for the proposed method when it is applied to error-diffused images.

Image resizing is to change an image size by upsampling or downsampling of a digital image. Most still images and video frames on digital media are given in a compressed domain. Image resizing of a compressed image can be performed in the spatial domain via decompression and recompression. In general, resizing of a compressed image in a compressed domain is much faster than that in the spatial domain. We propose a novel approach to resize images with L/M resizing ratio in the discrete cosine transform (DCT) domain, which exploits the multiplication-convolution property of DCT (multiplication in the spatial domain corresponds to symmetric convolution in the DCT domain). When an image is given in terms of its 8 x 8 block-DCT coefficients, its resized image is also obtained in 8 x 8 block-DCT coefficients. The proposed approach is computationally fast and produces visually fine images with high PSNR.

We propose a new method to compress error diffused images with spatial scalability. This method is a kind of combination of inverse halftone and rehalftone. During inverse halftoning, we combine four dots into a pixel of a reduced image. Then, the reduced image is rehalftoned using an error diffusion algorithm. We encode an error diffused image progressively from the lowest resolution image to higher resolution image. To code higher resolution images, we use the dots in the previously coded lower resolution image. The compression ratios of the proposed algorithm are similar to those of the JBIG (Joint Bilevel Image Processing Group) progressive mode. However, the reduced image quality from the proposed algorithm is better than that from the JBIG progressive mode.

The paper presents a segmentation algorithm to extract the endocardial and epicardial walls of the left ventricle in MR cardiac images. The algorithm is based on a generalized gradient vector flow (GGVF) snake and a prediction of initial contour (PIC). Especially, the proposed algorithm uses physical characteristics of endocardial and epicardial contours, cross profile correlation matching (CPCM), and a mixed interpolation model. The proposed method is applied to short axis MR cardiac image sets, which are obtained by Siemens, Medinus, and GE MRI systems. The experimental results show that the proposed algorithm can extract acceptable epicardial and endocardial walls. In addition, we calculate quantitative parameters from the segmented results, which are displayed graphically. The segmented left ventricle is visualized volumetrically by surface rendering. The proposed algorithm is implemented in the Windows environment using Visual C++.

Proposes an invariant feature set for recognizing nonoccluded military vehicles in natural FLIR (forward-looking infrared) images. The proposed feature set is extracted from global and local shape information to improve recognition performance. After segmenting a target, a radial function is defined from a target boundary to extract global shape features. Also, to extract local shape features of upper region of a target, a distance function is defined, which designates distance between boundary points of upper region and a line drawn by two extreme points. From two functions and target boundary, four global and four local shape features are extracted. They are more invariant to similarity transform than traditional feature sets. In the experiments, we show that the proposed features are superior to the traditional feature sets with respect to invariance and recognition performance.

In many image-coding applications such as Web browsing, image databases, and telemedicine, it is useful to reconstruct only a region of interest (ROI) before the rest of the image is reconstructed. In this paper, an ROI coding functionality is incorporated with the set partitioning in hierarchical trees (SPIHT) algorithm for wavelet-based image coding. By placing a higher emphasis on the transform coefficients pertaining to the ROI, the ROI is coded with higher fidelity than the rest of the image in earlier stages of progressive coding. The general thrust of this research is to identify necessary coefficients in the wavelet-transform domain for the decoder to reconstruct the desired region. This new method provides better performance than the previously presented methods.

Real-time processing and visualization of 3D image data are the most important requirements for medical imaging. Among various 3D visualization methods, maximum intensity projection (MIP) is a useful tool to visualize 3D medical images. The processing time of the MIP depends on the number of voxels of the 3D data. Therefore, it is very difficult to realize the MIP of the 3D medical images in real time. In order to overcome the large computation amount of the MIP, we have developed a hierarchical MIP method that can perform the first-step MIP with average sub-sampled images, which is made from sub-sampled data by averaging. In the second step, the low-resolution MIP image and maximum sub-sampled voxels can be used to save the computation time of the full-resolution MIP image. The hierarchical MIP method was implemented, and its computation complexity and image quality were analyzed.

We have developed a moving-object segmentation method for standard-compliant DCT-based video coder. To be compliant with current block-based video coding standards, the proposed algorithm uses the DCT coefficients and motion vector of a block, respectively, as spatial and temporal features of segmentation. Since a block-based motion vector may not be sufficient to represent the true motion of the object, the proposed algorithm overcomes this problem by using a statistical approach. Moreover, the proposed segmentation algorithm operates in parallel with the video coding. Therefore, the proposed segmentation algorithm can be efficiently integrated in parallel with the DCT-based video coder for the improvement of coding efficiency as well as segmentation accuracy.

In many image coding applications, such as Web browsing, image database and telemedicine, it is useful to reconstruct only a region of interest (ROI) before the rest of the image is reconstructed. An ROI coding functionality is incorporated with the set partitioning in hierarchical trees (SPIHT) algorithm (see Said and Pearlman, IEEE Trans. Circuits and Systems for Video Technology, vol.6, p.243-50, 1996). By emphasizing the ROI coefficients, they are coded with higher fidelity than the rest of the image in early stages of progressive reconstruction. The main thrust of this research is how to identify necessary coefficients for the decoder to reconstruct the desired region. The proposed method provides better performance than the previous method.

Active contours or snakes are an effective edge-based method in segmenting an object of interest. However, the segmented boundary of a moving object in one video frame may lie far from the same moving object in the next frame due to its rapid motion, causing the snake to converge on the wrong edges. To guide the snake toward the appropriate edges, we have added gradient directional information to the external image force to create a "directional snake." Thus, in minimizing the snake energy, the new method considers both the gradient strength and gradient direction of the image. Experimental results demonstrate that the directional snake can provide a better segmentation than the conventional method in certain situations, e.g., when there are multiple edge candidates in the neighborhood with different directions. The directional snake is significant because it provides a framework to incorporate directional information in digital video segmentation.

The maximum intensity projection (MIP) is a useful tool to visualize 3D images. The processing time of the MIP depends on the number of voxels of the 3D data. Therefore, it is very difficult to realize the MIP of the 3D medical image in real time. In order to overcome the large amount of computation for the MIP, we have developed a hierarchical MIP method that can perform the first-step MIP with sub-sampled image. At the second step, the low-resolution MIP image can be used to save the computation time of the full-resolution MIP image. The hierarchical MIP method was implemented and its computational complexity was analyzed.

Currently, the most popular method of converting 24 frames per second (fps) film to 60 fields/s video is to repeat each odd-numbered frame for 3 fields and each even-numbered frame for 2 fields. This method is known as 3:2 pulldown and is an easy and inexpensive way to perform 24 fps to 60 fields/s frame-rate conversion. However, the 3:2 pulldown introduces artifacts, which are especially visible when viewing on progressive displays and during slow-motion playback. We have developed a motion-compensated frame-rate conversion algorithm to reduce the 3:2 pulldown artifacts. By using frame-rate conversion with interpolation instead of field repetition, mean square error and blocking artifacts are reduced significantly. The techniques developed here can also be applied to the general frame-rate conversion problem.

This paper has developed a fast variable-length decoder which uses a plane separation technique to reduce the processing time of the feedback path in the decoder. The developed decoder performs two shift processes and a decision process concurrently. Therefore, the processing time in the feedback path of our developed variable length decoder can be improved and determined by the longest time among the three processes, not by the sum of their processing times together. Our simulation results show that the total processing time of our developed decoder makes about 30% improvement from that of the Sun and Lei's (1991, 1992) decoder and their modified decoder when they are implemented with field programmable logic device.

The discrete wavelet transform (DWT) has several advantages of multiresolution analysis and subband decomposition, which has been successfully used in image processing. However, the shift-variant property is intrinsic due to the decimation process of the wavelet transform, and it makes the wavelet-domain motion estimation and compensation inefficient. To overcome the shift-variant property, a low-band-shift method is proposed and a motion estimation and compensation method in the wavelet-domain is presented. The proposed method has a superior performance to the conventional motion estimation methods in terms of the mean absolute difference (MAD) as well as the subjective quality. The proposed method can be a model method for the motion estimation in wavelet-domain just like the full-search block matching in the spatial domain.

The decompressed images from highly compressed data have noticeable image degradations such as blocking artifacts, corner outliers, and ringing noise, because most image coding standards quantize the DCT coefficients of 8/spl times/8-pixel blocks independently. A loop-filtering algorithm and a post-filtering algorithm, which can reduce the quantization effects, are described, and the performance of both algorithms is compared with respect to the image quality, the computational complexity, and the compressed bit rates. Both the loop-filtering algorithm and the post-filtering algorithm reduce the quantization effects adaptively by using both DCT domain analysis and temporal information.

The reconstructed images from highly compressed MPEG data have noticeable image degradations, such as blocking artifacts near the block boundaries, corner outliers at crosspoints of blocks, and ringing noise near image edges because the MPEG quantizes the transformed coefficients of 8/spl times/8 pixel blocks. A postprocessing algorithm is proposed to reduce quantization effects, such as blocking artifacts, corner outliers, and ringing noise, in MPEG-decompressed images. The proposed postprocessing algorithm reduces the quantization effects adaptively by using both spatial frequency and temporal information extracted from the compressed data. The blocking artifacts are reduced by one-dimensional (1-D) horizontal and vertical low-pass filtering (LPF), and the ringing noise is reduced by two-dimensional (2-D) signal-adaptive filtering (SAF). A comparison study of the peak signal-to-noise ratio (PSNR) and the computation complexity analysis between the proposed algorithm and the MPEG-4 VM (verification model) postprocessing algorithm is performed by computer simulation with several image sequences. According to the comparison study of PSNR and computation complexity analysis, the proposed algorithm shows better performance than the VM postprocessing algorithm, whereas the subjective image qualities of both algorithms are similar.

Clustered microcalcifications on X-ray mammograms are an important sign for early detection of breast cancer. Texture-analysis methods can be applied to detect clustered microcalcifications in digitized mammograms. In this paper, a comparative study of texture-analysis methods is performed for the surrounding region-dependence method, which has been proposed by the authors, and conventional texture-analysis methods, such as the spatial gray-level dependence method, the gray-level run-length method, and the gray-level difference method. Textural features extracted by these methods are exploited to classify regions of interest (ROI's) into positive ROI's containing clustered microcalcifications and negative ROI's containing normal tissues. A three-layer backpropagation neural network is used as a classifier. The results of the neural network for the texture-analysis methods are evaluated by using a receiver operating-characteristics (ROC) analysis. The surrounding region-dependence method is shown to be superior to the conventional texture-analysis methods with respect to classification accuracy and computational complexity.

A programmable and scalable parallel architecture is proposed for the real-time encoding/decoding of HDTV images and for nonlinear editing of the compressed video data. It only uses the intra-mode compression/decompression so that nonlinear editing can be performed easily and high-quality images can be recovered. Spatially partitioned image data are concurrently processed by multiple parallel processing units (PUs). Each PU consists of a programmable parallel digital signal processor, called multimedia video processor (MVP; TMS320C80), and reconfigurable field programmable logic devices (FPLDs). The performance of the REDS is described in terms of the required MVP cycles for transform coding and the FPLDs throughput for entropy coding. Robust rate distortion-optimized quantization matrices for HDTV images are presented.

Clustered microcalcifications on X-ray mammograms are an important feature in the detection of breast cancer. For the detection of the clustered microcalcification on digitized mammograms, this paper proposes a texture analysis method called the surrounding region dependence method (SRDM), which is a statistical texture analysis based on the second-order histogram in two surrounding regions. Four textural features are extracted from the SRDM. These features are used to classify region of interests (ROIs) into positive ROIs containing clustered microcalcifications and negative ROIs of normal breast tissues. The three-layer backpropagation neural network is employed as a classifier with input data of four textural features. The classification performance of the proposed method is evaluated by using the round-robin method and the receiver operating-characteristics (ROC) analysis.

This paper proposes an adaptive image enhancement method for mammographic images, which is based on the first derivative and the local statistics. The adaptive enhancement method consists of three processing steps. The first step is to remove the film artifacts which may be misread as microcalcifications. The second step is to compute the gradient images by using the first derivative operators. The third step is to enhance the important features of the mammographic image by adding the adaptively weighted gradient images. Local statistics of the image are utilized for adaptive realization of the enhancement, so that image details can be enhanced and image noises can be suppressed. The objective performances of the proposed method were compared with those by the conventional image enhancement methods for a simulated image and the seven mammographic images containing real microcalcifications. The performance of the proposed method was also evaluated by means of the receiver operating-characteristics (ROC) analysis for 78 real mammographic images with and without microcalcifications.

Digital halftoning is a gray level rendition in devices that have limited gray representation capabilities. The major evaluation points of digital halftoning methods are the image quality and the processing speed. A new halftoning method using a non-thresholding process is proposed, which simply constructs a halftoning database to achieve real time halftoning as well as optimal image quality. The proposed algorithm is constructed by two major parts. The first part is to generate well-distributed binary patterns for every gray level. A contrast sensitivity function (CSF) is used as a filter to distribute void and cluster regions and as a criterion of iteration. The second part is to index the halftoning database to image pixel value. The halftoning database consists of the well-distributed binary patterns. By the proposed algorithm, halftoned images can be obtained faster than the other algorithms and produce similar image quality to those of the max energy mask.

A description is given of the computer graphics aspects of two architectures designed for imaging and graphics. The two systems use parallel and pipelined architectures for high-performance graphics operations. UWGPSP3 uses only commercially available off-the-shelf chips, and consists of a TM34020 graphics system processor and four TMS34082 floating point coprocessors that can be configured into pipelined or SIMD modes depending on the algorithm. UWGSP4 uses dedicated ASIC chips for higher performance, and consists of two main computational parts: a parallel vector processor with 16 vector processing units, used mainly for image processing, and a graphics subsystem which utilizes a parallel pipelined architecture for image synthesis.>