Inspired by humans comprehending speech in a multi-modal manner, various audio-visual datasets have been constructed. However, most existing datasets focus on English, developed from pre-existing videos using various prediction models, and have only a small number of multi-view videos. To mitigate the limitations, we constructed the Open Large-scale Korean Audio-Visual Speech (OLKAVS) dataset, which is the largest among publicly available audio-visual speech datasets. The dataset contains 1,150 hours of transcribed audio from 1,107 Korean speakers in a studio setup with nine different viewpoints and various noise situations. We also provide the pre-trained baseline models for two tasks: audiovisual speech recognition and lip reading. We conducted experiments based on the models to verify the effectiveness of multi-modal and multi-view training over uni-modal and frontal-view-only training. We expect the OLKAVS dataset to facilitate multi-modal research in broader areas.

This paper proposes a unified depth model based on the light field gradient, in which estimated disparity is represented by the complex number. The complex-valued disparity by the proposed depth model can be represented in both the Cartesian and polar coordinates. In the Cartesian representation, the proposed depth model is represented by real and imaginary parts of the disparity. The real part can be used for disparity estimation with respect to the in-focus plane, whereas the imaginary part represents the non-Lambertian-ness. In the polar representation, the proposed depth model is expressed by the disparity magnitude and disparity angle. The disparity magnitude shows the relationship among depth from stereo, depth from focus, and depth from defocus, whereas the disparity angle shows whether or not the bundles of rays are flipped with respect to the in-focus plane. For disparity analysis, we present the real response, imaginary response, magnitude response, and angle response, which are represented by the three-dimensional volume. Experimental results on synthetic and real light field images show that the real and magnitude responses of the proposed depth model are valid for local disparity estimation.

In this letter, we propose an attention network for object tracking. To construct the proposed attention network for sequential data, we combine long-short term memory (LSTM) and a residual framework into a residual LSTM (RLSTM). The LSTM, which learns temporal correlation, is used for a temporal learning of object tracking. In the proposed RLSTM method, the residual framework, which achieves the highest accuracy in ImageNet large scale visual recognition competition (ILSVRC) 2016, learns the variations of spatial inputs and thus achieves the spatio-temporal attention of the target object. Also, a rule-based RLSTM learning is used for robust attention. Experimental results on large tracking benchmark datasets object tracking benchmark (OTB)-2013, OTB-100, and OTB-50 show that the proposed RLSTM tracker achieves the highest performance among existing trackers including the Siamese trackers, attention trackers, and correlation trackers, and also has comparable performance with the state-of-the-art deep trackers.

A sign function approximation method for depth from light field (DFLF) based on the foreground–background separation (FBS) is proposed. From a signal processing viewpoint, the FBS-based method can be considered as a bridge between the cost-based DFLF methods and depth model-based ones. The proposed sign function approximation corresponds to the winner-takes-all (WTA) approaches as the cost-based methods do. Experimental results on the synthetic images show that the proposed method reasonably performs in terms of the mean squared error as the state-of-the-art methods do. Especially, by using a suitable WTA method in the framework of our previous FBS-based DFLF work, the proposed method effectively reduces the angular aliasing artifacts in the resulting disparity maps of both the synthetic and real images.

This study proposes a colour image dehazing method using a near‐infrared (NIR) image. Haze is a degradation in colour images caused by light scattering and is dependent on the wavelength of the light and the size of atmospheric particles. According to Rayleigh's law, an NIR image is less scattered than a red‐green‐blue (RGB) image, and in this study, it is observed that in a small patch of a haze‐free image, the high‐frequency (HF) components of the RGB and NIR images are similar to each other; in contrast, those in a small patch of a hazy image are different from each other. This study proposes a new dehazing method that fuses the HF components of the RGB and NIR images in a local hazy patch; the HF components in the patch are then used to generate the weight maps used to combine detail layers. The proposed method enhances the detail layer of a hazy RGB image by fusing the detail layers of the RGB and NIR images, while the base layer is restored using an existing single image dehazing method. Experimental results show that the proposed method is qualitatively better than existing methods.

This study proposes an unsupervised binary hashing (UBH) method and provides a comprehensive analysis and evaluation to the UBH method in image similarity. Using the orthogonal locality preserving projection, the proposed UBH method performs the dimensionality reduction (DR). To reduce the quantisation error between low‐dimensional vectors and binary hash codes generated in the DR, the proposed UBH method calculates the optimal parameters of rotation and offset. The authors use two test beds to evaluate the preservation of original feature space and the semantic consistency. Experimental results with two test beds show that the proposed UBH method is state of the art.

Mobile display technology has been developed fast and low‐power design for the display is more important than ever. Hardware techniques for low‐power design are not flexible and the existing software techniques are not considering the characteristics of the display panels. In this study, a histogram‐based contrast‐aware power control method for active‐matrix organic light‐emitting diode display is proposed. The proposed power control method solves an optimisation problem to achieve power control with maximum contrast available. A new histogram representation is also proposed to calculate the power consumption of an image efficiently in the optimisation process. Simulation results and hardware experiment results show that the proposed method provides both precise power control and better contrast of the output image.

This paper proposes an efficient and fast iris localization method. It uses support vector machine learning of iris features that represent closed outer and inner iris boundaries encompassing a low-intensity region. In addition, depending on the location of the iris in an eye image, an iris detection method is proposed based on three sub-datasets of eye images (middle, right, and left sub-datasets) with different iris features. The proposed method is implemented using fast sliding window and fast computation of the iris detection score with binary features. Compared with state-of-the-art methods, experimental results show that the proposed method is twice as fast and has comparable accuracy, even when factoring in head rotation, glasses, and highlights.

This paper proposes a new color fringe correction method that preserves the object color well by the color difference prediction using the logistic function. We observe two characteristics between normal edge (NE) and degraded edge (DE) due to color fringe: 1) the DE has relatively smaller R-G and B-G correlations than the NE and 2) the color difference in the NE can be fitted by the logistic function. The proposed method adjusts the color difference of the DE to the logistic function by maximizing the R-G and B-G correlations in the corrected color fringe image. The generalized logistic function with four parameters requires a high computational load to select the optimal parameters. In experiments, a one-parameter optimization can correct color fringe gracefully with a reduced computational load. Experimental results show that the proposed method restores well the original object color in the DE, whereas existing methods give monochromatic or distorted color.

Display capabilities of the smart mobile devices are highly developed for the last decade. Thus, it is necessary to consider both quality of the images and the power consumed by the display. In this study, a power‐constrained contrast enhancement method formulated as an optimisation problem (quadratically constrained quadratic programming) is proposed. The proposed locality condition is used to enhance the conventional histogram equalisation, where power constraint is used to specify the power‐consumption ratio of the enhanced image. The experimental results show that the proposed method effectively enhances the quality of single images with designated power reduction achieved. The proposed algorithm is also applied to video sequences with a simple and effective update algorithm in real‐time. The update algorithm gives successful results without flickering artefacts observed in existing methods.

High Efficiency Video Coding (HEVC) is the state-of-the-art video coding standard. It adopts a hierarchical quad-tree-based coding unit (CU) partitioning structure that is flexible in various texture and motion characteristics of a video signal. However, the exhaustive partitioning process for finding optimal CU partitions requires a dramatic increase in computational complexity of the HEVC encoder compared with previous video coding standards. In this paper, a fast CU partitioning algorithm is proposed for HEVC encoder, which early on terminates the CU partitioning process based on the Bayesian decision rule using joint online and offline learning. An online learning method is first presented based on the minimum error Bayesian decision rule using a training picture selection method with scene change detection. Next, a joint online and offline learning method is presented, which additionally trains the loss of decision making of the proposed method based on the minimum risk Bayesian decision rule. The proposed method is implemented on an HEVC test software 15.0. Experimental results show that the proposed method reduces the computational complexity of HEVC encoder to 53.6% on an average with a 0.71% acceptable Bjøntegaard delta bitrate loss in random access configuration. For other configurations, 48.4%, 48.5%, and 54.2% encoding time saving are obtained on an average for low delay, low delay-P, and all intra-configurations, respectively.

Biometric information is widely used in user identification systems. Iris is one of the most reliable and accurate biometric information. In an iris recognition system, the iris localisation is one the most important parts because the performance of an iris recognition system is highly dependent on the accuracy of iris localisation. If unreliable iris regions are used in the iris recognition system, the recognition rate may be degraded. Therefore many researchers have studied the iris localisation methods. Especially, localising an iris region from noisy images is one of the hot topics in the iris recognition researches. In this study, the authors are concentrating on the iris localisation, where an efficient iris localisation method for noisy iris images is proposed. The proposed iris localisation method consists of two steps: pupil boundary localisation and iris boundary localisation. To localise a pupil region, an efficient block‐based minimum energy detection method is used, in which specular reflection removal is performed as a preprocessing. Iris boundary is localised using a guided filter, the circular Hough transform and an ellipse fitting method. Experimental results with various test image sets show the effectiveness of the proposed method.

This study proposes a segmentation method using colour and depth images, from which saliency map is generated. With saliency map obtained from both images, salient foreground is extracted using adaptive thresholding of the saliency map, which reduces performance degradation of existing methods for images with complex background. Also, to enhance edges of the foreground, an adaptive guided filter is used. Normalised cut segmentation is performed with the extracted and enhanced foreground to separate different objects. Experimental results with three different types of datasets show that the proposed method gives better segmentation results than the existing methods, in which the complex background is well separated from the foreground.

As smart audio‐visual multimedia devices are developed for various applications, there has been a growing interest in effective human–computer interaction (HCI) interfaces for specific environments. There have also been great efforts to implement HCI interfaces into musical instruments, in which it would be possible to take intuitions, comfort and expressiveness into the musical instruments. However, most of the traditional HCI interfaces are not applicable because both hands are likely to be occupied while playing a musical instrument. In this environment, a lip reading method can be used. A lip reading method is a HCI method that analyses lip motion to recognise spoken words. In this study, a lip reading method is proposed with its application. As a specific example of the interface for musical instruments, the guitar effector application is presented. The proposed lip reading method uses a constrained local model instead of the conventional active appearance model for effective facial feature tracking. The proposed method also uses a dynamic time warping‐based classifier for word recognition which is effective for simple real‐time implementation of lip reading. The proposed lip reading method shows 85.0% word recognition accuracy on OuluVS database and is effectively applied to the proposed guitar effector application.

Biometric information is widely used in user identification system. Because of the unique and invariant properties of the iris through a lifetime, iris recognition is one of the most stable and reliable means in biometric identification. Extracting distinguishable iris features for iris recognition is very important. In this paper, for capturing effective texture features that represent the complex directional structures of an iris image, a new iris recognition method using the nonsubsampled contourlet transform (NSCT) features is proposed. With the shift-invariance, multiscale, and multidirection properties, significant NSCT coefficient features along the radial and angular directions in an iris image can be represented efficiently. Iris segmentation and normalization are considered at first as pre-processing. The modified normalized iris image is obtained from the normalized iris regions for extracting the robust iris features, and then is filtered with the NSCT to obtain the distinct coefficient features in each directional subband. Next, using the NSCT coefficients in each subband, an iris code vector is constructed for iris matching. Comparison of experimental results of the proposed and existing methods with three databases show the effectiveness of the proposed NSCT feature-based iris recognition algorithm, in terms of the three performance measures.

In this letter, a dual difference regression classification (DDRC) based face recognition method is proposed to reduce the effect of outliers in regression methods. DDRC consists of two parts: pixel difference binary pattern (PDBP) and difference regression classification (DRC). PDBP calculates the pixel differences in image domain, and then the pixel differences are binarized and converted to decimal values. DRC calculates the vector differences in feature domain to estimate an optimal predictor matrix. Experimental results on noiseless and noisy images of Yale B, Extended Yale B, JAFFE, AT&T, and GT databases with different image sizes show the effectiveness of the proposed face recognition method in terms of the recognition accuracy.

Histogram equalization (HE), a simple contrast enhancement (CE) method, tends to show excessive enhancement and gives unnatural artifacts on images with high peaks in their histograms. Histogram-based CE methods have been proposed in order to overcome the drawback of HE, however, they do not always give good enhancement results. In this letter, a histogram-based locality-preserving CE method is proposed. The proposed method is formulated as an optimization problem to preserve localities of the histogram for performing image CE. The locality-preserving property makes the histogram shape of the enhanced image to be similar to that of the original image. Experimental results show that the proposed histogram-based method gives output images with graceful CE on which existing methods give unnatural results.

This paper proposes an image fusion based method for color fringe correction. Color fringes are frequently observed at high contrast edges, i.e., a brighter image has more color fringe than a darker image. The proposed color fringe correction method fuses pixel values of the normal-exposure image with color fringe and the under-exposure image with no color fringe. First, each of normal- and under-exposure images is transformed using a color difference model. Second, color fringe regions are detected and used for calculating fusion weights. Third, the color difference values are fused with the weights refined by guided filtering. Experimental results show that the proposed fusion-based method preserves better an original color in color fringe regions than existing methods.

Colour fringe is an artefact that mainly appears at the boundary of an object in an image, where colours change abruptly and luminance value is large. Thus, colour fringe should be corrected in a colour digital image. This study proposes a method for detection and correction of colour fringe in YC b C r colour space. The luminance value as well as gradient magnitudes/directions of YC b C r are used to detect colour fringed region. Then, chrominance values in colour fringed regions are corrected using those of pixels in the neighbouring regions along the colour fringe direction. The experimental results show the effectiveness of the proposed colour fringe detection and correction method.

These days, a large number of human-computer interactive systems have been developed. This paper proposes a visualization system of reconstructed three-dimensional (3-D) scene, which is based on user-interactive control using the Kinect sensor and a webcam. First, we acquire depth and color images from a number of viewpoints, from which 3-D reconstruction of a scene is performed. We pre-process the acquired data to improve 3-D reconstruction result. Then, we reconstruct the 3-D scene for the visualization system that uses face detection and tracking with a webcam. Experimental results show that our system gives a visualized 3-D scene reconstruction simply using the Kinect sensor and a webcam. It can be applied to a human-interaction display system at museum, amusement park, and so on.

In this paper, we propose a new image quality metric using derivative filters in the context of compressive sensing (CS) that represents a sparse or compressible signal with a small number of measurements. In general, an arbitrary image is not sparse or compressible, however, its derivative image is compressible. In this paper, derivative images are obtained using first- and second-order derivative filters such as Sobel operators and Laplacian of Gaussian filters. Each derivative image of the reference and distorted images is measured via CS. Measurements of derivative images are compared for evaluating the image quality. Experiments with the laboratory for image and video engineering database show the effectiveness of the proposed method.

Multiple images with different exposures are used to produce a high dynamic range image (HDRI). When a user captures an image sequence, the ghost effect caused by moving objects and background change appears in an HDRI. This paper proposes a histogram based ghost removal method, in which object motion and background change between two exposures are detected using multi-level threshoding of the intensity histogram. The detected object motion is used to determine weighting factors in construction of the radiance map. Experiments with several sets of test images with different exposures show that the proposed method gives a better performance than existing methods in terms of visual results and computation time.

Image degradation deforms the structure of an image. Thus, to evaluate the quality of an image is equivalent to measuring the degree of deformation of the structure of the image. In this letter, we propose a new image quality metric based on the Harris response, which is computed from the gradient information matrix and its eigenvalues. When an image is degraded by image compression, noise, transmission error, and so on, gradient information of the image is changed, causing the Harris response to change. Therefore, the degree of change in the Harris response of the image is related to the quality degradation of the image. Experimental results with the laboratory for image and video engineering data set show the effectiveness of the proposed quality metric.

Eyelids and eyelashes occluding the iris region are noise factors that degrade the performance of iris recognition. If they are incorrectly classified as the iris region, the false iris pattern information will increase, decreasing the recognition rate. Thus, reliable detection of eyelids and eyelashes is required to improve the performance of iris recognition. The objective of this paper is to present a research direction for reduction of noise factors based on analysis and performance comparison of existing eyelid and eyelash detection algorithms. With CASIA version 3 database we compare six existing eyelid and eyelash detection algorithms and analyze the characteristics and performance of each algorithm. The performance of each algorithm is evaluated in terms of different performance measures such as the decidability, the equal error rate, and the detection error trade-off curve.

We propose a novel adaptive image interpolation method based on the weight classified vector quantization (CVQ) using optimal coefficients which are constrained by a quadratic signal class. It overcomes the computational complexity bottleneck of other second-order image interpolation methods such as an edge-directed and an optimal recovery interpolation. Our proposed interpolation method consists of three steps: the coefficient generation step constructing both image patches and constrained coefficient sequences from a set of training images, weight CVQ step generating codevectors from classified image patches, and interpolation step using an equal-average nearest neighbor search (ENNS). Simulation results with exemplary test images demonstrate that our proposed method is superior to a bilinear and a bicubic interpolation. Moreover, its visual quality is comparable to that of the edge-directed and the optimal recovery interpolation, with the much less computational load

In this letter, we propose a reconstruction method of intermediate views from stereoscopic images captured by parallel cameras for three-dimensional (3-D) display applications. Conventional methods to reconstruct intermediate views for 3-D videoconference systems ignore the background region and assume a single object in a scene; however, in 3-D display devices such as 3-D television (TV), geometrical relationships between an object and background regions and between one object and another must be importantly considered. The proposed algorithm makes use of the geometric constraint of objects in reconstructing intermediate views. First of all, using quadtree-based variable-size block matching, we estimate disparity vectors (DVs) in visible areas in both stereoscopic images. Next, to synthesize intermediate views, we propose a novel algorithm that assigns DVs in occlusion regions, based on the geometrical constraint that describes structures of objects. Finally, we present four different intermediate views between two stereoscopic images. Experimental results show that the proposed algorithm assigns more precise DVs to occlusion regions and obtains more accurate intermediate views than conventional algorithms.

This paper presents a trinocular stereo sequence-coding algorithm based on moving picture experts group (MPEG)-2. Trinocular stereo images are a set of three images; center, left, and right view images are taken at the same time. To reduce the redundancy of a trinocular stereo sequence, we use disparity information, as well as motion information. Three reference images are employed: two reference (previous and future) images for the forward and backward motion vectors and the center view image for the disparity vector (DV). We investigate three temporal prediction structures used for trinocular stereo sequence coding. New macroblock types and variable length code tables that include DV are designed. Experimental results with real test sequences show that the proposed algorithm gives a higher peak signal-to-noise ratio (PSNR) than MPEG-2 and MPEG-2 multiview profile.

Block-wise processing of motion compensated interlaced-to-progressive conversion (MC-IPC) can cause spurious errors that degrade converted image quality significantly. This paper presents a novel IPC method based on motion adaptive IPC (MA-IPC) with its adaptation assisted by the motion compensation. The motion adaptation is performed with respect to a motion selected from a limited set based on the accuracy evaluated with a reduced and overlapped block. The use of the reduced and overlapped blocks makes evaluation of the accuracy more precise, and implementation less costly. The motion adaptation structure ensures that any possible error occurs pixel-wise so that the block-wise spurious errors can be eliminated. The estimation of the motion is performed efficiently with data already available for the motion adaptation. The proposed motion compensation assisted MA-IPC method provides image quality similar to the MC-IPC, for those images, the MC-IPC offers significant improvement over the MA-IPC, yet at considerably lower computational and implementation costs.

This paper presents a novel interlaced-to-progressive conversion (IPC) method based on the motion adaptive IPC (MA-IPC) with its adaptation assisted by the motion compensation. The motion adaptation is performed with respect to a motion selected from a limited set based on the accuracy evaluated with a reduced and overlapped block. The use of the reduced and overlapped blocks makes the evaluation of the accuracy more precise, and the implementation less costly. The motion adaptation structure ensures that any possible error occurs pixel-wise, so that block-wise spurious error can be eliminated. The estimation of the motion is performed efficiently with data already available for the motion adaptation. The proposed motion compensation assisted motion adaptive IPC (MoMA-IPC) method provides image quality similar to the motion compensated IPC (MC-IPC) for those images the MC-IPC offers significant improvement over the MA-IPC, yet at the considerably lower computational and implementation cost.

Proposes a fast local motion-compensation algorithm that improves motion-compensation performance for video sequences with brightness variations. The brightness variation parameters, a multiplier and an offset field for image intensity, are robustly estimated and local motions are compensated. We also propose the frame classification method based on the cross entropy between histograms of two successive frames, to detect the frame with brightness variations. Simulation results show that the proposed method yields a higher peak signal-to-noise ratio than the conventional methods, with a low computational load, when the video scene contains large brightness changes.

Motion compensated interlaced-to-progressive conversion (MC-IPC) methods employ implicit decision on whether to use the estimated motion vectors in order to prevent spurious errors due to poorly estimated motion vectors. This often limits the performance of the MC-IPC. We present an IPC method based on the pattern matching assisted motion estimation and motion vector histogram analysis. The proposed method employs an explicit decision on whether to use the estimated motion vectors. Extensive experiments show that high quality IPC can be achieved without spurious errors.

Presents an integrated system for navigation parameter estimation using sequential aerial images, where the navigation parameters represent the positional and velocity information of an aircraft for autonomous navigation. The proposed integrated system is composed of two parts: relative position estimation and absolute position estimation. Relative position estimation recursively computes the current position of an aircraft by accumulating relative displacement estimates extracted from two successive aerial images. Simple accumulation of parameter values reduces the reliability of the extracted parameter estimates as an aircraft goes on navigating, resulting in a large positional error. Therefore, absolute position estimation is required to compensate for the positional error generated by the relative position estimation. Absolute position estimation algorithms using image matching and digital elevation model (DEM) matching are presented. In the image matching, a robust-oriented Hausdorff measure (ROHM) is employed, whereas in the DEM matching, an algorithm using multiple image pairs is used. Experiments with four real aerial image sequences show the effectiveness of the proposed integrated position estimation algorithm.

To manipulate a large video database, effective video indexing and retrieval are required. A large number of video retrieval algorithms have been presented for framewise user query or video content query, whereas few video-sequence matching algorithms have been investigated. In this paper, we propose an efficient algorithm for video sequence matching using the modified Hausdorff distance and the directed divergence of histograms between successive frames. To effectively match the video sequences with a low computational load, we use the key frames extracted by the cumulative directed divergence and compare the set of key frames using the modified Hausdorff distance. Experimental results with color video sequences show that the proposed algorithms for video sequence matching yield better performance than conventional algorithms such as histogram difference, histogram intersection, and chi-square test methods.

This paper points out the incorrect expressions of Uenohara and Kanade (see ibid., vol.7, p.116-19, 1998), in the context of the representation of the eigenvectors based on the discrete cosine transform (DCT). With the repeated eigenvalues, the eigenvector matrix of the P x P real symmetric circulant matrix can be constructed using the singular value decomposition (SVD), where P denotes the number of uniformly rotated images. Or equivalently it can be formulated in terms of the discrete Hartley transform (DHT). An example with P=4 is presented to show the correctness of our analysis.

This paper proposes the localization algorithm that estimates translation parameters of an aircraft by comparing the sampled elevation map recovered from aerial sequence images, and the digital elevation model (DEM) with the given orientation and altitude parameters obtained from a gyroscope. It consists of two stages: recovering the sampled elevation map from multiple aerial image pairs and matching of the relative recovered elevation map (REM) with the relative DEM. Computer simulations with three real aerial image sequences show the effectiveness of the proposed algorithm.

This paper proposes an oriented Hausdorff similarity (OHS) measure for robust object alignment. The OHS measure is introduced by replacing the distance concept of conventional Hausdoff distance (HD) algorithms by the similarity concept of the Hough transform (HT). The proposed algorithm can be considered as the modified directed HT using the distance transform (DT). The orientation information at each pixel is also used to remove incorrect correspondences.

Two coding methods for arbitrarily shaped objects in still images using the lapped orthogonal transform (LOT) are proposed. The LOT is applied to the projection onto convex sets (POCS) based algorithm and the shape adaptive-discrete cosine transform (SA-DCT) with the even number of basis vectors. Simulation results show improved reconstruction quality compared with the conventional methods.

In huge video databases, an effective video indexing method is required. While manual indexing is the most effective approach to this goal, it is slow and expensive. Thus automatic indexing is desirable, and previously various indexing tools for video databases have been developed. For efficient video indexing and retrieval, the similarity measure is an important factor. This paper presents new similarity measures between frames and proposes a new algorithm to detect scene changes using a cross entropy defined between two histograms. Experimental results show that the proposed algorithm is fast and effective compared with several conventional algorithms to detect abrupt scene changes and gradual transitions including fade in/out and flash light scenes.

This paper proposes a matrix which represents the reliability of the rotation components of the iterative closest point (ICP) algorithm in range image registration. We show that the reliability of the ICP algorithm depends on the surface normal vectors of the object.

This paper proposes two Hausdorff distance (HD) matching algorithms, in which robust HD measures are implemented in pyramidal structures. By computer simulations, the matching performance of the conventional HD measures and the proposed robust HD matching algorithms using pyramidal structures is compared, with real images which are degraded by noise and occlusions.

This work presents a hybrid method for navigation parameter estimation using sequential aerial images, where navigation parameters represent the position and velocity information of an aircraft for autonomous navigation. The proposed hybrid system is composed of two parts: relative position estimation and absolute position estimation. Computer simulation with two different sets of real aerial image sequences shows the effectiveness of the proposed hybrid parameter estimation algorithm.

A Hausdorff distance (HD) is one of commonly used measures for object matching. This work analyzes the conventional HD measures and proposes two robust HD measures based on m-estimation and least trimmed square (LTS) which are more efficient than the conventional HD measures. By computer simulation, the matching performance of the conventional and proposed HD measures is compared with synthetic and real images.

A fast coding algorithm for images using vector quantization (VQ) and pixelwise fractal approximation is proposed. The low frequency component of an input image is approximated and its residual is used to calculate the scaling factor of the fractal transform. The scaling factor is compressed by transform VQ (TVQ). In the proposed method, to encode a digital image by an iterated function system (IFS), we use the pixel-based IFS (PIFS) rather than the block-based IFS: the scaling factor is computed for each pixel. In the proposed method, the scaling factor of each pixel is calculated with the constraint of contraction mapping and it is transformed by wavelets and quantized by VQ. For approximation of an original image, the variable block-size segmentation using quadtrees is employed. Because the proposed method calculates the scaling factor using the PIFS, the encoding time is faster than the conventional algorithm using block-based IFS with search.

We propose a novel image segmentation technique using the robust, adaptive least kth order squares (ALKS) estimator which minimizes the kth order statistics of the squares of residuals. The optimal value of k is determined from the data, and the procedure detects the homogeneous surface patch representing the relative majority of the pixels. The ALKS shows a better tolerance to structured outliers than other recently proposed similar techniques. The performance of the new, fully autonomous, range image segmentation algorithm is compared to several other methods.

This paper proposes a motion estimation algorithm that is robust to motion discontinuity and noise. The proposed algorithm is constructed by embedding the least median squares (LMedS) of robust statistics into the maximum a posteriori (MAP) estimator. Difficulties in accurate estimation of the motion field arise from the smoothness constraint and the sensitivity to noise. To cope robustly with these problems, a median operator and the concept of reweighted least squares (RLS) are applied to the MAP motion estimator, resulting in the reweighted robust MAP (RRMAP). The proposed RRMAP motion estimation algorithm is also generalized for multiple image frame cases. Computer simulation with various synthetic image sequences shows that the proposed algorithm reduces errors, compared to three existing robust motion estimation algorithms that are based on M-estimation, total least squares (TLS), and Hough transform. It is also observed that the proposed algorithm is statistically efficient and robust to additive Gaussian noise and impulse noise. Furthermore, the proposed algorithm yields reasonable performance for real image sequences.

This paper proposes a two-stage global motion parameter estimation method using block-based motion vectors and a model failure (MF) object compensation algorithm by object-oriented fractal mapping of the residual image for an object-oriented coder. In the first stage of the two-stage motion parameter estimation algorithm, coarse motion parameters are estimated by fitting block-based motion vectors computed hierarchically to a six-parameter model, and in the second stage, the estimated motion parameters are refined by the gradient method using an image reconstructed by motion parameters detected in the first stage. The local prediction error by the six-parameter method is locally reduced by blockwise motion parameter correction using the residual image. Finally, the MF object is compensated by object-oriented fractal mapping of the previous residual image into the current one, in which geometric affine mapping is followed by the massic transformation. For MF object compensation, the selection between motion parameter compensation and fractal mapping is achieved by the validity test. Computer simulation results show that the proposed method gives better performance than the conventional ones in terms of the peak signal-to-noise ratio (PSNR) and compression ratio (CR).

The structure extraction problem is analyzed. The cooccurrence matrices (CMs) are the popular basis for this goal. We show that a binary preparation of an arbitrary periodical texture preserves its structure. This transformation decreases the computation time of analysis and the required memory. Twenty-two features adapted for detecting displacement vectors on binarized images are analyzed and compared. We suggest using the CM elements jointly as the united feature for this goal. We show that it is a stable detector for noisy images and simpler than well-known /spl chi//sup 2/ and /spl kappa/ statistics.

This paper proposes hardware architectures of adaptive equalizers applicable to both the quadrature amplitude modulation (QAM) and vestigial sideband modulation (VSB) systems. This paper presents digitization methods for QAM and VSB systems requiring a low hardware cost with the performance comparable to that of the algorithm employing floating-point operations. To reduce the hardware cost of the high definition television (HDTV) equalizers, we also propose a pipelined architecture that processes some parallel parts sequentially.

For quadrature-amplitude modulation (QAM) systems, the joint blind equalization (JBE) algorithm, concatenation of two different equalization modes, has been widely used. After the eye is opened by the blind equalization mode, the decision-directed (DD) mode follows to improve the equalization performance with fast convergence. This paper proposes a shell partition-based constant modulus algorithm (CMA) (SPCMA) which acts as the DD mode in the JBE. Computer simulation results for 16-QAM show the effectiveness of the proposed JBE.

This paper proposes a new systolic array architectures for computation of the 1-D discrete wavelet transform (DWT). The proposed systolic array consists of L processing element (PE) arrays, where L denotes the number of levels. The proposed PE array computes only the product terms that are required for further computation in higher levels, and the outputs of lowpass and highpass filters are computed in alternate clock cycles. Therefore, the proposed architectures can compute the one-level DWT using a single architecture. Note that the proposed architectures do not require extra processing units whereas the conventional architectures need ones. The required time and hardware cost for computation of the DWT using the proposed systolic arrays are comparable to that of the conventional architectures. The proposed architectures can also be applied to subband decomposition by simply changing the filter coefficients.

This paper presents two-layer object-oriented analysis-synthesis coding that makes the amount of transmitted data flexible and scalable at each layer. Object-oriented analysis-synthesis coding that estimates motion, shape, and color parameters for each object is considered as one of promising candidates for the MPEG-4 standard. The parameters are obtained through image analysis and the proposed two-layer coding algorithm is based on the object-oriented analysis-synthesis coding method. In this method, shape parameters are decomposed into two parts with a criterion of maximum area detection, by which the amount of data for shape parameters and the other parameters can be divided into two levels of priority. The proposed two-layer method shows the possibility of adaptation of MPEG-4 to asynchronous transfer mode (ATM) networks. Simulation results in terms of the bit rate and the peak signal to noise ratio (PSNR) show the functionality of the proposed two-layer coding method.

This paper proposes a method that combines 3-D compression with segmentation-based coding, in which 1-D temporal decomposition by the discrete cosine transform (DCT) and 1-D wavelet packet decomposition are employed. The proposed 1-D DCT/WT consists of a variable (2-, 3-, and 4-) point 1-D DCT for effective reduction of temporal redundancy and wavelet packet decomposition for adaptive selection of spatial frequency bands. Effectiveness of the proposed method is shown by computer simulation.

This paper proposes progressive image transmission (PIT) algorithms based on the wavelet packet transform. In the proposed methods quantization steps are optimized at each stage in an embedded manner for the given bit rate. After quantization selected subbands are ordered by their priorities for transmission. Subjective quality of the reconstructed image is improved by human visual system (HVS) weighting.

We propose unified systolic arrays for computation of the one-dimensional (1-D) and two-dimensional (2-D) discrete cosine transform/discrete sine transform/discrete Hartley transform (DCT/DST/DHT). By decomposing the transforms into even- and odd-numbered frequency samples, the proposed architecture computes the 1-D DCT/DST/DHT. Compared to the conventional methods, the proposed systolic arrays exhibit advantages in terms of the number of PE's and latency. We generalize the proposed structure for computation of the 2-D DCT/DST/DHT. The unified systolic arrays can be employed for computation of the inverse DCT/DST/DHT (IDCT/IDST/IDHT).

This paper proposes a multiresolution surface parameter estimation method for range images. Based on the robust estimation, it approximates a patch to a planar or quadric surface in the locally adaptive window. Selection of resolution is made pixelwise by comparing a locally computed homogeneity measure with the global threshold calculated by the distribution of the approximation error. Experimental results show that the proposed multiresolution method well preserves step and roof edges and is robust to noise compared with the conventional parameter estimation methods.

In transmitting color images represented by the palette, the indices corrupted by the bit error cause serious image quality degradation in the reconstructed images at a receiver. This paper proposes an ordered palette construction algorithm based on the hue-saturation-intensity (HSI) color system for minimizing the reconstruction error. We define the average Euclidean distance (AED) as a new measure for comparing the performance of color image palette construction algorithms and show the effectiveness of the proposed method in the case of the bit error. Experimental results show that the proposed algorithm effectively reduces the number of mismatched colors caused by the corrupted indices and thus the reconstruction error.

A Hausdorff distance (HD) is one of commonly used measures for object matching. It calculates the distance between two point sets of edges in two-dimensional binary images without establishing correspondences. This paper analyzes the conventional HD measures based on robust statistics, and proposes robust HD measures based on M-estimation, least trimmed square (LTS), and /spl alpha/-trimmed mean methods. The matching performance by the conventional and proposed HD measures is compared with synthetic and real images.

This paper presents a method for navigation parameter estimation using sequential aerial images, where navigation parameters represent the velocity and position information of an aircraft for autonomous navigation. The proposed navigation parameter estimation system is composed of two parts: relative position estimation and absolute position estimation. Relative position estimation recursively computes the current velocity and position of an aircraft by accumulating navigation parameters extracted from two successive aerial images. However, simple accumulation of parameter values decreases reliability of the extracted parameters as an aircraft goes on navigating, resulting in a large position error. Therefore absolute position estimation is required to compensate for position error generated in the relative position estimation step. A hybrid absolute position estimation algorithm combining image matching and digital elevation model (DEM) matching is presented. In image matching, line segment matching or Hausdorff distance (HD) matching is employed whereas in DEM matching a new algorithm for absolute position estimation by minimizing the variance of displacements is proposed. Computer simulation with real aerial image sequences shows the effectiveness of the proposed algorithm.

We investigate hardware implementation of block matching algorithms (BMAs) for motion estimation of moving sequences. Using systolic arrays, we propose VLSI architectures for the two-stage BMA and full search (FS) BMA. The two-stage BMA using integral projections reduces greatly the computational complexity with its performance comparable to that of the FS BMA. The proposed hardware architectures for the two-stage BMA and FS BMA are faster than the conventional hardware architectures with lower hardware complexity. Also, the proposed architecture of the first stage of the two-stage BMA is modeled in VHDL and simulated. Simulation results show the functional validity of the proposed architecture.

In this paper, we propose a coding algorithm for still images using vector quantization (VQ) and fractal approximation, in which low-frequency components of an input image are approximated by VQ, and its residual is coded by fractal mapping. The conventional fractal coding algorithms indirectly used the gray patterns of an original image with contraction mapping, whereas the proposed fractal coding method employs an approximated and then decimated image as a domain pool and uses its gray patterns. Thus, the proposed algorithm utilizes fractal approximation without the constraint of contraction mapping. For approximation of an original image, we employ the discrete cosine transform (DCT) rather than conventional polynomial-based transforms. In addition, for variable blocksize segmentation, we use the fractal dimension of a block that represents the roughness of the gray surface of a region. Computer simulations with several test images show that the proposed method shows better performance than the conventional fractal coding methods for encoding still pictures.

In transmitting moving pictures, interframe coding is shown to be effective for compressing video data. A hierarchical motion vector estimation algorithm using mean pyramid is proposed. Using the same measurement window at each level of a pyramid, the proposed algorithm, based on the tree pruning, reduces the computational complexity greatly with its performance comparable to that of the full search (FS). By varying the number of candidate motion vectors which are to be used as the initial search points for motion vector estimation at the next level, the mean squared error of the proposed algorithm varies, ranging between those of the FS and three step search (TSS) methods. Also, depending on the number of candidate motion vectors, the computational complexity of the proposed hierarchical algorithm ranges from 1/8-1/2 of that of the FS. The computer simulation results of the proposed technique compared with the conventional methods are given for various test sequences.>

We propose a coding algorithm for still images using fractal approximation and vector quantization (VQ). The conventional fractal coding algorithms indirectly used the gray patterns of an original image with contraction mapping, whereas the proposed fractal coding method employs a previously approximated image as a domain pool and uses its gray patterns. Thus, the proposed algorithm employs fractal approximation without the constraint of contraction mapping. To approximate an original image, we use an orthogonal polynomial transform and to encode the transform coefficients we employ VQ. Also, for variable block-size segmentation, we use the fractal dimension that represents the similarity of regions in a block and the roughness of the gray surface of a region. Computer simulations with several test images show that the proposed method shows better performance than the conventional fractal coding methods for encoding still pictures.>

Block-by-block motion compensation algorithms are studied for video-conference/video-telephone television signals. A fast feature-based block matching algorithm using integral projections for the motion vector estimation is proposed. The proposed algorithm reduces the motion estimation computations by a factor of two by calculating the one-dimensional cost functions rather than the two-dimensional ones. Also, the low sensitivity of the proposed algorithm to the presence of additive noise is shown experimentally. Simulation results based on the original and noisy image sequences are presented.>

Comments are made on the above named work by V. Lacroix (see ibid., vol.10, no.6, p.803-10, 1988). The commenters contend that the proposed orthogonal bases are the same as the edge subspace bases in the approach of W. Frei and C.C. Chen (IEEE Trans. Comput., vol.C-26, no.10, p.988-98, 1977) and that the ratio for detecting edges is the same as that proposed by Frei and Chen for special cases. In his reply, Lacroix explains the relationship between his method and that of Frei and Chen.>

The commenters point out explicitly that the fast search procedure of S.S. Redy et al. (see ibid., vol.SMC-14, no.4, p.661-5, (1984)) does not converge to the optimal threshold if the interclass variance, has not the unique maximum. The limitations of N. Otsu's technique (see ibid., vol.SMC-9, no.1, p.62-6, (1979)) are also discussed briefly.>