We construct a highly regular and simple structured class of sparsely connected convolutional neural networks with rectifier activations that provide universal function approximation in a coarse-to-fine manner with increasing number of layers. The networks are localized in the sense that local changes in the function to be approximated only require local changes in the final layer of weights. At the core of the construction lies the fact that the characteristic function can be derived from a convolution of characteristic functions at the next coarser resolution via a rectifier passing. The latter refinement result holds for all higher order univariate B-splines.

In this paper, we formulated the optimal bit-allocation problem for a scalable codec for images/videos as a graph-based constrained vector-valued optimization problem with many optimal solutions, which are referred to as Pareto points. Pareto points are generally derived using weighted sum scalarization; however, it has yet to be determined whether all Pareto points can be derived using this approach. This paper addresses this issue. When presented as a theorem, our results indicate that as long as the rate-distortion function of each resolution is strictly decreasing and convex and the Pareto points form a continuous curve, then all Pareto points can be derived using scalarization. The theorem is verified using the state-of-the-art scalable coding method H.264/SVC and a scalability extension of High Efficiency Video Coding (HEVC). We highlight a number of easily interpretable Pareto points that represent a good trade-off between candidate resolutions. The proximity point is defined as the Pareto point closest to the ideal performance for each resolution. We also model the Pareto points as a function of total bit rate and demonstrate that the Pareto points at other target bit rates can be predicted.

Frames are the foundation of the linear operators used in the decomposition and reconstruction of signals, such as the discrete Fourier transform, Gabor, wavelets, and curvelet transforms. The emergence of sparse representation models has shifted of the emphasis of signal representation in frame theory toward sparse$l_{1}$-minimization problems. In this paper, we apply frame theory to the sparse representation of signals in which a synthesis dictionary is used for a frame and an analysis dictionary is used for a dual frame. We developed a novel dictionary learning algorithm (called Parseval K-SVD) to learn a tight-frame dictionary. We then leveraged the analysis and synthesis perspectives of signal representation with frames to derive optimization formulations for problems pertaining to image recovery. Our preliminary results demonstrate that the images recovered using this approach are correlated to the frame bounds of dictionaries, thereby demonstrating the importance of using different dictionaries for different applications.

The blind source separation (BSS) problem extracts unknown sources from observations of their unknown mixtures. A current trend in BSS is the semiblind approach, which incorporates prior information on sources or how the sources are mixed. The constrained independent component analysis (ICA) approach has been studied to impose constraints on the famous ICA framework. We introduced an alternative approach based on the null space component (NCA) framework and referred to the approach as the c-NCA approach. We also presented the c-NCA algorithm that uses signal-dependent semidefinite operators, which is a bilinear mapping, as signatures for operator design in the c-NCA approach. Theoretically, we showed that the source estimation of the c-NCA algorithm converges with a convergence rate dependent on the decay of the sequence, obtained by applying the estimated operators on corresponding sources. The c-NCA can be formulated as a deterministic constrained optimization method, and thus, it can take advantage of solvers developed in optimization society for solving the BSS problem. As examples, we demonstrated electroencephalogram interference rejection problems can be solved by the c-NCA with proximal splitting algorithms by incorporating a sparsity-enforcing separation model and considering the case when reference signals are available.

Blind image restoration is a non-convex problem which involves restoration of images from an unknown blur kernel. The factors affecting the performance of this restoration are how much prior information about an image and a blur kernel are provided and what algorithm is used to perform the restoration task. Prior information on images is often employed to restore the sharpness of the edges of an image. However, no consensus is present regarding what prior information to use in restoring from a blur kernel due to complex image blurring processes. In this paper, we propose modelling of a blur kernel as a sparse linear combinations of basic 2-D patterns. Our approach has a competitive edge over the existing blur kernel modelling methods because our method has the flexibility to customize the dictionary design, which makes it well-adaptive to a variety of applications. As a demonstration, we construct a dictionary formed by basic patterns derived from the Kronecker product of Gaussian sequences. We also compare our results with those derived by other state-of-the-art methods, in terms of improvement SNR (ISNR).

The operator-based signal separation approach uses an adaptive operator to separate a signal into a set of additive subcomponents. In this paper, we show that differential operators and their initial and boundary values can be exploited to derive corresponding integral operators. Although the differential operators and the integral operators have the same null space, the latter are more robust to noisy signals. Moreover, after expanding the kernels of Frequency Modulated (FM) signals via eigen-decomposition, the operator-based approach with the integral operator can be regarded as the matched filter approach that uses eigen-functions as the matched filters. We then incorporate the integral operator into the Null Space Pursuit (NSP) algorithm to estimate the kernel and extract the subcomponent of a signal. To demonstrate the robustness and efficacy of the proposed algorithm, we compare it with several state-of-the-art approaches in separating multiple-component synthesized signals and real-life signals.

Multi-resolution and wavelet analysis have generated considerable interest in the field of mesh surface representation. In this paper, we propose a backward, coarse-to-fine framework that derives a semi-regular approximation of an original mesh, and demonstrate its effectiveness on level-of-detail and scalable coding applications. The framework is flexible and simple because the position of a new vertex at a finer resolution can be derived in a closed form, based on the affine combination of a subdivision scheme, the original mesh, and “new” information about the wavelet coefficients. We report the results of experiments on both applications; and also compare the scalable coding results with those of other methods.

Signal separation has a fundamental role in many image applications, such as noise removing (white noise, reflection, rain, etc), segmentation, and inpainting. To fulfill signal separation, morphological component analysis (MCA) has been widely deployed in a plenty of applications [1], [2], [3]. MCA uses dictionaries to model morphologies of subcomponents, but the coherence between dictionaries may cause the defect presenting in the obtained subcomponents [4], [5]. In this article, we replace the sparse coding of MCA with the weighted sparse coding, and by assigning heavier weights to dictionaries' highly coherent atoms, the defect presenting in the obtained subcomponents is reduced. The experimental results show that the proposed signal separation algorithm achieves a significant performance gain over MCA.

Morphological component analysis (MCA) for signal separation decomposes a signal into a super-position of morphological subcomponents, each of which is approximately sparse in a certain dictionary. Some of the dictionaries can also be modified to make them adaptive to local structure in images. We show that signal separation performance can be improved over the previous MCA approaches by replacing L1-norm optimization with “weighted” L1-norm optimization and replacing their dictionary adaptation with regularized dictionary adaptation. The weight on an atom for sparse coding is commonly derived from the corresponding coefficient's value. In contrast, the weight of an atom in a dictionary for signal separation is derived from the mutual coherence between the atom and the atoms in the other dictionaries. The proposed solution for regularized dictionary adaptation is an extension of the K-SVD method, where the dictionary and “weighted” sparse coefficients are estimated simultaneously. We present a series of experiments demonstrating the significant performance improvement of the proposed algorithm over the previous approaches for signal separation.

The operator-based signal separation approach uses an adaptive operator to separate a signal into additive subcomponents. And different types of operator can depict different properties of a signal. In this paper, we define a new kind of integral operator which can be derived from the second kind of Fredholm integral equation. Then, we analyze the properties of the proposed integral operator and discuss its relation to the second condition of Intrinsic Mode Function (IMF). To demonstrate the robustness and efficacy of the proposed operator, we incorporate it into the Null Space Pursuit algorithm to separate several multicomponent signals, including a real-life signal.

With the progress of model averaging algorithms, scientists in the field of brain research have an increasing demand for methods capable to register and warp source data to the pre-registered standard atlas. We here propose a thin-plate spline (TPS) based surface registration method to facilitate the registration and warping process of Drosophila brain data. Our contributions are twofold. First, the proposed method performs TPS-based registration in the parameterization domain, and hence it no longer needs a rigid transformation to globally align and scale the input models. Second, the obtained well-registered surface model can act as boundary constraints for further volumetric registration schemes. Experiments show that the proposed method is effective. For models with a 750-voxel-long bounding box diagonal, the average surface-to-surface distance is reduced to about 0.1-voxel-long after registration.

From the perspective of the Bayesian approach, the denoising problem is essentially a prior probability modeling and estimation task. In this paper, we propose an approach that exploits a hidden Bayesian network, constructed from wavelet coefficients, to model the prior probability of the original image. Then, we use the belief propagation (BP) algorithm, which estimates a coefficient based on all the coefficients of an image, as the maximum-a-posterior (MAP) estimator to derive the denoised wavelet coefficients. We show that if the network is a spanning tree, the standard BP algorithm can perform MAP estimation efficiently. Our experiment results demonstrate that, in terms of the peak-signal-to-noise-ratio and perceptual quality, the proposed approach outperforms state-of-the-art algorithms on several images, particularly in the textured regions, with various amounts of white Gaussian noise.

A number of techniques have been developed to deal with image denoising, which is regarded as the simplest inverse problem. In this paper, we propose an approach that constructs a Bayesian network from the wavelet coefficients of a single image such that different Bayesian networks can be obtained from different input images. Then, we utilize the maximum-a-posterior (MAP) estimator to derive the wavelet coefficients. Constructing a graphical model usually requires a large number of training images. However, we demonstrate that by using certain wavelet properties, namely, interscale data dependency, decorrelation between wavelet coefficients, and sparsity of the wavelet representation, a robust Bayesian network can be constructed from one image to resolve the denoising problem. Our experiment results show that, in terms of the peak-signal-to-noise-ratio (PSNR) performance, the proposed approach outperforms state-of-art algorithms on several images with various amounts of white Gaussian noise.

In this paper, we propose a spatially-varying deblurring method to remove the out-of-focus blur. Our proposed method mainly contains three parts: blur map generation, image deblurring, and scale selection. First, we derive a blur map using local contrast prior and the guided filter. Second, we propose our image deblurring method with L1-2 optimization to obtain a better image quality. Finally, we adopt the scale selection to ensure our output free from ringing artifacts. The experimental results demonstrate our proposed method is promising.

Recently developed were the Brainbow and Flybow techniques that can image and visualize a large number of neurons simultaneously; however, scientists still lack adequate tools to process this kind of colored multi-neuron image volumes. Due to dozens of colorized neuron fibers spreading densely in a very intricate structure, it is difficult to trace them by existing algorithms designed for single-neuron images. We proposed a framework to formulate and solve this issue, and the experimental results show that our method can successfully extract independent neurons from Flybow images. Consequently, the proposed procedure contributes to neuroscience by increasing the efficiency of collecting neuron information from Flybow images.

We investigate the wavelet-based scalable video coding problem and present a solution that takes account of each user's preferred resolution. Based on the preference, we formulate the bit allocation problem of wavelet-based scalable video coding. We propose three methods to solve the problem. The first is an efficient Lagrangian-based method that solves the upper bound of the problem optimally, and the second is a less efficient dynamic programming method that solves the problem optimally. Both methods require knowledge of the user's preference. For the case where the user's preference is unknown, we solve the problem by a min-max approach. Our objective is to find the bit allocation solution that maximizes the worst possible performance. We show that the worst performance occurs when all users subscribe to the same spatial, temporal, and quality resolutions. Thus, the min-max solution is exactly the same as the traditional bit allocation method for a non-scalable wavelet codec. We conduct several experiments on the 2D+t MCTF-EZBC wavelet codec with respect to various subscribers' preferences. The results demonstrate that knowing the users' preferences improves the coding performance of the scalable video codec significantly.

In this paper, we present a theoretical analysis of the distortion in multilayer coding structures. Specifically, we analyze the prediction structure used to achieve temporal, spatial, and quality scalability of scalable video coding (SVC) and show that the average peak signal-to-noise ratio (PSNR) of SVC is a weighted combination of the bit rates assigned to all the streams. Our analysis utilizes the end user's preference for certain resolutions. We also propose a rate-distortion (R-D) optimization algorithm and compare its performance with that of a state-of-the-art scalable bit allocation algorithm. The reported experiment results demonstrate that the R-D algorithm significantly outperforms the compared approach in terms of the average PSNR.

In this paper, we propose a multiple component predictive coding framework. We firstly separate the reconstructed image into several subcomponents; and then predict each subcomponent independently but encode them together. To separate image into multiple subcomponents, we also propose a fast operator-based image separation algorithm. With the help of multicomponent prediction strategy, our prediction results can achieve superior performance than the H.264/AVC intra frame prediction method for images containing rich textures. By adopting the residue coding method used in H.264/AVC, we compare the compression efficacy of our proposed algorithm with the state-of-art JPEG2000 and H.264/AVC intra frame compression algorithms in the experimental part. The numerical results show that our algorithm is better than both H.264/AVC intra frame coding algorithm and JPEG2000 algorithm for images with ample textures.

A texture representation should corroborate various functions of a texture. In this paper, we present a novel approach that incorporates texture features for retrieval in an examplar-based texture compaction and synthesis algorithm. The original texture is compacted and compressed in the encoder to obtain a thumbnail texture, which the decoder then synthesizes to obtain a perceptually high quality texture. We propose using a probabilistic framework based on the generalized EM algorithm to analyze the solutions of the approach. Our experiment results show that a high quality synthesized texture can be generated in the decoder from a compressed thumbnail texture. The number of bits in the compressed thumbnail is 400 times lower than that in the original texture and 50 times lower than that needed to compress the original texture using JPEG2000. We also show that, in terms of retrieval and synthesization, our compressed and compacted textures perform better than compressed cropped textures and compressed compacted textures derived by the patchwork algorithm.

In this paper, we propose a color image enhancement method that uses retinex with a robust envelope to improve the visual appearance of an image. The word “retinex” is hybird “retina” and “cortex”, suggesting that human visual perception is involved in this color image enhancement. To avoid the gray-world violation, a color-shifting problem, an input RGB color image is transformed into an HVS color image, but only the V component is enhanced. Furthermore, to prevent hallow artifacts, we construct a robust envelope with a gradient-dependent weighting to limit disturbances around intensity gaps such as edges and corners. Our experiment results show that the proposed method yields a better (almost hallow-free) performance than traditional image enhancement methods.

Performing optimal bit-allocation with 3-D wavelet coding methods is difficult because energy is not conserved after applying the motion-compensated temporal filtering (MCTF) process and the spatial wavelet transform. The problem cannot be solved by extending the 2-D wavelet coefficients weighting method directly and then applying the result to 3-D wavelet coefficients, since this approach does not consider the complicated pixel connectivity that results from the lifting-based MCTF process. In this paper, we propose a novel weighting method, which takes account of the pixel connectivity, to solve the problem and derive the effect of the quantization error of a subband on the reconstruction error of a group of pictures. We employ the proposed method on a 2-D + t structure with different temporal filters, namely the 5-3 filter and the 9-7 filter. Experiments on various coding parameters and sequences show that the proposed approach improves the bit-allocation performance over that obtained by using the weightings derived without considering the pixel connectivity in the MCTF process.

In this paper, we propose a novel way to derive the weighting for subbands that conserves energy. Once the weight of each subband has been calculated by motion-compensation temporal filtering, the effect of the quantization error of a subband on the reconstruction error of a group of pictures can be determined. Then, we weigh each subband and use the weighted coefficients to derive the optimal bit-allocation solution for 3-D wavelet coding methods. We apply the proposed method to bit-allocation on a 2D+t structure with a 5-3 temporal wavelet filter, and show that it achieves a 0.5–1.5 dB peak-signal-to-noise improvement over using the weighting of subbands on various coding parameters and sequences.

Improving the subjective quality and reducing the computational complexity of interpolation algorithms are important issues in video and network signal processing. To this end, we propose a fast adaptive image interpolation algorithm that classifies pixels and uses different linear interpolation kernels that are adaptive to the class of a pixel. Pixels are classified into regions relevant to the perception of an image, either in a texture region, an edge region, or a smooth region. Image interpolation is performed with Neville filters, which can be efficiently implemented by a lifting scheme. Since linear interpolation tends to over-smooth pixels in edge regions and texture regions, we apply the Laplacian operator to enhance the pixels in those regions. The results of simulations show that the proposed algorithm not only reduces the computational complexity of the process, but also improves the visual quality of the interpolated images.

Traditional approaches to receiver-driven layered multicast apply coarse-grain layered congestion control, which enables each receiver to adjust its receiving rate to match the available bandwidth. However, this restricts the scalability of fine-grain scalable video coding. In this paper, we present an approach that facilitates bandwidth inference congestion control in a hybrid coarse/fine layered multicast scheme for multimedia delivery. One-way delay trend detection has proven effective in explicitly or implicitly estimating end-to-end available bandwidth. Using a hierarchical layered probing scheme and a delay trend detection method, we have developed a congestion control protocol for fine-grain layered multicast. Combining this protocol with scalable video coding, we present a framework that achieves efficient scalable video streaming over heterogeneous networks. The performance of our hybrid coarse/fine layered multicast scheme is evaluated by network simulations and the video coding efficiency.

Inspired by Paragious and Deriche's work, which unifies boundary-based and region-based image partition approaches, we integrate the snake model and the Fisher criterion to capture, respectively, the boundary information and region information of microarray images. We then use the proposed algorithm to segment the spots in the microarray images, and compare our results with those obtained by commercial software. Our algorithm is automatic because the parameters are adaptively estimated from the data without human intervention.

Inspired by Paragious and Deriche's work, which unifies boundary-based and region-based image partition approaches, we integrate the active contour(snake) model and the Fisher criterion to capture, respectively, the boundary and region information of microarray images. We then use the proposed algorithm to automatically segment the spots in the microarray images, and compare our results with those obtained by commercial software.

We propose an AND-OR tree-based approach for structuring a variety of mode selections in a hybrid video coding system. The proposed approach can systematically analyze the input sequence and transform it into an AND-OR tree representation to allocate bits to each node of the tree. The motion vector of our system is estimated in the wavelet domain, where wavelet coefficients in different scales can be obtained by using different wavelets. We demonstrate the performance of the new features, and compare our codec's performance with that of an H.264/AVC-like implementation.

This paper proposes an automatic eye wink interpretation system for the severely handicapped people. First, we apply the support vector machine (SVM) and template matching algorithm to detect the eyes and then track the eye winks. Then, we convert the sequence of eye winks into code (binary digit) sequence. Finally, we use the dynamic programming to translate the eye wink sequence to a certain command for human-machine interface. In the experiments, our system demonstrates very good performance and high accuracy.

We propose a subspace-based maximum-likelihood approach for acquiring the timing of wavelet-based multirate transmission systems. The S-curve analysis shows that our approach can correctly acquire the initial timing of a symbol from anywhere within a symbol-time interval at a cost of increasing jitter variance.

In this paper, we propose a systematic approach that approximates a target dictionary to reduce the complexity of a matching pursuit encoder. We combine calculation of the inner products and maximum atom extraction of a matching pursuit video coding scheme based on eigendictionary approximation and tree-based vector quantization. The approach makes the codec design and optimization cleaner and more systematic than previous dictionary approximation methods. We vary the quality of approximation to demonstrate the tradeoff between computational complexity and coding efficiency. The experiment results show that our codec achieves speed-up factors of up to 100 with a performance loss of less than 0.1 dB. We use double-stimulus impairment scale scores to evaluate the perceptual quality of our approach for different levels of complexity.

We propose a new framework for multiple scalable bitstream video communications over lossy channels. The major feature of the framework is that the encoder estimates the effects of postprocessing concealment and includes those effects in the rate-distortion analysis. Based on the framework, we develop a rate-distortion optimization algorithm to generate multiple scalable bitstreams. The algorithm maximizes the expected peak signal-to-noise ratio by optimally assigning forward error control codes and transmission schemes in a constrained bandwidth. The framework is a general approach motivated by previous methods that perform concealment in the decoder, as in our special case. Simulations show that the proposed approach can be implemented efficiently and that it outperforms previous methods by more than 2 dB.

Research on texture synthesis has made substantial progress in recent years, and many patch-based sampling algorithms now produce quality results in an acceptable computation time. However, when such algorithms are applied, whether they provide good results for specific textures, and why they do so, are questions that have yet to be answered. In this article, we deal specifically with the second question by modeling the synthesis problem as one of learning from incomplete data, and propose an algorithm that is a generalization of patch-work approach. Through this algorithm, we demonstrate that the solution of patch-based sampling approaches is an approximation of finding the maximum-likelihood optimum by the generalized expectation and maximization (GEM) algorithm.

We propose an error control scheme for video communications over lossy channels. The proposed algorithm uses receiver post-processing in analysis to coordinate with multiple encoded streams which is capable of handling error concealment and error protection to achieve robust transmissions. Unlike previous methods, our algorithm focuses on joint optimizing multiple substreams distortion with concealment over error prone channels. The algorithm minimizes the expected rate-distortion function to achieve the optimal FEC result. In the experiments, we demonstrate the effectiveness of our method by using a 3-D SPIHT algorithm. Simulation results show that the proposed protection strategy achieves about 2 dB higher peak signal-to-noise ratio compared to conventional method.

In this paper, we propose an error control scheme for video communications over lossy channels. The proposed algorithm uses error concealment protection (ECP) approach to coordinate with multiple encoded streams which is capable of handling error concealment to achieve robust transmissions. Unlike previous methods, our algorithm focuses on joint optimizing multiple substreams distortion with concealment over error prone channels. The algorithm minimizes the expected rate-distortion function to achieve the optimal FEC result. In the experiments, we demonstrate the effectiveness of our method by using a 3-D SPIHT algorithm. Simulation results show that the proposed protection strategy achieves about 2 dB higher peak signal-to-noise ratio compared to conventional method.

Video coding schemes based on matching pursuits have been shown to have better coding efficiency and perceptual quality at low bit rates than DCT-based video coding schemes. Matching pursuit is a greedy algorithm that decomposes a signal into a linear combination of bases within an over complete dictionary. However, as the bases are not independent, the redundancy in linear combinations increases as the number of iterations of matching pursuit increases. In this paper, we introduce an orthonormal matching pursuit algorithm that reduces the redundancy between bases and represents a signal by a linear combination of bases more efficiently. Our proposed updated full search matching pursuit algorithm has been shown to be effective in both coding performance and computational complexity. Based on this algorithm, we present an orthonormal matching pursuit video coding scheme that achieves a more efficient coding performance. The coding efficiency and perceptual quality are evaluated and compared to traditional matching pursuit video coding schemes.

We compare, both objectively and subjectively, the performance of various advanced motion compensation methods, including overlapped block motion compensation (OBMC) and control grid interpolation (CGI), in a 3-D wavelet-based coding system. The motion vectors of the methods are obtained by using a sequence of 1-D dynamic programming algorithms that minimizes the cost function. Our experiment results indicate that an OBMC sequence usually has a higher peak signal-to-noise ratio (PSNR) than other methods, while a CGI sequence usually contains the fewest blocking artifacts. We provide a simple framework that combines the OBMC and CGI sequences. The proposed hybrid method removes more than 50% of the blocking artifacts of an OBMC sequence, while simultaneously maintaining a high PSNR performance.

Matching pursuit (MP) is a greedy algorithm that decomposes a signal into a linear combination of bases within an over-complete dictionary. The MP algorithm is usually only approximated due to its massive computational complexity. By approximating a residual in a subspace, we propose a multiple blocks MP update algorithm for video coding that achieves a faster and better MP approximation than traditional algorithms. As the subspace is dependent on a residual, our algorithm is content-dependent. We evaluate the coding performance, including the peak signal-to-noise ratio, runtime, and subjective quality, of our algorithm and compare it with traditional MP algorithms.

We use an optimization technique to accurately locate a distorted grid structure in a microarray image. By assuming that spot centers deviate smoothly from a checkerboard grid structure, we show that the process of gridding spot centers can be formulated as a constrained optimization problem. The constraint is equal to the variations of the transform parameter. We demonstrate the accuracy of our algorithm on two sets of microarray images. One set consists of some images from the Stanford Microarray Database; we compare our centers with those annotated in the Database. The other set consists of oligonucleotide images, and we compare our results with those obtained by GenePix Pro 5.0. Our experiments were performed completely automatically.

A prototype is representative of a set of similar objects. This paper proposes an approach that formulates the problem of prototype generation as finding the mean from a given set of objects, where the prototype solution must satisfy certain constraints. These constraints describe the important perceptual features of the sample shapes that the proposed prototype must retain. The contour prototype generated from a set of planar objects was used as an example of the approach, and the corners were used as the perceptual features to be preserved in the proposed prototype shape. However, finding a prototype solution for more than two contours is computationally intractable. A tree-based approach is therefore proposed in which an efficient greedy random algorithm is used to obtain a good approximation of the proposed prototype and analyze the expected complexity of the algorithm. The proposed prototype-generation process for hand-drawn patterns is described and discussed in this paper.

We propose an efficient error-resilient video transmission algorithm over packet erasure channels using optimal source and channel bit-allocation. This algorithm uses FEC and rate-distortion optimization to find the optimal allocation of source and channel bits in each quality layer. The packet loss probability is periodically reported to the video server. This method can also be incorporated with any coding structure that generates a set of independent compressed bitstreams. The algorithm efficacy is demonstrated by simulations in which the video compression is an error-resilient 3D-SPIHT algorithm and the channel protection is provided by Reed-Solomon (RS) codes.

This paper describes a new 3-D framework for the construction of a hybrid motion compensation model to reduce blocking artifacts in a highly scalable video coding system. Previous works have focused primarily on using control grid interpolation (CGI) or over-lapped block motion compensation (OBMC) to reduce blocking artifacts. However, both methods generate distinct side effects, such as PSNR degradation or poor de-blocking results, during the de-blocking process. The main objective of our proposed model is to achieve better visual quality and maintain satisfactory coding efficiency simultaneously by integrating the advantages of CGI and OBMC. The introduction of the dual mode selection between CGI and OBMC provides a more flexible mechanism to adjust our objective function and thereby achieves higher objective and subjective coding gains. Experimental results show that our proposed model achieves improved coding efficiency in terms of both PSNR and visual quality.

Matching pursuit (MP) video coding has been demonstrated to attain a better coding performance than DCT-based video coding in terms of PSNR and perceptual quality at very low bit rates. However, because of its massive computational complexity, the MP algorithm is usually only approximated. By approximating a residual in a subspace, we propose a multiple blocks search and update algorithm in MP video coding to achieve a faster and better MP approximation. In this paper, we evaluate the performance and compare it to the traditional one-block search algorithms. The experimental results show that our proposed algorithm can significantly improve the coding performance and encoding time.

We present an asymmetric watermarking method for copyright protection that uses different matrix operations to embed and extract a watermark. It allows for the public release of all information, except the secret key. We investigate the conditions for a high detection probability, a low false positive probability, and the possibility of unauthorized users successfully hacking into our system. The robustness of our method is demonstrated by the simulation of various attacks.

Complex-valued wavelets are normally used to measure instantaneous frequencies, while real wavelets are normally used to detect singularities. We prove that the wavelet modulus maxima with a complex-valued wavelet can detect and characterize singularities. This is an extension of the previous wavelet work of Mallat and Hwang on modulus maxima using a real wavelet. With this extension, we can simultaneously detect instantaneous frequencies and singularities from the wavelet modulus maxima of a complex-valued wavelet. Some results of singularity detection with the modulus maxima from a real wavelet and an analytic complex-valued wavelet are compared. We also demonstrate that singularity detection methods can be employed to detect the corners of a planar object.

We propose a clock acquisition algorithm for timing recovery for fractal modulation in additive white Gaussian noise (AWGN) and 1/f channels. Our acquisition algorithm uses exclusively the data redundancy inherently built in the fractal modulation to locate the beginning of timing of all the subbands simultaneously. Different diversity techniques can then be applied after the beginning of the data block is obtained. Our acquisition functions are derived by using the maximum-likelihood method. Using a serial search algorithm to maximize the acquisition function, we may find the timing point. Simulations have been conducted in evaluating the mean acquisition time of our algorithm.

We propose a new systematic approach to reduce matching pursuit (MP) encoder complexity. MP codecs are asymmetric as decoder complexity is low while the encoder complexity is extremely high. An MP encoder contains three components: the inner products, maximum atom extraction and atom encoding. We propose a new approach which combines the first two components using eigen-dictionary approximation and tree-based vector quantization (VQ). The advantages of this are a simpler design and a slower growth of computational costs as the target dictionary becomes large than traditional approaches. By varying the approximation accuracy, our algorithm can provide the trade-off between coding performance and speed-up of the MP encoder.

Image mosaicing is the act of combining two or more images and is used in many applications in computer vision, image processing, and computer graphics. It aims to combine images such that no obstructive boundaries exist around overlapped regions and to create a mosaic image that exhibits as little distortion as possible from the original images. In the proposed technique, the to-be-combined images are first projected into wavelet subspaces. The images projected into the same wavelet space are then blended. Our blending function is derived from an energy minimization model which balances the smoothness around the overlapped region and the fidelity of the blended image to the original images. Experiment results and subjective comparison with other methods are given.

The acquisition problem in wavelet-based modulation is very important. We discuss the problem of the ML-based method proposed by M. Luise et al. (see IEEE Trans. Commun., vol.48, p.1047-54, 2000) for timing acquisition and then we develop a novel acquisition algorithm. Our method uses the properties of the scaling function in the derivation of the acquisition function. We also show that the S-curve of our acquisition algorithm is smooth and has a unique zero during the signaling interval. Thus, we can acquire the correct symbol timing without ambiguity. The performance of our acquisition algorithm is evaluated by Monto-Carlo simulation using the Meyer wavelet.

We propose a naive algorithm for the timing recovery for fractal modulation. We first investigate the bit error rate due to the clock error of a fractal modulation. Then, we propose clock synchronization techniques for fractal modulation. Our acquisition algorithm uses exclusively the self-similar property of fractal for signal modulation. The performance of our algorithm is also demonstrated.

The prototype generation has been widely used in industrial design, medical imaging, computer animation, pattern recognition, and computer vision. We propose a new hierarchical tree based algorithm for generating a prototype contour from a class of similar planar solid objects. Our algorithm uses the wavelet moduli of a complex-valued wavelet to extract the contour features. Features of different objects are organized as a binary tree. The root of a subtree corresponds to the prototype contour for the contours of its children. We show that one can update this tree efficiently by only modifying the local subcontours. An experiment of our method to prototype generation and pattern recognition is demonstrated.

Since channel capacity varies depending on the network traffic and the capacity of each reception, fine granularity scalability (FGS) of video coding has emerged over the past years as an important area in multimedia streaming applications. We propose a FGS video codec and a two-layer SNR scalable video codec. Both of these are based on matching pursuits (MP). The bit plane atom coding is presented to achieve the SNR scalable matching pursuits. We also propose a differential quadtree-based algorithm to encode atom positions efficiently. The, performance of our methods are compared with those of H.263+ and DCT-based FGS coder.

Transmission of compressed images is challenging because the communication channels can be noisy and the compressed bitstreams are sensitive to channel errors. Most of the proposed methods in the literature do not guarantee the quality of the received image over noisy channels. We propose an ARQ-based diversity system as a solution, in which the use of the diversity system avoids the problems when one or some channels get congested, and the ARQ scheme guarantees the quality of the received image. A specific ARQ-based diversity system is designed for transmission of images compressed using EZW (embedded zerotree wavelet). Experiments performance results are shown.

The watermarking method has emerged as an important tool for content tracing, authentication, and data hiding in multimedia applications. We propose a watermarking strategy in which the watermark of a host is selected from the robust features of the estimated forged images of the host. The forged images are obtained from Monte Carlo simulations of potential pirate attacks on the host image. The solution of applying an optimization technique to the second-order statistics of the features of the forged images gives two orthogonal spaces. One of them characterizes most of the variations in the modifications of the host. Our watermark is embedded in the other space that most potential pirate attacks do not touch. Thus, the embedded watermark is robust. Our watermarking method uses the same framework for watermark detection with a reference and blind detection. We demonstrate the performance of our method under various levels of attacks.

We propose an integrated wavelet-based framework for an active contour model (snake) which is used for the motion tracking of deformable objects in a sequence of images. The input image frame is decomposed into a multiresolution representation using a wavelet transform. First, the wavelet transform coefficients are used in a multiresolution motion estimation process to find the initial contour in the frame. The presented multiresolution motion estimation method allows larger movement of the tracked object than the traditional image-based motion estimation. Secondly, the wavelet transform modulus at each scale is considered in the energy function of the active contour model. The application of biological cell tracking using the proposed method is shown as an example.

Image mosaic combines two or more images. It has found many applications in computer vision, image processing, and computer graphics. A common goal of the problem is to join two or more images such that there is an invisible boundary around the seam line and the mosaic image is as little distorted from the original images as possible. We propose a new image mosaic method by wavelet multiresolution analysis and variational calculus. We first project the images into wavelet spaces. The projected images at each wavelet space are then blended. Variational calculus techniques are applied to balance the quality between the smoothness around the seam line and the fidelity of the combined image relative to the original images in image blending. A mosaic image is finally obtained by summing the blended images at the wavelet spaces. Experimental results based on our method are demonstrated.

A pirate attack on an image aims to create an invisible modification of it. We propose a watermarking strategy which applies optimization techniques to the second order statistics of perceptually unaltered modifications of an image. The solution gives two orthogonal spaces. One of them characterizes most of the variations in the modification of the image. Our watermark is embedded in the other space that most potential pirate attacks do not touch. Thus, the embedded watermark is robust. We also show that our method is able to enhance many existing watermarking strategies. We also demonstrate the performance of our method.

We propose a codebook adaptation algorithm for matching pursuit low bit rate video compression. The matching pursuit dictionary is a set of basis functions. In practice, a function is approximated as a vector. According to the incoming data, the basis functions are modified similar to the minimum-point-finding algorithm in stochastic regression. Although matching pursuit low bit rate compressions have been studied intensively, bases adaptation in the dictionary is a new approach. We demonstrate the performance of our adaptation method on MPEG-4 video sequences.

We develop a multi-channel channel-optimized scheme for embedded zerotree wavelet (EZW) image compression in a noisy transmission environment. A block-based modification for EZW is applied to improve the robustness of EZW, and to produce several coded bitstreams for transmission over multiple channels with different noise conditions. Then the respective channel noise is considered in the rate-distortion analysis, and the resultant rate-distortion functions are used for optimal bit allocation among the coded bitstreams. The case of burst noise is analyzed as an example.

The two-dimensional (2-D) fractional Brownian motion (fBm) model is useful in describing natural scenes and textures. Most fractal estimation algorithms for 2-D isotropic fBm images are simple extensions of the one-dimensional (1-D) fBm estimation method. This method does not perform well when the image size is small (say, 32x32). We propose a new algorithm that estimates the fractal parameter from the decay of the variance of the wavelet coefficients across scales. Our method places no restriction on the wavelets. Also, it provides a robust parameter estimation for small noisy fractal images. For image denoising, a Wiener filter is constructed by our algorithm using the estimated parameters and is then applied to the noisy wavelet coefficients at each scale. We show that the averaged power spectrum of the denoised image is isotropic and is a nearly 1/f process. The performance of our algorithm is shown by numerical simulation for both the fractal parameter and the image estimation. Applications to coastline detection and texture segmentation in a noisy environment are also demonstrated.

We propose a new method that combines the use of a global threshold and a window-based scheme for computing local thresholds. The latter scheme compares the contrast of gray values within a neighborhood whose size varies with the scale of the objects being examined. To compute the scale quantity, a new wavelet model entitled Hadamard multiresolution analysis is also proposed. When the window-based scheme is applied to the areas where the global threshold is likely to fail, we obtain uniformly better binary results than using a global threshold only. Significant improvements to OCR performance can also be achieved by our binary results.

We show that the techniques of gain-shape VQ can be used in optimizing the dictionary of the matching pursuit codec. Various performance of our method were evaluated.

We apply the dynamic bit allocation to the block-based EZW algorithm for robust image compression. To optimize the performance of the bit allocation, the effect of the channel noise is concerned. The robustness of our method was evaluated.

This paper presents a study on statistical integration of temporal filter banks for robust speech recognition using linear discriminant analysis (LDA). The temporal properties of stationary features were first captured and represented using a bank of well-defined temporal filters. Then these derived temporal features can be integrated and compressed using the LDA technique. Experimental results show that the recognition performance can be significantly improved both in clean and in noisy environments.

In this correspondence, a method is proposed for estimating the surface orientation of a planar texture under perspective projection based on the ridge of a two-dimensional (2-D) continuous wavelet transform (CWT). We show that an analytical solution of the surface orientation can be derived from the scales of the ridge surface. A comparative study with an existing method is given.

A common assumption of the shape from texture problem is that a perceived image mainly contains only one type of texture with the same surface orientation. Unfortunately, a natural image is often composed of more than one textures. In order to solve the shape from texture problem in a practical manner, we need to segment 3D textured images. In this paper, we propose a new algorithm for the task. We estimate the local surface orientations from the the scales of the ridge points of continuous wavelet transform. Then, the local surface orientations are used as the features for texture segmentation. Textured images synthesized from Brodatz's album and several natural images demonstrate the performance of our method.

The designs and implementations of efficient aggregate data structures have been important issues in functional programming. It is not clear how to select a good representation for an aggregate when access patterns to the aggregate are highly variant, or even unpredictable. Previous approaches rely on compile--time analyses or programmer annotations. These methods can be unreliable because they try to predict a program's behavior before it is executed.We propose a probabilistic approach, which is based on Markov processes, for automatic selection of data representations. The selection is modeled as a random process moving in a graph with weighted edges. The proposed approach employs coin tossing at run--time to help choosing a suitable data representation. The transition probability function used by the coin tossing is constructed in a simple and common way from a measured cost function. We show that, under this setting, random selections of data representations can be quite effective. The probabilistic approach is used to implement bag aggregates, and the performance results are compared to those of deterministic selection strategies.

We propose a new shape from texture method based on the ridge of continuous wavelet transform. This method determines the orientations of a planar surface in a direct way under the perspective projection model. The variations of the image projected from a planar surface can be accurately characterized by the ridge of the continuous wavelet transform. The ridge of the 1-D signal and 2-D image are represented as a ridge curve and ridge plane, respectively. Ridges represent the energy concentration in the time-frequency plane where the energy is a local maxima. We show that the ridge of the projected image is a parabolic plane with a rotation angle equal to the tilt angle of the planar surface. The ridge is then rotated with the angle such that the slant effect appears in the X-axis and plays no role along the Y-axis. As a result, the rotated ridge plane can be regarded as the plane composed of many 1-D ridge curves. The slant angle of the 2-D image is thus obtained from the derived slant angle of the 1-D signal. A voting method and a curve fitting method are developed to obtain the slant angle of the 1-D signal. Several synthetic and real-world images have demonstrated the robustness and accuracy of our method.

An adaptive early jump-out technique for speeding up the block-based motion estimation is proposed. By using the new technique, we can speed up the full range search several times without losing the picture quality significantly. The proposed technique can also be embedded into almost all the existing fast motion estimation algorithms to speed up the computation further. Since the proposed technique can be embedded into the existing motion estimation algorithms, it can be applied to almost all the standard video codecs, such as the MPEG coder, and improve the coding speed of such codecs significantly. Our technique has been tested on the H.261 and the MPEG-I codecs, and the coding speed improves significantly.

This paper presents the use of a variety of filters in the temporal trajectories of frequency band spectrum to extract speech recognition features for environmental robustness.Three kind of filters for em- phasizing the statistically important parts of speech are proposed.First, a bank of RASTA-like band-pass filters to fit the statistical peaks of modulation frequency band spectrum of speech are used.Secondly, a three-channel octave band-iilter band with a smoothed rectangular window spline is applied.Thirdly, a datadriven filter is developed.Experimental results show that significant improvements for speech recognition using the proposed feature extraction approach under noisy environments can be achieved.

The authors present a new algorithmic procedure for the analysis of brain images. This procedure is specifically designed to image the activity and functional organization of the brain. The authors' results are tested on data collected and previously analyzed with the technique known as in vivo optical imaging of intrinsic signals. The authors' procedure enhances the applicability of this technique and facilitates the extension of the underlying ideas to other imaging problems (e.g., functional MRI). The authors' thrust is two fold. First, they give a systematic method to control the blood vessel artifacts which typically reduce the dynamic range of the image. They propose a mathematical model for the vibrations in time of the veins and arteries and they design a new method for cleaning the images of the vessels with the highest time variations. This procedure is based on the analysis of the singularities of the images. The use of wavelet transform is of crucial importance in characterizing the singularities and reconstructing appropriate versions of the original images. The second important component of the authors' work is the analysis of the time evolution of the fine structure of the images. They show that, once the images have been cleaned of the blood vessel vibrations/variations, the principal component of the time evolutions of the signals is due to the functional activity following the stimuli. The part of the brain where this function takes place can be localized and delineated with precision.

One can detect and characterize the singularities of a signal from the evolution of the wavelet transform coefficients across scales. The authors discriminate signal information from noise by using some prior knowledge of the properties of singularities. The wavelet transform of the signal is processed in order to remove the singularities created by the noise. The authors restore a sharp signal where part of the noise has been suppressed. Examples in one and two dimensions are shown.>

The mathematical characterization of singularities with Lipschitz exponents is reviewed. Theorems that estimate local Lipschitz exponents of functions from the evolution across scales of their wavelet transform are reviewed. It is then proven that the local maxima of the wavelet transform modulus detect the locations of irregular structures and provide numerical procedures to compute their Lipschitz exponents. The wavelet transform of singularities with fast oscillations has a particular behavior that is studied separately. The local frequency of such oscillations is measured from the wavelet transform modulus maxima. It has been shown numerically that one- and two-dimensional signals can be reconstructed, with a good approximation, from the local maxima of their wavelet transform modulus. As an application, an algorithm is developed that removes white noises from signals by analyzing the evolution of the wavelet transform maxima across scales. In two dimensions, the wavelet transform maxima indicate the location of edges in images.>