Most of previous research efforts on support vector machines (SVMs) were directed toward efficient implementations and practical applications. In this work, we concentrate on a different aspect of SVMs. Specifically, we investigate the non-uniqueness of SVM solutions. The key features of this work include (1) we concentrate on the cases where the dual solutions are not unique, whereas the primal solutions are unique; (2) our test for non-uniqueness can be directly applied to data points without solving the SVC optimization problem, namely, the non-uniqueness information is obtained before any numerical optimization procedure is employed.

Principal component analysis (PCA) has been shown to be very fruitful for extracting the most useful information from a given sequence of observations. Quite a number of PCA methods can be found in the literature. In this work, we concentrate on the derivation of optimal learning rates for some well-known adaptive PCA algorithms. A detailed derivation procedure is described which results in closed-form formulae of the optimal learning rates. These optimal learning rates can be obtained from the solution of either quadratic or cubic equations which can be analytically solved, where no numerical procedures are needed. The key advantage of the optimal learning rate is to offer a wise mechanism to automatically adjust the learning stepsize.

This work presents an adaptive principal component analysis (PCA) algorithm that equips the information theory-based algorithm with the momentum mechanism, which can be used to speedup the convergence and to stablize the weight trajectory. Issues such as equilibria, local stability, and convergence improvement are addressed. The theoretical analysis is largely facilitated by the ordinary differential equation (ODE) approach which characterizes the averaged convergence behavior of a nonlinear dynamic system. A demonstrative example is given to show the possible merits of this scheme. It is noteworthy to point out that the scope of this research is confined to local convergence improvement, whereas global convergence acceleration is a different issue that we have not covered.

Clustering is an unsupervised procedure to group objects in accordance with their similarities. For non-separable clusters, the concept of fuzziness is incorporated. Among other approaches, the fuzzy c-means algorithm is the most well-known fuzzy clustering method. In this work, we present a modified form of the fuzzy c-means based on a new definition of distance measure which can be considered as an extension of the conventional one. The key advantage of this new fuzzy clustering scheme is its ability to flexibly control the membership function curves. Analytical formulae have been derived for both cluster centers and the fuzzy partition matrix. Parameter effects related to the membership function curves have also been analyzed. Examples are given to demonstrate the clustering results of the newly presented scheme.

Support vector machines are currently a very active research area within machine learning, data mining, and other related research communities. This paper presents a new form of generalized support vector classifier that includes both the 1-norm and 2-norm SVCs as its special cases. We refer to this new SVC as the mixed-norm support vector classifier (or m-SVC). The dual form of the m-SVC optimization problem is explicitly derived. A decomposition-type algorithm is described to solve the large sample-size m-SVC problem. We give some examples to demonstrate the solvability of the m-SVC formulation and to illustrate the relations among the m-SVC and convectional 1-norm and 2-norm SVCs.

In this paper, we propose a fuzzy clustering (FC) approach to improve the template matching method described in (T. Wakahara et al., Proc. IEEE, vol.80, no.7, p.1181-1194, 1992) that demonstrated high recognition rates. Our focus is on the reduction of recognition time. The key idea is to organize the template pool in a structured manner so that matching can be performed more quickly. Through the hierarchical FC, the original template pool has the form of a decision tree where each internal node in the tree is represented by a cluster center. Since the number of class templates in each subcluster is less than the original cluster, the recognition time can be effectively reduced. An approximate formula for the number of distance calculations needed for the presented scheme is derived. The use of FC is to provide a mechanism for cluster overlapping so that in the middle of the recognition process almost 100% hit ratio can be obtained. In the experiments conducted, we obtained approximately a 3.4 times reduction of recognition time (using the simplified formula we derived) at only a 0.2% average loss of recognition rate.

Due to the seemingly uniqueness of physiological and behavioral characteristics of each individual, writer identification has shown to be a feasible task. Security is the primary reason to perform any kind of biometrics-related personal identification. There have been scarce research results for personal identification using online Chinese handwriting. In this paper, we present a novel approach for online writer identification based on the point distribution model (PDM). The PDM technique provides a means to describe the variations in shapes in a parametric form. As a statistical tool, the PDM combines the benefits of feature alignment and principal component analysis. By learning the eigenstructure of each writer’s handwriting, the writer’s specific style can be determined. Through projection onto the eigenspace of each individual’s handwriting, discriminative features are obtained and utilized in the recognition process. In this work, we use the sum of the strength of major eigenmodes as a similarity metric. From the twelve-people experiment conducted, the best writer identification rate obtained is 97.3% in average. The appealing results suggest that the proposed method is a promising approach.

On-line handwriting has been shown to be an effective tool for the input of Chinese characters. However, due to the high variabilities of handwriting, the recognition performance is somewhat limited. One approach to resolve this problem is to view the pattern mismatch as composed of two components: natural distortion component and residual discriminative component. By adopting a suitable deformation technique, the residual discriminative component can be extracted, which is then used for character recognition. The stroke-based affine transformation (SAT) is a prominent example of this approach. Although the SAT method proved to be successful, it did not fully exploit the structural nature of Chinese characters. This work extends Wakahara's SAT further by considering a higher-level character element, radical, which is commonly deemed as a true structural entity in Chinese characters. The proposed scheme contains two main parts: fuzzy stroke grouping and radical-based affine transformation (RAT). Simple yet effective fuzzy connectivity and transitivity concepts are adopted to extract pseudo-radicals. After the pseudo-radicals are found, an RAT is employed to deform each pseudo-radical so that the input test pattern can match the desired reference pattern more closely. A small-scale experiment was conducted to examine the feasibility of this approach. The results indicate some improvements of the presented approach over the SAT scheme and verify its applicability. Further performance improvement is expected if a writer has the tendency to write radicals more independently.

This paper presents a new analytic framework for dynamic location management of PCS networks. Based on the theory of hexagonal cellular patterns, a novel two-dimensional Markov walk model with six states is proposed to characterize the dynamic behavior of the intercell movements for a mobile station. We discover a broad class of isotropic processes having an identical uniform steady-state distribution of the six directions, but distinct circulant transition probability matrices. Six special isotropic processes exhibiting IID, directional, turning, ping-pong, h-spin, and t-spin mobility patterns are identified for performance comparison. We also generalize the selective paging strategy by introducing the concept of probabilistic selective paging (PSP) to reduce the paging cost. A unified analysis using recursive computations is carried out to compute the exact probability distribution for the number of per-call location updates. We calculate the location profile with respect to an incoming call and derive an exact expression of the average paging cost for PSP. With its ability to distinguish subtle performance variations and unveil new features of various dynamic location management schemes, our analytic framework is demonstrated to provide more precise and insightful results than conventional analysis.

The Karhunen-Loeve transform (KLT) is known to be the optimal transform for data compression. However, since it is signal dependent and lacks a fast algorithm, it is not used in practice. In this paper, a fast approximate Karhunen-Loeve transform (AKLT) is presented. This new transform is derived using perturbation theory of linear operators. Both the forward and inverse AKLT are analytically derived in closed forms. In addition, fast computational algorithms are developed for both the forward and inverse transforms. The order of computational complexity for the AKLT is N log2 N, which is the same as that of the DCT, the transform presently used in industrial practice. Performance comparisons reveal for a first-order Markov sequence that the AKLT performs better than the DCT in its energy compaction and signal decorrelation capabilities. Experiments on real images also demonstrate a definite superiority of the AKLT over the DCT when an adaptive scheme is used.

A new type of perfect reconstruction filter bank is presented in this paper., This new class of filter bank has the following distinct features at the same time: linear-phase, distortion-free, IIR-FIR filter pair, closed form expression, orthogonal projection, and factorization-free. In contrast, the other existing filter banks do not possess all these properties simultaneously. For example, the classic Quadrature Mirror Filter (QMF) does not have the perfect reconstruction nature. Another example is the Conjugate Quadrature Filter (CQF), which lacks the linear- phase property. A qualitative comparison of different types of filter banks is also included. This newly developed perfect reconstruction IIR-FIR filter bank is then extended to the 2-D case. Both separable and non-separable filters are considered. In addition, both rectangular downsampling and non-rectangular downsampling schemes are discussed. Numerical optimization procedures are used to compute the optimal filter coefficients. The results are listed in tables for quick reference.