Abstract In this paper, a novel tensor dimensionality reduction (TDR) approach is proposed, which maintains the local geometric structure of tensor data by tensor local linear embedding and explores the global feature by optimising global subspace projection. Firstly, we analyse the local linear feature of tensor data for learning the linear separable embedding of the tenor data. Furthermore, a global subspace projection distance minimisation strategy is introduced to extract the global characteristic of the tensor data. The aim of this strategy is to find an optimal low‐dimensional subspace for TDR. In particular, two novel TDR algorithms are developed by the ensemble of tensor local feature preservation and global subspace projection distance minimisation, which express the subspace projection optimisation as an iteration optimisation problem and a Rayleigh quotient problem, respectively. The extensive experimental results on tensor data classification and clustering have demonstrated the proposed algorithms performed well.

Manifold learning-based methods, such as LLE, capture the geometry of the data based on the assumption that the local structure of a manifold is linear. However, these methods may extract an inaccurate local structure when the nonlinearity of the data is obvious. In this paper, we propose a novel dimensionality reduction method with the ability to characterize the locally nonlinear geometry of the data by multilocal linearity. Specifically, we first construct a local area for each data point. And based on the overlapping of local areas, each data point will belong to and be linearly reconstructed from several local areas. Next, the set of linear coefficients used to reconstruct the data point constitutes the multilocal linear pattern (MLLP) which is used to characterize the local geometry of the data. Geometrically, the MLLP of a data point represents the hyperplanes in different directions passing through the current point. And the locally nonlinear surface where the data point is located is approximated by these hyperplanes, which is more accurate to reflect the geometry of the data. Then, MLLP is preserved to the embedding data space, and the dimension-reduced data can be obtained by minimizing the reconstruction errors. Finally, experiment results on various datasets demonstrate the effectiveness of the proposed method.

Abstract Tensor dimensionality reduction (TDR) is a hot research topic in machine learning, which learns data representations by preserving the original data structure while avoiding convert samples into vectors and solving the problem of the curse of dimensionality of tensor data. In the work, a novel TDR approach based on mode product and Hilbert–Schmidt Independence criterion (HSIC) is proposed. The contributions of authors' work is described as following: (1) HSIC measures the statistical correlation of two random variables. However, instead of measuring the statistical correlation of two random variables directly, HSIC first transforms the two random variables into two reproducing kernel Hilbert spaces (RKHSs), and then measures the statistical correlation of transformed random variables by using Hilbert–Schmidt operators between the two RKHSs. The exploitation of RKHS increases the flexibility and applicability of HSIC. Although HSIC is widely used in machine learning, the authors have not seen its application to dimensionality reduction (DR)(except for authors' previous work). (2) A novel HSIC‐based TDR approach is proposed, which first applies HSIC to capture statistical information of tensor data set for DR. The authors give the mathematical derivation of HSIC for tensor data and establish a framework of TDR based on HSIC, named HSIC‐TDR for short, which aims to improve the DR results of tensor by exploring and preserving the statistical information of original data set. (3) Furthermore, to solve the out‐of‐sample problem, the authors learn an explicit expression between the dimensionality‐reduced tensors and the higher‐dimensional tensors by introducing mode product to HSIC‐TDR. The experimental results between the proposed method and other state‐of‐the‐art algorithm on various datasets demonstrate the well performance of the proposed method.

In machine learning, the idea of maximizing the margin between two classes is widely used in classifier design. Enlighted by the idea, this paper proposes a novel supervised dimensionality reduction method for tensor data based on local decision margin maximization. The proposed method seeks to preserve and protect the local discriminant information of the original data in the low-dimensional data space. Firstly, we depart the original tensor dataset into overlapped localities with discriminant information. Then, we extract the similarity and anti-similarity coefficients of each high-dimensional locality and preserve these coefficients in the embedding data space via the multilinear projection scheme. Under the combined effect of these coefficients, each dimension-reduced locality tends to be a convex set where strongly correlated intraclass points gather. Simultaneously, the local decision margin, which is defined as the shortest distance from the boundary of each locality to the nearest point of each side, will be maximized. Therefore, the local discriminant structure of the original data could be well maintained in the low-dimensional data space. Moreover, a simple iterative scheme is proposed to solve the final optimization problem. Finally, the experiment results on 6 real-world datasets demonstrate the effectiveness of the proposed method.

In this article, a manifold learning algorithm based on straight-like geodesics and local coordinates is proposed, called SGLC-ML for short. The contribution and innovation of SGLC-ML lie in that; first, SGLC-ML divides the manifold data into a number of straight-like geodesics, instead of a number of local areas like many manifold learning algorithms do. Figuratively speaking, SGLC-ML covers manifold data set with a sparse net woven with threads (straight-like geodesics), while other manifold learning algorithms with a tight roof made of titles (local areas). Second, SGLC-ML maps all straight-like geodesics into straight lines of a low-dimensional Euclidean space. All these straight lines start from the same point and extend along the same coordinate axis. These straight lines are exactly the local coordinates of straight-like geodesics as described in the mathematical definition of the manifold. With the help of local coordinates, dimensionality reduction can be divided into two relatively simple processes: calculation and alignment of local coordinates. However, many manifold learning algorithms seem to ignore the advantages of local coordinates. The experimental results between SGLC-ML and other state-of-the-art algorithms are presented to verify the good performance of SGLC-ML.

Among the representative algorithms of manifold learning, Hessian locally linear embedding (HLLE) and local tangent space alignment (LTSA) algorithms haven been regarded as two different algorithms. However, in practice, the effects of these two algorithms are very similar and LTSA performs better than HLLE in some applications. This paper tries to account for this phenomenon from a mathematical point of view. There are only two differences between HLLE and LTSA. First, LTSA includes a data point into its neighborhood, while HLLE does not. Second, HLLE and LTSA use different methods to align the local coordinates of manifold. In this paper, we show that, the first difference between HLLE and LTSA is not essential. However, from the viewpoint of data utilization, LTSA does better than HLLE in the neighborhood construction. This may account for why LTSA can perform better than HLLE in some applications. As for the second difference between HLLE and LTSA, we first prove that, the alignment equations used by HLLE and LTSA are exactly the same. Second, we prove that, although HLLE and LTSA uses different methods to solve the alignment equation, their solutions are exactly the same, provided that HLLE adopts the same method as LTSA to construct the neighborhoods. Based on these arguments, we claim that HLLE and LTSA are equivalent to each other. This conclusion can also be verified experimentally by using manifold learning MATLAB demo (MANI), a widely-used experimental platform of manifold learning. When testing HLLE on MANI, if HLLE adopts the same method as LTSA to construct the neighborhoods, the experimental results presented by MANI will be the same as those of LTSA.

In manifold learning, alignment is performed with the objective of deriving the global low‐dimensional coordinates of input data from their local coordinates. In virtually all alignment processes, the relation between the local and global coordinates is designed intuitively, without mathematical deduction and detailed analysis. In this study, the authors propose a local non‐linear alignment manifold learning algorithm (LNA) for non‐linear dimensionality reduction, based on the concept of local pullback and the mathematical characteristics of a manifold. According to mathematical manifold theory, a function defined on a manifold cannot be differentiated directly on the manifold directly. Instead, it has to be pulled back to Euclidean space with the help of local homeomorphism between the manifold and Euclidean space, where it is then differentiated. In the authors’ proposed algorithm, the component functions of global homeomorphism are regarded as the functions defined on the manifold and pulled back to the Euclidean space. Then, Taylor expansion is utilised up to the second order to establish the relation between the global and local coordinates. The objective function in LNA is based on the alignment error and can be solved with an eigenvalue problem. The experimental results conducted on various datasets verify the validity of the authors’ method.

Histogram of oriented gradients (HOG) descriptor was initially applied to human detection and achieved great success. In recent years, HOG descriptor has also been applied to face recognition. However, comparing with other sophisticated feature descriptors such as LBP, Gabor and so on, there are still considerable research space on the application of HOG features for face recognition. There are two main contributions. On one hand, the main parameters are statistically analysed characterising HOG descriptor for face recognition, which seems to be not discussed clearly in literatures so far. On the other hand, a novel framework for face recognition based on the fusion of global and local HOG features has been proposed. Face images are first illumination normalised by the DoG filter. Secondly, global and local HOG features are extracted by PCA + LDA or LDA with different framework. Finally, in decision level, global and local classifiers are built by the nearest neighbour classifier, after that, two classifiers are fused by a weighted sum rule. Experimental results on two large‐scale face databases FERET and CAS‐PEAL‐R1 show that, in comparison with 12 state‐of‐the‐art approaches of face recognition, the proposed method achieves the highest average recognition rate.

In manifold learning, the neighborhood is often called a patch of the manifold, and the corresponding open set is called the local coordinate of the patch. The so-called alignment is to align the local coordinates in the d-dimensional Euclidean space to get the global coordinate of the manifold. There are two kinds of alignment methods: global and progressive alignment methods. The global alignment methods align the local coordinates of the manifold all at one time by solving an eigenvalue problem. The progressive alignment methods often take the local coordinate of a patch as the basic local coordinate and then attach other local ordinates to the basic local coordinate patch-by-patch until the basic local coordinate evolves into the global coordinate of the manifold. In this paper, a new progressive alignment method is proposed, where only the local coordinates of the two patches with the largest intersection at the current stage of progressive alignment will be aligned into a larger local coordinate. It is inspired by the famous Huffman coding, where two random events with the smallest probabilities at the current phase will be merged into a random event with a larger probability. Therefore, the proposed method is a Huffman-like alignment method. The experiments on benchmark data show that the proposed method outperforms both the global alignment methods and the other progressive alignment methods and is more robust to the changes of data size. The experiments on real-world data show the feasibility of the proposed method.

Dimensionality reduction is vital in many fields, and alignment-based methods for nonlinear dimensionality reduction have become popular recently because they can map the high-dimensional data into a low-dimensional subspace with the property of local isometry. However, the relationships between patches in original high-dimensional space cannot be ensured to be fully preserved during the alignment process. In this paper, we propose a novel method for nonlinear dimensionality reduction called local coordinates alignment with global preservation. We first introduce a reasonable definition of topology-preserving landmarks (TPLs), which not only contribute to preserving the global structure of datasets and constructing a collection of overlapping linear patches, but they also ensure that the right landmark is allocated to the new test point. Then, an existing method for dimensionality reduction that has good performance in preserving the global structure is used to derive the low-dimensional coordinates of TPLs. Local coordinates of each patch are derived using tangent space of the manifold at the corresponding landmark, and then these local coordinates are aligned into a global coordinate space with the set of landmarks in low-dimensional space as reference points. The proposed alignment method, called landmarks-based alignment, can produce a closed-form solution without any constraints, while most previous alignment-based methods impose the unit covariance constraint, which will result in the deficiency of global metrics and undesired rescaling of the manifold. Experiments on both synthetic and real-world datasets demonstrate the effectiveness of the proposed algorithm.

The goal of nonlinear dimensionality reduction is to find the meaningful low dimensional structure of the nonlinear manifold from the high dimensional data. As a classic method of nonlinear dimensional reduction, locally linear embedding (LLE) is more and more attractive to researchers due to its ability to deal with large amounts of high dimensional data and its noniterative way of finding the embeddings. However, several problems in the LLE algorithm still remain open, such as its sensitivity to noise, inevitable ill-conditioned eigenproblems, the inability to deal with the novel data, etc. The existing extensions are comprehensively reviewed and discussed classifying into different categories in this paper. Their strategies, advantages/disadvantages and performances are elaborated. By generalizing different tactics in various extensions related to different stages of LLE and evaluating their performances, several promising directions for future research have been suggested.