In the era of big data, there are increasing demands to forecast incomplete, sparse, and nonstationary data. The current research aims to solve these problems for multilinear time series through a combination of tensor autoregression and low-rank tensor factorization. We propose an expansion of TMF to second-order data: temporal tensor factorization (TTF). The current research aims to interpolate missing values via low-rank tensor factorization, which produces a latent time series. We perform forecasting in the latent space with a transform-based tensor autoregression ($\mathcal{L}$-TAR) process. We present experimental results of the proposed method with other state-of-the-art methods on the benchmark datasets which include video imaging, dynamic networks, and energy usage.

In the era of big data, there is massive demand for new techniques to forecast and analyze multi-dimensional data. One task that has seen great interest in the community is anomaly detection of streaming data. Toward this end, the current research develops a novel approach to anomaly detection of streaming 2-dimensional observations via multilinear time-series analysis and 3-dimensional tensor principal component analysis (3DTPCA). We approach this problem utilizing dimensionality reduction and probabilistic inference in a low-dimensional space. We first propose a natural extension to 2-dimensional tensor principal component analysis (2DTPCA) to perform data dimensionality reduction on 4-dimensional tensor objects, aptly named 3DTPCA. We then represent the sub-sequences of our time-series observations as a 4-dimensional tensor utilizing a sliding window. Finally, we use 3DTPCA to compute reconstruction errors for inferring anomalous instances within the multilinear data stream. Experimental validation is presented via MovingMNIST data. Results illustrate that the proposed approach has a significant speedup in training time compared with deep learning, while performing competitively in terms of accuracy.

Multilinear discriminantanalysis (MLDA), a novel approach based upon recent developments in tensor-tensor decomposition, has been proposed recently and showed better performance than traditional matrix linear discriminant analysis (LDA). The current paper presents a nonlinear generalization of MLDA (referred to as KMLDA) by extending the well known "kernel trick" to multilinear data. The approach proceeds by defining a new dot product based on new tensor operators for third-order tensors. Experimental results on the ORL, extended Yale B, and COIL-100 data sets demonstrate that performing MLDA in feature space provides more class separability. It is also shown that the proposed KMLDA approach performs better than the Tucker-based discriminant analysis methods in terms of image classification.

Two-directional two-dimensional principal component analysis ((2D$)^{2}$PCA) has shown promising results for it’s ability to both represent and recognize facial images. The current paper extends these results into a multilinear framework (referred to as two-directional Tensor PCA or 2DTPCA for short) using a recently defined tensor operator for 3rd-order tensors. The approach proceeds by first computing a low-dimensional projection tensor for the row-space of the image data (generally referred to as mode-l) and then subsequently computing a low-dimensional projection tensor for the column space of the image data (generally referred to as mode-3). Experimental results are presented on the ORL, extended Yale-B, COIL100, and MNIST data sets that show the proposed approach outperforms traditional “ tensor-based” PCA approaches with a much smaller subspace dimension in terms of recognition rates.

With the massive influx of 2-dimensional observational data, new methods for analyzing, modeling, and forecasting multidimensional data need to be developed. The current research aims to accomplish these goals through the intersection of time-series modeling and multi-linear algebraic systems. In particular, the current research, aptly named the $\mathcal{L}$-Transform Tensor Auto-Regressive ($\mathcal{L}$-TAR for short) model expands previous auto-regressive techniques to forecast data from multilinear observations as oppose to scalars or vectors. The approach is based on recent developments in tensor decompositions and multilinear tensor products. Transforming the multilinear data through invertible discrete linear transforms enables statistical Independence between observations. As such, can be reformulated to a collection of vector auto-regression problems for model learning. Experimental results are provided on benchmark datasets containing image collections, video sequences, sea surface temperature measurements, and stock closing prices.

The tensor singular value decomposition (t-SVD) based on an algebra of circulants is an effective multilinear sub- space learning technique for dimensionality reduction and data classification. Unfortunately, the computational cost associated with computing the t-SVD can become prohibitively expensive, particularly when dealing with very large data sets. In this paper, we present a computationally efficient approach for estimating the t-SVD by capitalizing on the correlations of the data in the temporal dimension. The approach proceeds by extending our prior work on fast eigenspace decompositions by transforming the tensor data from the spatial domain to the spectral domain in order to obtain reduced order harmonic tensor. The t-SVD can then be applied in the transform domain thereby significantly reducing the computational burden. Experimental results which are presented on the extended Yale-B, COIL-100, and MNIST data sets show the proposed method provides considerable computational savings with the approximated subspaces that are nearly the same as the true subspaces as computed via the t-SVD.

Advanced decision making using machine learning should be both accurate and interpretable. Many standard machine learning techniques suffer from an inherent lack of transparency with regard to how the resulting decision was made. In the current work we aim to overcome this issue by introducing a hybrid learning approach using classical decision trees alongside artificial neural networks, dubbed a "neural shrub". The Neural Shrub methodology presented in this paper aims to maintain as much interpretability as possible without sacrificing either classification or regression accuracy. Experimental results are presented on several benchmark data sets to validate the proposed approach as well as provide insight into future research directions.

The current paper presents a new approach to dimensionality reduction and supervised learning for classification of multi-class data. The approach is based upon recent developments in tensor decompositions and a newly defined algebra of circulants. In particular, it is shown that under the right tensor multiplication operator, a third order tensor can be written as a product of third order tensors that is analogous to a traditional matrix eigenvalue decomposition where the "eigenvectors" become eigenmatrices and the "eigenvalues" become eigen-tuples. This new development allows for a proper tensor eigenvalue decomposition to be defined and has natural extension to tensor linear discriminant analysis (LDA). Comparisons are made with traditional LDA and it is shown that the current approach is capable of improved classification results for benchmark datasets involving faces, objects, and hand written digits.

Kinematically redundant manipulators are inherently more robust to locked joint failures than non-redundant manipulators. However, if poorly designed, performance degradation may still occur in the presence of a single locked joint. This paper presents a technique for designing a desired operating workspace for a kinematically redundant manipulator that can be guaranteed after the occurrence of an arbitrary single locked joint failure. The existence of such a workspace, called a failure-tolerant workspace, will be guaranteed by imposing a suitable set of artificial joint limits prior to a failure. Conditions are presented that characterize end-effector locations within the failure-tolerant region. Based on these conditions, an algorithm for computing the failure-tolerant workspace is presented. The algorithm is based upon identifying the boundaries of the failure-tolerant workspace. Examples are presented to illustrate the application of the proposed algorithm to various manipulator design problems.

A tensor is a multidimensional array. First-order tensors and second-order tensors can be viewed as vectors and matrices, respectively. Tensors of higher order, with the ability to include more information, appear more frequently nowadays in image and signal processing, data mining, biomedical engineering, and so on. With the recent work of Kilmer and Martin, familiar matrix-based factorizations in linear algebra can be extended in a straightforward way to third-order tensors based on their new tensor multiplication and concepts. Our method has an advantage over a popular tensor-based face recognition algorithm called TensorFaces, which is based on the higher-order SVD of an arrangement of the images in a database, in that it does not require a least squares solve for the coefficients. In this paper, we give a brief introduction to the new tensor framework and apply the induced tensor decompositions to the application of facial recognition. In the numerical results, we compare our new approaches with the traditional principal component analysis method based on matrix decomposition (also known as eigenfaces) and with TensorFaces.

Dimensionality reduction and object classification (recognition and pose estimation) serve as important tools in robotics, robotic vision, and industrial automation. The current paper presents a new approach to dimensionality reduction and object classification of three-dimensional rigid objects. The approach is based upon recent developments in tensor decompositions and a newly defined algebra of circulants. In particular, it is shown that under the right tensor multiplication operator, a third order tensor can be written as a product of third order tensors in which the left and right tensors are tensor-orthogonal and the inner-tensor is a diagonal tensor of singular-tuples. This new development allows for a proper tensor singular value decomposition (SVD) to be defined and has natural extension to tensor principal component analysis (PCA). Comparisons are made with traditional PCA and it is shown that the current approach is capable of recovering significantly more information from an image sequence using a much smaller subspace dimension. Further, it is shown that for most objects, accurate pose estimation can be performed from a single subspace dimension.

Many appearance-based classification problems such as principal component analysis, linear discriminant analysis, and locally preserving projections involve computing the principal components (eigenspace) of a large set of images. Although the online expense associated with appearance-based techniques is small, the offline computational burden becomes prohibitive for practical applications. This paper presents a method to reduce the expense of computing the eigenspace decomposition of a set of images when variations in both illumination and pose are present. In particular, it is shown that the set of images of an object under a wide range of illumination conditions and a fixed pose can be significantly reduced by projecting these data onto a few low-frequency spherical harmonics, producing a set of "harmonic images." It is then shown that the dimensionality of the set of harmonic images at different poses can be further reduced by utilizing the fast Fourier transform. An eigenspace decomposition is then applied in the spectral domain at a much lower dimension, thereby significantly reducing the computational expense. An analysis is also provided, showing that the principal eigenimages computed assuming a single illumination source are capable of recovering a significant amount of information from images of objects when multiple illumination sources exist.

Eigendecomposition has been used to classify three-dimensional objects from two-dimensional images in a variety of computer vision and robotics applications. The biggest on-line computational expense associated with using eigendecomposition is the determination of the closest point on an image manifold embedded in a high-dimensional space. The dimensionality and complexity of the space is a result of the p principal eigenimages that are selected. Unfortunately, for some real-time applications, this search may be prohibitively expensive. This work presents a method to reduce the on-line expense associated with using eigendecomposition for pose estimation. The approach is based on selecting a linear combination of the principal eigenimages to design an eigenspace manifold having a desirable geometric structure that reduces the cost associated with classification.

Determining the pose of a three-dimensional object under unknown lighting conditions is a challenging problem. Eigenspace methods represent one computationally efficient method for doing illumination invariant pose estimation, and have been applied in a variety of application domains. Unfortunately, determining the appropriate eigenspace dimension, as well as the eigenspace itself, is computationally prohibitive for real-world applications. This paper presents a method to reduce this expense by using results from spectral theory. In particular, this paper shows that a set of images of an object under a wide range of illumination conditions and a fixed pose can be significantly reduced by projecting this data on to a few low-frequency spherical harmonics, producing a set of ¿harmonic images¿. It is then shown that the dimensionality of the set of harmonic images can be further reduced by utilizing the fast Fourier transform. An eigendecomposition is then applied in the spectral domain thus relieving the computational burden. Experimental results are presented to compare the proposed algorithm to the true eigendecomposition, as well as assess the computational savings.

Eigendecomposition represents one computationally efficient approach for dealing with object detection and pose estimation, as well as other vision-based problems, and has been applied to sets of correlated images for this purpose. The major drawback in using eigendecomposition is the off line computational expense incurred by computing the desired subspace. This off line expense increases drastically as the number of correlated images becomes large (which is the case when doing fully general 3-D pose estimation). Previous work has shown that for data correlated on S(1), Fourier analysis can help reduce the computational burden of this off line expense. This paper presents a method for extending this technique to data correlated on S(2) as well as SO3 by sampling the sphere appropriately. An algorithm is then developed for reducing the off line computational burden associated with computing the eigenspace by exploiting the spectral information of this spherical data set using spherical harmonics and Wigner-D functions. Experimental results are presented to compare the proposed algorithm to the true eigendecomposition, as well as assess the computational savings.

The pose detection of three-dimensional (3-D) objects from two-dimensional (2-D) images is an important issue in computer vision and robotics applications. Specific examples include automated assembly, automated part inspection, robotic welding, and human robot interaction, as well as a host of others. Eigendecomposition is a common technique for dealing with this issue and has been applied to sets of correlated images for this purpose. Unfortunately, for the pose detection of 3-D objects, a very large number of correlated images must be captured from many different orientations. As a result, the eigendecomposition of this large set of images is very computationally expensive. In this work, we present a method for capturing images of objects from many locations by sampling S2appropriately. Using this spherical sampling pattern, the computational burden of computing the eigendecomposition can be reduced by using the spherical harmonic transform to "condense" information due to the correlation in S2. We propose a computationally efficient algorithm for approximating the eigendecomposition based on the spherical harmonic transform analysis. Experimental results are presented to compare and contrast the algorithm against the true eigendecomposition, as well as quantify the computational savings.

In addition to possessing a number of other important properties, kinematically redundant manipulators are inherently more tolerant to locked-joint failures than non- redundant manipulators. However, a joint failure can still render a kinematically redundant manipulator useless if the manipulator is poorly designed or controlled. This paper focuses on the implementation issues involved in identifying a region of the workspace for which task completion is guaranteed in the event of a locked-joint failure for a general class of planar 3R manipulators. The existence of such a region, called a failure-tolerant workspace, will be guaranteed by imposing a suitable set of artificial joint limits prior to a failure. The authors have developed a graphical user interface (GUI) that computes all workspace boundaries of interest for the planar 3R manipulators. This GUI allows the user to explore different robot geometries, and adjust the artificial joint limits, in an attempt to gain further understanding of a manipulator's failure-tolerant workspace.