Abstract. Tensor robust principal component analysis (TRPCA), which aims to remove sparse noise or outliers of high-dimensional data with intrinsic low rank properties, has attracted extensive research and been widely used in various areas. In this paper, we focus on TRPCA based on the Tucker rank. Considering that computing singular value decompositions (SVDs) of all unfolding matrices in the convex relaxation model of TRPCA based on the Tucker rank is highly time consuming, we propose a two-layer TRPCA model (TTRPCA) based on the convex relaxation model. In TTRPCA, we select a mode according to the nuclear norm of all unfolding matrices, and only need to compute SVD of the matrix unfolded along this mode, which can capture more information of the original data with low-rankness compared with other unfolding matrices. Moreover, we establish a generalized nonconvex two-layer TRPCA model (NTRPCA). Unlike existing methods which usually use a specific nonconvex function, NTRPCA uses a class of nonconvex functions to approximate the rank function and the [Formula: see text] norm to more accurately capture the low rank structure and the sparsity. After that, we establish an error bound of the proposed NTRPCA model, which still holds for the TTRPCA model, and give some comparisons of cases using specific nonconvex functions. An alternating direction method of multipliers algorithm with convergence guarantee is then developed to solve the NTRPCA (as well as the TTRPCA) model. Finally, extensive numerical experiments on various datasets demonstrate the superior performance of proposed models in comparison with several state-of-the-art TRPCA methods.

Abstract. In this paper, a tensor completion method is proposed based on the Strassen–Ottaviani flattening, which can reveal the underlying tensor rank intrinsically. The resulting tensor completion optimization problem is formulated by spectral functions (convex or nonconvex) as surrogates for the rank function. An exact recovery result for the nuclear norm surrogate is given. An efficient method is proposed for this problem, and adaptive parameters for the spectral functions are allowed during the iterations. The global convergence is established under mild assumptions on the spectral functions and the adaptive parameter updates. In particular, we show that the class of weighted nuclear norms (either with nonincreasing or nondecreasing weights), the [Formula: see text]-sparsity index, and a class of weighted nuclear norms with adaptive weights, which are all widely employed in the literature, all fulfill the assumptions, and thus the global convergence is valid without any assumption. Numerical experiments on color images show that the proposed methods are promising, and always return images with better quality than some state-of-the-art methods.

Abstract. Tensor robust principal component analysis (TRPCA) is an important method to handle high-dimensional data and has been widely used in many areas. In this paper, we mainly focus on the TRPCA problem based on tensor fibered rank for sparse noise removal, which aims to recover the low-fibered-rank tensor from grossly corrupted observations. Usually, the [Formula: see text]-norm is used as a convex approximation of tensor rank, but it is essentially biased and fails to achieve the best estimation performance. Therefore, we first propose a novel nonconvex model named [Formula: see text], in which the [Formula: see text] norm ([Formula: see text]) is adopted to approximate tensor fibered rank and measure sparsity. Then, an error bound of the estimator of [Formula: see text] is established and this error bound can be better than those of similar models based on Tucker rank or tubal rank. Further, we use the alternating direction method of multipliers to solve [Formula: see text] and provide convergence guarantee. Finally, extensive experiments on color images, videos, and hyperspectral images demonstrate the effectiveness of the proposed method.

It is well-known that second-order optimizer can accelerate the training of deep neural networks, however, the huge computation cost of second-order optimization makes it impractical to apply in real practice. In order to reduce the cost, many methods have been proposed to approximate a second-order matrix. Inspired by KFAC, we propose a novel Trace-based Hardware-driven layer-ORiented Natural Gradient Descent Computation method, called THOR, to make the second-order optimization applicable in the real application models. Specifically, we gradually increase the update interval and use the matrix trace to determine which blocks of Fisher Information Matrix (FIM) need to be updated. Moreover, by resorting the power of hardware, we have designed a Hardware-driven approximation method for computing FIM to achieve better performance. To demonstrate the effectiveness of THOR, we have conducted extensive experiments. The results show that training ResNet-50 on ImageNet with THOR only takes 66.7 minutes to achieve a top-1 accuracy of 75.9 % under an 8 Ascend 910 environment with MindSpore, a new deep learning computing framework. Moreover, with more computational resources, THOR can only takes 2.7 minutes to 75.9 % with 256 Ascend 910.

Second-order optimization methods have the ability to accelerate convergence by modifying the gradient through the curvature matrix. There have been many attempts to use second-order optimization methods for training deep neural networks. In this work, inspired by diagonal approximations and factored approximations such as Kronecker-factored Approximate Curvature (KFAC), we propose a new approximation to the Fisher information matrix (FIM) called Trace-restricted Kronecker-factored Approximate Curvature (TKFAC), which can hold the certain trace relationship between the exact and the approximate FIM. In TKFAC, we decompose each block of the approximate FIM as a Kronecker product of two smaller matrices and scaled by a coefficient related to trace. We theoretically analyze TKFAC's approximation error and give an upper bound of it. We also propose a new damping technique for TKFAC on convolutional neural networks to maintain the superiority of second-order optimization methods during training. Experiments show that our method has better performance compared with several state-of-the-art algorithms on some deep network architectures.

In this paper, a novel feature extraction method based on an improved color local binary pattern (LBP) is proposed for color face recognition. Firstly, in a given neighborhood of every pixel, we choose some sampling points from three color channels simultaneously and the numbers of the sampling points from every channel may be different. Secondly, we use a new rule to select the threshold which does not always locate in the geometrical center of the given neighborhood. Thirdly, in order to excavate the potential of the proposed sampling method, we use the [Formula: see text]-uniform LBP to obtain the binary code of each pixel. In addition, we embed the Hamming distance into our method for improving the recognition rate of the proposed method. For evaluating the performance of our method, we implement the proposed method and several related methods on five public face databases: FERET, CMU-PIE, Georgia, FEI and Asian databases. Experimental results show that our method possesses higher recognition rates and lower computational cost than other related color face recognition methods.

In this paper, a customized sparse representation model is proposed to take advantage of the variational information for undersampled face recognition. The proposed model with the mixed norm is a generalization of the extended sparse representation-based classification model. This model guarantees the sparsity of representation coefficient and the robustness for the variational information from generic data set. The mixed norm well fits the distribution of variational information (such as illumination, expression, poses, and occlusion) and the interference information (somewhat face-specific in generic data set) simultaneously. We compare the proposed method with the related methods on several popular face databases, including AR, CMU-PIE, Georgia, and LFW databases. The experimental results show that the proposed method outperforms several popular face recognition methods.

Recently, a nonconvex relaxation of low-rank matrix recovery (LMR), called the Schatten-pquasi-norm minimization (0pp-isometry constants (0p≤ 1) and derive ap-RIP condition for exact reconstruction of LMR via Schatten-pquasi-norm minimization. In particular, we determine how many random, Gaussian measurements are needed for thep-RIP condition to hold with high probability, which gives a theoretical result that it needs fewer measurements with smallpfor exact recovery via Schatten-pquasi-norm minimization than whenp=1.