This work studies the low-rank tensor completion problem based on partially observed entries. Inspired by the success of matrix completion based on low-rank property and its tightest convex relaxation - nuclear norm, we borrow the idea of low-rank scheme for tensor recovery. However, different from the matrix case, singular values of tensors are not straightforward. As a contribution, we reformulate tensor’s nuclear norm into an equivalent form based on the unfoldings along each mode. We show that the new objective has nice properties by which we can make use of the alternating minimization method with Nesterov accelerated gradient descent. We prove the proposed algorithm has convergence rate. Numerical experiments demonstrate the superior performance of our proposed algorithm over its counterparts.

Sparse logistic regression is for classification and feature selection simultaneously. Although many studies have been done to solve $\ell_{1}$-regularized logistic regression, there is no equivalently abundant work on solving sparse logistic regression with nonconvex regularization term. In this paper, we propose a unified framework to solve $\ell_{1}$-regularized logistic regression, which can be naturally extended to nonconvex regularization term, as long as certain requirement is satisfied. In addition, we also utilize a different line search criteria to guarantee monotone convergence for various regularization terms. Empirical experiments on binary classification tasks with real-world datasets demonstrate our proposed algorithms are capable of performing classification and feature selection effectively at a lower computational cost.

Multi-task learning aims to boost the generalization performance of multiple related tasks simultaneously by leveraging information contained in those tasks. In this paper, we propose a multi-task learning framework, where we utilize prior knowledge in the relations between features. We also impose a penalty on the coefficients changing for each specific feature to ensure related tasks have similar coefficients on common features shared among them. In addition, we capture a common set of features via group sparsity. The objective is formulated as a non-smooth convex optimization problem, which can be solved with various methods, including (sub)gradient descent method, iterative shrinkage-thresholding algorithm (ISTA) with back-tracking, and its momentum variation - fast iterative shrinkage-thresholding algorithm (FISTA). In light of the sub-linear convergence rate of the methods aforementioned, we propose an asymptotically linear convergent algorithm with theoretical guarantee. Empirical experiments on both regression and classification tasks with real-world datasets demonstrate that our proposed algorithms are capable of improving the generalization performance of multiple related tasks.

Nonnegative matrix factorization (NMF) is widely used for clustering with strong interpretability. Among general NMF problems, symmetric NMF is a special one that plays an important role in graph clustering where each element measures the similarity between data points. Most existing symmetric NMF algorithms require factor matrices to be nonnegative, and only focus on minimizing the gap between similarity matrix and its approximation for clustering, without giving a consideration to other potential regularization terms which can yield better clustering. In this paper, we explore factorizing a symmetric matrix that does not have to be nonnegative, presenting an efficient factorization algorithm with a regularization term to boost the clustering performance. Moreover, a more general framework is proposed to solve symmetric matrix factorization problems with different constraints on the factor matrices.

Text generation tasks require that the generated text have certain diversity while ensuring the relevance. Traditional Seq2Seq models usually use cross entropy as the objective function. It demands the results keep strictly consistent with the ground truth texts, which easily leads to the lack of variability in generated texts. In this paper, we propose a novel framework, TransVAE, which applies Variational Auto-Encoder (VAE) to improve the Seq2Seq architecture. We design the Translator module to transform the latent variable spaces of origin input to target output, thus enhancing the diversity of generated texts and supporting semi-supervised learning. Moreover, we add attention and copy mechanisms to the TransVAE model to balance the relevance and diversity. Abundant experiments are carried out on three different string transduction tasks: dialogue generation, machine translation, and text summarization. The experiment results verify the effectiveness of our method.