The long-tailed data distribution frequently occurs in the real-world scenarios, whereas deep learning is not effective enough for such distribution. In order to improve the effectiveness for the long-tailed data, data augmentation is widely used to balance the distribution of classes by generating new samples. However, most existing studies are designed from the perspective of the class-independence assumption by default, ignoring the effect of interrelation among classes for data augmentation, which causes that some generated samples may be unrepresentative and useless for balancing the class-distribution. Inspired by this, we propose a new data augmentation method based the sparse class-correlation exploitation in this paper, which can generate more representative samples by utilizing the class-correlation, to effectively balance the class-distribution for the long-tailed data. In the proposed method, a sparse class-correlation exploration module is first proposed to explore the potential correlations among multiple classes for boosting the classification performance. Based on the class-correlations, the pivotal seed-samples are generated by maximizing the sparse representation of challenging samples. Meanwhile, an ambiguity-filtered translation module is designed to generate more representative new samples for the target classes based the obtained seed-samples by enhancing the class-consistency and suppressing the deviation from the target classes. In addition, we introduce the self-supervised feature and fuse it with the discriminative feature to explore more accurate class-correlations. Experimental results illustrate that the proposed method obtains better performance only with a small number of generated samples than the state-of-the-art methods.

Connections between visual components are ubiquitous. Graphs, as a highly flexible data structure, not only allow imposing relational induction bias on data, but can provide a completely distinct learning perspective for regular image data. In this paper, we propose a hierarchical dynamic graph clustering network (HDGCN) for visual feature learning. We construct hierarchical graph representations in graph domain in an adaptive, data-adaptive and task-adaptive manner. First, the initial graph is constructed in high-dimensional feature domain of images. To mine the hierarchical geometric features in latent graph space, adaptive clustering network (ClusterNet) is performed to learn discriminative clusters and generates cluster-based coarse graph. Then, graph convolutional networks (GCNs) are used to diffuse, transform and aggregate information among clusters. So, the intra-class and inter-class information is fully explored to increase the discriminativity of graph representations. Next, coarsened graph representations are mapped to grid based on its affinity with linear projection features. To further improve the task adaptation of clusters and hierarchical graph representations, ClusterNet and GCNs are fused in the same framework for end-to-end training and clusters is updated dynamically. We have conducted extensive experiments on classification and segmentation tasks. The experimental results fully validate the robustness of the proposed algorithm.

In this paper, we propose a simple variant of the original SVRG, called variance reduced stochastic gradient descent (VR-SGD). Unlike the choices of snapshot and starting points in SVRG and its proximal variant, Prox-SVRG, the two vectors of VR-SGD are set to the average and last iterate of the previous epoch, respectively. The settings allow us to use much larger learning rates, and also make our convergence analysis more challenging. We also design two different update rules for smooth and nonsmooth objective functions, respectively, which means that VR-SGD can tackle non-smooth and/or non-strongly convex problems directly without any reduction techniques. Moreover, we analyze the convergence properties of VR-SGD for strongly convex problems, which show that VR-SGD attains linear convergence. Different from most algorithms that have no convergence guarantees for nonstrongly convex problems, we also provide the convergence guarantees of VR-SGD for this case, and empirically verify that VR-SGD with varying learning rates achieves similar performance to its momentum accelerated variant that has the optimal convergence rate O(1=T2). Finally, we apply VR-SGD to solve various machine learning problems, such as convex and non-convex empirical risk minimization, and leading eigenvalue computation. Experimental results show that VR-SGD converges significantly faster than SVRG and Prox-SVRG, and usually outperforms state-of-the-art accelerated methods, e.g., Katyusha.

Learning data representation is a fundamental problem in data mining and machine learning. Spectral embedding is one popular method for learning effective data representations. In this paper we propose a novel framework to learn enhanced spectral embedding, which not only considers the geometrical structure of the data space, but also takes advantage of the given pairwise constraints. The proposed formulation can be solved by an iterative eigenvalue thresholding (IET) algorithm. Specially, we convert the problem of learning spectral embedding with pairwise constraints into the one of completing an "ideal" kernel matrix. And we introduce the spectral embedding of graph Laplacian as the auxiliary information and cast it as a small-scale positive semidefinite (PSD) matrix optimization problem with nuclear norm regularization. Then, we develop an IET algorithm to solve it efficiently. Moreover, we also present an effective semi-supervised clustering (SSC) approach with learned spectral embedding (LSE). Finally, we validate the proposed IET algorithm and LSE approach by extensive experiments on real-world data sets.

Integrating new knowledge sources into various learning tasks to improve their performance has recently become an interesting topic. In this paper we propose a novel semi-supervised learning (SSL) approach, called semi-supervised learning with Mixed Knowledge Information (SSL-MKI) which can simultaneously handle both sparse labeled data and additional pairwise constraints together with unlabeled data. Specifically, we first construct a unified SSL framework to combine the manifold assumption and the pairwise constraints assumption for classification tasks. Then we present a Modified Fixed Point Continuation (MFPC) algorithm with an eigenvalue thresholding (EVT) operator to learn the enhanced kernel matrix. Finally, we develop a two-stage optimization strategy and provide an efficient SSL approach that takes advantage of Laplacian spectral regularization: semi-supervised learning with Enhanced Spectral Kernel (ESK). Experimental results on a variety of synthetic and real-world datasets demonstrate the effectiveness of the proposed ESK approach.

Based on an analysis of immune phenomena in nature and utilizing performances of ANN, a novel network mode, i.e., an immune neural network (INN), is proposed which integrates the immune mechanism and the function of neural information processing. The learning algorithm of an INN selects an excitation function and adaptive algorithm of the network. This model makes it easy for a user to utilize directly the characteristic information of a problem and to simplify the original structure by adjusting the excitation function with prior knowledge, improving the working efficiency and searching accuracy. A theoretical analysis and a simulation test for the twin-spiral problem show that, compared with an artificial neural network, INN is not only effective but also feasible. INN can simplify the structure of the existent model and show good working performance.