Due to their scalability and energy efficiency, memristor-based analog neuromorphic chips (MANCs) have significant advantages in edge computing. Researchers use software to train network parameters, then in hardware deployment, weights are translated into memristor conductances and biases are provided by digital-to-analog converters (DACs). Analog operational amplifiers are utilized for neural signal summation. Nowadays, efficient design, training, and implementation of MANCs have not been thoroughly explored. Particularly, MANC co-design considering software training and hardware deployment has not been studied. To address this issue, we propose Constraints-Aware Training (CAT), which includes hardware constraints of memristor devices, operational amplifiers, and DACs in the training process, thereby enabling co-design between offline training and hardware deployment. To evaluate the proposed methodology, a 4-layer fully connected network (FC-4) and a convolutional network (LeNet-5) are trained by CAT on MNIST and CIFAR-10 datasets, and then deployed in hardware circuits. Experimental results demonstrate that incorporating hardware constraints during offline training through CAT enables convenient and successful MANC hardware solutions.

Since sparse neural networks usually contain many zero weights, these unnecessary network connections can potentially be eliminated without degrading network performance. Therefore, well-designed sparse neural networks have the potential to significantly reduce the number of floating-point operations (FLOPs) and computational resources. In this work, we propose a new automatic pruning method-sparse connectivity learning (SCL). Specifically, a weight is reparameterized as an elementwise multiplication of a trainable weight variable and a binary mask. Thus, network connectivity is fully described by the binary mask, which is modulated by a unit step function. We theoretically prove the fundamental principle of using a straight-through estimator (STE) for network pruning. This principle is that the proxy gradients of STE should be positive, ensuring that mask variables converge at their minima. After finding Leaky ReLU, Softplus, and identity STEs can satisfy this principle, we propose to adopt identity STE in SCL for discrete mask relaxation. We find that mask gradients of different features are very unbalanced; hence, we propose to normalize mask gradients of each feature to optimize mask variable training. In order to automatically train sparse masks, we include the total number of network connections as a regularization term in our objective function. As SCL does not require pruning criteria or hyperparameters defined by designers for network layers, the network is explored in a larger hypothesis space to achieve optimized sparse connectivity for the best performance. SCL overcomes the limitations of existing automatic pruning methods. Experimental results demonstrate that SCL can automatically learn and select important network connections for various baseline network structures. Deep learning models trained by SCL outperform the state-of-the-art human-designed and automatic pruning methods in sparsity, accuracy, and FLOPs reduction.

Matrix factorization (MF) is a major technique for collaborative filtering of recommender systems. However, in the traditional MF model, it is difficult to tune the regularization parameter, and the predicted ratings may not lie within the given range. In this paper, we propose a new MF approach, in which MF is modeled as a constrained optimization problem and the constraint conditions are given in terms of the range of the factorization matrices. Under the new model, the regularization parameter is not needed and the predicted ratings are limited in the given range. We further provide a feasible direction method to solve the new model. Experimental results demonstrate that our approach outperforms the traditional MF.

A novel model of support vector classification with partial empirical risks given (P-SVC) is proposed. A sequential minimal optimization for P-SVC is also provided. P-SVC is an extension of the classical support vector classification (C-SVC) and can be used in the case where partial empirical risks are requested. The experiments on some artificial and benchmark datasets show P-SVC obtains a better classification accuracy and a more stable classification result than C-SVC does when partial empirical risks are known.

Gridding is the first and most important step to separate the spots into distinct areas in microarray image analysis. Human intervention is necessary for most gridding methods, even if some so-called fully automatic approaches also need preset parameters. The applicability of these methods is limited in certain domains and will cause variations in the gene expression results. In addition, improper gridding, which is influenced by both the misalignment and high noise level, will affect the high throughput analysis. In this paper, we have presented a fully automatic gridding technique to break through the limitation of traditional mathematical morphology gridding methods. First, a preprocessing algorithm was applied for noise reduction. Subsequently, the optimal threshold was gained by using the improved Otsu method to actually locate each spot. In order to diminish the error, the original gridding result was optimized according to the heuristic techniques by estimating the distribution of the spots. Intensive experiments on six different data sets indicate that our method is superior to the traditional morphology one and is robust in the presence of noise. More importantly, the algorithm involved in our method is simple. Furthermore, human intervention and parameters presetting are unnecessary when the algorithm is applied in different types of microarray images.

The gene expression data are usually provided with a large number of genes and a relatively small number of samples, which brings a lot of new challenges. Selecting those informative genes becomes the main issue in microarray data analysis. Recursive cluster elimination based on support vector machine (SVM-RCE) has shown the better classification accuracy on some microarray data sets than recursive feature elimination based on support vector machine (SVM-RFE). However, SVM-RCE is extremely time-consuming. In this paper, we propose an improved method of SVM-RCE called ISVM-RCE. ISVM-RCE first trains a SVM model with all clusters, then applies the infinite norm of weight coefficient vector in each cluster to score the cluster, finally eliminates the gene clusters with the lowest score. In addition, ISVM-RCE eliminates genes within the clusters instead of removing a cluster of genes when the number of clusters is small. We have tested ISVM-RCE on six gene expression data sets and compared their performances with SVM-RCE and linear-discriminant-analysis-based RFE (LDA-RFE). The experiment results on these data sets show that ISVM-RCE greatly reduces the time cost of SVM-RCE, meanwhile obtains comparable classification performance as SVM-RCE, while LDA-RFE is not stable.

In smooth support vector machine (SSVM), the plus function must be approximated by some smooth function, and the approximate error will affect the classification ability. This paper studies the smooth approximation to the plus function by piecewise polynomials. First, some standard piecewise polynomial smooth approximation problems are formulated. Then, the existence and uniqueness of solution for these problems are proved and the analytic solutions are achieved. The comparison between the results in this paper and the previous ones shows that the piecewise polynomial functions in this paper achieve better approximation to the plus function

This paper discusses the strategies of maximizing the sum of margins for ranking multi classification problem. First, the strategy of maximizing the sum of margins (MSM) is extended to maximizing the sum of weighted margins (MSWM). Using MSWM, a mathematical model is established to deal with the ranking multi classification problems where the importance of margins between classes is different, and its dual model is deduced. Then, by introducing the concept of algebraic margin, which is a generalization of geometric margin, the MSWM is further extended to maximizing the sum of weighed algebraic margins (MSWAM). Based on the MSWAM, the deduced mathematical model of the ranking multi classification problem not only has positive generalization ability, but is also a simple linear programming model