Knowledge graph embedding is an effective way to represent knowledge graph, which greatly enhance the performances on knowledge graph completion tasks, e.g., entity or relation prediction. For knowledge graph embedding models, designing a powerful loss framework is crucial to the discrimination between correct and incorrect triplets. Margin-based ranking loss is a commonly used negative sampling framework to make a suitable margin between the scores of positive and negative triples. However, this loss can not ensure ideal low scores for the positive triplets and high scores for the negative triplets, which is not beneficial for knowledge completion tasks. In this paper, we present a double limit scoring loss to separately set upper bound for correct triplets and lower bound for incorrect triplets, which provides more effective and flexible optimization for knowledge graph embedding. Upon the presented loss framework, we present several knowledge graph embedding models including TransE-SS, TransH-SS, TransD-SS, ProjE-SS and ComplEx-SS. The experimental results on link prediction and triplet classification show that our proposed models have the significant improvement compared to state-of-the-art baselines.

Network embedding is a promising topic that maps the vertices to the latent space while keeps the structural proximity in the original space. The network embedding task is difficult since the network vertices have no specific time or space orders. Models that used to extract information from images and texts with regular space or time structures can not be directly applied in network heading. The key feature of network embedding methods should be further exploited. Previous network embedding reviews mainly focus on the models and algorithms used in different methods. In this survey, we review the network embedding works in the stochastic perspective either in data side or model side. Roughly, the network embedding methods fall into three main categories: matrix based methods, random walk based methods and aggregated based methods. We focus on the applications of stochastic models in solving the challenges of network embedding in data processing and modeling following the line of the three categories.

Deep neural networks (DNNs) has drawn considerable attention in recent years as a result of their remarkable performace in many visual and speech recognition assignments. As the scale of tasks that need to solve is increasingly big, the networks used also become wider and deeper, resulting in millions or even billions of parameters needed. Deep and wide networks with large number of parameters bring many problems, including memory requirement, computation cost and overfitting, which severely hinder the application of DNNs in practice. Therefore, a natural thought is to train sparse networks with less parameters and float operators while maintaining comparable performance. During past few years, a mass of research has been proposed in this area. In this paper, we survey sparsity-promoting techniques in DNNs proposed in recent years. These approaches are roughly divided into three categories, including pruning, randomly reducing the complexity and optimizing with sparse regularizer. Pruning techniques will be introduced first and others will be described in the following section. For each kind of methods, we present approaches in this category, strengths and drawbacks. In the final, we will discuss the relationship of these three categories of methods.

Ordinal regression has received increasing interest in the past years. It aims to classify patterns by an ordinal scale. With the the explosive growth of data, the method of SVM with ordinal partitioning called SVMOP highlights its advantages due to its convenience of dealing with large scale data. However, the method of SVMOP for ordinal regression has not been exploited much. As we know, the costs should be different when dealing with mislabeled samples and how to use them plays a dominant role in model building. However, L2-loss which could enlarge the cost sensitivity has not been applied into SVM ordinal partition yet. In this paper, we propose the method of SVMOP with L2-loss for ordinal regression. Numerical results show that our approach outperforms the method of SVMOP with L1-loss and other ordianl regression models.