Large-scale multiobjective optimization problems (LSMOPs), characterized by a substantial number of decision variables, pose significant challenges for many existing evolutionary algorithms. However, the search efficiency of these algorithms is not yet satisfactory. This is mainly because that the search efficiency of these algorithms may deteriorate dramatically since the search space increases exponentially with the number of decision variables. Having this in mind, we proposed a large-Scale multiobjective optimization framework named causal inference-based competitive swarm optimizer (CI-CSO). Specifically, a causal-information-(CI)-based operator is designed for competitive swarm optimizers. First, a causal inference technique named information geometric causal inference (IGCI) is introduced to adequately explore the CI between decision variables and fitness values. To further distinguish the positive or negative impacts of these critical variables on solution quality, a CI processing module is designed, facilitating targeted optimization. To enhance search efficiency, CI-based offspring generator are employed, leveraging the variance of causal effects to dynamically adjust the search step size and sampling range. To evaluate its performance, the proposed CI-based operator is embedded into two multiobjective evolutionary algorithms (MOEAs) (LSTPA and LMOCSO). To demonstrate the effectiveness of the proposed framework, experimental results are presented using the LSMOP test suite and five real-world problems, each involving up to 10 000 decision variables. In addition, six classic algorithms are included for comparison.

A recurrent neural network (RNN) has shown powerful performance in tackling various natural language processing (NLP) tasks, resulting in numerous powerful models containing both RNN neurons and feedforward neurons. On the other hand, the deep structure of RNN has heavily restricted its implementation on mobile devices, where quite a few applications involve NLP tasks. Magnitude-based pruning (MP) is a promising way to address such a challenge. However, the existing MP methods are mostly designed for feedforward neural networks that do not involve a recurrent structure, and, thus, have performed less satisfactorily on pruning models containing RNN layers. In this article, a novel stage-wise MP method is proposed by explicitly taking the featured recurrent structure of RNN into account, which can effectively prune feedforward layers and RNN layers, simultaneously. The connections of neural networks are first grouped into three types according to how they are intersected with recurrent neurons. Then, an optimization-based pruning method is applied to compress each group of connections, respectively. Empirical studies show that the proposed method performs significantly better than the commonly used RNN pruning methods; i.e., up to 96.84% connections are pruned with little or even no degradation of precision indicators on the testing datasets.

Stochastic gradient descent (SGD) is a popular and efficient method with wide applications in training deep neural nets and other nonconvex models. While the behavior of SGD is well understood in the convex learning setting, the existing theoretical results for SGD applied to nonconvex objective functions are far from mature. For example, existing results require to impose a nontrivial assumption on the uniform boundedness of gradients for all iterates encountered in the learning process, which is hard to verify in practical implementations. In this article, we establish a rigorous theoretical foundation for SGD in nonconvex learning by showing that this boundedness assumption can be removed without affecting convergence rates, and relaxing the standard smoothness assumption to Hölder continuity of gradients. In particular, we establish sufficient conditions for almost sure convergence as well as optimal convergence rates for SGD applied to both general nonconvex and gradient-dominated objective functions. A linear convergence is further derived in the case with zero variances.

Most heuristic algorithms for NP-hard combinatorial optimization problems require expertise in both the problem domains and heuristic methods. Recent research has begun to apply Deep Neural Network to learning heuristics for combinatorial optimization problems automatically. These works mainly focus problems with simple formulations, such as Travelling Salesman Problem and Vehicle Routing Problem defined on Euclidean graphs. This paper presents a novel deep reinforcement learning based algorithm for the Capacitated Arc Routing Problem which is defined on more complex non-Euclidean information graphs. The proposed approach is a combination of a Graph Convolutional Network and two encoder-decoder models. By regrading the negative objective values of CARP instances as the rewards, the proposed method optimizes the parameters with REINFORCE algorithm. In empirical experiments, the proposed method is able to generate solutions approximate optimal solutions well with much less time than heuristic algorithms.

Deep neural networks (DNNs) have achieved great success, but the applications to mobile devices are limited due to their huge model size and low inference speed. Much effort thus has been devoted to pruning DNNs. Layer-wise neuron pruning methods have shown their effectiveness, which minimize the reconstruction error of linear response with a limited number of neurons in each single layer pruning. In this paper, we propose a new layer-wise neuron pruning approach by minimizing the reconstruction error of nonlinear units, which might be more reasonable since the error before and after activation can change significantly. An iterative optimization procedure combining greedy selection with gradient decent is proposed for single layer pruning. Experimental results on benchmark DNN models show the superiority of the proposed approach. Particularly, for VGGNet, the proposed approach can compress its disk space by 13.6× and bring a speedup of 3.7×; for AlexNet, it can achieve a compression rate of 4.1× and a speedup of 2.2×, respectively.

Layer-wise magnitude-based pruning (LMP) is a very popular method for deep neural network (DNN) compression. However, tuning the layer-specific thresholds is a difficult task, since the space of threshold candidates is exponentially large and the evaluation is very expensive. Previous methods are mainly by hand and require expertise. In this paper, we propose an automatic tuning approach based on optimization, named OLMP. The idea is to transform the threshold tuning problem into a constrained optimization problem (i.e., minimizing the size of the pruned model subject to a constraint on the accuracy loss), and then use powerful derivative-free optimization algorithms to solve it. To compress a trained DNN, OLMP is conducted within a new iterative pruning and adjusting pipeline. Empirical results show that OLMP can achieve the best pruning ratio on LeNet-style models (i.e., 114 times for LeNet-300-100 and 298 times for LeNet-5) compared with some state-of-the- art DNN pruning methods, and can reduce the size of an AlexNet-style network up to 82 times without accuracy loss.

The subset selection problem that selects a few items from a ground set arises in many applications such as maximum coverage, influence maximization, sparse regression, etc. The recently proposed POSS algorithm is a powerful approximation solver for this problem. However, POSS requires centralized access to the full ground set, and thus is impractical for large-scale real-world applications, where the ground set is too large to be stored on one single machine. In this paper, we propose a distributed version of POSS (DPOSS) with a bounded approximation guarantee. DPOSS can be easily implemented in the MapReduce framework. Our extensive experiments using Spark, on various real-world data sets with size ranging from thousands to millions, show that DPOSS can achieve competitive performance compared with the centralized POSS, and is almost always better than the state-of-the-art distributed greedy algorithm RandGreeDi.

Object detection from still images has been among the most active and challenging area in computer vision recently. In contrast, fully supervised object detection from video has rarely been investigated. In this paper, we propose an algorithm to improve the performance of object detection from video. Our proposed method is based on an empirical property that the trajectory of an object is important for detection in videos. We use object trajectory to filter outliers, determine probable location of object and correct detection errors. We compare our method with baseline method which regard video frames as still images directly. Experiments show that our method outperform the compared baseline in terms of average precision while inducing moderate computation overhead.