In this study, a multivariate time series forecasting model applicable to high-dimensional short-term forecasting is proposed. The Reservoir Computing Network (RCN) state reduction method is investigated to address the dimensionality arising from high-dimensional data. To enable the states to express very distant dependencies in time so that the RCN satisfies the echo state property, a bidirectional RCN model is investigated to capture the dependencies of the data forward and backward in time. To solve the short-term data prediction, the spatiotemporal transformation theory is combined with the RCN to exploit the conjugate property between delayed and nondelayed embeddings of dynamical systems, and the correlation between variables in high-dimensional systems is used to compensate for the shortcomings of short-term series, design the method of the RCN training, and complete the prediction of future states. In the experiments, four existing models based on two different data sets were used for comparison, and the results showed that our method has better prediction performance.

Non-local sparsity has been widely concerned in image compressive sensing. Considering the difference of distribution characteristic of among group-based sparse coefficients of image, a new method for image compressive sensing reconstruction (ICSR) is proposed based on the z-scores standardized group sparse representation (ZSGSR). Here, the similar patch groups of the image are firstly extracted and decomposed by adaptive PCA dictionary, then the resulting coefficients are normalized using z-score standardization in component-wise, and used to regularize compressive sensing recovery with l1norm term. The reconstruction model is solved by splitting Bregman iteration and soft threshold shrinking algorithm. The z-score standardization in group-based transformation domain effectively can improve the sparse representation ability of the image and better restore the edges and texture details in ICSR. Using objective and subjective quality evaluation, extensive experimental results verify the effectiveness of this method.

Motivated by the growing popularity of time-variant evolutionary algorithms (EAs) in solving practical problems, this paper uses spectral analyses to study convergence in probability for a general class of time-variant EAs which can be asymptotically described by reducible Markov chains with multiple aperiodic recurrent classes, covering many existing concrete case studies as specific instantiations. We provide a universal yet easily checkable characteristic for time-variant EAs satisfying global convergence, by introducing the asymptotical elitism and asymptotical monotonicity. To illustrate the effectiveness of our result, we consider four specific EAs with distinct asymptotical behavior, and recover, under even mild conditions, the state-of-the-art result as simple applications of our general theorem. Besides, simulation experiments further verify these results.

The firefly algorithm (FA) is one of the swarm intelligence algorithms for which opposition-based learning (OBL) is an efficient method for improving performance. In most of the existing OBL schemes, the opposite solution is calculated simultaneously for all dimensions of the original solution. However, the opposite solution does not always offer a better value in every dimension than the original solution. This paper develops a new scheme by utilizing the orthogonal experiment design method to select a subset of elements of the individual to be changed into opposite values by the centroid opposition, while the rest remain unchanged. Useful information about the original individual and its opposite can be found by this method. This new scheme is named orthogonal centroid opposition-based learning (OCOBL) and is incorporated into FA to obtain an orthogonal centroid opposition-based firefly algorithm (OCOFA). OCOFA is tested on the CEC's 2013 benchmark suite and compared with state-of-the-art FA variants. The experimental results demonstrate the effectiveness of OCOBL and an improved performance for the proposed OCOFA.

This paper studies the generalization performance of radial basis function (RBF) networks using local Rademacher complexities. We propose a general result on controlling local Rademacher complexities with the L1 -metric capacity. We then apply this result to estimate the RBF networks' complexities, based on which a novel estimation error bound is obtained. An effective approximation error bound is also derived by carefully investigating the Hölder continuity of the lp loss function's derivative. Furthermore, it is demonstrated that the RBF network minimizing an appropriately constructed structural risk admits a significantly better learning rate when compared with the existing results. An empirical study is also performed to justify the application of our structural risk in model selection.

Marginal Fisher analysis (MFA) is a well-known linear dimensionality reduction method. However, MFA does not utilize the local diversity information of the training data, which will degrade its performance. In order to enhance the discriminant power of MFA, this paper considers introducing local variation quantity to enlarge the distances between local neighborhood embeddings and proposes a flexible and efficient implementation of MFA (F-MFA) within the regularization framework. Therefore, the discriminant structure and diversity of data are preserved in low-dimensional subspace. Computationally, F-MFA is formulated as a trace differential optimization problem which can completely avoids the singularity problem as it exists in MFA. Further, an efficient algorithm is developed for implementing F-MFA via QR-decomposition. Experimental results on four face data sets demonstrate the effectiveness of our approach.

We present a nearest nonlinear subspace classifier that extends ridge regression classification method to kernel version which is called Kernel Ridge Regression Classification (KRRC). Kernel method is usually considered effective in discovering the nonlinear structure of the data manifold. The basic idea of KRRC is to implicitly map the observed data into potentially much higher dimensional feature space by using kernel trick and perform ridge regression classification in feature space. In this new feature space, samples from a single-object class may lie on a linear subspace, such that a new test sample can be represented as a linear combination of class-specific galleries, then the minimum distance between the new test sample and class specific subspace is used for classification. Our experimental studies on synthetic data sets and some UCI benchmark datasets confirm the effectiveness of the proposed method.

This paper proposes a simple, yet very powerful local face representation, called the Gradient Orientations and Euler Mapping (GOEM). GOEM consists of two stages: gradient orientations and Euler mapping. In the first stage, we calculate gradient orientations of a central pixel and get the corresponding orientation representations by performing convolution operator. These representation results display spatial locality and orientation properties. To encompass different spatial localities and orientations, we concatenate all these representation results and derive a concatenated orientation feature vector. In the second stage, we define an explicit Euler mapping which maps the space of the concatenated orientation into a complex space. For a mapping image, we find that the imaginary part and the real part characterize the high frequency and the low frequency components, respectively. To encompass different frequencies, we concatenate the imaginary part and the real part and derive a concatenated mapping feature vector. For a given image, we use the two stages to construct a GOEM image and derive an augmented feature vector which resides in a space of very high dimensionality. In order to derive low-dimensional feature vector, we present a class of GOEM-based kernel subspace learning methods for face recognition. These methods, which are robust to changes in occlusion and illumination, apply the kernel subspace learning model with explicit Euler mapping to an augmented feature vector derived from the GOEM representation of face images. Experimental results show that our methods significantly outperform popular methods and achieve state-of-the-art performance for difficult problems such as illumination and occlusion-robust face recognition.

Estimating the Rademacher chaos complexity of order two is important for understanding the performance of multikernel learning (MKL) machines. In this letter, we develop a novel entropy integral for Rademacher chaos complexities. As compared to the previous bounds, our result is much improved in that it introduces an adjustable parameter ε to prohibit the divergence of the involved integral. With the use of the iteration technique in Steinwart and Scovel (2007), we also apply our Rademacher chaos complexity bound to the MKL problems and improve existing learning rates.

In this study, we introduce a Fuzzy Time-Dependent Network (FTDN) and analyze its shortest path problem. The FTDN is a network in which travel times are represented as fuzzy sets and are also time-dependent. Under these circumstances, the shortest path problem on the FTDN is far more complex in comparison with the shortest path problem on the existing networks. To highlight the complexity, we show that on the FTDN, "standard" shortest path algorithms (e.g., the well-known Dijkstra algorithm) are not able to come up with solutions. Subsequently, we construct a suitable method which is suitable to deal with the shortest problem. A fuzzy programming model is presented for finding the shortest path on the FTDN. The proposed model is handled through the techniques which combine mechanisms of fuzzy simulation and genetic optimization. In this particular setting, fuzzy simulation is exploited to estimate the value of uncertain functions, which do not exist in the general networks. The proposed model is evaluated with the use of numerical experimentation. A comparative analysis demonstrates that the proposed model leads to the shortest path while standard algorithms are not capable of finding the path when dealing with the shortest path problem on the FTDN.

The quality of an approximation set usually includes two aspects-- approaching distance and spreading diversity. This paper introduces a new technique for assessing the diversity of an approximation to an exact Pareto-optimal front. This diversity is assessed by using an ldquoexposure degreerdquo of the exact Pareto-optimal front against the approximation set. This new technique has three advantages: Firstly, The ldquoexposure degreerdquo combines the uniformity and the width of the spread into a direct physical sense. Secondly, it makes the approaching distance independent from the spreading diversity at the most. Thirdly, the new technique works well for problems with any number of objectives, while the widely used diversity metric proposed by Deb would work poor in problems with 3 objectives or over. Experimental computational results show that the new technique assesses the diversity well.

This paper proposes a new evolutionary algorithm with lower dimensional crossover and gradient-based mutation for real-valued optimization problems with constraints. The crossover operator of the new algorithm searches a lower dimensional neighbor of the parent points where the neighbor center is the barycenter of the parents, and therefore the new algorithm converges fast. The gradient-based mutation is used to converge fast for the problems with equality constraints and active inequality constraints. And the new algorithm is simple and easy to be implemented. We have used 24 constrained benchmark problems to test the new algorithm. The experimental results show it works better than or competitive to a known effective algorithm.

This paper proposes a new evolutionary algorithm, called lower-dimensional-search evolutionary algorithm (LDSEA). The crossover operator of the new algorithm searches a lower-dimensional neighbor of the parent points where the neighbor center is the barycenter of the parents therefore the new algorithm converges fast, especially for high-dimensional constrained optimization problems. The niche-impaction operator and the mutation operator preserve the diversity of the population to make the LDSEA algorithm not to be trapped in local optima as much as possible. What's more is that the LDSEA algorithm is simple and easy to be implemented. We have used the 24 constrained benchmark problems [18] to test the LDSEA algorithm. The experimental results show it works better than or competitive to a known effective algorithm [7] for higher-dimensional constrained optimization problems.

There are rather few articles in the literature so far that deal with dynamic multi-objective optimization problems. This article introduces a dynamic orthogonal multi-objective evolutionary algorithm called "DOMOEA", that generalizes an earlier paper of ours (on an orthogonal multi-objective evolutionary algorithm (OMOEA-II) (Zeng et al., 2005)) to dynamic environments. DOMOEA solves a particular class of dynamic multi-objective optimization problems, namely those that have continuous decision variables. This new algorithm uses the evolutionary results, before any environmental change, as the initial population after the environmental change. It applies an "orthogonal design method" to enhance the fitness of the population during the static stages between two successive changes of environment. We obtained satisfactory results when testing this algorithm against the benchmark problems proposed in the literature. Our new algorithm is based on an ordinary evolutionary algorithm that does not have the capacity to detect environmental changes. Hence it has a comparatively simple structure, making comparisons with other dynamic multi-objective evolutionary algorithms relatively easy.

An orthogonal hill-climbing algorithm for dynamic optimization problems with continuous variables (labeled ODHC ) is proposed in present paper. The local peak climber is not a solution x, but rather a "niche", a small hyperrectangle. An orthogonal design method is employed on the niches for the niche to climb a potentially peak fast. An archive is used to store the latest found higher peaks for the ODHC algorithm learning from the past search. The randomly creating niches implement the global search. Numerical experiments show that the ODHC algorithm performs a lot better than the SOS (self organizing scouts) algorithm [J. Branke, T. Kaufler, C. Schmidt, and H. Schmeck. A multipopulation approach to dynamic optimization problems. Adaptive Computing in Design and Manufacturing. Springer, 2000.].

A robust dynamic evolutionary algorithm (labeled RODEA), where both the robust calculation and mutation operator are based on an orthogonal design, is proposed in this paper. Previous techniques calculate the mean effective objective (for robust) by using samples without much evenly distributing over the neighborhood. The samples by using orthogonal array distribute evenly. Therefore the calculation of mean effective objective more robust. The new technique is generalized from the ODEA algorithm [1]. An orthogonal design method is employed on the niches for the mutation operator to find a potentially good solution that may become the representative in the niche. The fitness of the offspring is therefore likely to be higher than that of its parent. We propose a complex benchmark, consisting of moving function peaks, to test our new approach. Numerical experiments show that the moving solutions of the algorithm are a little worse in objective value but robust.

This paper focuses on the computation time of evolutionary algorithms. First, some exact expressions of the mean first hitting times of general evolutionary algorithms in finite search spaces are obtained theoretically by using the properties of Markov chain. Then, by introducing drift analysis and applying Dynkin’s Formula, the general upper and lower bounds of the mean first hitting times of evolutionary algorithms are given rigorously under some mild conditions. These results obtained in this paper, and the analytic methods used in this paper, are widely valid for analyzing the computation time of evolutionary algorithms in any search space(finite or infinite)as long as some simple technique processes are introduced.

In this paper, an orthogonal multi-objective evolutionary algorithm (OMOEA) is proposed for multi-objective optimization problems (MOPs) with constraints. Firstly, these constraints are taken into account when determining Pareto dominance. As a result, a strict partial-ordered relation is obtained, and feasibility is not considered later in the selection process. Then, the orthogonal design and the statistical optimal method are generalized to MOPs, and a new type of multi-objective evolutionary algorithm (MOEA) is constructed. In this framework, an original niche evolves first, and splits into a group of sub-niches. Then every sub-niche repeats the above process. Due to the uniformity of the search, the optimality of the statistics, and the exponential increase of the splitting frequency of the niches, OMOEA uses a deterministic search without blindness or stochasticity. It can soon yield a large set of solutions which converges to the Pareto-optimal set with high precision and uniform distribution. We take six test problems designed by Deb, Zitzler et al., and an engineering problem (W) with constraints provided by Ray et al. to test the new technique. The numerical experiments show that our algorithm is superior to other MOGAS and MOEAs, such as FFGA, NSGAII, SPEA2, and so on, in terms of the precision, quantity and distribution of solutions. Notably, for the engineering problem W, it finds the Pareto-optimal set, which was previously unknown.

A fast algorithm is proposed to find the nondominated set for multiobjective optimization problems in this paper. Two accelerated techniques are adopted in the algorithm. One is that the algorithm can yield an integer rank set after it indexes the search space. Based on this, the goal is changed into determination of the nondominated set of the integer rank set. The other is that the nondominated check sequence follows that the likely nondominated members are first checked, and that the check process is stopped when the remaining members in the nondominated check sequence are dominated. The computational complexity of the new algorithm is analyzed theoretically. Experimental results show that the new method performs much better than KLP (a famous effective algorithm) when the search space contains a large nondominated set. Moreover, the two new techniques introduced in this paper are very useful for multiobjective evolutionary algorithms (MOEAs) to improve the computational speed.

A new algorithm is proposed to solve constrained multiobjective problems. The constraints of the MOPs are taken account of in determining Pareto dominance. As a result, the feasibility of solutions is not an issue. At the same time, it takes advantage of both the orthogonal design method to search evenly, and the statistical optimal method to speed up the computation. The output of the technique is a large set of solutions with high precision and even distribution. Notably, for an engineering problem WATER, it finds the Pareto-optimal set, which was previously unknown.