In this article, we focus on a biobjective hot strip mill (HSM) scheduling problem arising in the steel industry. Besides the conventional objective regarding penalty costs, we have also considered minimizing the total starting times of rolling operations in order to reduce the energy consumption for slab reheating. The problem is complicated by the inevitable uncertainty in rolling processing times, which means deterministic scheduling models will be ineffective. To obtain robust production schedules with satisfactory performance under all possible conditions, we apply the robust optimization (RO) approach to model and solve the scheduling problem. First, an RO model and an equivalent mixed-integer linear programming model are constructed to describe the HSM scheduling problem with uncertainty. Then, we devise an improved Benders' decomposition algorithm to solve the RO model and obtain exactly optimal solutions. Next, for coping with large-sized instances, a multiobjective particle swarm optimization algorithm with an embedded local search strategy is proposed to handle the biobjective scheduling problem and find the set of Pareto-optimal solutions. Finally, we conduct extensive computational tests to verify the proposed algorithms. Results show that the exact algorithm is effective for relatively small instances and the metaheuristic algorithm can achieve satisfactory solution quality for both small- and large-sized instances of the problem.

Deep reinforcement learning (RL) agents are becoming increasingly proficient in a range of complex control tasks. However, the agent's behavior is usually difficult to interpret due to the introduction of black-box function, making it difficult to acquire the trust of users. Although there have been some interesting interpretation methods for vision-based RL, most of them cannot uncover temporal causal information, raising questions about their reliability. To address this problem, we present a temporal-spatial causal interpretation (TSCI) model to understand the agent's long-term behavior, which is essential for sequential decision-making. TSCI model builds on the formulation of temporal causality, which reflects the temporal causal relations between sequential observations and decisions of RL agent. Then a separate causal discovery network is employed to identify temporal-spatial causal features, which are constrained to satisfy the temporal causality. TSCI model is applicable to recurrent agents and can be used to discover causal features with high efficiency once trained. The empirical results show that TSCI model can produce high-resolution and sharp attention masks to highlight task-relevant temporal-spatial information that constitutes most evidence about how vision-based RL agents make sequential decisions. In addition, we further demonstrate that our method is able to provide valuable causal interpretations for vision-based RL agents from the temporal perspective.

Deep reinforcement learning (RL) has recently led to many breakthroughs on a range of complex control tasks. However, the agent's decision-making process is generally not transparent. The lack of interpretability hinders the applicability of RL in safety-critical scenarios. While several methods have attempted to interpret vision-based RL, most come without detailed explanation for the agent's behavior. In this paper, we propose a self-supervised interpretable framework, which can discover interpretable features to enable easy understanding of RL agents even for non-experts. Specifically, a self-supervised interpretable network (SSINet) is employed to produce fine-grained attention masks for highlighting task-relevant information, which constitutes most evidence for the agent's decisions. We verify and evaluate our method on several Atari 2600 games as well as Duckietown, which is a challenging self-driving car simulator environment. The results show that our method renders empirical evidences about how the agent makes decisions and why the agent performs well or badly, especially when transferred to novel scenes. Overall, our method provides valuable insight into the internal decision-making process of vision-based RL. In addition, our method does not use any external labelled data, and thus demonstrates the possibility to learn high-quality mask through a self-supervised manner, which may shed light on new paradigms for label-free vision learning such as self-supervised segmentation and detection.

Data augmentation is widely known as a simple yet surprisingly effective technique for regularizing deep networks. Conventional data augmentation schemes, e.g., flipping, translation or rotation, are low-level, data-independent and class-agnostic operations, leading to limited diversity for augmented samples. To this end, we propose a novel semantic data augmentation algorithm to complement traditional approaches. The proposed method is inspired by the intriguing property that deep networks are effective in learning linearized features, i.e., certain directions in the deep feature space correspond to meaningful semantic transformations, e.g., changing the background or view angle of an object. Based on this observation, translating training samples along many such directions in the feature space can effectively augment the dataset for more diversity. To implement this idea, we first introduce a sampling based method to obtain semantically meaningful directions efficiently. Then, an upper bound of the expected cross-entropy (CE) loss on the augmented training set is derived by assuming the number of augmented samples goes to infinity, yielding a highly efficient algorithm. In fact, we show that the proposed implicit semantic data augmentation (ISDA) algorithm amounts to minimizing a novel robust CE loss, which adds minimal extra computational cost to a normal training procedure. In addition to supervised learning, ISDA can be applied to semi-supervised learning tasks under the consistency regularization framework, where ISDA amounts to minimizing the upper bound of the expected KL-divergence between the augmented features and the original features. Although being simple, ISDA consistently improves the generalization performance of popular deep models (e.g., ResNets and DenseNets) on a variety of datasets, i.e., CIFAR-10, CIFAR-100, SVHN, ImageNet, and Cityscapes. Code for reproducing our results is available at https://github.com/blackfeather-wang/ISDA-for-Deep-Networks.

Scheduling of a hot strip mill is an important decision problem in the steel manufacturing industry. Previous studies on the hot strip mill scheduling problem have mostly neglected the random factors in production. However, random variations in processing times are inevitable due to unpredictable delays and disturbances. In this article, we adopt a robust optimization approach to deal with the uncertainty in processing times. The advantage is that no assumption has to be made regarding the distribution of random data, and the obtained schedule will remain strictly feasible when the variations have not exceeded a predefined uncertainty set. First, a mixed-integer linear programming model is presented to formulate the robust scheduling problem. Then, a hybrid metaheuristic algorithm, which combines ant colony system (ACS) and enhanced local search, is proposed to provide an efficient solution to the problem. Finally, extensive computational experiments involving both randomly generated and real-world instances have been conducted to verify the effectiveness of the proposed algorithm. It is shown that the algorithm achieves optimality for small instances and outperforms two state-of-the-art metaheuristics when used to solve large instances.

Domain adaptation aims to deal with learning problems in which the labeled training data and unlabeled testing data are differently distributed. Maximum mean discrepancy (MMD), as a distribution distance measure, is minimized in various domain adaptation algorithms for eliminating domain divergence. We analyze empirical MMD from the point of view of graph embedding. It is discovered from the MMD intrinsic graph that, when the empirical MMD is minimized, the compactness within each domain and each class is simultaneously reduced. Therefore, points from different classes may mutually overlap, leading to unsatisfactory classification results. To deal with this issue, we present a graph embedding framework with intrinsic and penalty graphs for MMD-based domain adaptation algorithms. In the framework, we revise the intrinsic graph of MMD-based algorithms such that the within-class scatter is minimized, and thus, the new features are discriminative. Two strategies are proposed. Based on the strategies, we instantiate the framework by exploiting four models. Each model has a penalty graph characterizing certain similarity property that should be avoided. Comprehensive experiments on visual cross-domain benchmark datasets demonstrate that the proposed models can greatly enhance the classification performance compared with the state-of-the-art methods.

This paper studies on sensorimotor control problems of Autonomous Underwater Vehicles (AUVs) using deep reinforcement learning. We design an end-to-end learning architecture mapping original sensor input to continuous control output without referring to the dynamics of vehicles. To avoid difficult and noisy underwater localization, we implement the learning without knowing the positions of AUVs by proposing novel state encoder and reward shaping strategies. Two distinct underwater tasks, obstacle avoidance with sonar sensor and pipeline following with visual sensor, are simulated to validate the effectiveness of proposed architecture and strategies. For the latter, we test the learned policy on realistic images of underwater pipelines to check its generalization ability.

Model-free deep reinforcement learning (RL) algorithms have been widely used for a range of complex control tasks. However, slow convergence and sample inefficiency remain challenging problems in RL, especially when handling continuous and high-dimensional state spaces. To tackle this problem, we propose a general acceleration method for model-free, off-policy deep RL algorithms by drawing the idea underlying regularized Anderson acceleration (RAA), which is an effective approach to accelerating the solving of fixed point problems with perturbations. Specifically, we first explain how policy iteration can be applied directly with Anderson acceleration. Then we extend RAA to the case of deep RL by introducing a regularization term to control the impact of perturbation induced by function approximation errors. We further propose two strategies, i.e., progressive update and adaptive restart, to enhance the performance. The effectiveness of our method is evaluated on a variety of benchmark tasks, including Atari 2600 and MuJoCo. Experimental results show that our approach substantially improves both the learning speed and final performance of state-of-the-art deep RL algorithms.

In this paper, we propose a novel implicit semantic data augmentation (ISDA) approach to complement traditional augmentation techniques like flipping, translation or rotation. Our work is motivated by the intriguing property that deep networks are surprisingly good at linearizing features, such that certain directions in the deep feature space correspond to meaningful semantic transformations, e.g., adding sunglasses or changing backgrounds. As a consequence, translating training samples along many semantic directions in the feature space can effectively augment the dataset to improve generalization. To implement this idea effectively and efficiently, we first perform an online estimate of the covariance matrix of deep features for each class, which captures the intra-class semantic variations. Then random vectors are drawn from a zero-mean normal distribution with the estimated covariance to augment the training data in that class. Importantly, instead of augmenting the samples explicitly, we can directly minimize an upper bound of the expected cross-entropy (CE) loss on the augmented training set, leading to a highly efficient algorithm. In fact, we show that the proposed ISDA amounts to minimizing a novel robust CE loss, which adds negligible extra computational cost to a normal training procedure. Although being simple, ISDA consistently improves the generalization performance of popular deep models (ResNets and DenseNets) on a variety of datasets, e.g., CIFAR-10, CIFAR-100 and ImageNet. Code for reproducing our results are available at https://github.com/blackfeather-wang/ISDA-for-Deep-Networks.

Rotating machinery prognostics plays an important role in promoting reliability and efficiency in the operation of machinery and reducing maintenance costs. This paper proposes a novel bearing remaining useful life prediction approach which puts emphasis on deriving effective features from raw vibration data. To enhance the degradation feature extraction, a non-recursive method named variational mode decomposition (VMD) is adopted to decompose the raw vibration data into several principal modes, then feature smoothing with a local regression filter is performed and some suitable features are selected by evaluating feature fitness using monotonicity and correlation analysis. Based on the selected features, long short-term memory (LSTM) network is introduced for bearings RUL prediction. Numerical experiments with real bearing dataset exhibit the effectiveness and superiority of the proposed approach in comparison to other data-driven approaches.

Service ecosystems, consisting of various kinds of services and mashups, usually keep evolving over time.Existing works on the evolution of service ecosystems focus on either evaluating the impacts of single services' changes on the usage of services and the stability of the whole ecosystem, or discovering co-occurrence relationship between services, but fail to disclose any knowledge from the aspect of the evolution of service composition patterns.Based on our previous work, this paper moves one step further, revealing the latent service composition trends in a service ecosystem and providing more distinct explanation of different topic evolution patterns.A novel methodology, named Extended Dependency-Compensated Service Co-occurrence LDA (EDC-SeCo-LDA), is developed to calculate the directed dependencies between different topics and build topic evolution graph.The evolution trend

Extreme learning machine (ELM) has been applied in a wide range of classification and regression problems due to its high accuracy and efficiency. However, ELM can only deal with cases where training and testing data are from identical distribution, while in real world situations, this assumption is often violated. As a result, ELM performs poorly in domain adaptation problems, in which the training data (source domain) and testing data (target domain) are differently distributed but somehow related. In this paper, an ELM-based space learning algorithm, domain space transfer ELM (DST-ELM), is developed to deal with unsupervised domain adaptation problems. To be specific, through DST-ELM, the source and target data are reconstructed in a domain invariant space with target data labels unavailable. Two goals are achieved simultaneously. One is that, the target data are input into an ELM-based feature space learning network, and the output is supposed to approximate the input such that the target domain structural knowledge and the intrinsic discriminative information can be preserved as much as possible. The other one is that, the source data are projected into the same space as the target data and the distribution distance between the two domains is minimized in the space. This unsupervised feature transformation network is followed by an adaptive ELM classifier which is trained from the transferred labeled source samples, and is used for target data label prediction. Moreover, the ELMs in the proposed method, including both the space learning ELM and the classifier, require just a small number of hidden nodes, thus maintaining low computation complexity. Extensive experiments on real-world image and text datasets are conducted and verify that our approach outperforms several existing domain adaptation methods in terms of accuracy while maintaining high efficiency.

This paper investigates trajectory tracking problem for a class of underactuated autonomous underwater vehicles (AUVs) with unknown dynamics and constrained inputs. Different from existing policy gradient methods which employ single actor critic but cannot realize satisfactory tracking control accuracy and stable learning, our proposed algorithm can achieve high-level tracking control accuracy of AUVs and stable learning by applying a hybrid actors-critics architecture, where multiple actors and critics are trained to learn a deterministic policy and action-value function, respectively. Specifically, for the critics, the expected absolute Bellman error-based updating rule is used to choose the worst critic to be updated in each time step. Subsequently, to calculate the loss function with more accurate target value for the chosen critic, Pseudo Q-learning, which uses subgreedy policy to replace the greedy policy in Q-learning, is developed for continuous action spaces, and Multi Pseudo Q-learning (MPQ) is proposed to reduce the overestimation of action-value function and to stabilize the learning. As for the actors, deterministic policy gradient is applied to update the weights, and the final learned policy is defined as the average of all actors to avoid large but bad updates. Moreover, the stability analysis of the learning is given qualitatively. The effectiveness and generality of the proposed MPQ-based deterministic policy gradient (MPQ-DPG) algorithm are verified by the application on AUV with two different reference trajectories. In addition, the results demonstrate high-level tracking control accuracy and stable learning of MPQ-DPG. Besides, the results also validate that increasing the number of the actors and critics will further improve the performance.

In this brief, we propose a novel nonparametric supervised linear dimension reduction (SLDR) algorithm that extracts the features by maximizing the pairwise separation probability. The separation probability, as a new class separability measure, describes the generalization accuracy when we use the obtained features to train a linear classifier. Obtaining high-quality features, the proposed method avoids the overlaps between classes that are close to each other in the input space and improves the subsequent classification performance. Experiments on benchmark data sets show the superiority of the proposed algorithm over some other state-of-the-art SLDR methods.

Service composition remains an important topic where recommendation is widely recognized as a core mechanism. Existing works on service recommendation typically examine either association rules from mashup-service usage records, or latent topics from service descriptions. This paper moves one step further, by studying latent topic models over service collaboration history. A concept of service co-occurrence topic is coined, equipped with a mechanism developed to construct service co-occurrence documents. The key idea is to treat each service as a document and its co-occurring services as the bag of words in that document. Four gauges are constructed to measure self-co-occurrence of a specific service. A theoretical approach, Service Co-occurrence LDA (SeCo-LDA), is developed to extract latent service co-occurrence topics, including representative services and words, temporal strength, and services' impact on topics. Such derived knowledge of topics will help to reveal the trend of service composition, understand collaboration behaviors among services and lead to better service recommendation. To verify the effectiveness and efficiency of our approach, experiments on a real-world data set were conducted. Compared with methods of Apriori, content matching based on service description, and LDA using mashup-service usage records, our experiments show that SeCo-LDA can recommend service composition more effectively, i.e., 5% better in terms of Mean Average Precision than baselines.

Extreme learning machines (ELMs), as “generalized” single hidden layer feedforward networks, have been proved to be effective and efficient for classification and regression problems. Traditional ELMs assume that the training and testing data are drawn from the same distribution, which however is often violated in real-world applications. In this paper, we propose a unified cross-domain ELM (CDELM) framework to address domain adaptation problems, in which the distributions of training data (source domain) and testing data (target domain) are different but related. CDELM not only fully leverages labeled source data and unlabeled target data simultaneously to construct an adaptive target classifier but also maintains the computational efficiency of ELMs. Specifically, CDELM adapts the source classifier to target domain by matching the projected means of both domains, and explores the structure property of target domain by using manifold regularization to make the final classifier more adaptable to target data. Based on the framework, two algorithms CDELM-M and CDELM-C are proposed, which aim at minimizing the marginal and conditional distribution distance between source and target domains, respectively. Moreover, CDELM-C can further enhance the classification accuracy by multiple iterations. Comprehensive experimental studies on artificial datasets and public text and image datasets demonstrate that both CDELM-M and CDELM-C are competitive with several state-of-the-art domain adaptation learning methods in terms of the classification accuracy and efficiency.

In this paper, we consider depth control problems of an autonomous underwater vehicle (AUV) for tracking the desired depth trajectories. Due to the unknown dynamical model and the coupling between surge and yaw motions of the AUV, the problems cannot be effectively solved by most of the model-based or proportional-integral-derivative like controllers. To this purpose, we formulate the depth control problems of the AUV as continuous-state, continuous-action Markov decision processes under unknown transition probabilities. Based on the deterministic policy gradient theorem and neural network approximation, we propose a model-free reinforcement learning (RL) algorithm that learns a state-feedback controller from sampled trajectories of the AUV. To improve the performance of the RL algorithm, we further propose a batch-learning scheme through replaying previous prioritized trajectories. We illustrate with simulations that our model-free method is even comparable to the model-based controllers. Moreover, we validate the effectiveness of the proposed RL algorithm on a seafloor data set sampled from the South China Sea.

Existing service ecosystems typically rely on some centralized service registries (e.g., ProgrammableWeb.com) as "middle people" to record service behaviors thus to provide service ranking and recommendation. Excessive centralization increasingly becomes the bottleneck and hinders the further growth of the service ecosystems. As the first attempt to apply the fundamental technique underneath the emerging Bitcoin network into the field of service oriented computing, this paper proposes to build a service ecosystem as a decentralized blockchain-oriented service network, called Decentralized Service Eco-System (DSES). Whenever any activity occurs in the system (e.g., APIs are used together in a published mashup), all involved parties will individually store and maintain a copy of the detailed record (provenance) at their local databases. Such a distributed database-oriented solution will enable services who do not fully trust each other to maintain a set of global states. In this way, service discovery and recommendation can be realized in a distributed manner that promises higher scalability and maintainability. As a proof of concept, a prototyping system of DSES is constructed using the real-world data from ProgrammableWeb.com, based on the INKchain, a newly open-source consortium blockchain mechanism extending the Hyperledger Fabric.

Deep neural networks have been successfully applied to numerous machine learning tasks because of their impressive feature abstraction capabilities. However, conventional deep networks assume that the training and test data are sampled from the same distribution, and this assumption is often violated in real-world scenarios. To address the domain shift or data bias problems, we introduce layer-wise domain correction (LDC), a new unsupervised domain adaptation algorithm which adapts an existing deep network through additive correction layers spaced throughout the network. Through the additive layers, the representations of source and target domains can be perfectly aligned. The corrections that are trained via maximum mean discrepancy, adapt to the target domain while increasing the representational capacity of the network. LDC requires no target labels, achieves state-of-the-art performance across several adaptation benchmarks, and requires significantly less training time than existing adaptation methods.

Domain adaptation manages to build an effective target classifier or regression model for unlabeled target data by utilizing the well-labeled source data but lying different distributions. Intuitively, to address domain shift problem, it is crucial to learn domain invariant features across domains, and most existing approaches have concentrated on it. However, they often do not directly constrain the learned features to be class discriminative for both source and target data, which is of vital importance for the final classification. Therefore, in this paper, we put forward a novel feature learning method for domain adaptation to construct both domain invariant and class discriminative representations, referred to as DICD. Specifically, DICD is to learn a latent feature space with important data properties preserved, which reduces the domain difference by jointly matching the marginal and class-conditional distributions of both domains, and simultaneously maximizes the inter-class dispersion and minimizes the intra-class scatter as much as possible. Experiments in this paper have demonstrated that the class discriminative properties will dramatically alleviate the cross-domain distribution inconsistency, which further boosts the classification performance. Moreover, we show that exploring both domain invariance and class discriminativeness of the learned representations can be integrated into one optimization framework, and the optimal solution can be derived effectively by solving a generalized eigen-decomposition problem. Comprehensive experiments on several visual cross-domain classification tasks verify that DICD can outperform the competitors significantly.

Routing service considering uncertainty is at the core of intelligent transportation systems and has attracted increasing attention. Existing stochastic shortest path models require the exact probability distributions of travel times and usually assume that they are independent. However, the distributions are often unavailable or inaccurate due to insufficient data, and correlation of travel times over different links has been observed. This paper presents a robust shortest path (RSP) model that only requires partial distribution information of travel times, including the support set, mean, variance, and correlation matrix. We introduce a concept of robust mean-excess travel time to hedge against the risk from both the uncertainty of the random travel times and the uncertainty in their distributions. To solve the RSP problem, an equivalent dual formulation is derived and used to design tight lower and upper bound approximation methods, which adopt the scenario approach and semi-definite programming approach, respectively. To solve large problems, we further propose an efficient primal approximation method, which only needs to solve two deterministic shortest path problems and a mean-standard deviation shortest path problem, and analyze its approximation performance. Experiments validate the tightness of the proposed bounds and demonstrate the impact of uncertainty on the relative benefit and cost of robust paths.

Service ecosystem consists of all kinds of services, and some of them may be composed by developers to create new mashups. Existing work on service recommendation and composition mine either frequent patterns from mashup-service usage records, or latent topics from service metadata. In this paper, we propose Service Co-occurrence LDA (SeCo-LDA), a novel approach that mines latent topic models over service co-occurrence patterns. The key idea is to treat each service as a document, and its bag of co-occurring services as the bag of words in that document. Using this model, we can analyze such service co-occurrence documents with a probabilistic topic model. We show how to derive service co-occurrence topics, and then validate our model on the real-world ProgrammableWeb.com dataset. We illustrate that SeCo-LDA can discover meaningful latent service composition patterns including their temporal strength and services' impacts, which conventional Apriori can not reveal. Comparing with Apriori, content matching based on service description and LDA directly using mashup-service usage records, we have demonstrated that SeCo-LDA can recommend service composition more effectively, 5% better in terms of MAP than the baseline approach.

The industrial sector is one of the largest energy consumers in the world. To alleviate the grid's burden during peak hours, time-of-use (TOU) electricity pricing has been implemented in many countries around the globe to encourage manufacturers to shift their electricity usage from peak periods to off-peak periods. In this paper, we study the unrelated parallel machine scheduling problem under a TOU pricing scheme. The objective is to minimize the total electricity cost by appropriately scheduling the jobs such that the overall completion time does not exceed a predetermined production deadline. To solve this problem, two solution approaches are presented. The first approach models the problem with a new time-interval-based mixed integer linear programming formulation. In the second approach, we reformulate the problem using Dantzig-Wolfe decomposition and propose a column generation heuristic to solve it. Computational experiments are conducted under different TOU settings and the results confirm the effectiveness of the proposed methods. Based on the numerical results, we provide some practical suggestions for decision makers to help them in achieving a good balance between the productivity objective and the energy cost objective.

This paper proposes a Block-shifting Simulated Annealing (BSA) algorithm for the no-wait flowshop scheduling problem (NWFSP) to minimize makespan. The proposed algorithm makes use of the objective incremental properties of NWFSP and embeds a block-shifting operator based on k-insertion moves into the algorithm framework of simulated annealing. A major advantage of the BSA algorithm lies in its easy implementation since it does not involve sophisticated evolutionary strategy and parameter tuning process. In addition to its simplicity, BSA is shown to be very effective. Through experimental comparisons, the effectiveness of the block-shifting operator is clearly revealed. In addition, the BSA algorithm is proved to be more effective and robust than the state-of-the-art algorithms for solving the NWFSP.

In the operational optimization and scheduling problems of actual industrial processes, such as iron and steel, and microelectronics, the operational indices and process parameters usually need to be predicted. However, for some input and output variables of these prediction models, there may exist a lot of uncertainties coming from themselves, the measurement error, the rough representation, and so on. In such cases, constructing a prediction interval (PI) for the output of the corresponding prediction model is very necessary. In this paper, two twin extreme learning machine (TELM) models for constructing PIs are proposed. First, we propose a regularized asymmetric least squares extreme learning machine (RALS-ELM) method, in which different weights of its squared error loss function are set according to whether the error of the model output is positive or negative in order that the above error can be differentiated in the parameter learning process, and Tikhonov regularization is introduced to reduce overfitting. Then, we propose an asymmetric Bayesian extreme learning machine (AB-ELM) method based on the Bayesian framework with the asymmetric Gaussian distribution (AB-ELM), in which the weights of its likelihood function are determined as the same method in RALS-ELM, and the type II maximum likelihood algorithm is derived to learn the parameters of AB-ELM. Based on RALS-ELM and AB-ELM, we use a pair of weights following the reciprocal relationship to obtain two nonparallel regressors, including a lower-bound regressor and an upper-bound regressor, respectively, which can be used for calculating the PIs. Finally, some discussions are given, about how to adjust the weights adaptively to meet the desired PI, how to use the proposed TELMs for nonlinear quantile regression, and so on. Results of numerical comparison on data from one synthetic regression problem, three University of California Irvine benchmark regression problems, and two actual industrial regression problems show the effectiveness of the proposed models.

Mashup has emeraged as a promising way to allow developers to compose existed APIs (services) to create new or value-added services. With the rapid increasing number of services published on the Internet, service recommendation for automatic mashup creation gains a lot of momentum. Since mashup inherently requires services with different functions, the recommendation result should contain services from various categories. However, most existing recommendation approaches only rank all candidate services in a single list, which has two deficiencies. First, ranking services without considering to which categories they belong may lead to meaningless service ranking and affect the recommendation accuracy. Second, mashup developers are not always clear about which service categories they need and services in which categories cooperate better for mashup creation. Without explicitly recommending which service categories are relevant for mashup creation, it remains difficult for mashup developers to select proper services in a mixed ranking list, which lower the user friendliness of recommendation. To overcome these deficiencies, a novel category-aware service clustering and distributed recommending method is proposed for automatic mashup creation. First, a Kmeans variant(vKmeans) method based on topic model Latent Dirichlet Allocation is introduced for enhancing service categorization and providing a basis for recommendation. Second, on top of vKmeans, a service category relevance ranking (SCRR) model, which combines machine learning and collaborative filtering, is developed to decompose mashup requirements and explicitly predict relevant service categories. Finally, a category-aware distributed service recommendation (CDSR) model, which is based on a distributed machine learning framework, is developed for predicting service ranking order within each category. Experiments on a real-world dataset have proved that the proposed approach not only gains significant improvement at precision rate but also enhances the diversity of recommendation results.

This paper proposes a tabu mechanism improved iterated greedy (TMIIG) algorithm to solve the no-wait flow-shop scheduling problem with makespan criterion. The motivation of seeking for further improvement in the iterated greedy (IG) algorithm framework is based on the observation that the construction phase of the original IG algorithm may lead to repeated search when applying the insertion neighborhood search. To overcome the drawback, we modified the IG algorithm by a tabu-based reconstruction strategy to enhance its exploitation ability. A powerful neighborhood search method which involves insert, swap, and double-insert moves is then applied to obtain better soluions from the reconstructed solution in the previous step. Numerical computations verified the advantages of utilizing the new reconstruction scheme. In addition, comparisons with other high-performing algorithms demonstrated the effectiveness and robustness of the proposed algorithm.

Service compositions inherently require multiple services each with its domain-specific functionality. Therefore, how to mine matching patterns between services in relevant domains and compositions becomes crucial to service recommendation for composition. Existing methods usually overlook domain relevance and domain-specific matching patterns, which restrict the quality of recommendations. In this paper, a novel approach is proposed to offer domain-aware service recommendation. First, a K Nearest Neighbor variant (vKNN) based on topic model Latent Dirichlet Allocation (LDA) is introduced to cluster services into semantically coherent domains. On top of service domain clustering results by vKNN, a probabilistic matching model Domain Router (DR) based on Extreme Learning Machine (ELM) is developed for decomposing a requirement to relevant domains. Finally, a comprehensive Domain Topic Matching (DTM) model is built to mine relevant domain-specific matching patterns to facilitate service recommendation. Experiments on a large-scale real-world dataset show that DTM not only gains significant improvement at precision rate but also enhances the diversity of results.

Extreme learning machines (ELMs) have proven to be efficient and effective learning mechanisms for pattern classification and regression. However, ELMs are primarily applied to supervised learning problems. Only a few existing research papers have used ELMs to explore unlabeled data. In this paper, we extend ELMs for both semi-supervised and unsupervised tasks based on the manifold regularization, thus greatly expanding the applicability of ELMs. The key advantages of the proposed algorithms are as follows: 1) both the semi-supervised ELM (SS-ELM) and the unsupervised ELM (US-ELM) exhibit learning capability and computational efficiency of ELMs; 2) both algorithms naturally handle multiclass classification or multicluster clustering; and 3) both algorithms are inductive and can handle unseen data at test time directly. Moreover, it is shown in this paper that all the supervised, semi-supervised, and unsupervised ELMs can actually be put into a unified framework. This provides new perspectives for understanding the mechanism of random feature mapping, which is the key concept in ELM theory. Empirical study on a wide range of data sets demonstrates that the proposed algorithms are competitive with the state-of-the-art semi-supervised or unsupervised learning algorithms in terms of accuracy and efficiency.

In this paper, a robust support vector regression (RSVR) method with uncertain input and output data is studied. First, the data uncertainties are investigated under a stochastic framework and two linear robust formulations are derived. Linear formulations robust to ellipsoidal uncertainties are also considered from a geometric perspective. Second, kernelized RSVR formulations are established for nonlinear regression problems. Both linear and nonlinear formulations are converted to second-order cone programming problems, which can be solved efficiently by the interior point method. Simulation demonstrates that the proposed method outperforms existing RSVRs in the presence of both input and output data uncertainties.

Elastic compute clouds are best represented by the virtual clusters in Amazon EC2 or in IBM RC2. This paper proposes a simulation based approach to scheduling scientific workflows onto elastic clouds. Scheduling multitask workflows in virtual clusters is a NP-hard problem. Excessive simulations in months of time may be needed to produce the optimal schedule using Monte Carlo simulations. To reduce this scheduling overhead is necessary in real-time cloud computing. We present a new workflow scheduling method based on iterative ordinal optimization (IOO). This new method outperforms the Monte Carlo and Blind-Pick methods to yield higher performance against rapid workflow variations. For example, to execute 20,000 tasks on 128 virtual machines for gravitational wave analysis, an ordinal optimized schedule can be generated in a few minutes, which is O(103)~O(104) faster than using Monte Carlo simulations. The ordinal optimized schedule results in higher throughput with lower memory demand. The cloud experimental results being reported verified our theoretical findings on the relative performance of three workflow scheduling methods studied in this paper.

Virtual machines are allocated on demand in virtualized cloud platforms to provide flexible and reliable services. The major difficulty lies in satisfying the conflicting objectives of reducing response time while lowering resource costs. In this paper, a mathematical multi-tier framework for virtual machine allocation is proposed, which can be used to capture the performance of the cloud platform. We first use simulations to derive virtual resource allocation policies, and later use real benchmarking applications to verify the effectiveness of this framework. Experimental results show that the model can be simply and effectively used to satisfy the response time requirement as well as lowering the cost of using the virtual machine resources.

Parameter prediction is an important data mining problem and has many applications. Considering the difficulty for the conventional parameter prediction methods to deal with numeric and categorical inputs, this paper proposes a new Adaptive-Network-based Fuzzy Inference System (ANFIS)-based parameter prediction method that can well tackle such inputs. First, it introduces a Firing-strength Transform Matrix (FTM) into the generation mechanism of firing strengths of fuzzy rules in standard ANFIS in order that the categorical inputs can be handled. Next, a new training algorithm of the structural parameters in the premise/consequent parts of fuzzy rules and FTM in the new ANFIS is proposed. Moreover, to reduce the number of structural parameters to be learned in the new ANFIS with high-dimensional inputs, this paper presents a fuzzy c-means method based on a binary tree linear division method for identifying the structure of the new ANFIS. Then, numerical comparisons are made, and the results show that the performance of the new ANFIS has significant advantages over that of the Multilayer-Perceptron (MLP)-based parameter prediction method. Finally, the proposed method is applied to predict the trim-beam numbers in an industrial textile scheduling process.

A decomposition and optimization algorithm is presented for large-scale job shop scheduling problems in which the total weighted tardiness must be minimized. In each iteration, a new subproblem is first defined by a heuristic approach and then solved using a genetic algorithm. We construct a fuzzy controller to calculate the characteristic values which describe the the bottleneck jobs in different optimization stages. Then, these characteristic values are used to guide the process of subproblem-solving in an immune mechanism. Numerical computational results show that the proposed algorithm is effective for solving large-scale scheduling problems.

Genetic programming (GP) has been rarely applied to scheduling problems. In this paper the use of GP to learn single-machine predictive scheduling (PS) heuristics with stochastic breakdowns is investigated, where both tardiness and stability objectives in face of machine failures are considered. The proposed bi-tree structured representation scheme makes it possible to search sequencing and idle time inserting programs integratedly. Empirical results in different uncertain environments show that GP can evolve high quality PS heuristics effectively. The roles of inserted idle time are then analysed with respect to various weighting objectives. Finally some guides are supplied for PS design based on GP-evolved heuristics.

Ranking and comparing fuzzy numbers is an important part in many fuzzy optimization problems such as intelligent control and manufacturing system production line scheduling with uncertainty environments. In this paper, based on the level characteristic function and α -average of level cut sets of fuzzy number, we establish the IM α -metric method for measuring fuzzy number as a whole, and introduce the concept of ID α -difference that describes the reliability of IM α -metric value. Further, the basic properties and the separability of IM α -metric and ID α -difference are discussed. Finally, we give a mathematical model to solve fuzzy optimization problems by means of IM α -metric.