An important challenge in the cyber-space is the effective identification of network anomalies, often caused by malicious activities. With the remarkable advances, machine learning algorithms have widely been studied for network intrusion and anomaly detection. In particular, deep learning based on neural network structures has recently been given a greater attention to deal with the growing complexity of data with higher dimensions and non-linearity. Convolutional Neural Networks (CNNs) is one of the widely employed deep learning methods. In this work, we introduce a new encoding technique that enhances the performance for the identification of anomalous events using a CNN structure. To evaluate, we utilize three different datasets for the extensive analysis. The experimental results show that our method consistently outperforms the gray-scale encoding technique previously proposed over the datasets employed in the evaluation.

Deep learning has been widely applied to network anomaly detection to improve performance. In our past research, we empirically evaluated a set of deep learning models, including Fully Connected Network (FCN), Variational Auto Encoder (VAE), and Sequence to Sequence model with Long Short-Term Memory (Seq2Seq-LSTM), for network anomaly detection. Additionally, we evaluate Convolution Neural Networks (CNNs) for network anomaly detection in this paper. We set up three simple CNN models with different internal depths (shallow CNN, moderate CNN, and deep CNN) to see the impact of the depth to the performance. We evaluate the models using three different types of traffic datasets. Our experimental results show that deeper structures do not make any performance improvement. In addition, we observed that the evaluated CNN models occasionally outperform the VAE models, but do not work better than the other deep learning models based on FCN and Seq2Seq-LSTM.

Identifying anomalous events in the network is one of the vital functions in enterprises, ISPs, and datacenters to protect the internal resources. With its importance, there has been a substantial body of work for network anomaly detection using supervised and unsupervised machine learning techniques with their own strengths and weaknesses. In this work, we take advantage of the both worlds of unsupervised and supervised learning methods. The basic process model we present in this paper includes (i) clustering the training data set to create referential labels, (ii) building a supervised learning model with the automatically produced labels, and (iii) testing individual data points in question using the established learning model. By doing so, it is possible to construct a supervised learning model without the provision of the associated labels, which are often not available in practice. To attain this process, we set up a new property defining anomalies in the context of clustering, based on our observations from anomalous events in network, by which the referential labels can be obtained. Through our extensive experiments with a public data set (NSL-KDD), we will show that the presented method perform very well, yielding fairly comparable performance to the traditional method running with the original labels provided in the data set, with respect to the accuracy for anomaly detection.

Active learning enables supervised classifiers to learn using fewer labeled samples, by actively selecting samples for human labeling. Most Active Learning approaches can be categorized as pool-based or stream-based. Pool-based strategies select instances to be labeled from the available pool of unlabeled data, by evaluating each instance, whereas stream-based strategies examine every instance in the incoming stream of unlabeled data and decide sequentially whether they want that instance to be labeled or not. Stream-based strategies enable the ability to adapt the classifier model more quickly as the incoming data changes, while pool-based strategies often exhibit better learning rates. In this paper, we propose a framework and method for Hybrid Active Learning that integrates pool-based and stream-based strategies to harvest the benefits of both, in a streaming data classification scenario where concept drift may be prevalent, and labeling is asynchronous. In addition, we propose (i) prioritized aggregation of selection to combine selected instances for labeling from the pool-based and stream-based strategies, and (ii) batch period adaptation to dynamically change the triggering pattern of the pool-based strategy based upon the detection of concept drift.

Federated cloud enables a workflow to be deployed in multiple private and public clouds. By facilitating external cloud-based services to execute sub-tasks of the workflow, service workflow owners can reduce the cost of executing the workflow, while meeting a performance requirement, since those cloud-based services can be more cost efficient and have better performance than internal ones. However, due to inter-dependencies between sub-tasks, the complexity of the workflow, and the heterogeneity of clouds, it is a challenge to achieve the optimal tradeoff between cost and performance. This paper presents a novel workflow scheduler designed to achieve the optimal end-to-end execution time and cost when deploying such complex workflows in heterogeneous computing nodes in clouds. Specifically, our scheduling algorithm addresses the tradeoff between the execution cost, the computing time, and the data transfer delay between sub-tasks. Our scheduler can handle complex workflows that contain recursively paralleled sub-flows caused by branch and merging sub-tasks. Experiments indicate that our scheduler can efficiently compute the near optimal deployment compared with greedy and evolutionary algorithms for both end-to-end execution time and corresponding cost.

The proliferation of cloud computing can imply a barrier to cloud users. When deploying their complex workloads into clouds, cloud users are typically overwhelmed by too many technical choices. Moreover, underlying technologies and pricing mechanisms of clouds vary and are not transparent to them. Consequently, it is hard for cloud users to capture the monetary and performance implications of their workload deployments. This paper introduces a cloud recommendation platform, referred to as Cloud Advisor. It allows cloud users to explore various cloud configurations recommended based on user preferences such as budget, performance expectation, and energy saving for given workload. Then, it allows cloud users to compare offered price and performance with other clouds' offerings for the workload. By providing transparent comparisons, it can also support cloud provider to develop a competitive pricing strategy such as price reduction driven by energy efficiency. We have applied the proposed platform for recommendation from a real data center and some external clouds.

MapReduce-Hadoop has emerged as an effective framework for large-scale data analytics, providing support for executing jobs and storing data in a parallel and distributed manner. MapReduce has been shown to perform very well on large datacenters running applications where the data can be effectively divided into homogeneous chunks running across homogeneous hardware. However, the performance of MapReduceHadoop is far from ideal when either or both hardware and datasets are heterogeneous. Such heterogeneity is unavoidable in many academic computing environments that use multiple generations of hardware, and share resources among users. Heterogeneity is also unavoidable in scientific applications that process a varying number of datasets of different sizes. In these cases, the performance of MapReduce-Hadoop can be a concern. In this paper, we implement MapReduce on top of CometCloud to address the issue of heterogeneity and support applications classes that involve irregular datasets (e.g. large number of small data files or datasets of varying sizes). Furthermore, we develop an autonomic manager that can schedule MapReduce tasks based on user objective, provision resources accordingly, and support on-demand scale up and cloudbursts. These resources can be selected from a hybrid infrastructure such as local clusters, data centers, and public clouds. The performance of the developed solution is verified using a protein data mining application operating on data from the Protein Data Bank. The application is deployed, based on deadline and budget constraints, on a cluster at Rutgers and/or Amazon EC2 resources. The experimental results show that the MapReduce-CometCloud framework can effectively support applications operating on large numbers of small data files on a heterogeneous and distributed environment, and satisfy user objective autonomically using cloudbursts.

Clouds enable novel execution modes often supported by advanced capabilities such as autonomic schedulers. These capabilities are predicated upon an accurate estimation and calculation of runtimes on a given infrastructure. Using a well understood high-performance computing workload, we find strong fluctuations from the mean performance on EC2 and Eucalyptus-based cloud systems. Our analysis eliminates variations in IO and computational times as possible causes, we find that variations in communication times account for the bulk of the experiment-to-experiment fluctuations of the performance.

Clouds are emerging as an important class of distributed computational resources and are quickly becoming an integral part of production computational infrastructures. An important but oft-neglected question is, what new applications and application capabilities can be supported by clouds as part of a hybrid computational platform? In this paper we use the ensemble Kalman-filter based dynamic application workflow and investigate how clouds can be effectively used as an accelerator to address changing computational requirements as well as changing Quality of Service constraints (e.g., deadlines). Furthermore, we explore how application and system-level adaptivity can be used to improve application performance and achieve a more effective utilization of the hybrid platform. Specifically, we adapt the ensemble Kalman-filter based application formulation (serial versus parallel, different solvers etc.) so as to execute efficiently on a range of different infrastructure (from High Performance Computing grids to clouds that support single core and many-core virtual machines). Our results show that there are performance advantages to be had by supporting application and infrastructure level adaptivity. In general, we find that grid-cloud infrastructure can support novel usage modes, such as deadline-driven scheduling, for applications with tunable characteristics that can adapt to varying resource types.

In todays turbulent market conditions, the ability to generate accurate and timely risk measures has become critical to operating successfully, and necessary for survival. Value-at-risk (VaR) is a market standard risk measure used by senior management and regulators to quantify the risk level of a firm's holdings. However, the time-critical nature and dynamic computational workloads of VaR applications, make it essential for computing infrastructures to handle bursts in computing and storage resources needs. This requires on-demand scalability, dynamic provisioning, and the integration of distributed resources. While emerging utility computing services and clouds have the potential for cost-effectively supporting such spikes in resource requirements, integrating clouds with computing platforms and data centers, as well as developing and managing applications to utilize the platform remains a challenge. In this paper, we focus on the dynamic resource requirements of online risk analytics applications and how they can be addressed by cloud environments. Specifically, we demonstrate how the CometCloud autonomic computing engine can support online multi-resolution VaR analytics using and integration of private and Internet cloud resources.

Clouds are rapidly joining high-performance Grids as viable computational platforms for scientific exploration and discovery, and it is clear that production computational infrastructures will integrate both these paradigms in the near future. As a result, understanding usage modes that are meaningful in such a hybrid infrastructure is critical. For example, there are interesting application workflows that can benefit from such hybrid usage modes to, per- haps, reduce times to solutions, reduce costs (in terms of currency or resource allocation), or handle unexpected runtime situations (e.g., unexpected delays in scheduling queues or unexpected failures). The primary goal of this paper is to experimentally investigate, from an applications perspective, how autonomics can enable interesting usage modes and scenarios for integrating HPC Grid and Clouds. Specifically, we used a reservoir characterization application workflow, based on Ensemble Kalman Filters (EnKF) for history matching, and the CometCloud autonomic Cloud engine on a hybrid platform consisting of the TeraGrid and Amazon EC2, to investigate 3 usage modes (or autonomic objectives) - acceleration, conservation and resilience.

To provide on-demand video streaming services through the network, video objects with both high and low client request rates must be served efficiently. In this paper, we propose a dynamic scheme transition to provide on-demand streaming services efficiently regardless of video popularity. This approach can maintain quality-of-service (QoS) by transitioning the service scheme according to the request rate. The server provides services by heuristically broadcasting video segments when the video popularity is low and by a periodic broadcast when the video popularity is high. The server identifies the variations in client request rates from the number of service channels and determines transitions to more efficient service schemes autonomously. We evaluated this scheme transition for various parameters and traces from a company providing streaming services. The results show that the performance of scheme transition is very efficient.

Peer-to-peer (P2P) networks are attracting considerable research interest because of their scalability and high performance relative to cost. One of the important services on a P2P network is the streaming service. However, because each node in the P2P network is autonomous, it is difficult to provide a stable streaming service on the network. Therefore, for a stable streaming service on the P2P network, a fault-tolerant scheme must be provided. In this paper, we propose two new node selection schemes, playback node first (PNF) and playback node first with prefetching (PNF-P), that can be used for a service migration-based fault-tolerant streaming service. The proposed schemes exploit the fact that the failure probability of a node currently being served is lower than that of a node not being served. Simulation results show that the proposed schemes outperform traditional node selection schemes.

The reliable execution of a mobile agent is a very important design issue to build a mobile agent system and many fault-tolerant schemes have been proposed. Hence, we present the experimental evaluation of the performance of the fault-tolerant schemes for the mobile agent environment. Our evaluation focuses on the checkpointing schemes and deals with the cooperating agents.

We evaluate the performance of checkpointing and replication schemes for the fault tolerant mobile agent system. For the quantitative comparison, we have implemented an experimental system on top of the Mole mobile agent system and also built a simulation system to include various failure cases. Our experiment aims to have the insight into the behavior of agents under two schemes and provide a guideline for the fault tolerant system design. The experimental results show that the checkpointing scheme shows a very stable performance; and for the replication scheme, some controllable system parameter values should be chosen carefully to achieve the desirable performance.