Network and Service analytics can harness the immense stream of operational data from clouds, to services, to social and communication networks. In the era of big data and connected devices of all varieties, analytics and machine learning have found ways to improve reliability, configuration, performance, fault and security management. In particular, we see a growing trend towards using machine learning, artificial intelligence and data analytics to improve operations and management of information technology services, systems and networks.

Large enterprises require reliable and scalable network connectivity which relies heavily on correct network design for LAN and ample bandwidth on WAN. The latter is mostly affected by external market-defined prices, which, absent careful optimization and estimation, can result in unnecessary business expenses. This paper presents a framework for network capacity forecast of a large enterprise to enable accurate and reliable prediction of WAN requirements for all enterprise offices. Quarterly forecasts are generated for individual offices in an enterprise network using historical bandwidth utilization for each office and their associated usage headcount. This framework is currently used to inform WAN circuit upgrade/downgrade decisions for more than 70 offices, and more than 200 associated circuits. The framework uses statistical regression models to create 6, 12, and 24 months forecast for each office, and rigorously evaluates the forecast accuracy with real data going back to 2014Q1. This office-centric approach makes the framework applicable to any corporate network or any large/distributed network-dependent organization.

The paper describes the implementation and experiences gathered from the Spade project which is a decentralized mechanism for data management that has been used in High Energy and Light Source physics experiments. The background and motivations that shaped the Spade project are covered, followed by an outline of the current design of the deployed application. The operational scenarios that have been developed from the use of Spade in a number of difference contexts are enumerated, after which there is a short discussion on how the implementation of the application has progressed. The paper finishes with some highlight of Spade's performance and a conclusion about what has been learnt from this project.

Observational data are fundamental for scientific research in almost any domain. Recent advances in sensor and data management technologies are enabling unprecedented amounts of observational data to be collected and analyzed. However, an essential part of using observational data is not currently as scalable as data collection and analysis methods: data quality assurance and control. While specialized tools for very narrow domains do exist, general methods are harder to create. This paper explores the identification of data issues that lead to the creation of data tests and tools to perform data quality control activities. Developing this identification step in a systematic manner allows for better and more general quality control tools. As our case study, we use carbon, water, and energy fluxes as well as micro-meteorological data collected at field sites that are part of FLUXNET, a network of over 400 ecosystem-level monitoring stations. In an effort toward the release of a new global data set of fluxes, we are doing data quality control for these data. The experience from this work led to the creation of a catalog of issues identified in the data. This paper presents this catalog and its generalization into a set of patterns of data quality issues that can be detected in observational data.

In the past decade, scanning has been widely used as a reconnaissance technique to gather critical network information to launch a follow up attack. To combat, numerous intrusion detectors have been proposed. However, scanning methodologies have shifted to the next-generation paradigm to be evasive. The next-generation reconnaissance techniques are intelligent and stealthy. These techniques use a low volume packet sequence and intelligent calculation for the victim selection to be more evasive. Previously, we proposed models for firewall policy reconnaissance that are used to set bound for learning accuracy as well as to put minimum requirements on the number of probes. We presented techniques for reconstructing the firewall policy by intelligently choosing the probing packets based on the responses of previous probes. In this paper, we show the statistical analysis of these techniques and discuss their evasiveness along with the improvement. First, we present the previously proposed two techniques followed by the statistical analysis and their evasiveness to current detectors. Based on the statistical analysis, we show that these techniques still exhibit a pattern and thus can be detected. We then develop a hybrid approach to maximize the benefit by combining the two heuristics.

In this work we describe a method to automatically detect errors in de novo assembled genomes. The method extends a Bayesian assembly quality evaluation framework, ALE, which computes the likelihood of an assembly given a set of unassembled data. Starting from ALE output, this method applies outlier detection algorithms to identify the precise locations of assembly errors. We show results from a microbial genome with manually curated assembly errors. Our method detects all deletions, 82.3% of insertions, and 88.8% of single base substitutions. It was also able to detect an inversion error that spans more than 400 bases.

Gene synthesis is a key step to convert digitally predicted proteins to functional proteins. However, it is a relatively expensive and labor-intensive process. About 30-50% of the synthesized proteins are not soluble, thereby further reduces the efficacy of gene synthesis as a method for protein function characterization. Solubility prediction from primary protein sequences holds the promise to dramatically reduce the cost of gene synthesis. This work presents a framework that creates models of solubility from sequence information. From the primary protein sequences of the genes to be synthesized, sequence features can be used to build computational models for solubility. This way, biologists can focus the effort on synthesizing genes that are highly likely to generate soluble proteins. We have developed a framework that employs several machine learning algorithms to model protein solubility. The framework is used to predict protein solubility in the Escherichia coli expression system. The analysis is performed on over 1,600 quantified proteins. The approach successfully predicted the solubility with more than 80% accuracy, and enabled in depth analysis of the most important features affecting solubility. The analysis pipeline is general and can be applied to any set of sequence features to predict any binary measure. The framework also provides the biologist with a comprehensive comparison between different learning algorithms, and insightful feature analysis.

Configuring a large number of routers and network devices to achieve quality of service (QoS) goals is a challenging task. In a differentiated services (DiffServ) environment, traffic flows are assigned specific classes of service, and service level agreements (SLA) are enforced at routers within each domain. We present a model for QoS configurations that facilitates efficient property-based verification. Network configuration is given as a set of policies governing each device. The model efficiently checks the required properties against the current configuration using computation tree logic (CTL) model checking. By symbolically modeling possible decision paths for different flows from source to destination, properties can be checked at each hop, and assessments can be made on how closely configurations adhere to the specified agreement. The model also covers configuration debugging given a specific QoS violation. Efficiency and scalability of the model are analyzed for policy per-hop behavior (PHB) parameters over large network configurations.

Monitoring and managing multi-gigabit networks requires dynamic adaptation to end-to-end performance characteristics. This paper presents a measurement collection and analysis framework that automates the troubleshooting of end-to-end network bottlenecks. We integrate real-time host, application, and network measurements with a common representation (compatible with perfSONAR) within a flexible and scalable architecture. Our measurement architecture is supported by a light-weight eXtensible Session Protocol (XSP), which enables context-sensitive adaptive measurement collection. We evaluate the ability of our system to analyze and detect bottleneck conditions over a series of high-speed and I/O intensive bulk data transfer experiments and find that the overhead of the system is very low and that we are able to detect and understand a variety of bottlenecks.

The deployment of ubiquitous distributed monitoring infrastructure such as perfSONAR is greatly increasing the availability and quality of network performance data. Cross-cutting analyses are now possible that can detect anomalies and provide real-time automated alerts to network management services. However, scaling these analyses to the volumes of available data remains a difficult task. Although there is significant research into offline analysis techniques, most of these approaches do not address the systems and scalability issues. This work presents an analysis framework incorporating industry best-practices and tools to perform large-scale analyses. Our framework integrates the expressiveness of Pig, the scalability of Hadoop, and the analysis and visualization capabilities of R to achieve a significant increase in both speed and power of analysis. Evaluation of our framework on a large dataset of real measurements from perfSONAR demonstrate a large speedup and novel statistical capabilities.

Policy-based network management is a necessity for managing large-scale environments. It provides the means for separating high-level system requirements from the actual implementation. As the network size increases, the need for automated tools to perform management becomes more apparent. But configuring routers and network devices to achieve QoS goals is a challenging task. Using Differentiated Services to dynamically perform this configuration involves defining policies on different network nodes in multiple domains. Policy aggregation across domains requires a unified policy model that can overcome the challenge of conflict detection and resolution. In this work, we propose a unified model to represent and encode QoS policies. This model enables efficient and flexible conflict analysis. The representation utilizes a bottom-up approach, from the base policy parameters to the aggregation of policies across domains with respect to traffic classes. We also present a classification of these conflicts and a measure of conflicts to assess the severity of any misconfiguration. The model and the conflict measure are evaluated with large networks and different topologies.

Scientific workflows are an enabler of complex scientific analyses. Large-scale scientific workflows are executed on complex parallel and distributed resources, where many things can fail. Application scientists need to track the status of their workflows in real time, detect execution anomalies automatically, and perform troubleshooting -- without logging into remote nodes or searching through thousands of log files. As part of the NSF-funded Synthesized Tools for Archiving Monitoring Performance and Enhanced DEbugging (STAMPEDE) project, we have developed an infrastructure to answer these needs by integrating detailed workflow and resource monitoring. On top of this infrastructure, we have developed analysis techniques for online detection of a wide variety of "hard" and "soft" types of failures. We use these detected failures to derive higher-level statistics about the status of the resources and the workflow as a whole. In this paper, we describe our techniques and evaluate their effectiveness in the context of real application logs.

Configuring routers and network devices to achieve quality of service (QoS) goals is a challenging task. In a DiffServ environment, traffic flows are assigned specific classes of service, and service level agreements (SLA) are enforced at routers within the domain. We present a model for QoS policy con figurations that facilitates efficient property-based verification. Network configuration is given as a set of policies governing each device. The model efficiently checks the SLA against the current configuration using computation tree logic model checking. By following possible decision paths for a specific flow from source to destination, properties can be checked at each hop, and assessments can be made on how well configurations adhere to the specified agreement. The model also covers configuration debugging given a specific QoS violation.

Network security devices such as firewalls and intrusion detection systems are constantly updated in their implementation to accommodate new features, performance standards and to utilize new hardware optimization. Reliable, yet practical, testing techniques for validating the configuration enforcement after every new software and firmware update become necessary to assure correct configuration realization. Generating random traffic to test the firewall configuration enforcement is not only inaccurate but also impractical as it requires an infeasible number of test cases for a reasonable testing coverage. In addition, in most cases the policies used during testing are manually generated or have limited configuration profiles. We present a framework for automatic testing of the firewall configuration enforcement using efficient and flexible policy and traffic generation. In a typical test session, a large set of different policies are generated based on the access-control list (ACL) grammar and according to custom profiles. Test packets are generated to particularly consider critical segments of the tested policies and to achieve high coverage of the testing space. We also describe our implementation of a fully-automated framework, which includes ACL grammar modeling, the policy generation, test cases generation, capturing and analyzing firewall output, and creating detailed test reports. Our evaluation results show that our security configuration testing is not only achievable but it also offers high coverage with significant degree of confidence.

A firewall policy that is correct and complete is crucial to the safety of a computer network. An adversary will benefit a lot from knowing the policy or its semantics. In this paper, we propose a framework that could be used to blindly discover a firewall policy remotely as a black box and without prior knowledge about the network configuration. We show how an attacker can reconstruct a firewall's policy by probing the firewall with tailored packets into a network and forming an idea of what the policy looks like. The proposed methodology shows how to discover a policy that is semantically equivalent to the original one used in the deployed firewall. Three techniques are proposed for reconstructing the policy as well as to intelligently choose the probing packets adaptively based on the firewall response. We show the possibility of obtaining the deployed policy in a feasible time with acceptable accuracy.

Packet classification plays a critical role in many of the current networking technologies, and efficient yet lightweight packet classification techniques are highly crucial for their successful deployment. Most of the current packet classification techniques exploit the characteristics of classification policies, without considering the traffic behavior in optimizing their search data structures. In this paper, we present novel techniques that utilize traffic characteristics coupled with careful analysis of the policy to obtain adaptive methods that can accommodate varying traffic statistics while maintaining a high throughput. The first technique uses segmentation of the traffic space to achieve disjoint subsets of traffic properties and build bounded depth Huffman trees using the statistics collected for these segments. The second technique simplifies the structure maintenance by keeping the segments ordered in a most-recently-used (MRU) list instead of a tree. The techniques are evaluated and their performance are compared. Moreover, attacks targeting the firewall performance are discussed and corresponding protection schemes are presented.