Data preparation is the first and a very important step towards any Large Language Model (LLM) development. This paper introduces an easy-to-use, extensible, and scale-flexible open-source data preparation toolkit called Data Prep Kit (DPK). DPK is architected and designed to enable users to scale their data preparation to their needs. With DPK they can prepare data on a local machine or effortlessly scale to run on a cluster with thousands of CPU Cores. DPK comes with a highly scalable, yet extensible set of modules that transform natural language and code data. If the user needs additional transforms, they can be easily developed using extensive DPK support for transform creation. These modules can be used independently or pipelined to perform a series of operations. In this paper, we describe DPK architecture and show its performance from a small scale to a very large number of CPUs. The modules from DPK have been used for the preparation of Granite Models [1] [2]. We believe DPK is a valuable contribution to the AI community to easily prepare data to enhance the performance of their LLM models or to fine-tune models with Retrieval-Augmented Generation (RAG).

Recently, the needs of data required for data analysis are becoming more diversified, and research on data extraction and analysis methods has been continuously made in order to effectively respond to various needs. Process mining is a solution that analyzes various system logs built by companies or healthcare institutions so that they can be used for process improvement. From the process model extracted from the system logs, it is possible not only to grasp the exact flow of the current business process, but also to acquire additional information such as repetitive execution of activities in the process where the bottleneck occurs in the business process flow. The manufacturing industry has made great efforts to improve the process management, and as many companies are paying attention to big data these days, various data-related technologies are emerging in the healthcare industry as well to properly provide patients with the care needed. Process mining tools allow users to pull data by programming in a process mining query language using the APIs provided with the process mining tool, or by manually creating reusable analytical documents using user friendly tool. However, these tasks require the users to be familiar with the query language APIs and understand the data model and its relationships with respect to creating analytical documents. This paper proposes a methodology that allows users to easily extract desired data through natural language interface, which relieves nonprofessional users of the burden of programming in a process mining query language. The process mining query engine with natural language interface presented in this paper consists of four major components. Among them, the natural language processing pipeline that not only extracts intermediate representation of entities used when constructing a process mining query language report from natural language queries, but also effectively extracts a query hint from the context of natural language query. The query hint is used to select a process-specific function from the library that fits the context of the user query while transforming a natural language query into a process mining query report. The method proposed in this study has the advantage of being able to roughly grasp the process state for the user just by entering a query in natural language. The proposed system provides users with four query process options. That is, the user 1) retrieves intermediate representation of entities and query hints from the NLP pipeline, 2) retrieves the process mining query language from the query language generator, 3) submits the query language to the process mining engine and execute the query, 4) retrieves description of intermediate representation of entity and query hints in natural language to confirm that the query is processed correctly. The contents proposed in this paper were constructed and executed, and the query reports in process mining query language programmatically generated by the proposed query engine were also executed in a process mining engine and the query results were verified.

Database management systems offer an efficient way of managing huge amount of data such as financial and healthcare data and he data retrieval from databases requires knowledge of Structured Query Language (SQL). In this paper, an Automatic SQL Generation System is proposed to help users who are inexperienced in querying database with SQL. The proposed SQL generation system reads formatted data items in the query report from the user and converts the data items into SQL statements programmatically with the help of a data model that is pulled from a database. The SQL generation system can handle simple queries composed of a query block with a SELECT statement as well as complex queries composed of multiple query blocks containing multiple SELECT statements. The proposed system is integrated with an NLIDB (Natural Language Interface for Database) system to translate data items (or tokens) extracted from queries in natural languages into SQL query language, and the system is also integrated and adapted with various types of databases and use cases that include financial and healthcare use cases. The experiment results show that the proposed system correctly handles user queries in natural language just like any other neural model based system and more importantly, the proposed SQL generation engine generates SQL queries without syntactic problems with various databases for all queries.

A large number of time series forecasting models including traditional statistical models, machine learning models and more recently deep learning have been proposed in the literature. However, choosing the right model along with good parameter values that performs well on a given data is still challenging. Automatically providing a good set of models to users for a given dataset saves both time and effort from using trial-and-error approaches with a wide variety of available models along with parameter optimization. We present AutoAI for Time Series Forecasting (AutoAI-TS) that provides users with a zero configuration (zero-conf) system to efficiently train, optimize and choose best forecasting model among various classes of models for the given dataset. With its flexible zero-conf design, AutoAI-TS automatically performs all the data preparation, model creation, parameter optimization, training and model selection for users and provides a trained model that is ready to use. For given data, AutoAI-TS utilizes a wide variety of models including classical statistical models, Machine Learning (ML) models, statistical-ML hybrid models and deep learning models along with various transformations to create forecasting pipelines. It then evaluates and ranks pipelines using the proposed T-Daub mechanism to choose the best pipeline. The paper describe in detail all the technical aspects of AutoAI-TS along with extensive benchmarking on a variety of real world data sets for various use-cases. Benchmark results show that AutoAI-TS, with no manual configuration from the user, automatically trains and selects pipelines that on average outperform existing state-of-the-art time series forecasting toolkits.

Multimodal analysis that incorporates time series and textual corpora as input data sources is becoming a promising approach, especially in the financial industry. However, the main focus of such analysis has been on achieving high prediction accuracy rather than on understanding the association between the two data modalities. In this work, we address the important problem of automatically dis- covering a small set of top news articles associated with a given time series. Towards this goal, we pro- pose a novel multi-modal neural model called MSIN that jointly learns both the numerical time series and the categorical text articles in order to unearth the correlation between them. Through multiple steps of data interrelation between the two data modalities, MSIN learns to focus on a small subset of text articles that best align with the current performance in the time series. This succinct set is timely discovered and presented as recommended documents for the given time series, offering MSIN as an automated information filtering system. We empirically evaluate its performance on discovering daily top relevant news articles collected from Thomson Reuters for two given stock time series, AAPL and GOOG, over a period of seven consecutive years. The experimental results demonstrate MSIN achieves up to 84.9% and 87.2% respectively in recalling the ground truth articles, superior to SOTA algorithms that rely on conventional attention mechanisms in deep learning.

Discovering temporal lagged and inter-dependencies in multivariate time series data is an important task. However, in many real-world applications with big data, such as commercial cloud management or predictive maintenance in manufacturing, such dependencies can be time-variant and non-linear, which makes it more challenging to extract such dependencies through traditional methods like Granger causality or statistical models. In this work, we present a novel deep learning model that uses multiple layers of adapted gated recurrent units (GRUs) for discovering both time lagged behaviors and inter-timeseries dependencies, representing them in the form of directed weighted graphs. Each individual time series is first analyzed by a pair of encoding-decoding GRUs in order to discover the time lagged dependencies and representing its samples as high dimensional vectors. Such vectors collected from all component time series are then analyzed by a decoding network component to discover inter-dependencies across all time series while forecasting their next values in the multivariate time series. Though the discovery of two types of dependencies are separated at two levels of our neural network, they are tightly connected and jointly trained in an end-to-end manner. With this joint training, improvement in learning of one type of dependency immediately impacts the learning process of the other one, leading to the overall highly accurate dependencies discovery. We empirically test our model on synthetic time series data in which the exact form of dependencies are known. We also practically evaluate its performance on two real-world applications, (i) dynamic multivariate performance monitoring data with high volatility from a commercial cloud provider and, (ii) multivariate time series generated by sensors for a manufacturing plant. We show how our approach is capable of capturing these dependency behaviors via intuitive and interpretable dependency graphs and use them to generate forecasting values.

Analysis of legal and contract documents often requires both the discovery of document structure, as well as the accurate identification of important elements such as party (buyer, supplier), nature (obligation, right) and category (warranties, delivery, etc.). Hence, exploring novel features that lead to better element classification accuracy as well as better document structure discovery is particularly important. In this paper, we develop and present novel unsupervised learning techniques to analyze a large scale corpus of contract documents with the goal of learning and deriving new features to enhance classification accuracy over the elements of interest, and to extract relevant features leading to meaningful clusterings over contract structures. Particularly, we propose a novel t-threshold autoencoder neural network that flexibly controls the number of active neurons in response to sentences of different lengths at the network's input. Such an adaptive sparseness threshold enforces competition and specialization among encoding neurons and hence results in better features learning. We also present an extension of the convolutional neural network classifier that allows for the incorporation of these novel augmented features and show that higher classification accuracies over various classes of contract elements can be achieved. We further present a practical pipeline of deriving features from contract documents along with a clustering solution based on the K-means algorithm that leads to the separation among different types of sentences in the contract documents. We empirically demonstrate the performance of our developed techniques on a novel data corpus of Software Procurement contracts.

Change detection in system behavior and its root cause detection is essential for many large-scale systems such as, manufacturing plants, in order to keep systems running uninterrupted and avoid costly machine breakdown via predictive maintenance. In this paper, we present a novel graph based technique that uses time variant interdependencies and lagged dependencies among different components of a system to detect changes in the system behavior. We further find the root causes for these detected changes by pointing out the component and its historical values that are responsible for initiating and changing the system to the new state. The proposed mechanism extracts these time variant dependencies using a deep learning system and converts them into weighted directed graphs and applies graph based techniques for change detection. For each detected change, our system uses graph theoretic techniques to uncover the root causes for the change. Such a mechanism provides us with valuable insights about the inner workings of a system from a different perspective as opposed to traditional techniques for root cause analysis that directly apply statistical models to the time series data for analysis. Experimental results on real manufacturing data show, that we can detect changes in system behavior and accurately identify the root causes in almost 71% of the cases for which we have the ground truth. For synthetic data, our system can correctly identify root causes in 87% of the cases.

Automated Ontology Learning systems are nowadays practical and used in a variety of domains. By using these systems, subject matter experts (SMEs) and ontology designers can readily construct very large ontologies consisting of tens of thousands of concepts and their relations based on a corpus. However, ontologies of this size make it extremely challenging for such SMEs to understand and further tune these ontologies. Prior studies have proposed techniques for concept ranking based solely on the analysis of the structure of the ontology graphs. In this paper, we propose a novel approach, which further exploits a word-level summarization technique applied to the source documents used to generate the ontology. Using the document summarization technique, we devise features that measure concept importance based on source documents where concepts are extracted. We demonstrate the effectiveness of our approach by comparing with existing ranking methods and by devising a scalable evaluation process inspired from the document retrieval domain.

Performance monitoring of cloud-native applications that consist of several micro-services involves the analysis of time series data collected from the infrastructure, platform, and application layers of the cloud software stack. The analysis of the runtime dependencies amongst the component microservices is an essential step towards performing cloud resource management, detecting anomalous behavior of cloud applications, and meeting customer Service Level Agreements (SLAs). Finding such dependencies is challenging due to the non-linear nature of interactions, aberrant data measurements and lack of domain knowledge. In this paper, we propose a novel use of the modeling capability of Long-Short Term Memory (LSTM) recurrent neural networks, which excel in capturing temporal relationships in multi-variate time series data and being resilient to noisy pattern representations. Our proposed technique looks into the LSTM model structure, to uncover dependencies amongst performance metrics, which were learned during training. We further apply this technique in three monitoring use cases, namely finding the strongest performance predictors, discovering lagged/temporal dependencies, and improving the accuracy of forecasting for a given metric. We demonstrate the viability of our approach, by comparing the results of our proposed method in the three use cases with those obtained from previously proposed methods, such as Granger causality and the classical statistical time series analysis models, such as ARIMA and Holt-Winters. For our experiments and analysis, we use performance monitoring data collected from two sources: a controlled experiment involving a sample cloud application that we deployed in a public cloud infrastructure and cloud monitoring data collected from the monitoring service of an operational, public cloud service provider.

Apache Spark enables fast computations and greatly accelerates analytics applications by efficiently utilizing the main memory and caching data for later use. At its core Apache Spark uses data structures called RDDs (Resilient Distributed Datasets) to give a unified view to the distributed data. However, the data represented in the RDDs remain unencrypted which can result in leakage of confidential data produced or processed by applications. Apache Spark persists (unencrypted) RDDs to the disk storage under various circumstances including but not limited to caching, RDD checkpointing and data spill during the data shuffling operations, etc. This lack of security makes Apache Spark unsuitable for processing of sensitive information that should be secured at all times. Moreover, RDDs stored in the main memory are prone to main-memory attacks such as RAM-scrapping. In this paper, we propose and develop solutions to fill-up such security lapses in the current Apache Spark framework. We present three different approaches to incorporate security in the Apache Spark framework. These approaches are designed to limit the exposure of unencrypted data during data processing, caching and data spill to disk. We use combination of cryptographic splitting and encryption to secure data stored and spilled by Apache Spark, both to the disk as well as to the main memory. Our approaches provide strong security by incorporating combination of Information Dispersal Algorithm (IDA) and Shamir's Perfect Secret Sharing (PSS). Extensive experimentation show that with appropriately chosen parameters our security approaches provide high security at a performance penalty between 10%-25%.

The increasing interest of researchers in service oriented architecture (SOA) for wireless sensor networks (WSNs) is opening new unexplored venues in the field of WSNs. In service oriented systems, services are configured and composed of various other services and thus perform complex tasks. In such composite services, the geospatial locations of services and their coverage is of vital importance as they signify the geospatial relevance of the service to the area of interest to the user. In this paper, we present a service-oriented system for WSNs that is capable of performing service configuration under geospatial and relevancy constraints. We present and evaluate “Cost Based Model (CBM)” and “Gain Based Model (GBM)” approaches to capture the relevancy of services hosted on WSN nodes in composite service configuration. The system is resilient to failures and can operate in manual or autonomous recovery modes. The system supports three service configuration methods namely, distributed, centralized and hybrid. Furthermore, we present a novel emulation mechanism for testing the performance of our proposed relevancy models and show that our system efficiently configures services.

Cloud service providers are offering the popular Hadoop analytics platform following an "as-a-service" model, i.e. clusters of machines in their cloud infrastructures pre-configured with Hadoop software. Such offerings lower the cost and complexity of deploying a comparable system on-premises, however security considerations and in particular data confidentiality hamper wider adoption of such services by enterprises that handle data of sensitive nature. In this paper, we describe our efforts in providing security for data-at-rest (i.e. data that is stored) when Hadoop is offered as a cloud service. We analyze the requirements and architecture for such service and further describe a new distributed file system that we developed for Hadoop called SDFS, towards supporting this premise. We analyze parameter tuning for SDFS and through experiments on a real test-bed we evaluate its performance. We further present simulation results that explore the parameter space and can guide tuning.

Accurate, reliable and actionable intelligence produced by smart environment assets is essential in order for dynamic operations such as emergency response or humanitarian relief to be effective. Leveraging Future Internet building block technologies, smart environments are ecosystems which seamlessly embed IT assets into physical world's objects that collect, process and disseminate operational data and insights. The management of smart environment assets towards an efficient collaboration in multi-partner, dynamic scenarios where assets are owned and operated by different partners is a non-trivial problem, due to partners' restrictive sharing policies. In this work we compare two asset sharing approaches; the first is based on a traditional asset ownership model, while the second novel one, is based on an edge team-based model where users are grouped into cross-partner teams and as team members they have access to assets belonging to all the partners participating in team. We further experiment with the second, unexplored team-based sharing model by testing its behavior under different user mobility patterns and extreme asset ownership models investigating its impact on MSTA-P, a policy-regulated version of an existing asset-task assignment protocol. For the protocol's evaluation we implement a multi-partner operation scenario using an open source, agent-based and discrete time simulation environment.

Smart environments are ecosystems, which seamlessly embed IT assets into physical world's objects and hold promise for improving the services we receive from our social and economic ecosystems. The management of smart environment assets in multi-partner, dynamic collaboration scenarios where different sets of assets are owned and operated by different partners is a non-trivial problem, due to restrictive asset sharing policies applied by collaborating partners. In this work we formalize, evaluate and compare two asset sharing policies, investigating their impact on MSTA-P, a policy-regulated version of an existing asset-task assignment protocol. The first sharing policy is based on a traditional asset ownership model while the second is based on an edge model allowing asset sharing among collaborating partnes through cross-partner team formations. We find that while the traditional ownership model allows slightly better performance, the difference is only marginal, so a team-sharing model offers a viable alternative sharing approach.

Service modeling and service composition are software architecture paradigms that have been used extensively in web services where there is an abundance of resources. They mainly capture the idea that advanced functionality can be realized by combining a set of primitive services provided by the system. Many efforts in web services domain focused on detecting the initial composition, which is then followed for the rest of service operation. In sensor networks, however, communication among nodes is error-prone and unreliable, while sensor nodes have constrained resources. This dynamic environment requires a continuous adaptation of the composition of a complex service. In this paper, we first propose a graph-based formulation for modeling sensor services that maps to the operational model of sensor networks and is amenable to analysis. Based on this model, we formulate the process of sensor service composition as a cost-optimization problem and show that it is NP-complete. Two heuristic methods are proposed to solve the composition problem: the top-down and the bottom-up approaches. We discuss centralized and distributed implementations of these methods. Finally, using ns-2 simulations, we evaluate the performance and overhead of our proposed methods.

While virtual machine (VM) migration is allowing data centers to rebalance workloads across physical machines, the promise of a maximally utilized infrastructure is yet to be realized. Part of the challenge is due to the inherent dependencies between VMs comprising a multi-tier application, which introduce complex load interactions between the underlying physical servers. For example, simply moving an overloaded VM to a (random) underloaded physical machine can inadvertently overload the network. We introduce AppAware-a novel, computationally efficient scheme for incorporating (1) inter-VM dependencies and (2) the underlying network topology into VM migration decisions. Using simulations, we show that our proposed method decreases network traffic by up to 81%compared to a well known alternative VM migration method that is not application-aware.

There is a significant challenge in designing, optimizing, deploying and managing complex sensor networks over heterogeneous communications infrastructures. The ITA Sensor Fabric addresses these challenges in the areas of sensor identification and discovery, sensor access and control, and sensor data consumability, by extending the message bus model commonly found in commercial IT infrastructures out to the edge of the network. In this paper we take the message bus model further into a semantically rich, model-based design and analysis approach that considers the sensor network and its contained services as a Service Oriented Architecture. We present an application of a hierarchic schema for nested service definitions together with an initial ontology that describes the assets and services deployed in a sensor network infrastructure.

The vitality and utility of a network are affected significantly by the network management system (NMS) that is used to administer and monitor the network. However, models that can characterize the quality of a NMS are generally missing in the literature. In this paper, we introduce the concept of quality of monitoring (QoM), provide a mathematical formulation based on stochastic processes that can be used to model a network monitoring system and define QoM metrics based on this formulation. A formal analysis of the proposed framework along various metrics is also provided, along with a case study of its application to network monitoring in a mobile ad hoc network.

Security and privacy are two major concerns in supporting roaming users across administrative domains. In current practices, a roaming user often uses encrypted tunnels, e.g., Virtual Private Networks (VPNs), to protect the secrecy and privacy of her communications. However, due to its encrypted nature, the traffic flowing through these tunnels cannot be examined and regulated by the foreign network's firewall, which may lead the foreign network widely open to various attacks from the Internet. This threat can be alleviated if the users reveal their traffic to the foreign network or the foreign network reveals its firewall rules to the tunnel endpoints. However, neither approach is desirable in practice due to privacy concerns. In this paper, we propose a Cross-Domain Cooperative Firewall (CDCF) that allows two collaborative networks to enforce each other's firewall rules in an oblivious manner. In CDCF, when a roaming user establishes an encrypted tunnel between his home network and the foreign network, the tunnel endpoint (e.g., a VPN server) can regulate the traffic and enforce the foreign network's firewall rules, without knowing these rules. The key ingredients in CDCF are the distribution of firewall primitives across network domains, and the enabling technique of efficient oblivious membership verification. We have implemented CDCF and integrated it with the OpenVPN software, and evaluated its performance using extensive experiments. Our results show that CDCF can protect the foreign network from encrypted tunnel traffic with minimal overhead.

SMS has been arguably the most popular wireless data service for cellular networks. Due to its ubiquitous availability and universal support by mobile handsets and cellular carriers, it is also being considered for emergency notification and other mission-critical applications. Despite its increased popularity, the reliability of SMS service in real-world operational networks has received little study so far. In this work, we investigate the reliability of SMS by analyzing traces collected from a nationwide cellular network over a period of three weeks. Although the SMS service incorporates a number of reliability mechanisms such as delivery acknowledgement and multiple retries, our study shows that its reliability is not as good as we expected. For example the message delivery failure ratio is as high as 5.1% during normal operation conditions. We also analyze the performance of the service under stressful conditions, and in particular during a "flash-crowd" event that occurred in New Year's Eve of 2005. Two important factors that adversely affect reliability of SMS are also examined: bulk message delivery that may induce network-wide congestion, and the topological structure of the social network formed by SMS users, which may facilitate quick propagation of viruses or other malware.

A global-scale low cost outdoor Internet access infrastructure is finally attainable. Emerging projects are leveraging the proliferation of private Wi-Fi networks to build a global-scale ubiquitous access infrastructure from autonomous, independently owned Internet connections at homes and other private properties. To ensure the trace ability and accountability required by the broadband ISPs and private owners of these Wi-Fi networks, reliable authentication and authorization are needed. This paper describes authentication on the edge, a localized and distributed authentication method for open Wi-Fi networks. Three main ideas are used to adapt to the variability and unreliability of these networks: the use of certificate-based authentication, the distribution of a segmented certificate revocation list to all entities and the self organization of access points into a social lookup network. These methods achieve the scalability needed for the overwhelming size and volume of a global network and increase resiliency against temporary failures in the infrastructure.

In recent years, cellular networks have experienced an astronomical increase in the use of Short Message Service (SMS), making it a popular communication means for inter-personal as well as content provider-to-person usage. Yet little is known about the traffic and message user behavior in real SMS systems. In this paper, we present a measurement study of SMS based on traces collected from a nationwide cellular carrier during a three-week period. We characterize message traffic at both the message level and the conversation thread level. We also examine the "store-and-forward" mechanism of SMS and present initial measurements on how messages are actually delivered.

Routers are expected to play an important role in the Internet protocol-based wireless data network. Although a substantial number of adaptive and intercell coordination techniques have been proposed to improve wireless network performance under dynamic wireless channel conditions and host mobility, a system support framework is still missing. In this paper, we describe DIRAC, a software-based router system that is designed for wireless networks to facilitate the implementation and evaluation of various channel-adaptive and mobility-aware protocols. DIRAC adopts a distributed architecture that is composed of two parts: a router core (RC) shared by the wireless subnets, and a router agent (RA) at each access point/base station. RAs expose wireless link-layer information to the RC and enforce the control commands issued by the RC. This approach allows the router to make adaptive decisions based on link-layer information feedback on both data and control planes. It also permits the router to enforce its policies (e.g., policing) more effectively through underlying link-layer mechanisms. It further enables interaccess-point coordination at the RC. As showcases, we implement under DIRAC the prototypes of three wireless network services: link-layer assisted fast handover, channel-adaptive scheduling, and link-layer enforced policing. Our implementation and experiments show that our distributed wireless router provides a flexible framework, which enables advanced network-layer wireless services that are adaptive to channel conditions and host mobility.

This paper studies TCP performance in a stationary multihop wireless network using IEEE 802.11 for channel access control. We first show that, given a specific network topology and flow patterns, there exists an optimal window size W* at which TCP achieves the highest throughput via maximum spatial reuse of the shared wireless channel. However, TCP grows its window size much larger than W* leading to throughput reduction. We then explain the TCP throughput decrease using our observations and analysis of the packet loss in an overloaded multihop wireless network. We find out that the network overload is typically first signified by packet drops due to wireless link-layer contention, rather than buffer overflow-induced losses observed in the wired Internet. As the offered load increases, the probability of packet drops due to link contention also increases, and eventually saturates. Unfortunately the link-layer drop probability is insufficient to keep the TCP window size around W'*. We model and analyze the link contention behavior, based on which we propose link RED that fine-tunes the link-layer packet dropping probability to stabilize the TCP window size around W*. We further devise adaptive pacing to better coordinate channel access along the packet forwarding path. Our simulations demonstrate 5 to 30 percent improvement of TCP throughput using the proposed two techniques.

Restricting network access of routing and packet forwarding to well-behaving nodes and denying access from misbehaving nodes are critical for the proper functioning of a mobile ad-hoc network where cooperation among all networking nodes is usually assumed. However, the lack of a network infrastructure, the dynamics of the network topology and node membership, and the potential attacks from inside the network by malicious and/or noncooperative selfish nodes make the conventional network access control mechanisms not applicable. We present URSA, a ubiquitous and robust access control solution for mobile ad hoc networks. URSA implements ticket certification services through multiple-node consensus and fully localized instantiation. It uses tickets to identify and grant network access to well-behaving nodes. In URSA, no single node monopolizes the access decision or is completely trusted. Instead, multiple nodes jointly monitor a local node and certify/revoke its ticket. Furthermore, URSA ticket certification services are fully localized into each node's neighborhood to ensure service ubiquity and resilience. Through analysis, simulations, and experiments, we show that our design effectively enforces access control in the highly dynamic, mobile ad hoc network.

This paper studies TCP performance over multihop wireless networks that use the IEEE 802.11 protocol as the access method. Our analysis and simulations show that, given a specific network topology and flow patterns, there exists a TCP window size W*, at which TCP achieves best throughput via improved spatial channel reuse. However, TCP does not operate around W*, and typically grows its average window size much larger; this leads to decreased throughput and increased packet loss. The TCP throughput reduction can be explained by its loss behavior. Our results show that network overload is mainly signified by wireless link contention in multihop wireless networks. As long as the buffer size at each node is reasonably large (say, larger than 10 packets), buffer overflow-induced packet loss is rare and packet drops due to link-layer contention dominate. Link-layer drops offer the first sign for network overload. We further show that multihop wireless links collectively exhibit graceful drop behavior: as the offered load increases, the link contention drop probability also increases, but saturates eventually. In general, the link drop probability is insufficient to stabilize the average TCP window size around W*. Consequently, TCP suffers from reduced throughput due to reduced spatial reuse. We further propose two techniques, link RED and adaptive pacing, through which we are able to improve TCP throughput by 5% to 30% in various simulated topologies. Some simulation results are also validated by real hardware experiments.

Routers are expected to play an important role in the IP-based wireless data network. Although a substantial number of techniques have been proposed to improve wireless network performance under dynamic wireless channel conditions and host mobility, a system support framework is still missing. In this paper, we describe DIRAC, a software-based router system that is designed for wireless networks to facilitate the implementation and evaluation of various channel-adaptive and mobility-aware protocols. DIRAC adopts a distributed architecture that is composed of two parts: a Router Core (RC) shared by the wireless subnets, and a Router Agent (RA) at each access point/base station. RAs expose wireless link-layer information to the RC and enforce the control commands issued by the RC. This approach allows the router to make adaptive decisions based on link-layer information feedback. It also permits the router to enforce its policies (e.g., policing) more effectively through underlying link-layer mechanisms. As showcases, we implement under DIRAC the prototypes of three wireless network services: link-layer assisted fast handover, channel-adaptive scheduling, and link-layer enforced policing. Our implementation and experiments show that our distributed wireless router provides a flexible framework, which enables advanced network-layer wireless services that are adaptive to channel conditions and host mobility.

Mobile ad hoc networking offers convenient infrastructure-free communication over the shared wireless channel. However, the nature of ad hoc networks makes them vulnerable to security attacks. Examples of such attacks include passive eavesdropping over the wireless channel, denial of service attacks by malicious nodes as well as attacks from compromised nodes or stolen devices. Unlike their wired counterpart, infrastructureless ad hoc networks do not have a clear line of defense, and every node must be prepared for encounters with an adversary. Therefore, a centralized or hierarchical network security solution does not work well. This work provides scalable, distributed authentication services in ad hoc networks. Our design takes a self-securing approach, in which multiple nodes (say, k) collaboratively provide authentication services for any node in the network. This paper follows the design guidelines of [7] and makes several new contributions. We first formalize a localized trust model that lays the foundation for the design, and then expand the adversary model that the system should handle. We further propose refined localized certification services, and develop a new scalable solution of share updates to resist more powerful adversaries. Finally, the new solution is evaluated through simulations. 1

Providing security support for mobile ad-hoc networks is challenging for several reasons: (a) wireless networks are susceptible to attacks ranging from passive eavesdropping to active interfering, occasional break-ins by adversaries may be inevitable in a large time window; (b) mobile users demand "anywhere, anytime" services; (c) a scalable solution is needed for a large-scale mobile network. In this paper, we describe a solution that supports ubiquitous security services for mobile hosts, scales to network size, and is robust against break-ins. In our design, we distribute the certification authority functions through a threshold secret sharing mechanism, in which each entity holds a secret share and multiple entities in a local neighborhood jointly provide complete services. We employ localized certification schemes to enable ubiquitous services. We also update the secret shares to further enhance robustness against break-ins. Both simulations and implementation confirm the effectiveness of our design.