Federated learning (FL) is a promising distributed machine learning approach with enhanced data privacy protection. However, wireless communication remains a key bottleneck, directly affecting the efficiency and performance of FL. In this paper, we introduce a device-centric cell-free network to mitigate the negative effects of random fading and limited radio resources on FL. The convergence gap, representing the difference between the FL model’s performance and that of the optimal model, is analyzed to evaluate the impact of communication and computation factors, including inter-device interference, on FL performance. Then, access point (AP)-device association, transmission power, and computation frequency are jointly optimized to minimize the convergence gap. Lyapunov techniques are employed to decouple the long-term optimization into a series of online solvable problems. A deep reinforcement learning-based scheme is proposed to optimize the AP association and transmission power for devices, reducing the computational complexity from a prohibitive level to a real-time feasible quadratic level. Additionally, a closed-form solution for the optimal device computation frequency is derived. Simulation results show that the proposed scheme significantly outperforms the traditional cell-free FL and cellular FL schemes in both model training performance and energy efficiency.

Coordinated Multipoint technology plays a key role in improving cellular network performance by allowing user equipment to connect to multiple base stations simultaneously. However, dynamically selecting the best combination of base stations for each user poses a complex NP-hard problem. This decision must adapt in real-time as users move and wireless channel conditions change. Recent reinforcement learning approaches have shown promising results.This paper introduces a multi-agent reinforcement learning approach using transformers to extract the latent graph structure features from user’s observations. This approach can selfadapt to various scenarios and out-perform existing approaches, especially when the number of user equipment (UEs) scales up.

Remaining useful life (RUL) prediction is an important problem in the Prognostics and Health Management (PHM) of industrial machines. Recently, the proliferation of Internet of Things (IoT) has resulted in the availability of a large amount of sensor data obtained from the monitoring of industrial machines. This has greatly enhanced the performance of deep learning based RUL prediction methods. RUL prediction is a multivariate time series forecasting problem where state-of-the-art results have been achieved using transformer architectures and its variants. This success is based on their long sequence modelling ability. Two important characteristics of transformers is that they have a large number of free parameters and require a lot of data for training, and any single organization may not have access to the required data volume. In addition, operational sensor data is private, which organizations may not be willing to share. The Federated Learning (FL) is a suitable method to tackle this issue as it enables model training using data distributed over several sources without explicit data sharing. In many situations, FL client devices connect to the FL coordinator over wireless network and hence the performance achieved will be affected by wireless loss. Furthermore, FL clients may want to use UDP over TCP to exchange the large number of transformer model parameters. In this work, we first propose a novel transformer architecture for RUL prediction. Secondly, we propose multi-agent reinforcement learning (MARL) methods to mitigate the effect of wireless loss on FL based transformer model training. Our system formulation and experiments comprehensively demonstrate the effectiveness of our approach.

The evolution of manufacturing systems towards distributed architectures presents unique challenges in optimising production rates across multiple machines in various locations. To address this, effective coordination of production rates at each stage is crucial to ensure the production capacity of these distributed systems, minimise production-related expenses, and account for the impact of traffic conditions on inter-factory material transportation. In this paper, we explore the use of Multi-Agent Soft Actor-Critic (MASAC), a relatively under-explored Multi-Agent Reinforcement Learning (MARL) algorithm, for controlling production rates in a distributed manufacturing environment. Unlike traditional methods that address the credit assignment problem through value decomposition, this framework leverages a multi-headed Q-network to effectively identify and evaluate individual contributions of agents to the global reward. The MASAC algorithm is evaluated against MADDPG and DDQN algorithms, with simulation results showing that MASAC achieves high production output while significantly reducing energy, storage and computational costs. In addition, we compare the performance of MASAC with MAPPO.

In recent years, the transformer architecture and its variants have been widely applied for multivariate time series forecasting. They have achieved state-of-the-art performance across many domains, e.g. road traffic prediction, weather forecasting, exchange rates forecasting etc. This success is based on their long sequence modelling ability. The self-attention mechanism, the core of the standard transformer architecture and its variants, is position invariant. This is typically dealt with by incorporating positional encoding in the input. However, this alone is insufficient in the case of time series forecasting where the relative position of occurrence of various features is crucial. In this work we propose a new transformer architecture for traffic state forecasting that is more position sensitive. More specifically, we build position awareness into the attention weighted value computation process. We refer to our model as the Position Aware Transformer (PATr). Our experiments validate the performance of PATr extensively using the well-known PeMS traffic dataset. Our results show appreciable improvement in performance compared to the previous state-of-the-art.

Manufacturing processes of assembly lines are now changing from a single factory to distributed multi-factories for minimizing costs and meeting different customer demands. To achieve the production capacity of the distributed manufacturing system, minimize costs incurred during production as well as deal with the influence of traffic conditions on inter-factory material handling, the production rates of machines at each production stage need to be adjusted in a coordinated manner. With the emerging Industry 4.0 technologies, reinforcement learning techniques can be used for optimizing resources usage during production. To achieve better coordination, a multi-agent deep reinforcement learning (MADRL) based framework is proposed to deal with the distributed manufacturing resources optimization problem. A multi-agent PPO based algorithm MAPPO is developed in this framework and evaluated by comparing with the MADDPG and DDQN algorithms. Our simulation results show that MAPPO is able to achieve the production capacity with lower energy, storage, computational and communication costs.

Federated Learning (FL) is a widely used distributed learning framework for training a deep-learning global model while preserving client data privacy. This research extends FL's application into the burgeoning Internet of Things (loT) realm, particularly focusing on the Internet of Vehicles (IoV). In IoV, besides traditional privacy concerns, there is a crucial need for both global model generalization and instant localized response. We adopt a combined 3-tier FL framework and propose: (a) a Federated learning with Selected-Layer- Transmission (FedSLT) algorithm that allows partial model transmission and aggregation to cope with unreliable wireless links (which gives rise to system heterogeneity); and (b) two edge initialization algorithms Edge Aggregation (EA) and Warm-up (WU) that let edge servers shoulder more responsibility to give each edge cluster a better starting point with more customized initial model parameters (which gives rise to statistical heterogeneity) when each global round begins. Empirical experiments using different amounts of non-independent and identically distributed (non-IID) data have been done to support our theoretical analysis and illustrate the advantages of FedSLT and edge initialization algorithms over vanilla FedAvg, especially in severely heterogeneous situations.

The Internet of Vehicles (IoV) enhances data availability by equipping a plethora of sensors, driving the automotive industry towards data-driven Predictive Maintenance (PreM) models. However, traditional centralized PreM solutions, requiring complete access to training data, raise concerns about data privacy. PreM in the automotive domain is more challenging than in many other fields, partly due to the varying distribution nature of data samples and the limited network connectivity time caused by vehicle mobility. To address these challenges, we propose the PreM-FedIoV framework, extending single-agent Double Deep Q-Network (DDQN) to Multi-Agent Double Deep Q-Network (MADDQN). In each round, each vehicle client uploads a data packet to the server based on the current contention window, containing its local model, local test Mean Absolute Error (MAE), and a timestamp. The server initially performs federated aggregation on the received local models. The MADDQN module then dynamically adjusts the contention window of each vehicle for the next round based on the local test MAE and communication statistical state, aiming to optimize communication costs and predictive performance. Additionally, we utilize NS-3 to create IoV simulations and deploy the PreM-FedIoV framework within NS3-gym. We choose Federated Averaging (FedAvg) and FedAdam following the IEEE 802.11p standard as baselines. The experiments demonstrate significant improvements in our framework compared to state-of-the-art algorithms. On the C-MAPSS dataset, we achieve reductions of up to 10.2% in MAE, 26.31% in average communication clock time per round, and 65.6% in the number of participating clients per round. For the Random Battery Usage dataset, with up to 4.55%, 24.44%, and 36.58% improvements in the respective metrics.

The advent of Industry 4.0 has resulted in a significant increase in data availability, leading to the development and deployment of data-driven models for predicting the Remaining Useful Life (RUL) of machines. However, traditional centralized Predictive Maintenance (PreM) solutions that require complete access for training data give raise to concerns regarding data privacy. To address this challenge, Federated Learning (FL) has emerged as a promising and practical approach to enhance task performance while preserving data privacy within network nodes. Nevertheless, the presence of Non-Independent and Identically Distributed (Non-IID) data samples across devices can present challenges in terms of the convergence and speed of FL. Additionally, the heterogeneity of devices can lead to issues such as local model discarding and high communication costs, which are important considerations in FL. To address these challenges, this paper proposes Fedrated Predictive Maintenance (FedPreM), a novel federated reinforcement learning-based PreM scheme. FedPreM selectively involves a subset of devices in each communication round and employs an improved Perturbed Gradient Descent (PGD) optimizer to achieve flexible workload distribution among participating devices. By conducting experiments on a widely used turbofan dataset, our results demonstrate the effectiveness of FedPreM in reducing the number of communication rounds and minimizing prediction errors in distributed Industry 4.0 scenarios.

Supply chain disruptions caused by breakdown of freight vehicles lead to delayed deliveries which cost companies millions of dollars and loss of customer goodwill. Breakdowns can be reduced through predictive maintenance, which has become a mature field with solutions offered by various vendors. In this paper, we propose a prescriptive maintenance approach that leverages deep reinforcement learning (DRL) to directly make maintenance decisions for a fleet of freight vehicles such as trucks. Proximal Policy Optimization (PPO) is a state-of-the-art reinforcement learning (RL) algorithm based on policy gradient and uses function approximators like deep neural networks (DNNs) to store the policy and value functions. In order to introduce long timescale memory that can lead to superior policies in complex problems, we integrate the PPO with a Long Short Term Memory (LSTM) network. The resulting PPO-LSTM scheme requires careful handling of sequences of observations. We investigated the performance of the PPO-DNN and PPO-LSTM schemes in making prescriptive maintenance decisions for a fleet of trucks transporting goods between factories. From sensor readings indicating the condition of the truck transmission system which deteriorate under use, good maintenance decisions are made that enable a large number of trucks to remain active. The performance of these schemes were evaluated in realistic simulations of different traffic conditions using the SUMO simulator working in conjunction with realistic truck transmission system and factory production simulators. Our results show that the proposed schemes outperformed baseline schemes and achieved a significant increase in production throughput under different traffic conditions and maintenance and repair durations.

The Industrial IoT era has benefitted immensely from increased data collection and processing, which enables Industry 4.0 and predictive maintenance. Predictive maintenance tries to predict when different types of failures will occur. Prescriptive maintenance goes further by optimizing the maintenance decisions, i.e. what to do and when to do it. This paper considers a distributed factory environment, and firstly, proposes a model-based approach to prescriptive maintenance using the Partially Observable Markov Decision Process (POMDP) framework. A particle filter algorithm performs online estimation of a POMDP model to enable it to adapt to each machine over time. The POMDP is then solved using the point based value iteration (PBVI) method to obtain maintenance decisions. Next, we present a model-free approach to prescriptive maintenance using the Deep Q-Network (DQN) reinforcement learning method to obtain maintenance decisions. The POMDP and DQN methods are implemented on GPU-accelerated edge computing nodes and their performance in terms of reward and downtime are compared.

The Internet of Things (IoT) has become an important enabler for vehicular network applications, primarily the Internet of Vehicles (IoV). With the increase in the use of IoV, there is a potential increase in vulnerabilities to attacks and faults on vehicular networks. These misbehaviors or anomalies can vary from wrongly broadcasted data to more intense attacks like Denial of Service (DoS). It has become necessary to protect these vehicular networks through anomaly or misbehavior detection mechanisms. Deep learning models can be used for anomaly detection, considering the large volume of vehicular data available to train them. However, this gives rise to a need for privacy and security against data theft or information leaks of vehicular data. Hence, privacy preserving approaches like federated learning can be leveraged for anomaly detection. In this paper, we develop three federated learning (FL) schemes based on the federated averaging (FedAvg), FedAvg with Adam optimizer (FedAvg-Adam) and FedProx algorithms to acquire deep learning models in a distributed manner to perform anomaly detection in the IoV setting. The federated learning tasks run on local nodes deployed at the network edge, and models are combined on a global server deployed on the cloud. Our evaluation results using a publicly available IoV-relevant dataset show that these schemes were able to learn accurate models which permit effective anomaly detection in vehicular networks under different data distributions and network architectures.

Traffic forecasting is an important part of many future intelligent transportation systems and can be particularly useful for planning and navigation applications. The task is challenging because of the complex spatial patterns of road networks and dynamic temporal nature of traffic conditions. Recently, deep learning architectures with specially designed graph convolution layers to extract spatial patterns and recurrent or temporal convolution layers to extract temporal patterns have achieved good results for this task. In this paper, we propose a Mixture of Experts (MoE) based model integration framework to enhance the performance of these state-of-the-art traffic prediction models. In addition, we propose a novel entropy based loss function to improve the training of the MoE ensemble. Our experiments show that the performance of the Spatio-Temporal Graph Convolution Network (STGCN), a state of the art model, can be significantly improved.

Edge computing has become a major trend in networking research. The rapid growth in the number of data analytics based mobile applications has resulted in an exponential rise in processing demand. One way to cope with this increased processing demand is to use edge networks (EN), which is a wireless ad-hoc network of mobile cloudlets, vehicular cloudlets, dedicated edge devices, and cloud platforms. Typically these devices have different owners and service providers. In this work, we propose a distributed incentive mechanism for an EN, which does not require a trusted third party (TTP). We consider a multi hop EN where a user offloads tasks to neighboring nodes, which may further offload them to their neighbours or the cloud. Our scheme is computationally efficient and helps each node decide on the incentives and workload distribution. We conducted simulations to study the performance of our scheme in different scenarios, including those of untruthful behavior by some nodes. Results show the benefit of using multi hop offloading and that our scheme discourages the untruthful behavior of nodes by assigning them lesser workload. We also propose a partially distributed incentive mechanism and compared its performance to our fully distributed scheme.

Mobile crowdsensing (MCS) is an emerging human-powered service for large-scale sensing and data collection. Most existing frameworks for controlling MCS services focus on a system-initiated setting where the crowdsourcer selects a subset of contributors and incentivizes them to collect the sensing data after the queries from the consumers arrive at the system. Such a system-initiated setting may cause large delays for the system to answer the consumers’ queries, which is not suitable for many real-time sensing applications. In this article, we propose contributor-initiated proactive sensing (CIPS) frameworks for MCS where the sensing data are collected in a proactive manner before the consumers’ queries arrive. In CIPS, the consumers can get answers about their queries with virtually no delay, which opens the door for MCS to many real-time applications. We first propose a centralized algorithm called C-CIPS as a benchmark for sensing scheduling by assuming the contributors are truthful and the consumer queries are knowna priori. Next, we propose a distributed algorithm called D-CIPS to deal with strategic contributors and unknown consumer queries. Through rigorous theoretical analysis, we prove that both C-CIPS and D-CIPS can achieve near-optimal solutions. Furthermore, D-CIPS is proved to be truthful. Through comprehensive simulations with both synthetic and real-world data sets, we demonstrate the effectiveness of the proposed algorithms.

The Internet of Things (IoT) has emerged as a key networking infrastructure that connects a large number of sensors, thereby allowing the collection and processing of large amounts of sensor data efficiently. Edge computing involves the deployment of computing devices close to sensor locations in order to process sensor data and derive useful information quickly and efficiently near the source. This improves the quality of the collected data and reduces the strain on the underlying communication networks. The obtained data can be used for a number of inference tasks using machine learning. The majority of machine learning research efforts assume that the collected data is untainted by various effects in the data collection or transmission process. The data-driven models generated with this assumption and the inferences drawn are frequently unusable in practical scenarios. Predictive maintenance is the process of utilizing sensor data obtained from equipment to determine and predict the optimal time for maintenance of the equipment. Using predictive maintenance, the time and money spent on maintaining equipment can be greatly reduced. In this paper, we perform distributed modeling using edge computing to determine the Remaining Useful Lifetime (RUL) of machines used in a manufacturing plant. We analyze the effects of wireless network degradation on the accuracy of the distributed models and propose active learning algorithms to mitigate these effects, thereby improving the usefulness and robustness of predictive maintenance in realistic settings.

The Industrial IoT era has seen an outburst of areas benefiting from collecting more data. This includes Industry 4.0 and predictive maintenance, which have benefited from advancements in edge and fog computing. Predictive maintenance aims to minimize the downtime due to maintenance of machinery, while simultaneously minimizing the risk of unforeseen failures. This paper proposes a method to aid industries to make maintenance scheduling decisions that can be adopted in a distributed factory environment. The Partially Observable Markov Decision Process (POMDP) approach is used to determine the optimal time for maintenance for a machine. We first put forward an offline method for learning the Markov model parameters using historical sensor data. To allow for continual learning, an algorithm based on particle filters is proposed to provide online estimation of parameters of a Partially Observable MDP model. The particle filter algorithm allows the framework to adapt uniquely to each machine. The relative benefits of the POMDP model over a standard MDP model in the presence of noisy sensor data are evaluated through simulations which show significant improvements in revenue and reduced downtime. The POMDP and particle filter computations are executed on GPU-accelerated edge devices which achieve a speed-up of around 4 times compared to the CPU implementation.

Graphical models have been widely applied in distributed network computation problems such as inference in large-scale sensor networks. While belief propagation (BP) based on message passing is a powerful approach to solving such distributed inference problems, one major challenge, in the context of wireless sensor networks, is how to systematically address the trade-off between energy efficiency and inference performance. In this paper, we consider a distributed structure optimization problem and investigate the impacts of graphical model structure on energy consumption and inference performance. We first formulate the problem as a multi-objective constrained combinatorial optimization problem and prove its NP-hardness. Then, we propose an efficient distributed heuristic to solve the problem in polynomial time. Through extensive simulations, using both real-world sensor network data and synthetic data, we empirically evaluate our proposed graphical model structure optimization framework. The simulation results demonstrate that the graphical model constructed by the proposed framework can efficiently trades off the performance of the inference algorithm (measured by the mean squared error) with the energy consumed by the inference algorithm (measured by the energy used in communication). In addition, our proposed framework provides valuable insights for network designers on designing efficient model selection algorithms for distributed inference problems.

Ensemble learning (EL)methods have been shown to be effective for diagnostics and prognostics in industrial systems. By combining the learning ability of different base learners, EL has the potential to reduce the total complexity of the learning system while solving a difficult problem satisfactorily. Recent advantages in Industrial Internet of Things (IIoT)and edge computing technologies have started a new paradigm of distributed diagnostics and prognostics. However, existing EL methods that mainly focus on a centralized setting cannot adapt to the edge computing scenario, which significantly constrains the application of these EL methods in real industrial environments. In this paper, we present a new approach to EL called Graph-based Ensemble Learning (GraphEL)to enable distributed diagnostics and prognostics in real industrial environments. Comparing with existing methods, the proposed GraphEL framework builds different base learners for different subsystems. Furthermore, a graphical model is constructed to define the correlation structures among the outputs of different base learners in the ensemble such that they can be collaboratively trained to optimize the learning performance of the ensemble. Via simple message passing, the proposed GraphEL framework can be executed in a fully distributed manner which is suitable for edge computing. The performance of the proposed GraphEL framework is evaluated using two real-world industrial data sets where we demonstrate the advantages of GraphEL compared to existing EL methods.

Distributed Bayesian inference or estimation in Internet of Things (IoT) has recently received much attention due to its broad application in the areas of object classification, target tracking and medical diagnosis etc. In many distributed IoT systems with limited resources, e.g. sensor networks and crowdsourcing systems, it is likely that only a few agents will have valuable information at any given time. Therefore, the paradigm of information-driven distributed sensing (IDDS) is essential to achieve efficient inference, where the resources are spent only on sensing and communicating valuable information. In this paper, we consider the problem of IDDS for efficient Bayesian inference with exponential family distributions. We first propose a centralized algorithm (C-IDDS) where a centralized controller exists to make sensing decisions for the sensing agents. As the centralized algorithm does not scale well in large systems, we continue to design a distributed algorithm (D-IDDS) where each individual sensing agents can make their own sensing decisions independently. Both C-IDDS and D-IDDS are online algorithms which can adapt to stochastic system conditions without any future information. Through rigorous theoretical analysis, we prove that the proposed algorithms can achieve an asymptotically optimal system-wide utility. A real testbed has been built to evaluate the performance of the proposed algorithms in real-world environments. Using the data from the real-world testbed and comparing with some baseline methods, we demonstrate the effectiveness of the proposed C-IDDS and D-IDDS algorithms.

By taking advantage of pervasive mobile devices and their pairwise encounters, Mobile Cloud Computing (MCC) offers an efficient solution for mobile devices to execute complex applications in a collaborative manner. In this paper, we consider the problem of distributed task offloading in MCC systems with deadline constraints. We propose an online distributed task offloading (DTO) algorithm for practical MCC systems where each mobile user can dynamically make offloading decisions to nearby mobile devices in order to process computation tasks in a collaborative manner. The DTO scheme is lightweight and fully distributed. Through rigorous theoretical analysis, we prove that the proposed DTO algorithm can meet the deadline constraints of the computation tasks and achieve a near-optimal system-wide utility. Furthermore, through real testbed experiments and trace-driven simulations, we compare the DTO scheme with several baseline methods and demonstrate its effectiveness.

In order to achieve better system performance, the concept of an ad-hoc mobile cloud, whereby a mobile device can access resources such as processing, data or storage at other neighbouring nodes, has been proposed. The difficulty that arises with this concept is the mobility of nearby devices, i.e., a neighboring device may move out of range before it can communicate its results back to the source device. In this paper, we propose a workload assignment scheme between a source device and nearby mobile devices that takes into account the randomness of the connection time between these devices. In order to cope with this randomness, we adopt a multi-stage stochastic programming approach which is able to take posterior recourse actions to compensate for inaccurate predictions. Moreover, in order to motivate the available mobile devices to cooperate, we formulate a distributed multi-stage stochastic buyer-seller game (MSSBSG) in which different mobile devices attempt to maximize their utilities. Our results show that the stochastic programming approach outperforms several baseline schemes and the MSSBSG approach effectively promotes cooperation between mobile devices and achieves the best overall performance compared to simpler approaches that do not take stochastic operating conditions into account.

An ad-hoc mobile cloud allows a mobile user to access a potential cloud resource with a short offloading latency and a low bandwidth consumption from nearby mobile devices, namely mobile cloudlets via short-range communications. However, due to the user and cloudlet mobility, the time-varying wireless channel properties, and the cloudlet's limited computation resource, designing an effective offloading algorithm involves challenges in workload distribution, cost estimation and energy minimization. In this paper, to address these challenges, we develop an optimization-based offloading algorithm that enables the mobile user to make an optimal offloading decision. The proposed algorithm takes into account effects of the user's workload, the diverse connectivity of cloudlets, and the wireless environment on the offloading action. More specifically, we formulate and solve a Markov decision process (MDP) scheme to achieve an optimal offloading policy for the mobile user with the objective of maximizing the user's utility while minimizing the offloading cost. Extensive simulations were performed to evaluate the performance of the proposed MDP scheme. The simulation results show that the proposed scheme outperforms baseline schemes.

Mobile cloud computing (MCC) leverages the advantages of both cloud computing and mobile computing. With trusted, resource-rich and Internet-connected computing units, referred to as cloudlets, MCC brings cloud resources closer to the mobile users (MU) at the network edge. While MCC has plenty of advantages, unique challenges exist for efficient operation in practical MCC systems. First, the system-wide utility depends on both the offloading policy at the MUs and the task admission policy at the cloudlets. When the computation tasks have heterogeneous deadlines, determining optimal policies at both the MUs and the cloudlets in a distributed manner is nontrivial. Additionally, truthful mechanisms are needed for MCC markets in order to achieve desirable trading behaviors between MUs and cloudlets. In this paper, we present a novel scheme, called Stochastic Offloading in Mobile cloud computing (SOM), to address the above challenges through an online auction approach. SOM achieves desirable economic properties, such as truthfulness, individual rationality and budget balance. Furthermore, by leveraging Lyapunov optimization techniques, the proposed SOM scheme can drive the long-term system-wide utility towards a near-optimum. Through rigorous theoretical analysis and comprehensive simulations, we demonstrate the effectiveness of SOM.

Due to its advantages of providing a large geographical coverage and having no strict limits on energy and sensing and processing capabilities, a vehicular sensor network (VSN) has recently emerged as a promising paradigm for air quality monitoring in an urban area. However, designing an efficient VSN-based air monitoring system has challenges due to the vehicles' heterogeneous temporal and spatial coverage and the relatively expensive communication cost over cellular networks. In this paper, we propose a machine learning (ML)-based Air quality Monitoring (MLAirM) system which aims at reducing communication and sensing costs by allowing vehicles to process the collected data in a distributed fashion. More specifically, in MLAirM, vehicles are first assigned to take measurements at sets of locations in a sensing area. The vehicle then utilizes a distributed machine learning algorithm to learn a local model of air quality based on its collected data. Finally, the vehicle sends parameters of its model to a monitoring center which combines multiple local models to build a global air quality map. Furthermore, assigning the sensing locations to vehicles can be viewed as a successful measurement probability aware location assignment problem. An integer linear optimization problem is formulated and a heuristic algorithm is proposed to find the solution. Simulations based on realistic vehicular traces are performed to compare the proposed MLAirM system with other approaches. The simulations results show that the MLAirM can achieve a similar accuracy of building the global air quality map with a significant reduction in communication and sensing costs compared to other approaches.

The pervasiveness of mobile phones and the increasing sensing capabilities of their built-in sensors have made mobile crowd sensing (MCS) a promising approach for large-scale event detection and collective knowledge formation. In a typical MCS system, the crowdsourcer purchases sensing data from some mobile phone users (i.e., contributors) and sells it to consumers for revenue. This kind of sensing data exchange has its unique challenges in practical MCS systems. On one hand, the crowdsourcer wants to maximize the information utility to get the most revenue under heterogeneous requests from the consumers while offering incentives to strategic contributors. On the other hand, the contributors need to make optimal real-time sensing and data selling decisions by considering their real-time sensing cost and quality of information, in order to maximize their own profit. In this paper, we propose a novel Information-driven Data Auction (IDA) scheme for data exchange in practical stochastic MCS systems, which offers optimal strategies for both the crowdsourcer and the contributors. By applying stochastic Lyapunov optimization and mechanism design theory, IDA is able to achieve a near-optimal time-averaged system-wide utility, while offering incentives to the contributors. Moreover, IDA achieves favourable economic properties including truthfulness, individual rationality, and budget balance. We demonstrate the efficacy of IDA through rigorous theoretical analysis and comprehensive simulations.

The processing time of real-time sensing, data analytics and other applications is an important performance metric. When the application is being executed on a distributed system, the load balancing scheme among processing nodes significantly affects the total processing time of the application. We consider a load balancing scheme for distributed computing at the edge of the network. In the edge model considered, a group of nodes, either mobile or static, with processing and sensing capabilities, are connected to each other over a wireless ad-hoc network. Load balancing among edge nodes is formulated as a min-max optimization problem, with the objective of minimizing the overall processing time of the application while still satisfying the wireless channel capacity and link contention constraints. We use an aggregate utility method to convert the min-max problem into a convex optimization problem. The obtained constrained convex optimization problem is then relaxed with Lagrangian dual decomposition and solved with gradient descent. This form of the formulation can be implemented in a fully distributed manner among the edge nodes, which is consistent with the decentralized nature of edge networks. However, the convergence of this scheme may be slow. We further propose a heuristic algorithm which achieves fast convergence. Our simulation results show that it can give near-optimal performance most of the time.

We study the problem of dynamically migrating a service in the cloud to satisfy an online sequence of mobile batch-request demands in a cost-effective way. The service may have single or multiple replicas, each running on a virtual machine. As the origin of mobile accesses frequently changes over time, this problem is particularly important for time-bounded services to achieve enhanced Quality of Service and cost effectiveness. Moving the service closer to the client locations not only reduces the service access latency but also minimizes the network costs for service providers. However, these benefits are not free. The migration comes at a cost of bulk-data transfer and service disruption, and hence, increasing the overall service costs. To gain the benefits of service migration while minimizing the caused monetary costs, we propose an efficient search-based algorithm Dmig to migrate a single server, and then extend it as a scalable algorithm, called mDmig , to the multi-server situation, a more general case in the cloud. Both algorithms are fully distributed, symmetric, and characterized by the effective use of historical access information to conduct virtual migration so that the limitations of local search in the cost reduction can be overcome. To evaluate the algorithms, we compared them with some existing algorithms and an off-line algorithm. Our simulation results showed that the proposed algorithms exhibit better performance in service migration by adapting to the changes of mobile access patterns in a cost-effective way.

Graphical models have been widely applied in distributed network computation problems such as inference in large-scale sensor networks. While belief propagation (BP) based on message passing is a powerful approach to solving such distributed inference problems, one major challenge, in the context of wireless sensor networks, is how to systematically address the trade-off between energy efficiency and inference performance. Although various energy-efficient message passing algorithms based on a given graphical model have been proposed in the literature, little work has been done to optimize the graphical model structure to achieve good energy efficiency and inference performance at the same time. In this paper, we propose an efficient distributed algorithm for optimizing the graphical model structure in order to minimize the communication cost required by the inference algorithm without incurring significant performance loss. We first formulate the problem as a multi-objective constrained problem and prove its NP-hardness. Then, we propose an efficient heuristic to solve the problem in polynomial time. Through extensive simulations, using both real-world sensor network data and synthesized data, we empirically evaluate our proposed graphical model structure optimization framework. The simulation results demonstrate that the optimized graphical model efficiently balances the performance of the inference algorithm (measured by mean squared error) and the energy consumed by the inference algorithm (measured by energy used in communication). These highlight the advantages of our proposed framework.

Participatory sensing facilitates smartphone users to contribute data to form a body of knowledge. A significant issue which determines the success of participatory sensing is the incentive for contributors. In this paper, we propose a spatio-temporal incentive scheme with consumer demand awareness for participatory sensing. The proposed incentive scheme incentivizes the contributors according to the consumer demand considering the spatio-temporal relevance between contributors and consumers. According to the incentive scheme, contributors and consumers determine their participation levels respectively to maximize their utilities, and a resulting market equilibrium has been achieved where the optimal contribution rate and the optimal consumption rate are employed. The performance of the proposed incentive scheme is investigated through extensive simulations, which show an improved coverage and quality of consumption.

It is valuable for the public to get access to real-time noise level information. Unfortunately, it is generally difficult for ordinary people to access real-time noise level information because of limited noise information stations and increased burden of carrying professional noise level meters. Being equipped with a high-quality microphone, a smartphone can potentially serve as a handy noise level meter. However, the straightforward use of sound measurements from smartphones leads to large measurement errors. As a result, it is essential to calibrate a smartphone before it can be used for noise level measurements. Little work has been done on automatic smartphone calibration for noise measurement purposes. In this paper we design a system called iCal for calibrating smartphones for accurate noise level measurements. The system consists of two key components: node-based calibration and crowdsourcing-based calibration. The node-based calibration enables an individual smartphone to do offline calibration, but suffers a slow-start issue. Complementing the node-based calibration, the crowdsourcing-based calibration leverages the power of crowdsourcing to maintain a lookup table, which a smartphone user can consult to find an approximate offset specific to its smartphone model. Thus, the slow-start issue can be effectively mitigated. The salient feature of iCal is human intervention free. We have implemented iCal on the android platform and experimental results show that the calibration error is as low as 3 dbA.

Participatory sensing facilitates smartphone users to contribute data to form a body of knowledge. A significant issue which determines the success of participatory sensing is the incentive for contributors. In this paper, we propose an information-driven incentive scheme with consumer demand awareness for participatory sensing. An information utility metric is introduced to measure the quality of sensor information from contributors according to the consumer demand. The contributors with the most information utility are selected and incentivized, which allows us to maximize the satisfaction of the consumers as well as the profits of the service provider. Based on the incentive scheme, contributors and consumers determine their participation levels respectively to maximize their utilities. The performance of the proposed incentive scheme is investigated through extensive simulations, which show an improved information utility and sensing coverage.

User-contributed or crowd-sourced information is becoming increasingly common. In this paper, we consider the specific case of participatory sensing whereby people contribute information captured by sensors, typically those on a smartphone, and share the information with others. We propose a new metric called quality of contributed service (QCS) which characterizes the information quality and timeliness of a specific real-time sensed quantity achieved in a participatory manner. Participatory sensing has the problem that contributions are sporadic and infrequent. To overcome this, we formulate a market-based framework for participatory sensing with plausible models of the market participants comprising data contributors, service consumers and a service provider. We analyze the market equilibrium and obtain a closed form expression for the resulting QCS at market equilibrium. Next, we examine the effects of realistic behaviors of the market participants and the nature of the market equilibrium that emerges through extensive simulations. Our results show that, starting from purely random behavior, the market and its participants can converge to the market equilibrium with good QCS within a short period of time.

Mobile devices like smartphones have become the computing device of choice for many users, heralding the era of mobile computing. Many applications have been developed to run on mobile devices. However, despite the increased processing and wireless network speeds of mobile devices, their resources are still limited in terms of processing capacity and battery lifetime. Some applications, in particular computationally intensive ones such as multimedia processing, often require more resources than a mobile device can afford. To overcome this hurdle, we propose a mobile ad-hoc cloud in which a mobile device can access resources from other sources, such as nearby mobile devices, to share the workload. The difficulty that arises with this concept is the mobility of nearby devices, i.e. A neighbouring device may move out of range before it can communicate its results back to the source node. In this paper, we propose a workload distribution scheme among these nearby mobile devices that takes into account the randomness of the connection time between cooperating devices. In order to cope with this randomness, we adopt a multi-stage stochastic programming approach which is able to take posterior recourse actions to compensate for inaccurate predictions. Numerical studies and simulations were carried out to evaluate the performance of this scheme. The results show that the stochastic programming approach outperforms a naive scheme and a baseline scheme that only considers the average connection time.

The significant development of mobile cloud computing allows a mobile user to access resources of the nearby mobile devices, i.e., Cloudlets, for processing tasks by using the offloading mechanism. However, due to the mobility of the user and cloudlets, the connection between the user's device and cloudlets may be interrupted since cloudlets move out of transmission range of the user's device. Consequently, the task transmission may fail, forcing the user to re-offload the task to another cloudlet or process on the local device. In this paper, we propose a dynamic opportunistic offloading algorithm which allows the user to make the decision of offloading or deferring the processing of each task in a set of parallel tasks. We formulate and solve a Markov Decision Process (MDP) model for the mobile user to obtain an optimal offloading policy while minimizing the offloading and processing cost. We extend the MDP model to a constrained MDP to solve the offloading problem when the user has a processing deadline. Numerical studies and simulations were carried out to evaluate the performance of the proposed model. The results show that the proposed model outperforms conventional baseline schemes.

We study the resource scheduling for Wireless Sensor Network (WSN) based on Single Carrier Frequency Division Multiple Access (SC-FDMA) technique. This work distinguishes itself from the existing work in: 1) the scheduling for a relay network, 2) the scheduling for multimedia-enabled sensor nodes taking into account diverse QoS characteristics such as delay sensitive (DS), rate sensitive (RS), best effort (BE) and different BER requirements, and 3) more comprehensive resource scheduling considering frame segmentation, power, modulation, resource unit allocation, rate, delay, BER, etc. We propose to maximize the effective transmission rate of a relay network by jointly considering frame segmentation, resource block allocation, power distribution and modulation selection. We cater to the heterogeneous QoS requirements for a widespread kind of wireless sensor nodes. We prove the NP-hardness of the problem and propose an adaptive resource scheduling algorithm (ARSA) with superior performance. We demonstrate the superiority of the proposed ARSA scheme in terms of throughput, packet drop rate, QoS satisfaction and the computational complexity by comparing it with several state-of-the-art resource allocation mechanisms.

The emergence of mobile cloud computing enables mobile users to dynamically offload applications to nearby mobile resource-rich devices (i.e., cloudlets) to reduce energy consumption and improve execution efficiency. However, due to mobility, the connections between a mobile user and mobile cloudlets can be intermittent. As a result, offloading actions taken by a mobile user may fail (e.g., the user moves out of transmission range of cloudlets). In this paper, we model and develop an optimal offloading algorithm for the mobile user, considering the users' local load and availability of cloudlets. We formulate and solve a Markov decision process (MDP) model to obtain an optimal policy for the mobile user with an objective to minimize the computation and offloading cost. The numerical results show that the proposed dynamic offloading algorithm outperforms conventional baseline schemes.

In environmental monitoring, such as air quality monitoring, accurate and continuous data collection through a wireless sensor network is of significance. In this paper, we describe a deployed and participatory sensing system which monitors the environmental parameters through deployed sensor nodes or crowd-owned sensors, e.g., smartphones with affordable additional sensors. The noisy sensory information is processed using a Kalman filter and an information quality metric is computed and used in an incentive scheme. In this demonstration, we show that the proposed system effectively monitors the environmental parameters with improved accuracy and coverage.

Having received significant attention in the industry, the cloud market is nowadays fiercely competitive with many cloud providers. On one hand, cloud providers compete against each other for both existing and new cloud users. To keep existing users and attract newcomers, it is crucial for each provider to offer an optimal price policy which maximizes the final revenue and improves the competitive advantage. The competition among providers leads to the evolution of the market and dynamic resource prices over time. On the other hand, cloud providers may cooperate with each other to improve their final revenue. Based on a service level agreement, a provider can outsource its users' resource requests to its partner to reduce the operation cost and thereby improve the final revenue. This leads to the problem of determining the cooperating parties in a cooperative environment. This paper tackles these two issues of the current cloud market. First, we solve the problem of competition among providers and propose a dynamic price policy. We employ a discrete choice model to describe the user's choice behavior based on his obtained benefit value. The choice model is used to derive the probability of a user choosing to be served by a certain provider. The competition among providers is formulated as a noncooperative stochastic game where the players are providers who act by proposing the price policy simultaneously. The game is modelled as a Markov Decision Process whose solution is a Markov Perfect Equilibrium. Then, we address the cooperation among providers by presenting a novel algorithm for determining a cooperation strategy that tells providers whether to satisfy users' resource requests locally or outsource them to a certain provider. The algorithm yields the optimal cooperation structure from which no provider unilaterally deviates to gain more revenue. Numerical simulations are carried out to evaluate the performance of the proposed models.

Event detection is an important issue in sensor networks for a variety of real-world applications. Many events in real world are often correlated on a complex spatio-temporal level whereby they are manifested via observations over time and space proximities. In order to predict events in these spatiotemporal observations, the prediction model should be capable of modeling codependencies between data observed at various locations. In this paper, we propose a Dynamic Bayesian Network (DBN) with such spatio-temporal event prediction capability in sensor networks deployed for sensing environmental data. More specifically, we develop a DBN model with mixture distribution and a novel learning algorithm, for water level data prediction for different canals, using rainfall data at multiple locations. Experiments on real data demonstrates that our model and training method can provide accurate event prediction in real time for spatio-temporal sensor networks.

In this paper, we address the issue of human motion tracking in a smart space using a wireless sensor network with a small number of ultrasonic sensors. Ultrasonic sensing is preferable in situations where video monitoring is prohibited due to privacy concerns or is ruled out due to its higher cost and energy consumption. Unlike other common tracking techniques, the schemes proposed in this paper do not require the tracked person to wear a tag. In order to conserve energy, a single ultrasonic sensor that provides maximum information gain is selected. We use the Extended Kalman Filter (EKF) which provides robust state estimates from noisy signals as well as an uncertainty measure in the form of the state covariance, and propose the use of a process model which copes better with missed detections compared to the commonly used constant velocity process model. We propose two sensor selection schemes: (i) Current Node Sensor Selection (CNSS), and (ii) Distributed Neighbourhood node Sensor Election (DNSE), and evaluate their performance in terms of tracking accuracy, target detection ratio and sensor network lifetime.

User-contributed or crowd-sourced information is becoming increasingly common. In this paper, we consider the specific case of participatory sensing whereby people contribute information captured by sensors, typically those on a smartphone, and share the information with others. We propose a new metric called Quality of Contributed Service (QCS) which characterizes the information quality and timeliness of a specific real-time sensed quantity achieved in a participatory manner. Participatory sensing has the problem that contributions are sporadic and infrequent. To overcome this, we formulate a market-based framework for participatory sensing with plausible models of the market participants comprising data contributors, service consumers and a service provider. We analyze the market equilibrium and obtain closed form expressions for the resulting QCS at market equilibrium. Next, we examine the effects of realistic behaviors of the market participants and the nature of the market equilibrium that emerges through extensive simulations. Our results show that, starting from purely random behavior, the market and its participants can converge to the market equilibrium with good QCS within a short period of time.

Cloud computing is emerging as a paradigm for large-scale data-intensive applications. Cloud infrastructures allow users to remotely access to computing power and data over the Internet. Beside the huge economical impact, data centers consume enormous amount of electrical energy, contributing to high operational cost and carbon footprints to the environment. An advanced resource allocation model is therefore needed to not only reduce the energy consumption of data centers but also provide incentives to users to optimize their resource utilization and decrease the amount of energy consumed for executing their application. In particular, we present in this paper a novel resource allocation model using combinatorial auction mechanisms and taking into account the energy parameter. Based on this model, we propose three monotone and truthful algorithms used for winners determination and payments computation, namely exhaustive search algorithm (ESA), linear relaxation based randomized algorithm (LRRA) and green greedy algorithm (GGA). We perform numerical simulations to evaluate the performance of three proposed algorithms. Our numerical simulations show that the green greedy algorithm can significantly reduce the amount of consumed energy while generating higher revenue for cloud providers.

In this paper, we study the strategies for efficiently achieving data staging and caching on a set of vantage sites in a cloud system with a minimum cost. Unlike the traditional research, we do not intend to identify the access patterns to facilitate the future requests. Instead, with such a kind of information presumably known in advance, our goal is to efficiently stage the shared data items to predetermined sites at advocated time instants to align with the patterns while minimizing the monetary costs for caching and transmitting the requested data items. To this end, we follow the cost and network models in [1] and extend the analysis to multiple data items, each with single or multiple copies. Our results show that under homogeneous cost model, when the ratio of transmission cost and caching cost is low, a single copy of each data item can efficiently serve all the user requests. While in multicopy situation, we also consider the tradeoff between the transmission cost and caching cost by controlling the upper bounds of transmissions and copies. The upper bound can be given either on per-item basis or on all-item basis. We present efficient optimal solutions based on dynamic programming techniques to all these cases provided that the upper bound is polynomially bounded by the number of service requests and the number of distinct data items. In addition to the homogeneous cost model, we also briefly discuss this problem under a heterogeneous cost model with some simple yet practical restrictions and present a 2-approximation algorithm to the general case. We validate our findings by implementing a data staging solver, whereby conducting extensive simulation studies on the behaviors of the algorithms.

Distributed and renewable-energy resources are likely to play an important role in the future energy landscape as consumers and enterprise energy users reduce their reliance on the main electricity grid as their source of electricity. Environmental or ambient sensing of parameters such as temperature and humidity, and amount of sunlight and wind, can be used to predict electricity demand from users and supply from renewable sources, respectively. In this paper, we describe a Smart Grid Cyber-Physical System (SG-CPS) comprising sensors that transmit real-time streams of sensed information to predictors of demand and supply of electricity and an optimization-based decision maker that uses these predictions together with real-time grid electricity prices and historical information to determine the quantity and timing of grid electricity purchases throughout the day and night. We investigate two forms of the optimization-based decision maker, one that uses linear programming and another that uses multi-stage stochastic programming. Our results show that sensing-driven predictions combined with the optimization-based purchasing decision maker hosted on the SG-CPS platform can cope well with uncertainties in demand, supply, and electricity prices and make grid electricity purchasing decisions that successfully keep both the occurrence of electricity shortfalls and the cost of grid electricity purchases low. We then examine the computational and memory requirements of the aforementioned prediction and optimization algorithms and find that they are within the capabilities of modern embedded system microprocessors and, hence, are amenable for deployment in typical households and communities.

Making sense of big data and big metadata remains a challenge as more and more data are churned out every day. The problem of adding value to unstructured data requires the application of computationally intensive algorithms to discover useful patterns in the data. In terms of data streams from public transport such as buses, we address the problem of performing time-consuming algorithms to model the data while still being able to process abnormal events in real-time. We propose using Hidden Markov Models (HMMs) for identifying conditions for an abnormal event in bus journeys and methods for isolating HMM computations from real-time event processing. Results show that training HMMs with even noisy metadata can generate models that can recognize an abnormal event in a parallel and distributed manner in the cloud.

Participatory sensing has emerged recently as a promising approach to large-scale data collection. However, without incentives for users to regularly contribute good quality data, this method is unlikely to be viable in the long run. In this paper, we link incentive to users' demand for consuming compelling services, as an approach complementary to conventional credit or reputation based approaches. With this demand-based principle, we design two incentive schemes, Incentive with Demand Fairness (IDF) and Iterative Tank Filling (ITF), for maximizing fairness and social welfare, respectively. Our study shows that the IDF scheme is max-min fair and can score close to 1 on the Jain's fairness index, while the ITF scheme maximizes social welfare and achieves a unique Nash equilibrium which is also Pareto and globally optimal. We adopted a game theoretic approach to derive the optimal service demands. Furthermore, to address practical considerations, we use a stochastic programming technique to handle uncertainty that is often encountered in real life situations.

Implementing incremental relaying in a practical system is not a trivial task because it requires the destination node to provide feedback on success or failure of a transmission. In practice, the feedback may be affected by propagation impairments and collisions. In this case, the relay and the source should coordinate among themselves in the absence of perfect feedback information such that packet collision will not occur. In view of the challenge, this paper proposes a novel and practical incremental relaying scheme that deals with the imperfect feedback. The key idea is to have pessimistic source and optimistic relay. The proposed scheme is simple and easily implementable. Simulation results show that the proposed scheme can outperform amplify-and-forward (AF), as well as approaching the performance of an idealized incremental AF with perfect feedback information.

Cloud computing allows the users to efficiently and dynamically provision computing resources to meet their IT needs. Most cloud providers offer two types of payment plans to the user, i.e., reservation and on-demand. The reservation plan is typically cheaper than the on-demand plan but reservation plan has to be provisioned in advance. Reserving the resources would be straightforward if the actual computing demand (e.g., job processing) is known in advance. However, in reality, the actual computing demand can be observed only at the point of actual usage. Therefore, it is difficult to reserve the correct amount of resources during the reservation to meet the computing demands of the users. In this paper, we propose an evolutionary optimal virtual machine placement (EOVMP) algorithm with a demand forecaster. First, a demand forecaster predicts the computing demand. Then, EOVMP uses this predicted demand to allocate the virtual machines using reservation and on-demand plans for job processing. The performance of the proposed schemes is evaluated by simulations and numerical studies. The evaluation result shows that the EOVMP algorithm can provide the solution close to the optimal solution of stochastic integer programming (SIP) and the prediction of the demand forecaster is of reasonable accuracy.

Cooperation can greatly improve the performance of wireless networks, but not all the time. Due to the dynamics of the wireless network and incomplete information, in many cases, only some uncertain opportunities of cooperation can be found. When these situations arise, opportunistic cooperation techniques, such as algorithms for opportunistic routing, opportunistic spectrum sharing etc. have been designed and studied. Although these techniques have been designed for different problems, the fundamental strategies are similar. One of these strategies, studied in this paper, is the use of a priority list to facilitate cooperation. By studying the benefits and costs incurred in the cooperation process, we propose a general framework to calculate and predict the long term benefits and costs of using a priority list for cooperation. We then unify both benefits and costs into a single metric called cost-aware utility (CAU). In this paper, an algorithm with polynomial time complexity is derived to find the priority list which can give the optimal CAU. We analyze and compare the performance of several priority list-based opportunistic methods in the literature with the CAU-based optimal priority list, for an opportunistic forwarding scenario in an IEEE 802.11g network. Our results show that the proposed CAU-based algorithm discovers an optimal priority list that leads to significantly better network performance than other algorithms such as legacy WiFi and AnyCast.

Abstract The inherent temporal connectivity and existence of impairments in wireless channels pose challenges to network performance. Cooperative communication has been proposed as an effective technique to mitigate the imperfections of the wireless medium by exploiting channel diversity and availability of neighboring nodes that can act as relays. Although numerous cooperative communication techniques have been proposed in the literature, most of them do not consider Quality of Service (QoS) issues in a wireless sensor network. In this work, we study how cooperative communication can be applied to achieve differentiated QoS in a sensor network that uses Ultra‐Wideband (UWB) as its underlying PHY layer technology. SAUCeR is a slotted‐aloha based ultra‐wideband medium access control protocol with cooperative retransmissions that provides differentiated QoS in networks with varying traffic classes. Despite the high transmission rates provided by UWB, its impulse‐based nature renders many conventional carrier sensing MAC protocols incompatible. Consequently, SAUCeR utilizes slotted‐aloha to reduce packet collisions without the need for carrier sensing. Differentiated QoS is provided by allocating different resources (time slots) to varying traffic classes to segregate the contention between them. A QoS‐aware cooperative retransmission technique and two distributed relay selection schemes are also introduced to improve overall traffic throughput and reduce end‐to‐end delay, while preventing the starvation of any traffic class. Copyright © 2010 John Wiley & Sons, Ltd.

This paper exploits optimal position of the mobile sensor to improve the target tracking performance of wireless sensor networks and simultaneously localize both of the static sensor nodes and mobile sensor nodes when tracking the human motion. In our approach, mobile sensors collaborate with static sensors and move optimally to achieve the required detection performance. The accuracy of final tracking result is then improved as the measurements of mobile sensors have higher signal-to-noise ratios after the movement. Specifically, we can simultaneously localize the mobile sensor and static sensors position when localizing the human's position based on augmented extended Kaiman filters (EKF). In the algorithm, we develop a sensor movement optimization algorithm that achieves near-optimal system tracking performance. We also presented an sensor nodes management scheme in order to deduce the computation complexity when localizing the static sensor nodes. The effectiveness of our approach is validated by extensive simulations.

Upon the occurrence of a phenomenon of interest in a wireless sensor network, multiple sensors may be activated, leading to data implosion and redundancy. Data aggregation and/or fusion techniques exploit spatio-temporal correlation among sensory data to reduce traffic load and mitigate congestion. However, this is often at the expense of loss in Information Quality (IQ) of data that is collected at the fusion center. In this work, we address the problem of finding the least-cost routing tree that satisfies a given IQ constraint. We note that the optimal least-cost routing solution is a variation of the classical NP-hard Steiner tree problem in graphs, which incurs high overheads as it requires knowledge of the entire network topology and individual IQ contributions of each activated sensor node. We tackle these issues by proposing: (i) a topology-aware histogram-based aggregation structure that encapsulates the cost of including the IQ contribution of each activated node in a compact and efficient way; and (ii) a greedy heuristic to approximate and prune a least-cost aggregation routing path. We show that the performance of our IQ-aware routing protocol is: (i) bounded by a distance-based aggregation tree that collects data from all the activated nodes; and (ii) comparable to another IQ-aware routing protocol that uses an exhaustive brute-force search to approximate and prune the least-cost aggregation tree.

Indoor wireless localization has emerged as a key wireless network technology and has been used for a variety of applications. In this paper, we examine the possibility to use one RF-based fingerprint system for indoor wireless localization, and show that there is room for improvement in its location sensing approach. We propose an indoor wireless localization solution by improving a fingerprinting localization algorithm with Multidimensional Scaling (MDS). In our approach, we configure RFID readers to receive signal strengths from both RFID tags and reference points, and use a fingerprinting localization algorithm for initial location estimation. We preprocess the received signal strength information to obtain the pairwise distances' estimation between the RFID tags and the reference points. Having estimated the pairwise squared distances, we apply MDS to reconstruct the RFID tags' distribution, and we subsequently use Procrustes analysis to refine the previously obtained fingerprinting location estimation. Simulation results show that our proposed localization algorithm improves the localization accuracy of the fingerprinting approach under different wireless network conditions.

We are motivated to exploit the broadcast nature of a radio medium in combating its unreliable and time-varying characteristics. This leads to cooperative transmission where a neighbor node keeps a copy of an overheard packet and helps in retransmitting the overheard packet to its intended destination when the original transmission fails. Cooperative transmission improve wireless network performance through diversity gain, and can be performed in the forms of amplify-and-forward. Existing amplify-and-forward schemes either: (a) use multiple relays but in a non-incremental way, or (b) use only a single relay in an incremental way. Non-incremental (simple) schemes do not provide a good spectral efficiency. Also, using multiple relays can further improve diversity. This paper aims at proposing an incremental amplify-and-forward scheme that uses n relays, where n >; 1. In the proposed scheme, a relay will retransmit an overheard packet in a time slot, only if all the previous transmissions are not successful. The transmission cycle of a packet ends as soon as a packet has been received successfully by the intended destination or all the n relays has retransmitted their overheard packets. Analytical and numerical results show that the proposed scheme outperforms both the simple amplify-and-forward scheme and the single-relay incremental scheme in terms of a lower outage probability for a given spectral efficiency.

IEEE 802.11 distributed coordination function (DCF) medium access control (MAC) does not provide airtime fairness for all stations in a multi-rate scenario as it only provides max-min throughput fairness. This gives rise to the rate anomaly problem where the maximum throughput is limited by the slowest transmitting station. In our paper, we propose airtime fairness in a rate separation IEEE 802.11b MAC. Stations are grouped according to their transmission rates for transmitting their packets in different data transmission periods (DTPs) for the different groups of stations. The analytical framework is formulated for N stations, including an access point (AP). The state transition diagram is modeled by a two-dimensional discrete-time Markov chain. One dimension of the Markov chain is for the backoff stage and the second dimension is for the value of the backoff counter. The saturated throughput is approximated by the sum of the product of a weighted ratio of the throughput of the DTP under consideration and the throughput of the DTP minus the period necessary to transmit a packet before the end of the current DTP, the probability of the number of devices in the DTP and the number of DTPs. The DTPs to achieve airtime fairness are formulated as non-linear equations, which are solved using Newton-Raphson method with Jacobian functions. Numerical results of the saturated throughput corresponding to typical parameter values are presented. These results show the advantage of the proposed rate separation IEEE 802.11b MAC with airtime fairness in achieving airtime fairness and high saturated throughput.

Wireless link is not reliable and has a time-varying quality. When a link quality is bad, all transmissions over the link are not successful and persistent retransmissions of the failed packet over the same link are not productive. Consider the broadcast nature of radio communications, a packet that fails to reach its intended receiver may be successfully received by neighbors. The idea of cooperative retransmission suggests that a neighbor should help in retransmitting an overheard packet to its intended receiver, instead of letting the original sender to persistently retransmit the failed packet. This paper proposes a simple MAC layer cooperation retransmission scheme, called CoRex. The proposed scheme deals with the three critical issues, namely relay selection, packet selection and fairness, all together. The performance of CoRex has been analyzed theoretically and evaluated through random event simulations. The results show that CoRex always outperforms the non-cooperative scheme in terms of throughput without compromising fairness.

CSMA/CA wireless MAC protocol is adopted in many standards. However, its performance at high contention level is less than desirable due to prolonged backoff period and collisions. We propose a novel cooperative MAC based on the Interweave Cognitive framework to combat high contention. In our setup, we incorporate a subset of self-organizing advanced nodes that opportunistically switch to a contention-free MAC protocol to ease network congestion in a cognitive and cooperative manner. Consequently, the whole network benefits from a higher and more stable throughput even at high contention level. Our theoretical and simulation results demonstrate that IC-MAC is effective in reducing the number of backlogged nodes and improving the throughput of up to 20% relative to CSMA/CA MAC.

This paper considers a wireless sensor network with Time-Hopping-Impulse-Radio Ultra Wide Band as the physical layer. The network is structured as a group of interconnected cells that are not synchronized with each other. In such an unsynchronized environment, inter-cell interference is not Gaussian, but generalized Gaussian. For such a non-Gaussian random interference, its impulsiveness is determined by the Kurtosis of the distribution. This paper shows that the Kurtosis can be controlled by adjusting some practical parameters, such as pulse shape, pulse length, slot duration, number of slots per frame, number of users, etc. Further, this paper shows that network throughput can be maximized by controlling the impulsiveness. The finding has been validated through simulations.

Cooperation between wireless nodes to retransmit data for the other users introduces multiuser diversity to a wireless network and increases the system throughput. In this paper, the optimal cooperative relaying strategies in the MAC layer are analyzed while considering the UWB unique properties such as fine ranging and immunity to small scale fading. Specifically, the optimal cooperation strategies in the absence of coordination message passing between relays are determined in order to maximize the system throughput while reducing the control packet overhead. Mobile networks are also considered, in which the relays should exchange their ranging information together in some update intervals. The optimal update interval length is calculated in order to maximize the system throughput. More importantly, we show that if this optimal update interval is used, the optimal cooperation strategies in the mobile case will be similar to those in the static network. Two different relay selection schemes, namely proactive and reactive settings, are considered. Analysis and simulations confirm that the proposed UWB-based Cooperative Relaying Scheme, UCoRS, can achieve a considerable diversity gain in spite of its implementation simplicity. UCoRS also minimizes the number of control packets that are required for the optimal cooperation, which leads to the energy efficiency in the UWB costly data-receiving process.

Mobile healthcare (m-Healthcare) systems are regarded as a solution to address skyrocketing healthcare costs without reducing the quality of patient care. It is our aim to build an m-Healthcare platform based on the service-oriented computing paradigm. We are developing a Service-oriented Architecture (SOA) for m-Healthcare services platform, called SOAMOH that shall also support interfacing with the HL7 standard and facilitate the provisioning of healthcare to people anywhere, anytime using mobile devices that are connected through wireless communication technologies.

Latency is a critical concern in interactive or delay-sensitive services such as interactive IPTV and VoIP. This is especially so when using network coding as a means to conserve bandwidth in these bandwidth-hungry services. In practical network coding, packets are coded in batches and thus suffer a large average delay per packet when packets get decoded after the whole batch is received. A larger batch size, however, also gives the highest bandwidth savings. In this paper, we analyze the achievable upper and lower bounds of the average delay per packet as well as packet loss rates due to finite-sized queue in a multicast downlink transmission from the system and client perspectives. We validate our analysis with simulation results and characterize the queueing and transmission delays, and packet loss performance with respect to (i) the maximum size of a batch and (ii) packet arrival rates. We find that random linear coding is an upper bound in delay performance and other hybrid network coding method might achieve better delay gains.

Wireless community networks, where users share wireless bandwidth is attracting tremendous interest from academia and industry. Companies such as FON have been successful in attracting large communities of users. However, solutions such as FON either require users to buy specialized FON routers or firmware modifications to existing routers. In this paper we propose a solution which requires no such sophisticated hardware. An alternative is to provide a solution which requires users to download a client software on to their PCs. While the solution appears simple it raises several issues of incentivizing users to share their bandwidth and also issues of preventing users from cheating behaviors which give them an unfair advantage. In this paper, we propose a system and solution which (i) requires only software downloads on PCs, (ii) is robust to tampering of the software, and intermittent monitoring of an access point by the owner, (iii) a credit based mechanism whereby users earn credits for sharing bandwidth and punishment and pricing mechanism whereby users are charged at a higher price whenever they are caught misbehaving. By making simple but plausible assumptions about user behavior, we show via analysis and extensive simulations that the system converges to a Pareto optimal Nash equilibrium. We further validate our system model, by running trace driven simulations on real world data. We believe that the solution provided by Wi-Sh is an attractive and more credible alternative to solutions such as FON.

Network coding is a method that promises to achieve the min-cut capacity in multicasts. However, pushing towards this gain in throughput comes with two sacrifices. Delay suffers as the decoding procedure requires buffering and is performed in batches of coded packets, and unfairness prevails in terms of delay increases between receivers with worse channel conditions and those with better channel conditions. In this paper, we focus on optimizing the delay performance in reliably multicasting a data stream to a set of one-hop receivers from the receiver perspective. We analyze the system based on queueing theory using semi-Markov chains from both the system-wide and receiver perspectives. We find that the average delay per received packet at the receivers' end can be minimized by appropriate scheduling of data packets and appropriate size of the coding buffer, which depends on the rate of incoming data stream and capacities of the receivers. To circumvent unduly computational complexities, we design a heuristic scheme which can achieve significant performance gain when compared to an existing method. Our scheme readily adapts the coding size to the dynamics of the system, and schedules data packets to be coded via some strict priority measure for optimized delay performance. We show through extensive simulations that our scheme gives low average delay at high streaming rates and narrows the performance gap between receivers with bad and good channel conditions.

Applications for wireless sensor networks may be decomposed into the deployment of tasks on different sensor nodes in the network. Task allocation algorithms assign these tasks to specific sensor nodes in the network for execution. Given the resource-constrained and distributed nature of wireless sensor networks (WSNs), existing static (offline) task scheduling may not be practical. Therefore there is a need for an adaptive task allocation scheme that accounts for the characteristics of the WSN environment such as unexpected communication delay and node failure. In this paper, we focus on task allocation in WSNs which is performed with the aim of achieving a fair energy balance amongst the sensor nodes while minimizing delay using a market-based architecture. In this architecture, nodes are modeled as sellers communicating a deployment price for a task to the consumer. To address this task allocation problem, proposed price formulation is used as it continuously adapts to changes of the availabilities of resources. This scheme also accommodates for the node failure during task assignment. The centralized and distributed message exchanged mechanisms between the nodes (sellers) and task allocator (consumer) are proposed to determine the winner among the sellers with the goal of reducing overhead and energy consumption. Simulation results show that, compared with a static scheduling scheme with an objective in energy balancing, the proposed scheme adapts to new environmental changes and uncertain network condition more dynamically and achieves a much better performance on energy balancing.

It is well-known that throughput of wired multi-hop, multi-commodity networks can be maximized by employing back-pressure scheduling. With this approach, packets belonging to different destinations are dynamically routed/scheduled in a network based on buffer occupancies and link quality. There has been a considerable amount of research on applying back-pressure scheduling in the wireless environment; usually by abstracting each wireless channel as a point-to-point link and therefore, ignoring the fundamental wireless broadcast property. In this paper, we consider a two-hop wireless cooperative network and characterize the minimum throughput gain obtained by exploiting broadcast property in back-pressure scheduling. Numerical results are provided to support our analysis.

In cooperative retransmissions, nodes with better channel qualities help other nodes in retransmitting a failed packet to its intended destination. In this paper, we propose a cooperative retransmission scheme where each node makes local decision to cooperate or not to cooperate at what transmission power using a Markov decision process with reinforcement learning. With the reinforcement learning, the proposed scheme avoids solving an Markov decision process with a large number of states. Through simulations, we show that the proposed scheme is robust to collisions, is scalable with regard to the network size, and can provide significant cooperative diversity.

In cooperative wireless networks, the source node transmits the packets to its destination with the help from the cooperative nodes. However, existing routing metrics in the literature do not take into account the cooperative gain in choosing the next hop in a multi-hop network. In this paper, we propose a new routing metric that accounts for the potential cooperative gain a candidate next hop may receive from its neighbors. As such, a more efficient route selection can be done for better end-to-end performance. We call the proposed routing metric, Expected Transmission Time with Cooperation, and show that it outperforms hop count as a routing metric in terms of end-to-end transmission time (packet delay).

We propose a MAC for wireless ad hoc cognitive radio networks where secondary users employ cooperative spectrum sensing to mitigate the degradation of the channel between primary transmitter and secondary users. The sensing reports and fused decisions are transmitted based on random access of CSMA/CA and 802.11e EDCA on the control channel, whose access scheme determines the overall achievable throughput among the multi-channels. We propose several schemes and derive the upper bound of overall throughput. The saturation problem is also studied to address the optimization of the channel selection and the trade-off between cooperative sensing gain and channel reuse efficiency.

The challenging problem of cooperative retransmission in the wireless networks is investigated in this paper. This paper introduces the centralized and distributed Markov decision process (MDP) frameworks in the context of cooperative retransmission. Specifically, a MDP model with the global channel information is first constructed for the cooperation problem in the MAC layer. It is shown that this global MDP is able to perform optimally, where the objective is to minimize the total number of required transmissions for a successful packet delivery to the destination. When the global information is unavailable, we show that the suitable distributed MDP models can replace the global model for a near-optimal performance. Furthermore, the reinforcement learning methods are investigated when the MDP model is unavailable. Interestingly, simulation results confirm that the learning methods also provide an acceptable performance despite their simplicity and low overhead.

A transmitted packet that fails to reach its intended destination may be correctly received by neighbor nodes due to the broadcast nature of the wireless medium. In a cooperative retransmission scheme, these neighbor nodes, known as relays, can retransmit the failed packet on behalf of the original source node. The challenge is that multiple concurrent transmissions may lead to collision at the destination, and thus the problem is to decide which relay should help in retransmitting the failed packet so that the destination can successfully receive it. This paper proposes a decentralized partially observable Markov decision process (DEC-POMDP) model for selecting the relays to perform the cooperative retransmission. The proposed DEC- POMDP model does not require global channel state information (CSI). In addition, it is robust to noise in CSI measurements. Furthermore, the proposed DEC-POMDP scheme utilizes the gradient descent learning method to eliminate the need for a wireless channel model. We show that the proposed learning method based on the DEC-POMDP model can perform near optimally in the absence of a channel model and despite its implementation simplicity.

In wireless LANs, throughput is one of, if not the most, important performance metric. This metric becomes more critical at the bottleneck area of the network, which is normally the area around the access point (AP). In this paper, we propose SI-CCMAC, a sender initiating concurrent cooperative MAC for wireless LANs. It is designed to improve the throughput performance in the region near the AP through cooperative communication, where data is forwarded through a two-hop high data-rate link instead of a low data-rate direct link. Further-more, nodes are coordinated to enable concurrent transmissions to further increase throughput. The coordination part of SICCMAC is modeled as a vertex coloring problem, a maximum independent set problem and a MDP problem, depending on different scenarios. For all three modeling, solutions can be found, based on the existing algorithms to optimize the throughput performance while guaranteeing a max-min fairness. Through simulation, we show that SI-CCMAC can significantly shorten the transmission time for stations with low data rate links to the AP and it has better throughput performance than other MAC protocols, such as CoopMAC and legacy IEEE 802.11.

Fault-tolerant scheduling is an imperative step for large-scale computational grid systems, as often geographically distributed nodes co-operate to execute a task. By and large, primary-backup approach is a common methodology used for fault tolerance wherein each task has a primary copy and a backup copy on two different processors. In this paper, we identify two cases that may happen when scheduling dependent tasks with primary-backup approach. We derive two important constraints that must be satisfied. Further, we show that these two constraints play a crucial role in limiting the schedulability and overloading efficiency of backups of dependent tasks. We then propose two strategies to improve schedulability and overloading efficiency, respectively. We propose two algorithms (MRC-ECT and MCT-LRC), to schedule backups of independent jobs and dependent jobs, respectively. MRC-ECT is shown to guarantee an optimal backup schedule in terms of replication cost for an independent task, while MCT-LRC can schedule a backup of a dependent task with minimum completion time and less replication cost. We conduct extensive simulation experiments to quantify the performance of the proposed algorithms.

Coordination is one of the key components in distributed multi-agent systems. Establishing a coordination scheme with minimum communication requirements and robustness to communication failure is a difficult task. A new multi-agent architecture based on type-2 fuzzy decision making is proposed here for achieving coordination with minimum communication. The decision making module has been designed to achieve the coordination between agents by calculating the weight of the input to be used for deciding on the action plans in a dynamic manner. The effectiveness of the coordination scheme proposed was tested by applying it to a complex, non-linear and stochastic application of the traffic signal control. The size of the network chosen also serves to show the scalability of the agent architecture. The results obtained were compared with adaptive systems, fixed coordination schemes and no coordination schemes. Considerable improvement in the time delay was achieved while using the dynamic coordination scheme proposed.

This paper describes HOF, a hybrid opportunistic forwarding protocol for multi-hop wireless mesh networks. It combines the concept of traditional "best" path routing, with the concept of opportunistic routing, which explores and determines the next hop relay node at each hops. Some common problems of existing opportunistic routing protocols are studied in this paper. HOF can largely solve such problems by using the most preferred node (MPN) and weighted-Sift techniques, proposed in this paper. The HOF scheme is efficient, robust against channel or node failures and incurs little network overhead. The performance of HOF is compared with OSPF and GeRaF. Results show that, HOF performs almost as well as OSPF and better than GeRaF under a perfect environment. When more node and channel failures happen, it outperforms both of them.

Ultra-wideband (UWB) is a physical (PHY) layer technology that promises high transmission rates, as well as high resistance to noise and multipath effects. However, the impulse-based nature of UWB, coupled with its low transmission power, makes it difficult to enable efficient detection of the signals. Consequently, conventional carrier-sensing based MAC protocols cannot be used with a UWB PHY. In this paper, we propose SASW-CR - a Slotted Aloha MAC protocol for UWB networks with Sliding contention Window and Coperative Retransmissions, which provides QoS support without the use of carrier sensing. SASW-CR utilizes the slotted-Aloha technique to avoid carrier sensing and reduce packet collisions. In addition, it makes use of differentiated contention windows to provide varying classes of QoS for different traffic classes. A cooperative retransmission technique is also introduced to improve the overall traffic throughput and reduce end-to-end delay. The efficacy of our protocol is demonstrated through simulations.

In wireless LANs, one of the main concerns is throughput performance. When there is only one Access Point (AP) in a wireless LAN, the bottleneck is normally at the region near the AP. In this paper, we propose CCMAC, a coordinated cooperative MAC for wireless LANs. It is designed to improve the throughput performance in the region near the AP through cooperative communication, where data is forwarded through a two-hop high data-rate link instead of a low data-rate direct link. Furthermore, it can coordinate nodes to perform concurrent transmissions in order to further increase throughput. This coordination is done by modeling the problem as a POMDP (Partially Observable Markov Decision Process) and using a Reinforcement Learning (RL) algorithm to solve it. Through analysis and simulation, we show that CCMAC can significantly shorten the transmission time for stations with low data rate links to the AP and it has better throughput performance than other MAC protocols, such as CoopMAC and legacy IEEE 802.11.

The inherent near-far effect in wireless networks causes nodes that are further away from the receiver to suffer from throughput degradation, as packets from nodes that are nearer are typically received with greater signal strengths. This unfair situation is traditionally overcome by power control. However, when power control is not feasible, for example in tiny sensor nodes with power-limited batteries, alternative solutions have to be utilized to achieve fairness in the network. In this paper, we propose U-LiBRA - an UWB Location Based Resource Allocation scheme to alleviate the contention between near and far nodes in a TH-UWB sensor network. U-LiBRA allocates different time slots to nodes that are at varying distances from the receiver, so that nodes that are further away from the receiver can achieve higher throughput than what they would typically obtain under the influence of the near-far effect. Simulation results show that U-LiBRA can effectively mitigate the near-far effect and improve fairness in the absence of power control.

Most of the target tracking algorithms proposed for wireless sensor networks (WSNs) so far have been relying on sensors of single modality. To integrate multiple sensing modalities (e.g., by using the proximity sensors and ranging sensors together) to improve the tracking performance, the tracking algorithm shall be capable to deal with non-linear and non-Gaussian nature of the tracking problem. In this paper, we present a particle filter algorithm for multi-modality target tracking in WSNs. Simulation results show that the proposed algorithm can provide a good balance among sensor costs and tracking accuracy.

For a given multi-hop route in an IEEE 802.16 mesh network, we are interested in finding its end-to-end capacity so that admission control can be performed. The end-to-end capacity is difficult to determine due to the interference between communicating nodes caused by the broadcast nature of radio propagation. In this paper, we first propose a method to determine link capacity between two nodes, after which a zone-based method is used to obtain the end-to-end capacity of a route. We demonstrate the effectiveness of the link capacity and end-to-end capacity computing methods through simulations.

In this work, we study the time-slot allocation problem in a multi-sink single-hop TH-UWB network scenario, where the traffic from a sensor node is anycasted via a single hop to any one of multiple sinks. The slot allocation problem is formulated as an optimization problem and shown to be NP-hard. We then present a heuristic to increase network throughput and fairness as compared to a random allocation. In the proposed heuristic, nodes that are of similar distances to any sinks are grouped together to utilize the same set of TH slots for transmissions. Simulations show that the proposed heuristic improves both throughput and fairness, scales with multiple sinks and can be used as a simple admission control mechanism.

In this paper, we investigate the coverage problem in wireless sensor networks using a game theory method. We assume that nodes are randomly scattered in a sensor field and the goal is to partition these nodes into K sets. At any given time, nodes belonging to only one of these sets actively sense the field. A key challenge is to achieve this partition in a distributed manner with purely local information and yet provide near optimal coverage. We appropriately formulate this coverage problem as a coverage game and prove that the optimal solution is a pure Nash equilibrium. Then, we design synchronous and asynchronous algorithms, which converge to pure Nash equilibria. Moreover, we analyze the optimality and complexity of pure Nash equilibria in the coverage game. We prove that, the ratio between the optimal coverage and the worst case Nash equilibrium coverage, is upper bounded by 2 - 1/m+1 (m is the maximum number of nodes, which cover any point, in the Nash equilibrium solution s*). We prove that finding pure Nash equilibria in the general coverage game is PLS-complete, i.e. "as hard as that of finding a local optimum in any local search problem with efficient computable neighbors". Finally, via extensive simulations, we show that, the Nash equilibria coverage performance is very close to the optimal coverage and the convergence speed is sublinear. Even under the noisy environment, our algorithms can still converge to the pure Nash equilibria.

High-rate streaming in WSN is required for future applications to provide high-quality information of battlefield hot spots. Although recent advances have enabled large-scale WSN to be deployed supported by high-bandwidth backbone network for high-rate streaming, the WSN remains the bottleneck due to the low-rate radios used and the effects of wireless interferences. First, we propose a technique to evaluate the quality of a pathset for multipath load balancing, taking into consideration the effects of wireless interferences and that nodes may interfere beyond communication ranges. Second, we propose an interference- minimized multipath routing (I2MR) protocol that increases throughput by discovering zone-disjoint paths for load balancing, requiring minimal localization support. Third, we propose a congestion control scheme that further increases throughput by loading the paths for load balancing at the highest possible rate supportable. Finally, we validate thepath-set evaluation technique and also evaluate the I2MR protocol and congestion control scheme by comparing with AODV protocol and node-disjoint multipath routing (NDMR) protocol. Simulation results show that I2MR with congestion control achieves on average 230% and 150% gains in throughput over AODV and NDMR respectively, and consumes comparable or at most 24% more energy than AODV but up to 60% less energy than NDMR.

Road Traffic congestion can occur anywhere from normal city roads, freeways to even highways. Traffic congestion can also be accentuated by incidents like terrorist attacks, accidents and breakdowns. This paper summarizes the use of various evolutionary techniques for traffic management and congestion avoidance in Intelligent Transportation Systems. Evolutionary algorithms with their inherent strength as optimization techniques are good candidates for solutions to road traffic management and congestion avoidance problems. A number of approaches involving the use of Genetic algorithms, Learning Classifier Systems and Genetic programming have been discussed for solutions to different problems in this domain. This paper proposes a multi-agent based real-time centralized evolutionary optimization technique for urban traffic management in the area of traffic signal control. This scheme uses evolutionary strategy for the control of traffic signal. The total vehicle mean delay in a six junction network was reduced by using evolutionary strategy. In order to achieve this the green signal time was optimized in an online manner. Comparison with a fixed time based traffic controller has been made and was found to produce better results.

Real-world applications are bound to have certain level of uncertainty inherent in them. Among this noise is one of the most predominant factors affecting the optimization process whether it is conventional or evolutionary techniques. The evolutionary optimization techniques are found to be inherently stronger and robust to noisy environments but they are robust for lower noise levels, higher noise requires corrections to be made to the algorithm. This paper attempts to provide a comprehensive overview of the different correction methods used for optimizing noisy objective functions or fitness functions that creates uncertain environment and also provide with an brief overview of the other issues involved while using evolutionary computational methods for optimizing applications in uncertain environment.

This paper presents the application of two co- evolutionary algorithms for evolving buyers' bidding strategies in a restructured pool-type electrical power market. A "greedy" algorithm which always aims to get higher power and pay less Ideational marginal price, as well as a "demand- driven" algorithm which aims to follow closely the individual demand, have been analyzed and implemented in simulations under different market scenarios. The two distinctive algorithms were compared against each other in a simulated power market of a reasonably large scale with 7 buyers and 20 sellers in an IEEE 14 bus network. The PowerWorldreg simulator has been used as a tool to ensure that the system validity and various constraints have been met. The simulation results suggest that a "demand-driven" co-evolutionary algorithm is more effective as it does not only help buyers to save cost when supply in the market is sufficient, but also enables them to outbid their opponents easily during tougher situations, such as when supply is in great shortage.

Fault-tolerant scheduling is an imperative step for large-scale computational Grid systems, as often geographically distributed nodes co-operate to execute a task. By and large, the primary-backup approach is a common methodology used for fault tolerance where in each task has a primary copy and a backup copy on two different processors. Backup overloading has been proposed to reduce replication cost by allowing the backup copy to overload with other backup copies on the same processor. In this paper, we consider two classes of independent tasks where in both the classes have fault-tolerance requirements. Furthermore, Class 1 tasks require the response time to be as short as possible when a fault occurs, while Class 2 tasks prefer backups with minimum replication cost. We propose two algorithms, called the MRC-ECT algorithm and the MCT-LRC algorithm. Algorithm MRC-ECT is shown to guarantee an optimal backup schedule in terms of replication cost, while MCT-LRCcan schedule a backup with minimum completion time and low replication cost. We conduct extensive simulation experiments to quantify the performance of the proposed algorithms.

Quality of Service (QoS) support for wireless sensor networks (WSN) is a fairly new topic that is gaining more and more interest. This paper introduces a method for configuring the nodes of a WSN such that application-level QoS constraints are met. This is a complex task, since the search space is typically extremely large. The method is based on a recent algebraic approach to Pareto analysis, that we use to reason about QoS trade-offs. It features an algorithm that keeps the working set of possible configurations small, by analysing parts of the network in a hierarchical fashion, and meanwhile discarding configurations that are inferior to other configurations. Furthermore, we give WSN models for two different applications, in which QoS trade-offs are made explicit. Test results show that the models are accurate and that the method is scalable and thus practically usable for WSN, even with large numbers of nodes.

Power control for forward link soft handoff users in cellular CDMA systems is important in order to obtain good forward link performance. The two existing schemes in the 3GPP specification, balancing power control (BPC) and site selection diversity transmission (SSDT), both have shortcomings due to their static properties. In this paper, we propose a dynamic power control scheme which can modify the power control policy according to changing environment situations. We apply a distributed reinforcement learning (DRL)-based coordinated decision-making method to achieve dynamic power control. The main idea is to learn the optimal transmission power levels from BSs under different environmental situations with multiple mobile users. Our simulation results show that the proposed scheme effectively combines the advantages of existing schemes in order to maximize the forward link capacity.

Calculations of service availability of a high-availability (HA) cluster are usually based on the assumption of load- independent machine availabilities. In this paper, we study the issues and show how the service availabilities can be calculated under the assumption that machine availabilities are load dependent. We present a Markov chain analysis to derive the steady-state service availabilities of a load-dependent machine availability HA cluster. We show that with a load-dependent machine availability, the attained service availability is now policy dependent. After formulating the problem as a Markov decision process, we proceed to determine the optimal policy to achieve the maximum service availabilities by using the method of policy iteration. Two greedy assignment algorithms are studied: least load and first derivative length (FDL) based, where least load corresponds to some load balancing algorithms. We carry out the analysis and simulations on two cases of load profiles: In the first profile, a single machine has the capacity to host all services in the HA cluster; in the second profile, a single machine does not have enough capacity to host all services. We show that the service availabilities achieved under the first load profile are the same, whereas the service availabilities achieved under the second load profile are different. Since the service availabilities achieved are different in the second load profile, we proceed to investigate how the distribution of service availabilities across the services can be controlled by adjusting the rewards vector.

10.1109/INFOCOM.2006.118

Over the last few years, grid technologies have progressed towards a service-oriented paradigm that enables a new way of service provisioning based on utility computing models. Users consume these services based on their QoS (quality of service) requirements. In such "pay-per-use" grids, workflow execution cost must be considered during scheduling based on users' QoS constraints. In this paper, we propose a cost-based workflow scheduling algorithm that minimizes execution cost while meeting the deadline for delivering results. It can also adapt to the delays of service executions by rescheduling unexecuted tasks. We also attempt to optimally solve the task scheduling problem in branches with several sequential tasks by modeling the branch as a Markov decision process and using the value iteration method.

Classical tracking methods are not concerned with energy efficiency and require precise localisation. We addressed these in our previous work through HMTT (hierarchical Markov decision process for target tracking) that tracks single targets at location granularity. HMTT conserves energy by reducing the rate of sensing but preserves acceptable tracking accuracy through trajectory prediction. In this paper, HMTT is extended for the multiple targets case where the state of clusters could be affected by multiple incoming targets and where multiple updates are required at the lower level. The theoretical performance of HMTT in the multiple targets case is derived and simulations demonstrate its effectiveness against 2 other predictive tracking algorithms with up to 200% improvement.

The paper presents a joint bandwidth allocation and buffer management scheme for QoS provisioning in a differentiated services framework on a wireless ad hoc network. Our proposed scheme models the system as a semi-Markov decision process (SMDP) and uses a novel model-free average reward reinforcement learning (RL) algorithm that considers maximizing the average long term reward for our network and, at the same time, minimizing QoS violations with respect to bandwidth, queueing delay and buffer loss. Due to the nature of the provisioning problem of having continuous and multi-dimensional state and action spaces, we also present a novel function approximation technique (wire-fitted CMAC) that uses a linear tile-coding structure together with a wire-fitted interpolation. Using a linear approximator facilitates the convergence of our proposed RL algorithm. The wire-fitted CMAC also generalizes both the state and action values. Simulation results show the effectiveness and convergence of the proposed provisioning scheme with respect to the average long term reward.

The complexity involved in implementing multi-objective routing in computer networks has led to many researchers exploring alternate solutions with the use of heuristic based techniques. The rationale underlying the use of heuristic based priorities in achieving multiple objectives appears to be ad hoc and unclear due to the complex interactions among the various objectives. However these uncertainties can be effectively modeled using fuzzy set theory. This work introduces the notion of multi-objective route selection in mobile ad hoc networks (MANET) using a evolutionary fuzzy cost function to deliberately calculate cost adaptively. The fuzzy cost function is a continuous function of the metrics describing the state of a route. Simulation results demonstrate the superiority of the proposed technique over conventional MANET routing schemes.

The issue of bandwidth provisioning for Per Hop Behavior (PHB) aggregates in Differentiated Services (DiffServ) networks has received a lot of attention from researchers. However, most proposed methods need to determine the amount of bandwidth to provision at the time of connection admission. This assumes that traffic in admitted flows always conforms to predefined specifications, which would need some form of traffic shaping or admission control before reaching the ingress of the domain. This paper proposes an adaptive provisioning mechanism based on reinforcement-learning principles, which determines at regular intervals the amount of bandwidth to provision to each PHB aggregate. The mechanism adjusts to maximize the amount of revenue earned from a usage-based pricing model. The novel use of a continuous-space, gradient-based learning algorithm, enables the mechanism to require neither accurate traffic specifications nor rigid admission control. Using ns-2 simulations, we demonstrate using Weighted Fair Queuing, how our mechanism can be implemented in a DiffServ network.

A novel routing scheme for mobile ad hoc networks (MANETs), which combines the on-demand routing capability of Ad Hoc On-Demand Distance Vector (AODV) routing protocol with a distributed topology discovery mechanism using ant-like mobile agents is proposed in this paper. The proposed hybrid protocol reduces route discovery latency and the end-to-end delay by providing high connectivity without requiring much of the scarce network capacity. On the one side the proactive routing protocols in MANETs like Destination Sequenced Distance Vector (DSDV) require to know, the topology of the entire network. Hence they are not suitable for highly dynamic networks such as MANETs, since the topology update information needs to be propagated frequently throughout the network. These frequent broadcasts limit the available network capacity for actual data communication. On the other hand, on-demand, reactive routing schemes like AODV and Dynamic Source Routing (DSR), require the actual transmission of the data to be delayed until the route is discovered. Due to this long delay a pure reactive routing protocol may not be applicable for real-time data and multimedia communication. Through extensive simulations in this paper it is proved that the proposed Ant-AODV hybrid routing technique, is able to achieve reduced end-to-end delay compared to conventional ant-based and AODV routing protocols.

With the development of high-speed backbone networks, more and more traffic load is pushed to the Internet edge equipment and end hosts. Newly emerged bottleneck problems in end systems ask for quality of service (QoS) to be deployed in them. Meanwhile, the tremendous traffic brought by multimedia communications asks for end-to-end QoS. Facing these facts and challenges, a framework of deploying QoS in end systems is presented. The framework aims at both relieving bottleneck problems through utilizing limited resources efficiently and guaranteeing end-to-end QoS by integrating network QoS and end system QoS. In addition, the framework combines the functions of managing both network QoS and end system QoS.

The paper presents the development of an enterprise application using distributed object technology that integrates material requirements planning (MRP) to a job shop (simulator). The application aims to realize an integrated system that responds rapidly to changing requirements and is able to integrate heterogeneous manufacturing facilities. The application accepts the customer's order and performs material requirements planning. The MRP system then sends daily planned orders to a job shop that carries out real-time scheduling and production. At the same time, orders are sent to suppliers to purchase raw materials. After completing the production task, the job shop returns the relevant information about the finished parts. Once the customer adds or modifies orders, the MRP system will update pertinent data automatically to respond to the changes in customer's requirements rapidly. The Unified Modeling Language (UML) is applied for the analysis and design of the total application. In this integrated application, the MRP system uses Enterprise Java Beans (EJB) that are deployed in a J2EE compliant application server to perform business-to-customer transactions and MRP logic. The job shop system uses Common Object Request Broker Architecture (CORBA) specification as communication platform.

Multi-Protocol Label Switching is a technology that combines the simplicity of IP routing with the high-speed switching of ATM. Mobile IP is a protocol that allows users to move around and yet maintain continuous IP network connectivity. In this paper, we propose a scheme to integrate the Mobile IP and MPLS protocols. The integration improves the scalability of the Mobile IP data forwarding process by leveraging on the features of MPLS which are fast switching, small state maintenance and high scalability. In addition, we have removed the need for IP-in-IP tunneling from HA to FA under this scheme. This paper covers some issues regarding Mobile IP scalability and also defines the signaling and control mechanisms required to integrate MPLS and Mobile IP. Keywords---Multi-protocol Label Switching (MPLS), Label Distribution Protocol (LDP), Mobile IP (MIP) I.

This paper proposes a novel scheduling discipline for service differentiation in multi-service networks, which is referred to as the probabilistic priority (PP) discipline. The PP is based on the strict priority (SP) discipline with the difference that each priority class is assigned a parameter. The parameter determines the probability with which its corresponding queue is served when it is polled by the server. Service differentiation as well as fairness among traffic classes can be achieved in PP by setting the assigned parameters properly. In addition, PP can be easily reduced to the ordinary SP or to the reverse SP. Moreover, PP can provide service segregation among groups of traffic classes while at the same time provide service differentiation among classes within each group.

Future integrated services networks will need to provide quality of service (QoS) guarantees to multimedia applications. To ensure that the contracted QoS is sustained, it is not sufficient to just commit resources. QoS monitoring is required to detect and locate the degradation of QoS performance. In addition, the distribution of QoS, instead of simply end-to-end QoS, needs to be monitored. In QoS distribution monitoring, the distribution of QoS experienced by a real-time flow in different network segments is monitored. This paper presents a brief survey of current QoS monitoring-related mechanisms, followed by a discussion of the challenges involved in providing QoS distribution monitoring. Several approaches are then proposed to meet these challenges.

In this paper, we propose a generalised minimum queuing delay (GMQD) service discipline for high speed networks, mainly asynchronous transfer mode (ATM) networks. This proposed scheme is similar to service disciplines based on fair queuing, but instead of using only a single service rate for each session for its entire connection lifetime, multiple service rates are used. The service rate of any session at any point in time is computed efficiently based on the number of bits backlogged in the queues of the session and another imaginary reference session at that point in time. The main advantage of this scheme is that the queuing delays suffered by all the sessions connected to a single output node are minimised, leading to a smaller delay variation. In addition, this smaller delay variation also implies a smaller variance in the maximum queue length, thereby, reducing the possibility of buffer overflow.

Although neural networks have been applied for traffic and congestion control in ATM networks, most implementations use multi-layer perceptron (MLP) networks which are known to converge slowly. In this paper, we present a connection admission control (CAC) scheme which uses a modular neural network with fast learning ability to predict the cell loss ratio (CLR) at each switch in the network. A special type of OAM cell travels from the source node to the destination node and back in order to gather information at each switch. This information is used at the source to make CAC decisions such that quality of service (QoS) commitments are not violated. Experimental results which compare the performance of the proposed method with other CAC methods which use the peak cell rate (PCR), average cell rate (ACR) and equivalent bandwidth are presented.