Policy evaluation algorithms are essential to reinforcement learning due to their ability to predict the performance of a policy. However, there are two long-standing issues lying in this prediction problem that need to be tackled: off-policy stability and on-policy efficiency. The conventional temporal difference (TD) algorithm is known to perform very well in the on-policy setting, yet is not off-policy stable. On the other hand, the gradient TD and emphatic TD algorithms are off-policy stable, but are not on-policy efficient. This paper introduces novel algorithms that are both off-policy stable and on-policy efficient by using the oblique projection method. The empirical experimental results on various domains validate the effectiveness of the proposed approach.

Abstract IEEE 802.11 distributed coordination function (DCF) is a popular protocol used for the physical and medium access control layers in most ad hoc networks. DCF employs carrier sense multiple access with collision avoidance and a binary slotted exponential back off. It has been observed that the hidden and exposed terminal problems among stations degrade DCF's performance in terms of throughput and fairness. Hence, the effectiveness of the IEEE 802.11 DCF mechanism in ad hoc networks has attracted many research studies. Many collision avoidance schemes were proposed to address the IEEE 802.11 DCF weaknesses as the performance of the medium access control layer directly impacts the performance of higher‐layer protocols and hence the entire network. An evaluation of representative schemes under the same conditions will be helpful in understanding the limitations and strengths of these schemes. This paper surveys various collision avoidance schemes, classify them on the basis of their mechanism, and then provide a comparative study of representative schemes based on their performance. They are evaluated on a chain topology, a pair topology, and a random topology with static environment to measure their throughput, fairness, collision probabilities, and delay. On the basis of the evaluation, we conclude that gentle DCF is the best scheme that has lesser collisions with improved throughput and fairness. A comparison with the legacy carrier sense multiple access with collision avoidance suggests that these proposed schemes do tend to be promising and would inspire future researchers who are interested to find solutions to the age‐old collision and fairness issues in ad hoc networks. Copyright © 2014 John Wiley & Sons, Ltd.

This paper proposes a bit rate adaptation scheme based on frame transmission statistics. The bit adaptation scheme judiciously exploits the fragmentation mechanism in the IEEE 802.11 to accurately diagnose the cause of a frame transmission failure and accordingly adjusts the frame delivery statistics. We analytically prove that the fragmentation mechanism incurs less overhead and is more effective than the use of RTS/CTS for rate adaptation in a collisions dominated environments. We implemented and evaluated this scheme and four other recent and representative rate adaptations schemes on a Linux based testbed. We also discuss some anomalies plaguing these rate adaptation schemes observed while implementing them. With extensive experiments, we observe that these rate adaptation schemes have strengths and weaknesses under different scenarios, although our scheme performs very well in most of cases.

Wireless communication is indispensable to Internet of Things (IoT). Carrier sensing multiple access/collision avoidance (CSMA/CA) is a well-proven wireless random access protocol and allows each node of equal probability in accessing wireless channel, which incurs equal throughput in long term regardless of the channel conditions. To exploit node diversity that refers to the difference of channel condition among nodes, this paper proposes two opportunistic random access mechanisms: overlapped contention and segmented contention, to favor the node of the best channel condition. In the overlapped contention, the contention windows of all nodes share the same ground of zero, but have different upper bounds upon channel condition. In the segmented contention, the contention window upper bound of a better channel condition is smaller than the lower bound of a worse channel condition; namely, their contention windows are segmented without any overlapping. These algorithms are also polished to provide temporal fairness and avoid starving the nodes of poor channel conditions. The proposed mechanisms are analyzed, implemented, and evaluated on a Linux-based testbed and in the NS3 simulator. Extensive comparative experiments show that both opportunistic solutions can significantly improve the network performance in throughput, delay, and jitter over the current CSMA/CA protocol. In particular, the overlapped contention scheme can offer 73.3% and 37.5% throughput improvements in the infrastructure-based and ad hoc networks, respectively.

Routing metrics are critical to select paths with maximal throughput in multi-rate multi-hop wireless networks. Simple path selection based on minimal hop count often leads to poor performance due to the fact that paths with low hop count often have higher packet loss rates. Better paths can be obtained by characterizing the actual quality of wireless link. A number of link quality aware routing metrics such as per hop round trip time (RTT), expected transmission count (ETX), expected transmission time (ETT) have been developed and explored. This study highlights some shortcomings of these routing metrics and proposes the design of a novel metric called improved ETT metric, or iETT, that addresses the discussed weaknesses and works more efficiently under various link quality conditions. The performance of the iETT metric is compared against the normal ETT metric over a DSR-based routing protocol. In extensive simulations, iETT metric outperforms the normal ETT metric in terms of network throughput and average packet delay.

Presents the introductory welcome message from the conference proceedings.

Rate adaptation is the determination of the optimal data transmission rate most appropriate for current wireless channel conditions. It consists of assessing channel conditions and accordingly adjusting the rate. Rate adaptation is fairly challenging due to wild channel conditions fluctuations. In the last decade, rate adaptation for IEEE 802.11 networks has been extensively investigated. This paper presents a comprehensive and detailed study of the advances of rate adaptation schemes proposed for IEEE 802.11 networks, and summarizes their characteristics. We also categorize these rate adaptation schemes based on their support of loss differentiation and their methods to sense the channel conditions. Then, this paper compares the performance of three representative schemes through simulations. Finally, open issues for rate adaptation are raised.

This paper is concerned with blind separation of convolutive mixtures of spatially independent, and temporally possible non linear processes. We consider the MIMO extraction based on the maximization of a contrast function. A new self-styled referenced contrast (RC) function is proposed, which is based on cross-trispectra between the estimated output and a reference signal. Using Parsevals formula, the former criterion yields a new class of time-domain contrast. It presents two main advantages over other more traditional contrasts. Firstly, it concerns the computational cost, and secondly the extension takes into consideration the extraction of the independent sources in the presence of Gaussian sources by making some constraints on the reference signals. There is no comparison with other methods because this is the first technique that deals with this kind of signals in the convolutif mixtures.

Routing metrics are critical to select a path that yields a high throughput in multi-rate multi-hop wireless networks. This paper illustrates the weaknesses of existing routing metrics and proposes a new routing metric called Improved Expected Transmission Time (iETT). iETT addresses two key characteristics of a routing path that other routing metrics ignored: 1) a high discrepancy in packet loss rates of links and 2) the position of the links with different packet loss rates. By capturing these two characteristics, the iETT metric is able to choose a route with a better performance than shortest path, Expected Transmission Count (ETX) and Expected Transmission Time (ETT) metrics.

With opportunistic transmissions, a node exploits a stable high data rate to transmit multiple frames (instead of one) whenever it captures a contended medium. The IEEE 802.11 networks could support such multi-frame transmissions at high data rates if the channel remains stable long enough. Opportunistic transmission has been studied and shown to improve IEEE 802.11 networks performance. However, to our knowledge all opportunistic schemes require RTS/CTS control frames while RTS/CTS control frames are usually not used. First, this work introduces a Basic Opportunistic Transmission (BOT) scheme that works without RTS/CTS control frames. However, the lack of RTS/CTS control frames may lead to high congestion loss rates that defeat any opportunistic scheme and drastically lower network performance. Channel fading exacerbates the frame loss. This work addresses this weakness by proposing an effective loss recovery strategy to enable BOT to cope with frame losses from congestion and channel fading. BOT with loss recovery constitutes the OTLR. Extensive ns-2 simulations with CBR and TCP traffic illustrate that 1) BOT yields a dramatic improvement over traditional single frame transmission, 2) BOT is vulnerable to high frame loss rates stemming from collisions or channel fading, and 3) OTLR improves BOT's performance in lossy environments.

Data rate adaptation aims to select the optimal data rates for current channel conditions leading to substantial performance improvement. This paper proposes a data rate adaptation technique that: (1) exploits the periodic IEEE 802.11 beacons; (2) discriminates between frame losses due to channel fading and those due to collisions and takes actions appropriate for each type of loss; and (3) recommends and justifies the use of the lowest data rate for the very first retransmission after a frame loss. The last feature - namely retransmitting at the lowest data rate - helps in diagnosing the real cause of a frame loss. Moreover, this work analytically shows that retransmitting at the lowest data rate is more efficient, especially in poor SNR environments or when there is no knowledge of the cause of a loss (channel degradation or transmission collision). This scheme, dubbed loss differentiated rate adaptation (LDRA), is extensively evaluated through simulations and shown to perform better especially when network traffic is heavy.

This paper presents the design, implementation and evaluation of the FreeMobility location determination system. FreeMobility is a distributed system that allows Access Points (AP) or cellular towers to dynamically determine the location of a mobile user using the received signal strength and the mobility information (the direction and traveled distance) from the mobile. FreeMobility does not require the database calibration nor does it require special infrastructure or installation, and it is readily applicable to commodity mobile devices like laptops, PDAs and cell phones. Using both actual measurements (from the University of South Carolina Beaufort and a residential neighborhood in Columbia, South Carolina) and online data resources (from the war-driving community and other research groups), this paper presents detailed analysis and experimental results that explore practical issues in implementing the FreeMobility system. The results demonstrated that FreeMobility provides reliable location estimates and the best localization error is to within 1.0 meter. With the assistance of the Geographic Information System (GIS), FreeMobility is able to provide meaningful location attributes in addition to basic coordinates from most other systems. Also with GIS, FreeMobility does not require dense tower deployment in order to pinpoint a mobile's location.

Location determination of indoor mobile users is challenging due the complex and volatile indoor radio propagation signals. A radio-frequency (RF) based indoor localization system, like RADAR or ARIADNE, typically operates by first constructing a lookup table mapping the radio signal strength at different known locations in the building, and then a mobile user's location at an arbitrary point in the building is determined by measuring the signal strength at the location in question and searching the corresponding location from the above lookup table. Usually, the mobile's signal strength is measured by three or more sniffers deployed inside the building. Obviously, the number of sniffers and their positions greatly affect the localization performance. This paper presents a detailed analysis and experimental results that explore the impact of the sniffers deployment on the performance of the indoor localization. The results demonstrate that the best localization performance is obtained when the center of gravity of the equilateral triangular (formed by three sniffers) coincides with that of the floor plan; and in order to provide optimal localization for all positions of a large floor, it is necessary to deploy more than three sniffers in a semi-mesh style such that any position in the building is always covered by three nearby sniffers.

Location determination of mobile users within a building has attracted much attention lately due to its many applications in mobile networking including network intrusion detection problems. However, it is challenging due to the complexities of the indoor radio propagation characteristics exacerbated by the mobility of the user. A common practice is to mechanically generate a table showing the radio signal strength at different known locations in the building. A mobile user's location at an arbitrary point in the building is determined by measuring the signal strength at the location in question and determining the location by referring to the above table using a LMSE (least mean square error) criterion. Obviously, this is a very tedious and time consuming task. This paper proposes a novel and automated location determination method called ARIADNE. Using a two dimensional construction floor plan and only a single actual signal strength measurement, ARIADNE generates an estimated signal strength map comparable to those generated manually by actual measurements. Given the signal measurements for a mobile, a proposed clustering algorithm searches that signal strength map to determine the current mobile's location. The results from ARIADNE are comparable and may even be superior to those from existing localization schemes.

The demand for location based applications including network intrusion detection has grown tremendously lately. However, location determination of indoor mobile users is challenging because of the complexities of the indoor radio propagation characteristics. A radio-frequency (RF) based indoor localization system, like ARIADNE, typically operates by first constructing a lookup table mapping the radio signal strength at different known locations in the building, and then a mobile user's location at an arbitrary point in the building is determined by measuring the signal strength at the location in question and searching the corresponding location from the above lookup table. Usually, the mobile user's signal strength is measured by three or more sniffers mechanically deployed inside the building. Obviously, the sniffers position configuration and the number of available sniffers greatly affect the localization performance. This paper presents experimental results that explore the impact of the sniffers deployment on the localization based on the ARIADNE system. The results demonstrate that the triangular deployment and extra available sniffers generally generate better localization performance.

The performance of IEEE 802.11 in multi-hop wireless networks depends on the characteristics of the protocol itself, and on those of the upper layer routing protocol. Extensive work has been done to analyze and evaluate the performance of single hop networks under saturated traffic conditions, either through simulations or mathematical modeling. Little work has been done on the analysis of the performance of IEEE 802.11 protocol under the unsaturated traffic conditions that arise in multi-hop networks. The paper proposes analytical models and scenarios for such an analysis. ns-2 simulations with different network configurations validate the proposed models for performance metrics such as throughput, message delay, average queue length, and energy consumption. Simulation results show that the proposed models work well. Key to the modeling of the multi-hop networks is a treatment of the upper layer routing protocols that can affect the network performance through the way they forward packets and the impact of that on the traffic load. The model proposed takes into account the impact of the upper layer routing protocol by introducing a packet acceptance factor with which each relay station accepts packets from the wireless medium before forwarding the same.

We propose reliable and precise distributed localization algorithms for wireless networks, iterative multidimensional scaling (IT-MDS) and simulated annealing multidimensional scaling (SA-MDS). We use only radio communication constraints to infer a node's distances, and uniquely combine a multidimensional scaling algorithm (MDS) with a lateration algorithm. We analytically establish the upperbound on the estimation error. The proposed techniques can estimate all nodes' positions, even with limited and imprecise network knowledge. Analysis and test runs show that the proposed methods are independent of the topology randomness and the range measurement errors. We attempt to bound the localization imprecision. Simulation results yield an average estimation error of about 25% of radio transmission range. The highest estimation error observed is 40% radio range for most networks.

Currently, a TCP sender considers all losses as congestion signals and reacts to them by throttling its sending rate. With Internet becoming more heterogeneous with more and more wireless error-prone links, a TCP connection may unduly throttle its sending rate and experience poor performance over paths experiencing random losses unrelated to congestion. The problem of distinguishing congestion losses from random losses is particularly hard when congestion is light: congestion losses themselves appear to be random. The key idea is to "de-randomize" congestion losses. This paper proposes a simple biased queue management scheme that "de-randomizes" congestion losses and enables a TCP receiver to diagnose accurately the cause of a loss and inform the TCP sender to react appropriately. Bounds on the accuracy of distinguishing wireless losses and congestion losses are analytically established and validated through simulations. Congestion losses are identified with an accuracy higher than 95% while wireless losses are identified with an accuracy higher than 75%. A closed form is derived for the achievable improvement by TCP endowed with a discriminator with a given accuracy. Simulations confirm this closed form. TCP-Casablanca, a TCP-Newreno endowed with the proposed discriminator at the receiver, yields through simulations an improvement of more than 100% on paths with low levels of congestion and about 1% random wireless packet loss rates. TCP-Ifrane, a sender-based TCP-Casablanca yields encouraging performance improvement.

TCP uses packet loss as a feedback from the network to adapt its sending rate. TCP keeps increasing its sending rate as long as no packet loss occurs (unless constrained by buffer size). Alternative congestion avoidance techniques (CATs) have been proposed to avoid such "aggressive" behavior. These CATs use simple statistics on observed round-trip times and/or throughput of a TCP connection in response to variations in congestion window size. These CATs have a supposed ability to detect queue build-up.The objective of this paper is to question the ability of these CATs to reliably detect queue build-up under real network conditions. For this purpose, the sample coefficient of correlation between round-trip time and the number of packets in flight is analyzed for 14,218 connections over 737 Internet paths. These coefficients of correlation were extracted from a set of tcpdump traces collected by Vern Paxson.The coefficients of correlation measured confirm that the correlation between RTT and window size is often weak.

The TCP is a popular transport protocol used in the present-day Internet. When packet losses occur the TCP assumes that the packet losses are due to congestion, and responds by reducing its congestion window. When a TCP connection traverses a wireless link, a significant fraction of packet losses may occur due to transmission errors. The TCP responds to such losses also by reducing the congestion window. This results in unnecessary degradation in the TCP performance. We define a class of functions named loss predictors which may be used by a TCP sender to guess the actual cause of a packet loss (congestion or transmission error) and take appropriate actions. These loss predictors use simple statistics on round-trip times and/or throughput, to determine the cause of a packet loss. We investigate their ability to determine the cause of a packet loss. Unfortunately, our simulation measurements suggest that the three loss predictors do not perform too well.

For mobile users who move frequently but receive relatively rare calls, a forwarding scheme has been shown to outperform the normal IS-41 location management scheme. But the forwarding scheme is more vulnerable to failure of intermediate Visitor Location Registers (VLRs) than the IS-41 scheme. We propose two simple variations to the forwarding scheme to address the fault tolerance weakness. One is based on the idea of maintaining two paths from the home location server to the last VLR. The second scheme is based on the knowledge of the neighbors of the faulty VLR. We evaluate and compare the performance of these location management schemes.