As robots (edge-devices, agents) find uses in an increasing number of settings and edge-cloud resources become pervasive, wireless networks will often be shared by flows of data traffic that result from communication between agents and their corresponding edge-cloud nodes (cloud compute or data resource accessed by an agent). In such a setting, any agent communicating with the edge-cloud is unaware of the state of the network resource, which evolves in response to not just the agent’s own communication at any given time but also to communication by the other agents, which stays unknown to the agent. We address the challenge of an agent learning a policy that allows it to decide whether or not to communicate with its cloud node, using limited feedback it obtains from its own attempts to communicate, with the goal of optimizing its utility. The policy must generalize well to any number of other agents sharing the network and must not be trained for any particular network configuration. Our proposed policy is a deep reinforcement learning model Query Net (QNet) that we train using a proposed simulation-to-real framework. Our simulation model has just one parameter and is agnostic to specific configurations of any wireless network. It however allows training an agent’s policy over a wide range of outcomes that an agent’s communication with its edge-cloud node may face when using a shared network, by suitably randomizing the simulation parameter. We propose a learning algorithm that addresses the challenges we observe in training QNet. We validate our simulation-to-real driven approach through experiments conducted on real wireless networks including WiFi and cellular. We compare QNet with other policies to demonstrate its efficacy. Our WiFi experiments involved as few as five agents, resulting in barely any contention for the network, to as many as 50 agents, resulting in severe contention. The cellular experiments spanned a broad range of network conditions, with baseline network round-trip times ranging from a low of 0.07 s to a high of 0.83 s.

Data-driven models for state estimation using measurements obtained over a wireless network are essential to cyber-physical systems. Learning a data-driven model for an estimator using real wireless network deployments is, however, impractical as it would require data that captures varied wireless network conditions and their impact on estimation. We propose and evaluate a simulation to real-world transfer of a data-driven model for state estimation. Specifically, we train an estimator model using only data generated by a low-fidelity simulation of networks. Our choice of network simulation is a first-come-first-served single server queue, which is a network model with the two parameters of arrival rate of measurement packets into the queue and packet service rates. We employ domain randomization to bridge the gap between simulation and the real world, appropriately randomizing the network model parameters during training. The efficacy of the resulting estimator model is demonstrated by testing it over two deployments of real wireless networks. In one, the estimator model estimates vehicles’ positions and speeds using data from vehicular trajectories received by it over a shared WiFi network, with up to seventy sources sending the measurements. In the other, GPS coordinates are communicated by public transit buses over city-wide cellular networks. The estimator uses the received measurements to estimate locations of the buses.

We present the age control protocol ACP$+$, a transport layer protocol that regulates the rate at which update packets carrying information from a source are sent over the Internet to a monitor. The source would like to minimize the average age of information at the monitor. Extensive experimentation helps shed light on age control over the current Internet and its implications for sources sending updates over a shared wireless access to monitors in the cloud. Surprisingly, age minimizing rates over fast Internet paths are about 0.5 Mbps, which is a small fraction, for example, of link rates supported by WiFi wireless access technology. We also show that congestion control algorithms employed by the Transmission Control Protocol (TCP), including hybrid approaches that achieve higher throughputs at lower delays than traditional loss-based congestion control, are unsuitable for age control.

We consider selfish sources that send updates to a monitor over a shared wireless access. The sources would like to minimize the age of their information at the monitor. Our goal is to devise strategies that incentivize such sources to use the shared spectrum cooperatively. Earlier work has modeled such a setting using a non-cooperative one-shot game, played over a single access slot, and has shown that under certain access settings the dominant strategy of each source is to transmit in any slot, resulting in packet collisions between the sources' transmissions and causing all of them to be decoded in error at the monitor. We capture the interaction of the sources over an infinitely many medium access slots using infinitely repeated games. We investigate strategies that enable cooperation resulting in an efficient use of the wireless access, while disincentivizing any source from unilaterally deviating from the strategy. Formally, we are interested in strategies that are a subgame perfect Nash equilibrium (SPNE). We begin by investigating the properties of the one-stage (slot) optimal and access-fair correlated strategies. We then consider their many-slot variants, the age-fair and access-fair strategies, in the infinitely repeated game model. We prove that the access-fair and age-fair strategies are SPNEs for when collision slots are longer than successful transmission slots. Otherwise, neither is a SPNE. We end with simulations that shed light on a possible SPNE for the latter case.

Camera Selection Decisions (CSD) are highly useful for several applications in a multi-camera network. For example, CSD benefit multi-camera target tracking by reducing the number of candidate cameras to look for the target’s next location. The correct candidate cameras, decreases the number of false Re-ID queries as well as the computation time. Also, in multi-camera trajectory forecasting (MCTF) to predict where a person will re-appear in the camera network along with the transition time. These applications require a large amount of annotated data for training. In this paper, we use state-representation learning with a reinforcement learning based policy to effectively and efficiently make camera selection decisions. We further demonstrate that by using learned state representations, as opposed to hand-crafted state variables, we are able to achieve state-of-the-art results on camera selection, while reducing the training time for the RL policy. Along with this, we use a reward function that helps to reduce the amount of supervision in training the policy in a semi-supervised way. We report our results on four datasets: NLPR_MCT, DukeMTMC, CityFlow, and WNMF dataset. We show that an RL policy reduces unnecessary Re-ID queries and therefore the false alarms, scales well to larger camera networks, and is target-agnostic.

We summarize recent contributions in the broad area of age of information (AoI). In particular, we describe the current state of the art in the design and optimization of low-latency cyberphysical systems and applications in which sources send time-stamped status updates to interested recipients. These applications desire status updates at the recipients to be as timely as possible; however, this is typically constrained by limited system resources. We describe AoI timeliness metrics and present general methods of AoI evaluation analysis that are applicable to a wide variety of sources and systems. Starting from elementary single-server queues, we apply these AoI methods to a range of increasingly complex systems, including energy harvesting sensors transmitting over noisy channels, parallel server systems, queueing networks, and various single-hop and multi-hop wireless networks. We also explore how update age is related to MMSE methods of sampling, estimation and control of stochastic processes. The paper concludes with a review of efforts to employ age optimization in cyberphysical applications.

We investigate the coexistence of an age optimizing network (AON) and a throughput optimizing network (TON) that share a common spectrum band. We consider two modes of long run coexistence: (a) networks compete with each other for spectrum access, causing them to interfere and (b) networks cooperate to achieve non-interfering access. To model competition, we define a non-cooperative stage game parameterized by the average age of the AON at the beginning of the stage, derive its mixed strategy Nash equilibrium (MSNE), and analyze the evolution of age and throughput over an infinitely repeated game in which each network plays the MSNE at every stage. Cooperation uses a coordination device that performs a coin toss during each stage to select the network that must access the medium. Networks use the grim trigger punishment strategy, reverting to playing the MSNE every stage forever if the other disobeys the device. We determine if there exists a subgame perfect equilibrium, i.e., the networks obey the device forever as they find cooperation beneficial. We show that networks choose to cooperate only when they consist of a sufficiently small number of nodes, otherwise they prefer to disobey the device and compete.

Sensor sources submit updates to a monitor through an unslotted, uncoordinated, unreliable multiple access collision channel. The channel is unreliable; a collision-free transmission is received successfully at the monitor with some transmission success probability. For an infinite-user model in which the sensors collectively generate updates as a Poisson process and each update has an independent exponential transmission time, a stochastic hybrid system (SHS) approach is used to derive the average age of information (AoI) as a function of the offered load and the transmission success probability. The analysis is then extended to evaluate the individual age of a selected source. When the number of sources and update transmission rate grow large in fixed proportion, the limiting asymptotic individual age is shown to provide an accurate individual age approximation, even for a small number of sources.

We consider a network of status updating sensors whose updates are collected and sent to a monitor by a gateway. The monitor desires as fresh as possible updates from the network of sensors. The gateway may either poll a sensor for its status update or it may transmit collected sensor updates to the monitor. We derive the average age at the monitor for such a setting. We observe that increasing the frequency of transmissions to the monitor has the upside of resetting sensor age at the monitor to smaller values. However, it increases the length of time that elapses before a sensor is polled again. This motivates our investigation of policies that fix the number of sensors s the gateway polls before transmitting to the monitor.For any s, we show that when sensor transmission times to the gateway are independent and identically distributed (iid), for independent but possibly non-identical transmission times to the monitor, it is optimal to poll a sensor with the maximum age at the gateway first. Also, under simplifying assumptions, the optimal value of s increases as the square root of the number of sensors. For non-identical sensor transmission times, we consider a policy that polls a sensor such that the resulting average change in age is minimized. We compare our policy proposals with other policies, over a wide selection of transmission time distributions.

Real-time monitoring applications have Internet-of- Things (IoT) devices sense and communicate information (status updates) to a monitoring facility. Such applications desire the status updates available at the monitor to be fresh and would like to minimize the age of delivered updates. Networks of such devices may share wireless spectrum with WiFi networks. Often, they use a CSMA/CA based medium access similar to WiFi. However, unlike them, a WiFi network would like to provide high throughputs for its users.We model the coexistence of such networks as a repeated game with two players, an age optimizing network (AON) and a throughput optimizing network (TON), where an AON aims to minimize the age of updates and a TON seeks to maximize throughput. We define the stage game, parameterized by the average age of the AON at the beginning of the stage, and derive its mixed strategy Nash equilibrium (MSNE). We study the evolution of the equilibrium strategies over time, when players play the MSNE in each stage, and the resulting average discounted payoffs of the networks. It turns out that it is more favorable for a TON to share spectrum with an AON in comparison to sharing with another TON. The key to this lies in the equilibrium strategy of the AON that occasionally refrains all its nodes from transmitting during a stage. Such stages allow the TON competition free access to the medium.

Internet-of-Things (IoT) applications have sources sense and send their measurement updates over the Internet to a monitor (control station) for real-time monitoring and actuation. Ideally, these updates would be delivered fresh, at a high rate constrained only by the supported sensing rate. However, such a rate may lead to network congestion related delays in delivery of updates at the monitor that make the freshest update at the monitor unacceptably old for the application. Alternately, at low rates, while updates arrive at the monitor with smaller delays, new updates arrive infrequently. Thus, both low and high rates may lead to an undesirably aged freshest update at the monitor. We propose a novel transport layer protocol, namely the Age Control Protocol (ACP), which enables timely delivery of such updates to monitors over the Internet in a network-transparent manner. ACP adapts the rate of updates from a source such that the average age of updates at the monitor is minimized. We detail the protocol and the proposed control algorithm. We demonstrate its efficacy using extensive simulations and realworld experiments, including wireless access for the sources and an end-to-end connection with multiple hops to the monitor.

We examine multiple independent sources providing status updates to a monitor through simple queues. We formulate an age of information (AoI) timeliness metric and derive a general result for the AoI that is applicable to a wide variety of multiple source service systems. For first-come first-served and two types of last-come first-served systems with Poisson arrivals and exponential service times, we find the region of feasible average status ages for multiple updating sources. We then use these results to characterize how a service facility can be shared among multiple updating sources. A new simplified technique for evaluating the AoI in finite-state continuous-time queuing systems is also derived. Based on stochastic hybrid systems, this method makes AoI evaluation to be comparable in complexity to finding the stationary distribution of a finite-state Markov chain.

Independent sources send their status updates to a server for delivery to a monitor. We analyze an age of information timeliness metric when sources are assigned different priorities. We consider two service facilities (a) there is no waiting room and an update in service is preempted on arrival of an equal or higher priority update, and (b) there is a waiting room for at most one update and preemption is allowed in waiting but not in service. We model the age process as a stochastic hybrid system.

Edge clouds are an attractive platform to support latency-sensitive applications by providing computations on servers deployed close to end-users. These servers aim to employ MPTCP to leverage multiple connections including wireless over a public network. In this paper, we show that the default MPTCP design does not adequately support reliability in these environments, which makes it unfit for use in edge clouds. We propose RAMPTCP, an extension to MPTCP which focuses on adding reliability over network paths.

Real-time monitoring is characterized by a source repeatedly sending updates over the Internet to a monitor, which desires the sensed information at it to be as fresh (of small age) as possible, given network constraints. We propose the Age Control Protocol (ACP), which, in a network-transparent manner, enables a source to keep the age at the monitor small. We evaluate it using simulations and real-world experiments.

Applications like environmental sensing, and health and activity sensing, are supported by networks of devices (nodes) that send periodic packet transmissions over the wireless channel to a sink node. We look at simple abstractions that capture the following commonalities of such networks (a) the nodes send periodically sensed information that is temporal and must be delivered in a timely manner, (b) they share a multiple access channel and (c) channels between the nodes and the sink are unreliable (packets may be received in error) and differ in quality. We consider scheduled access and slotted ALOHA-like random access. Under scheduled access, nodes take turns and get feedback on whether a transmitted packet was received successfully by the sink. During its turn, a node may transmit more than once to counter channel uncertainty. For slotted ALOHA-like access, each node attempts transmission in every slot with a certain probability. For these access mechanisms we derive the age of information (AoI), which is a timeliness metric, and arrive at conditions that optimize AoI at the sink. We also analyze the case of symmetric updating, in which updates from different nodes must have the same AoI. We show that ALOHA-like access, while simple, leads to AoI that is worse by a factor of about 2e, in comparison to scheduled access.

White-Fi networks are CSMA/CA networks that are deployed in TV white spaces. In outdoor deployments, nodes in such networks can have a communication range of several kilometers. As a result, nodes in range have heterogeneous channel gains from the TV transmitters. Also, they have heterogeneous channel gains to TV receivers. While the nodes in the network may be configured as an ad hoc network of peers or in infrastructure (access point based) mode, they use the 802.11 distributed control function (DCF) to gain access to the medium. We investigate optimization of the DCF throughput in such a White-Fi network when there are restrictions on the aggregate interference (specifically, we consider its expectation and maximum) that it can create at TV receivers. To gain insights into the interplay between the TV networks and the White-Fi network, we allow nodes in the White-Fi network to choose their payload rates, access probabilities, overhead transmission rates, and channels. The resulting optimization problem is non-convex. Our proposed solution methodology obtains large gains (about 200% for 50 nodes) in throughput over a baseline that respects restrictions on aggregate interference but assigns the same power and probability of medium access, across available channels, to all nodes in the network.

Devices like smart phones come with 3G/4G and WiFi radios, which creates possibilities of heterogeneous network access. We investigate scenarios where Internet access to a device is available only via the cellular network. However, not every user may connect directly to it. Users in the network may be split into hotspots and clients. Hotspots are the users that connect directly to the cellular network and may provide connectivity to the internet to other users by allowing them to connect to their WiFi interface. Clients connect to the cellular network only via hotspots. The optimization problem is to find the split of hotspots and clients, and the association between clients and hotspots, that maximizes the sum of the link rates of users. Importantly, the users must get at least the link rate they get when all are directly connected to the cellular network. In this paper, we formulate the optimization problem. We provide insights into the interplay of WiFi connectivity amongst users, their link rates to the cell tower, and the split that maximizes sum rate. We propose a novel heuristic approach to split the network. Median gains of 1.5× are observed over networks of up to 40 nodes.

Smartphones can enable monitoring signs of distress as a human goes about his daily routine. Motivated by this possibility of 24x7 distress detection, we investigate detection of screaming and crying in urban environments, which we categorize into the contexts of indoors (home and office), outdoors, human conversation, large human gatherings, machinery, and audio from multimedia devices. Prior works are often restricted to specific environments or controlled settings. We propose a novel two-stage supervised learning based method, with tunable decision parameters for each stage, to achieve a desired true distress (scream and cry) detection rate (DR) and false alarm rate (FAR). We observe that the choice of the parameters is a function of the signal-to-noise ratio (SNR) of the distress signal, which is the ratio of the power of the distress signal to the power of the context audio. In the absence of SNR information, we show that a simple SNR estimation scheme performs well. Alternately, we show how the decision parameters can be selected based on the context estimated by the method. We show the results of testing the proposal over hundred hours of audio data recorded by the smartphones of ten volunteers as they went about their daily routines. Achieved performance is exemplified by a DR of 93.16% and a FAR of 4.76% at a SNR of 20 dB. The corresponding values for a SNR of 10 dB are 84.13% and 4.77%. Finally, we compare with, which also deals with audio event detection in noisy environments.

IEEE 802.11 WLAN (Wireless or WiFi LAN) clients discover neighboring APs (Access Points) by active or passive scanning. Such an active scan of WLAN injects probe frames in the network. Network conditions like packet losses, roaming, etc. result in increased active scanning and hence, an excessive increase of the probe traffic. Of the several causes inherent to WLANs like interference, we find an excessive probe traffic also has a potential of hampering goodput of a WiFi network. We confirm this behavior in a controlled home environment as well as in an uncontrolled enterprise environment. Our analysis of 36 hours of wireless traffic collected over a period of 5 months with approximately 45 million wireless frames reveals that goodput of a WLAN drops exponentially with increase in the probe traffic. Therefore, realtime detection of increase in probe traffic and knowledge of a threshold for acceptable probing is crucial for WLAN's performance. In this paper, we formulate a metric to measure the increase in probe traffic in realtime and evaluate its functioning empirically. The metric not only reflects increase in probe traffic correctly, it is even simple enough to allow its realtime measurement.

Increasingly ubiquitous communication networks and connectivity via portable devices have engendered a host of applications in which sources, for example people and environmental sensors, send updates of their status to interested recipients. These applications desire status updates at the recipients to be as timely as possible; however, this is typically constrained by limited network resources. In this paper, we employ a time-average age metric for the performance evaluation of status update systems. We derive general methods for calculating the age metric that can be applied to a broad class of service systems. We apply these methods to queue-theoretic system abstractions consisting of a source, a service facility and monitors, with the model of the service facility (physical constraints) a given. The queue discipline of first-come-first-served (FCFS) is explored. We show the existence of an optimal rate at which a source must generate its information to keep its status as timely as possible at all its monitors. This rate differs from those that maximize utilization (throughput) or minimize status packet delivery delay. While our abstractions are simpler than their real-world counterparts, the insights obtained, we believe, are a useful starting point in understanding and designing systems that support real time status updates.

We examine multiple independent sources providing status updates to a monitor through a first-come-first-served M/M/1 queue. We formulate a status-age timeliness metric and find the region of feasible average status ages for a pair of updating sources. In the presence of interfering traffic with a given offered load, we show the existence of an optimal rate at which a source should generate its updates.

This work is motivated by network applications that require nodes to disseminate their state to others. In particular, vehicular nodes will host applications that periodically disseminate time-critical state across the network to help improve on-road safety. In this work, we want to minimize the average age of state information that a node observes from any other node in networks with hundreds to thousands of nodes. We explore the benefits, vis-a-vis reducing age, of a multi-hop wireless network over a fully-connected one, for a physical network of on-road vehicles, by allowing nodes to piggyback other nodes' states. We show that for a large road network and a chosen schedule, there exists an optimal fraction of connected neighbor nodes, which, for a fixed signal-to-noise ratio between most distant nodes, is invariant to the size of the network. Via simulation we confirm that significant reductions in age are obtained via piggybacking for network sizes of interest.

Emerging applications rely on wireless broadcast to disseminate time-critical information. For example, vehicular networks may exchange vehicle position and velocity information to enable safety applications. The number of nodes in one-hop communication range in such networks can be very large, leading to congestion and undesirable levels of packet collisions. Earlier work has examined such broadcasting protocols primarily from a MAC perspective and focused on selective aspects such as packet error rate. In this work, we propose a more comprehensive metric, the average system information age, which captures the requirement of such applications to maintain current state information from all other nearby nodes. We show that information age is minimized at an optimal operating point that lies between the extremes of maximum throughput and minimum delay. Further, while age can be minimized by saturating the MAC and setting the CW size to its throughput-optimal value, the same cannot be achieved without changes in existing hardware. Also, via simulations we show that simple contention window size adaptations like increasing or decreasing the window size are unsuitable for reducing age. This motivates our design of an application-layer broadcast rate adaptation algorithm. It uses local decisions at nodes in the network to adapt their messaging rate to keep the system age to a minimum. Our simulations and experiments with 300 ORBIT nodes show that the algorithm effectively adapts the messaging rates and minimizes the system age.

Antenna diversity is a well-known technique used to improve the quality and reliability of a wireless link. In vehicular networks, a different approach to antenna diversity is needed due to their unique channel characteristics. However, this issue has not been actively researched, especially for the positioning of antennas. In this paper, we highlight the benefit of vertical diversity over traditional horizontal diversity techniques in vehicular network environments. Through experiments using IEEE 802.11a radios in the 5.2GHz band, we first show the difference of attenuation patterns from various antenna positions installed in a vehicle, then we show the benefit of vertical diversity by quantifying the diversity gains and combined error rates. This finding has implications for the future position of antenna installation in vehicles.