In wireless ad hoc networks, autonomous nodes are reluctant to forward others' packets because of the nodes' limited energy. However, such selfishness and noncooperation deteriorate both the system efficiency and nodes' performances. Moreover, the distributed nodes with only local information may not know the cooperation point, even if they are willing to cooperate. Hence, it is crucial to design a distributed mechanism for enforcing and learning the cooperation among the greedy nodes in packet forwarding. In this paper, we propose a self- learning repeated-game framework to overcome the problem and achieve the design goal. We employ self-transmission efficiency as the utility function of individual autonomous node. The self transmission efficiency is defined as the ratio of the power for self packet transmission over the total power for self packet transmission and packet forwarding. Then, we propose a framework to search for good cooperation points and maintain the cooperation among selfish nodes. The framework has two steps: First, an adaptive repeated game scheme is designed to ensure the cooperation among nodes for the current cooperative packet forwarding probabilities. Second, self-learning algorithms are employed to find the better cooperation probabilities that are feasible and benefit all nodes. We propose three learning schemes for different information structures, namely, learning with perfect observability, learning through flooding, and learning through utility prediction. Starting from noncooperation, the above two steps are employed iteratively, so that better cooperating points can be achieved and maintained in each iteration. From the simulations, the proposed framework is able to enforce cooperation among distributed selfish nodes and the proposed learning schemes achieve 70% to 98% performance efficiency compared to that of the optimal solution.

In this paper, we consider a class of energy-aware routing algorithm that explicitly takes into account the connectivity of the remaining sensor network. In typical sensor network deployments, some nodes may be more important than other nodes because the failure of these nodes causes the network disintegration, which results in early termination of information delivery. To mitigate this problem, we propose a class of routing algorithms called keep-connect algorithms, that use computable measures of network connectivity in determining how to route packets. The proposed algorithms embed the importance of the nodes in the routing cost/metric. The importance of a node is characterized by the algebraic connectivity of the remaining graph when that node fails. We prove several properties of the proposed routing algorithm including the energy consumption upper bound. Using extensive simulations, we demonstrate that the proposed algorithm achieves significant performance improvement compared to the existing routing algorithms. More importantly, we show that our proposed algorithm is more robust in terms of algebraic network connectivity compared to the existing algorithm. Finally, we present the distributed implementation of our proposed algorithm.

Cognitive radio has been proposed as a novel approach for improving the utilization of the precious limited radio resources by dynamically accessing the spectrum. One of the major design challenges is to coordinate and cooperate in accessing the spectrum opportunistically among multiple distributive users with only local information. In this paper, we propose a game theoretical approach with a new solution concept, the correlated equilibrium, which is better compared to the non-cooperative Nash equilibrium in terms of spectrum utilization efficiency and fairness among the distributive users. To achieve this correlated equilibrium, we construct an adaptive algorithm based on no-regret learning that guarantees convergence. From the simulation results, the optimal correlated equilibria achieve better fairness and 5% ~15% performance gain, compared to the Nash equilibria.

In this paper, we study the impact of cooperative routing for maximizing the network lifetime in sensor network applications. We assume nodes in the network are equipped with a single omnidirectional antenna and they perform cooperative transmission to achieve transmit diversity. We propose a joint cooperative transmission and energy aware routing algorithm to prolong the network lifetime. In contrast to the previous works, our approach uses the maximum lifetime power allocation, instead of minimum power allocation. Using the maximum lifetime power allocation, the cooperative nodes allocate their transmit power according to the channel condition and the residual energy in the nodes. Our proposed scheme combines the maximum lifetime power allocation and the energy aware routing to maximize the network lifetime. We study the performance of the cooperative routing in terms of network lifetime (defined as the time until the first node dies) and the total delivered packets before the first node dies. We demonstrate that the proposed solution achieves 1 ~ 3.5 and 1 ~ 2 times longer network lifetime and more total delivered packets compared to noncooperative routing, when it is used with MTE and FA algorithms, respectively. Furthermore, the maximum lifetime power allocation achieves 1 ~ 2 times longer lifetime, compared to minimum power allocation in MTE and FA routing schemes. We also provide distributed implementation of the proposed algorithm.

In this paper, we consider the energy-aware routing algorithm that explicitly takes into account the connectivity of the sensor networks. In typical sensor network deployment, some nodes may be more important than other nodes because the death of these nodes cause the network disintegration. The network disintegration causes early termination of information delivery. To overcome this problem, we propose a routing algorithm called keep connect algorithm that explicitly considers the connectivity of the network while making the routing decision. The algorithm can be used along with the existing routing algorithms. When doing the routing decision, the keep connect algorithm embeds the importance of the nodes in the routing cost. The importance of a node is quantified by how severe the remaining network becomes disconnected/disintegrated when that particular node dies. In particular, the importance of a node is characterized by the algebraic connectivity of the remaining graph. Compared to the existing routing algorithms, the proposed method achieves up to two times improvement in terms of the network lifetime and the number of successfully delivered packets in the special grid network. In random network, the proposed algorithm achieves 20% improvement of network lifetime, 30% less energy per packet, and 33% more successfully delivered packets.

For networks with packet forwarding, distributed control to enforce cooperation for node packet forwarding probabilities is essential to maintain the connectivity. In this paper, we propose a novel self-learning repeated game framework to optimize packet forwarding probabilities of distributed users. The framework has two major steps: first, an adaptive repeated game scheme ensures the cooperation among users for the current cooperative packet forwarding probabilities; second, a self-learning scheme tries to find better cooperation probabilities. Some special cases are analyzed to evaluate the proposed framework. From the simulation results, the proposed framework demonstrates the near optimal solutions in both symmetrical and asymmetrical networks.

We consider the problem of average throughput maximization per total consumed energy in packetized sensor communications. Our study results in a near-optimal transmission strategy that chooses the optimal modulation level and transmit power while adapting to the incoming traffic rate, buffer condition, and the channel condition. We investigate the point-to-point and multinode communication scenarios. Many solutions of the previous works require the state transition probability, which may be hard to obtain in a practical situation. Therefore, we are motivated to propose and utilize a class of learning algorithms [called reinforcement learning (RL)] to obtain the near-optimal policy in point-to-point communication and a good transmission strategy in multinode scenario. For comparison purpose, we develop the stochastic models to obtain the optimal strategy in the point-to-point communication. We show that the learned policy is close to the optimal policy. We further extend the algorithm to solve the optimization problem in a multinode scenario by independent learning. We compare the learned policy to a simple policy, where the agent chooses the highest possible modulation and selects the transmit power that achieves a predefined signal-to-interference ratio (SIR) given one particular modulation. The proposed learning algorithm achieves more than twice the throughput per energy compared with the simple policy, particularly, in high packet arrival regime. Beside the good performance, the RL algorithm results in a simple, systematic, self-organized, and distributed way to decide the transmission strategy.

We consider the problem of average throughput maximization per total consumed energy in packetized point-to-point wireless sensor communications. Our study results in an optimal transmission strategy that chooses the optimal modulation level and transmit power while adapting to the incoming traffic rate, buffer condition, and channel condition. We formulate the optimization problem as a Markov decision process (MDP). When the state transition probability of MDP is available, the optimal policy of MDP can be obtained using dynamic programming (DP). Since in practical situations, the state transition probability may not be available when the optimization is done, we propose to learn the near-optimal policy through the reinforcement learning (RL) algorithm. We show that the RL algorithm learns a policy that achieves almost the same throughput as the optimal one, and the learned policy obtains more than twice average throughput compared to the simple constant signal to noise ratio (CSNR) policy, particularly in high packet arrival rate. Moreover, the learning algorithm is robust in tracking the variation of the governing probability.

We investigate the problem of average throughput maximization per total expended energy in packetized multinode wireless sensor communications. We consider a CDMA like multiple access scenario where multi transmitters are communicating with one receiving node. Since the transmitting nodes are typically distributed in different locations for wireless sensor applications, a good transmission strategy should be employed in each node to maximize throughput per unit energy in the node. We propose to use a distributed actor-critic algorithm to independently learn a good transmission strategy for each transmitting node. The resulting strategy adapts to the incoming traffic rate, buffer condition, and channel condition, which is influenced by both the location of the transmitting node and the other nodes' transmission strategy. Our proposed method achieves 1.5 to 6 times throughput per energy compared to the simple constant signal to interference ratio (CSIR) policy, particularly in high packet arrival. Moreover, the proposed algorithm is robust in tracking packet arrival rate variation.

Orthogonal frequency division multiplexing (OFDM) has been adopted in many broadband wireless communication standards. In OFDM systems, channel estimation is a crucial module in coherent signal detection and receiver diversity combining. In this paper, we investigate the leakage effect of pilot-symbol-assisted channel estimation techniques and propose a channel-aware unequal error protection (UEP) scheme for multimedia communications in OFDM systems. The proposed method exploits the variations of (he bit error rate in different OFDM subchannels caused by the leakage effect, and provides UEP by transmitting the more important data through the subchannels experiencing lower channel estimation error. Compared with the systems that do not jointly consider the channel estimation in the transmission of multimedia data, the proposed scheme can improve the peak-signal-to-noise-ratio (PSNR) of reconstructed images by up to 3 dB.