This paper addresses a problem of event-triggered state estimation for a linear time-varying Gaussian system over the Internet of Underwater Things (IoUT), where the IoUT is a hybrid topology including underwater acoustic wireless sensors and surface wireless network. Distributed state estimation aims to reconstruct the system state using noisy measurements and local neighbor information, both of which are transmitted via the IoUT. However, IoUT is subject to limited energy and communication bandwidth. Each node, especially applied in underwater case, selectively transmits necessary data to maintain a minimum communication load, and thus to improve energy efficiency and prolong network lifetime. To achieve this goal, we design two event-triggered strategies for typical types of wireless communication channels, of which one channel is used to transmit the local posterior information pair of filter and the other is used to transmit measurement of sensor. Then, based on the covariance intersection fusion rule and the event-triggered strategies, we develop a novel consensus-based distributed state estimator with dual event-triggered communication in a recursive form. Further, after guaranteeing the network connectivity and system collective observability constraints, we derive the uniformly mean-square upper bound of the estimation error of each node. Finally, we provide an example about underwater target tracking to illustrate the effectiveness of the proposed approach.

In the process industry, long-term and efficient optimization of production lines requires real-time monitoring and analysis of operational states to fine-tune production line parameters. However, complexity in operational logic and intricate coupling of production process parameters make it difficult to develop an accurate mathematical model for the entire process, thus hindering the deployment of efficient optimization mechanisms. In view of these difficulties, we propose to deploy a digital twin (DT) of the production line by encoding its operational logic in a data-driven approach. By iteratively mapping the real-world data reflecting equipment operation status and product quality indicators in the DT, we adopt a quality prediction model for production process based on self-attention-enabled temporal convolutional neural networks (NNs). This model enables the data-driven state evolution of the DT. The DT takes a role of aggregating the information of actual operating conditions and the results of quality-sensitive analysis, which facilitates the optimization of process production with virtual-reality evolution. Leveraging the DT as an information-flow carrier, we extract temporal features from key process indicators and establish a production process quality prediction model based on the proposed deep NN. Our operation experiments on a specific tobacco shredding line demonstrate that the proposed DT-based production process optimization method fosters seamless integration between virtual and real production lines. This integration achieves an average operating status prediction accuracy of over 98% and a product quality acceptance rate of over 96%.

Industrial systems require reliable predictive maintenance strategies to enhance operational efficiency and reduce downtime. Existing studies rely on heuristic models which may struggle to capture complex temporal dependencies. This paper introduces an integrated framework that leverages the capabilities of the Transformer and Deep Reinforcement Learning (DRL) algorithms to optimize system maintenance actions. Our approach employs the Transformer model to effectively capture complex temporal patterns in IoT sensor data, thus accurately predicting the Remaining Useful Life (RUL) of equipment. Additionally, the DRL component of our framework provides cost-effective and timely maintenance recommendations. Numerous experiments conducted on the NASA C-MPASS dataset demonstrate that our approach has a performance similar to the ground-truth results and could be obviously better than the baseline methods in terms of RUL prediction accuracy as the time cycle increases. Additionally, experimental results demonstrate the effectiveness of optimizing maintenance actions.

To alleviate the spectrum scarcity problem, exploiting the vast available spectrum provided by the Millimeter-Wave (mmWave) frequency band and underlaying cellular network by Device-to-Device (D2D) communication are two promising solutions. In this paper, we focus on D2D-underlaid cellular network, where the D2D communication is performed on a hybrid manner (i.e., operating over either mmWave or microwave frequency band). To secure the hybrid D2D communication against vigilant adversary, we apply covert communication to hide its presence. In particular, the D2D transmitters perform power control and communication mode switch as well as leveraging the cellular signal to avoid the transmission detection by the adversaries. We model the conflict between the D2D transmitters and adversaries in the framework of a two-stage Stackelberg game. The D2D transmitters are the leaders to maximize their utility subject to the constraints on communication covertness at the upper stage. The adversaries are the followers to minimize their detection errors at the lower stage. We apply stochastic geometry to mathematically characterize the network spatial configuration and consider a large-scale D2D-underlaid network, enabling the study from system-level perspective. We analyze the game equilibrium and obtain it by adopting a bi-level algorithm. Numerical results are provided and insightful conclusions are drawn. Compared with the conventional D2D communication, hybrid D2D communication shows a significant advantage regarding throughput under the same security requirement while weak resistance to the more stringent security requirement.

Vertically distributed learning exploits the local features collected by multiple learning workers to form a better global model. However, data exchange between the workers and the model aggregator for parameter training incurs a heavy communication burden, especially when the learning system is built upon capacity-constrained wireless networks. In this paper, we propose a novel hierarchical distributed learning framework, where each worker separately learns a low-dimensional embedding of their local observed data. Then, they perform communication-efficient distributed max-pooling to efficiently transmit the synthesized input to the aggregator. For data exchange over a shared wireless channel, we propose an opportunistic carrier sensing-based protocol to implement the max-pooling of the output of all the workers. Our simulation experiments show that the proposed learning framework is able to achieve almost the same model accuracy as the learning model using the concatenation of all the raw outputs from the learning workers while significantly reducing the communication load.

Generalized Nash games differ from strategic form games by allowing the strategy set available for each player to depend on the strategies selected by the other players. The strong dependence of the strategies of the players make these generalized games harder to analyze. While convex generalized games are well understood, the case where the constraint sets and rewards are non-convex is significantly more complicated. In this paper we analyze a family of monotonic generalized games (not necessarily convex). We provide uniqueness and existence theorems for these games as well as rapidly converging algorithm for obtaining a Nash equilibrium. We then use the proposed solution to optimize access probability and energy consumption in ALOHA networks, where users have fixed but heterogeneous QoS requirements.

The Industrial Internet of Things (IIoT) is crucial for accurately assessing the state of complex equipment in order to perform predictive maintenance (PdM) successfully. However, existing IIoT-based PdM frameworks do not consider the influence of various practical yet complex system factors, such as the real-time production states, machine health, and maintenance manpower resources. For this reason, we propose a generic PdM optimization framework to assist maintenance teams in prioritizing and resolving maintenance task conflicts under real-world manufacturing conditions. Specifically, the PdM framework aims to jointly optimize the edge-based machine network uptime and the allocation of manpower resources in a stochastic IIoT-enabled manufacturing environment using the model-free deep reinforcement learning (DRL) methods. Since DRL requires a significant amount of training data, we propose and demonstrate the use of the transfer learning (TL) method to assist DRL in learning more efficiently by incorporating expert demonstrations, termed TL with demonstrations (TLDs). TLD reduces training wall time by 58% compared to baseline methods, and we conduct numerous experiments to illustrate the performance, robustness, and scalability of TLD. Finally, we discuss the general benefits and limitations of the proposed TL method, which are not well addressed in the existing literature but could be beneficial to both researchers and industry practitioners.

Unplanned breakdown of critical equipment interrupts production throughput in Industrial IoT (IIoT), and data-driven predictive maintenance (PdM) becomes increasingly important for companies seeking a competitive business advantage. Manufacturers, however, are constantly faced with the onerous challenge of manually allocating suitably competent manpower resources in the event of an unexpected machine breakdown. Furthermore, human error has a negative rippling impact on both overall equipment downtime and production schedules. In this article, we formulate the complex resource management problem as a resource optimization problem to determine if a model-free deep reinforcement learning (DRL)-based PdM framework can be used to automatically learn an optimal decision policy from a stochastic environment. Unlike the existing PdM frameworks, our approach considers PdM sensor information and the resources of both physical equipment and human as part of the optimization problem. The proposed DRL-based framework and proximal policy optimization long short term memory (PPO-LSTM) model are evaluated alongside baselines results from human participants using a maintenance repair simulator. Empirical results indicate that our PPO-LSTM efficiently learns the optimal decision-policy for the resource management problem, outperforming comparable DRL methods and human participants by 53% and 65%, respectively. Overall, the simulation results corroborate the proposed DRL-based PdM framework’s superiority in terms of convergence efficiency, simulation performance, and flexibility.

We study a decentralized channel allocation problem in an ad-hoc Internet of Things network underlaying on the spectrum licensed to a primary cellular network. In the considered network, the impoverished channel sensing/probing capability and computational resource on the IoT devices make them difficult to acquire the detailed Channel State Information (CSI) for the shared multiple channels. In practice, the unknown patterns of the primary users' transmission activities and the time-varying CSI (e.g., due to small-scale fading or device mobility) also cause stochastic changes in the channel quality. Decentralized IoT links are thus expected to learn channel conditions online based on partial observations, while acquiring no information about the channels that they are not operating on. They also have to reach an efficient, collision-free solution of channel allocation with limited coordination. Our study maps this problem into a contextual multi-player, multi-armed bandit game, and proposes a purely decentralized, three-stage policy learning algorithm through trial-and-error. Theoretical analyses shows that the proposed scheme guarantees the IoT links to jointly converge to the social optimal channel allocation with a sub-linear (i.e., polylogarithmic) regret with respect to the operational time. Simulations demonstrate that it strikes a good balance between efficiency and network scalability when compared with the other state-of-the-art decentralized bandit algorithms.

Daily decision-making circumstances often include risk and uncertainty, and the decision-maker’s risk preference constantly influences the course of action taken. Modeling how individuals make decisions in real-world situations remains a key challenge, and the use of machine learning techniques to model and augment individuals’ decision-making processes has garnered little attention. In this paper, we propose a new framework to evaluate the feasibility of inferring risk attitudes from human behavior modeled using the Reinforcement Learning (RL) model. The RL-based model framework is then generalized by including a situational or contextual variable that is indicative of cognitive biases observed in human decision-making, using an equipment maintenance scenario as a case study. Interestingly, when compared to human participants with working experience, the mean performance of a trained RL agent bears some semblance to the surveyed human participants. Riding on such strong empirical results, we also discover that the risk-seeking group of human participants are 9 times more likely to engage in risk-seeking associated actions. Besides, our exploratory simulations show that near optimal performance can be consistently achieved with several enhancements to state-of-the-art advantage actor-critic RL algorithms under the situational context of BUSY and IDLE. Overall, our proposed RL-based model framework can categorically infer the user-defined human risk attitude, and the trained RL agent demonstrates its rational decision-making prowess by outperforming the majority of human participants.

Open-access blockchains based on proof-of-work protocols have gained tremendous popularity for their capabilities of providing decentralized tamper-proof ledgers and platforms for data-driven autonomous organization. Nevertheless, the proof-of-work based consensus protocols are vulnerable to cyber-attacks such as double-spending. In this paper, we propose a novel approach of cyber risk management for blockchain-based service. In particular, we adopt the cyber-insurance as an economic tool for neutralizing cyber risks due to attacks in blockchain networks. We consider a blockchain service market, which is composed of the infrastructure provider, the blockchain provider, the cyber-insurer, and the users. The blockchain provider purchases from the infrastructure provider, e.g., a cloud, the computing resources to maintain the blockchain consensus, and then offers blockchain services to the users. The blockchain provider strategizes its investment in the infrastructure and the service price charged to the users, in order to improve the security of the blockchain and thus optimize its profit. Meanwhile, the blockchain provider also purchases a cyber-insurance from the cyber-insurer to protect itself from the potential damage due to the attacks. In return, the cyber-insurer adjusts the insurance premium according to the perceived risk level of the blockchain service. Based on the assumption of rationality for the market entities, we model the interaction among the blockchain provider, the users, and the cyber-insurer as a two-level Stackelberg game. Namely, the blockchain provider and the cyber-insurer lead to set their pricing/investment strategies, and then the users follow to determine their demand of the blockchain service. Specifically, we consider the scenario of double-spending attacks and provide a series of analytical results about the Stackelberg equilibrium in the market game.

With the development of decentralized consensus protocols, permissionless blockchains have been envisioned as a promising enabler for the general-purpose transaction-driven, autonomous systems. However, most of the prevalent blockchain networks are built upon the consensus protocols under the crypto-puzzle framework known as proof-of-work. Such protocols face the inherent problem of transaction-processing bottleneck, as the networks achieve the decentralized consensus for transaction confirmation at the cost of very high latency. In this paper, we study the problem of consensus formation in a system of multiple throughput-scalable blockchains with sharded consensus. Specifically, the protocol design of sharded consensus not only enables parallelizing the process of transaction validation with sub-groups of processors, but also introduces the Byzantine consensus protocols for accelerating the consensus processes. By allowing different blockchains to impose different levels of processing fees and to have different transaction-generating rate, we aim to simulate the multi-service provision eco-systems based on blockchains in real world. We focus on the dynamics of blockchain-selection in the condition of a large population of consensus processors. Hence, we model the evolution of blockchain selection by the individual processors as an evolutionary game. Both the theoretical and the numerical analysis are provided regarding the evolutionary equilibria and the stability of the processors' strategies in a general case.

Embedding sensors on wireless devices for collaborative environment sensing has been envisioned as a cost-effective solution for IoT applications. However, existing IoT platforms faces challenges, e.g., unsustainablility due to the limited on-device battery and tremendous cost of deploying middlewares for centralized task dispatching. In this paper, we employ wireless energy transfer and permissionless blockchains to construct a sustainable and decentralized IoT crowdsensing platform. Therein, IoT sensing cloud composed of multiple co-located sensors is wirelessly powered by RF-energy beacons for data sensing and transmission. The data is then forwarded to the blockchain for distributed data/transaction verification and trading. The data users access the crowdsensing service by renting sensors from the sensing clouds. Both the sensing clouds and data users are self-interested and aim to maximize their individual profits. The sensing clouds handle the interference of concurrent wireless transmissions and the on-chain transaction cost. Meanwhile, each user distributes its limited budget over the sensing clouds to optimize the service quality. We formulate a Stackelberg differential game to analyze the interaction among the sensing clouds and data users. Then, we investigate the Stackelberg equilibrium by capitalizing on Pontryagin's maximum principle. Furthermore, we provide a series of insightful numerical results about the Stackelberg equilibrium.

Public blockchain networks using proof of work (PoW)-based consensus protocols are considered as a promising platform for decentralized resource management with financial incentive mechanisms. In order to maintain a secured, universal state of the blockchain, PoW-based consensus protocols financially incentivize the nodes in the network to compete for the privilege of block generation through cryptographic puzzle solving. For rational consensus nodes, i.e., miners with limited local computational resources, offloading the computation load for PoW to the cloud/fog providers (CFPs) becomes a viable option. In this paper, we study the interaction between the CFPs and the miners in a PoW-based blockchain network using a game theoretic approach. In particular, we propose a lightweight infrastructure of the PoW-based blockchains, where the computation-intensive part of the consensus process is offloaded to the cloud/fog. We formulate the computation resource management in the blockchain consensus process as a two-stage Stackelberg game, where the profit of the CFP and the utilities of the individual miners are jointly optimized. In the first stage of the game, the CFP sets the price of offered computing resource. In the second stage, the miners decide on the amount of service to purchase accordingly. We apply backward induction to analyze the subgame perfect equilibria in each stage for both uniform and discriminatory pricing schemes. For uniform pricing where the same price applies to all miners, the uniqueness of the Stackelberg equilibrium is validated by identifying the best response strategies of the miners. For discriminatory pricing where the different prices are applied, the uniqueness of the Stackelberg equilibrium is proved by capitalizing on the variational inequality theory. Further, the real experimental results are employed to justify our proposed model.

In sponsored content and service markets, the content and service providers are able to subsidize their target mobile users through directly paying the mobile network operator to lower the price of the data/service access charged by the network operator to the mobile users. The sponsoring mechanism leads to a surge in mobile data and service demand, which in return compensates for the sponsoring cost and benefits the content/service providers. In this paper, we study the interactions among the three parties in the market, namely, the mobile users, the content/service providers, and the network operator, as a two-level game with multiple Stackelberg (i.e., leader) players. Our study is featured by the consideration of global network effects owning to consumers' grouping. Since the mobile users may have bounded rationality, we model the service-selection process among them as an evolutionary-population follower sub-game. Meanwhile, we model the pricing-then-sponsoring process between the content/service providers and the network operator as a non-cooperative equilibrium searching problem. By investigating the structure of the proposed game, we reveal a few important properties regarding the equilibrium existence and propose a distributed, projection-based algorithm for iterative equilibrium searching. Simulation results validate the convergence of the proposed algorithm and demonstrate how sponsoring helps improve both the providers' profits and the users' experience.

Blockchains enable tamper-proof, ordered logging for transactional data in a decentralized manner over open-access, overlay peer-to-peer networks. In this paper, we propose a decentralized framework of proactive caching in a hierarchical wireless network based on blockchains. We employ the blockchain-based smart contracts to construct an autonomous content caching market. In the market, the cache helpers are able to autonomously adapt their caching strategies according to the market statistics obtained from the blockchain, and the truthfulness of trustless nodes are financially enforced by smart contract terms. Further, we propose an incentive-compatible consensus mechanism based on proof-of-stake to financially encourage the cache helpers to stay active in service. We model the interaction between the cache helpers and the content providers as a Chinese restaurant game. Based on the theoretical analysis regarding the Nash equilibrium of the game, we propose a decentralized strategy-searching algorithm using sequential best response. The simulation results demonstrate both the efficiency and reliability of the proposed equilibrium searching algorithm.

This paper studies the problem of spectrum-aware routing in a multi-hop, multi-channel cognitive radio network when malicious nodes in the secondary network attempt to block the path with mixed attacks. Based on the location and time-variant path delay information, we model the path discovery process as a non-cooperative stochastic game. By exploiting the structure of the underlying Markov Decision Process, we decompose the stochastic routing game into a series of stage games. For each stage game, we propose a distributed strategy learning mechanism based on stochastic fictitious play to learn the equilibrium strategies of joint relay-channel selection in the condition of both limited information exchange and potential routing-toward-primary attacks. We also introduce a trustworthiness evaluation mechanism based on a multi-arm bandit process for normal users to avoid relaying to the sink-hole attackers. Simulation results show that without the need of information flooding, the proposed algorithm is efficient in bypassing the malicious nodes with mixed attacks.

In this paper, we study the transmission strategy adaptation problem in an RF-powered cognitive radio network, in which hybrid secondary users are able to switch between the harvest-then-transmit mode and the ambient backscatter mode for their communication with the secondary gateway. In the network, a monetary incentive is introduced for managing the interference caused by the secondary transmission with imperfect channel sensing. The sensing-pricing-transmitting process of the secondary gateway and the transmitters is modeled as a single-leader-multi-follower Stackelberg game. Furthermore, the follower sub-game among the secondary transmitters is modeled as a generalized Nash equilibrium problem with shared constraints. Based on our theoretical discoveries regarding the properties of equilibria in the follower sub-game and the Stackelberg game, we propose a distributed, iterative strategy searching scheme that guarantees the convergence to the Stackelberg equilibrium. The numerical simulations show that the proposed hybrid transmission scheme always outperforms the schemes with fixed transmission modes. Furthermore, the simulations reveal that the adopted hybrid scheme is able to achieve a higher throughput than the sum of the throughput obtained from the schemes with fixed transmission modes.

The sponsored content/service market is an emerging platform, where the Content/Service Providers (CSPs) pay the Mobile Network Operator (MNO) and subsidize the Mobile Users (MUs) to access their services at a lower price. The sponsoring mechanism leads to a surge in mobile data and service demand, which in return compensates for the sponsoring cost and benefits the CSPs. In this paper, we study the interactions among the three entities in the market, namely, the MUs, the CSPs and the MNO, as a two-level hierarchical game. Our study is featured by the consideration of global network effects owning to consumers' grouping. We model the service- selection process among the MUs as an evolutionary population sub-game, and the sponsoring-pricing process between the CSPs and the MNO as a non- cooperative sub-game. By investigating the structure of the proposed game, we discover a few important properties regarding the existence of the hierarchical equilibrium, and propose a distributed, projection-based algorithm for iterative equilibrium searching. Simulation results validate the convergence property of the proposed algorithm, and demonstrate how sponsoring helps to improve both the CSPs' profits and the MUs' experience.

In wireless communication systems, channel state feedback (CSF) is widely used to improve link performance. However, CSF consumes extra system resources and results in transmission overhead. In this paper, we evaluate such resource consumption in terms of bit rate and provide an explicit expression for the lower bound of the CSF overhead. We propose an overhead optimization mechanism under the constraint of channel state reconstruction accuracy. The numerical simulations show that our paper is beneficial to select the optimum CSF parameters, including the average bit number and channel state feedback period. It is also shown that the proposed overhead optimization scheme is able to reduce the system resource consumption with a guaranteed reconstruction accuracy.

In this paper, we study the power allocation problem for the downlink transmission in a set of closed-access femtocells which underlay a number of macrocells. We introduce a mutli-step pricing mechanism for the macrocells to control the cross- tier interference by femtocell transmissions without explicit coordination. We model the cross- tier joint power allocation process in the heterogeneous network as a non-cooperative, average-reward Markov game. By investigating the structure of the instantaneous payoff functions in the game, we propose a self-organized strategy learning scheme based on learning automata for both the macrocell base stations and the femtocell access points to adapt their transmit power simultaneously. We prove that the proposed learning scheme is able to find a pure-strategy Nash equilibrium of the game without the need for the femtocell access points to share any local information. Simulation results show the efficiency of the proposed learning scheme.

This paper addresses the problem of dynamic, real-time video transmission control over a time-varying wireless channel. The problem of adaptive source coding control is studied on the basis of scalable video coding schemes. A video-layer-based hybrid automatic repeat request scheme is adopted to achieve adaptive error protection allocation. The problem of joint source-channel resource allocation over a time-varying wireless channel is posed as a constrained Markov Decision Process (MDP). The goal of the proposed video streaming MDP is to minimize the average end-to-end frame distortion with the constraint on the average transmission time for each layer. In order to address the issue of unknown channel dynamics and inaccurate distortion model, the R-learning-based on-line algorithm is adopted for learning the transmission police. To handle the constrained MDP with the standard R-learning algorithm, the constrained MDP is converted into an unconstrained MDP through the Lagrangian approach. The efficiency and convergence of the proposed learning algorithm are demonstrated with the simulation results.

A green-energy-powered, self-sustainable cellular network is studied in this paper. To address the problem of the energy variability in the green sources, a joint energy-traffic management mechanism is proposed based on game theory. The interaction between the cellular network and a microgrid power controller is modeled as a two-level Stackelberg game. On the network level, an iterative combinatorial auction mechanism is proposed to solve the joint power-subcarrier allocation problem. On the microgrid level, an adaptive pricing mechanism is adopted for the power controller to balance between the transmission demand and the energy supply. Simulation results show that the proposed resource allocation mechanism achieves a significant improvement in the network sustainability and throughput when compared with the conventional resource allocation method.

This paper investigates the energy-efficient power allocation for a two-tier, underlaid femtocell network. The behaviors of the Macrocell Base Station (MBS) and the Femtocell Users (FUs) are modeled hierarchically as a Stackelberg game. The MBS guarantees its own QoS requirement by charging the FUs individually according to the cross-tier interference, and the FUs responds by controlling the local transmit power non-cooperatively. Due to the limit of information exchange in intra-and inter-tiers, a self-learning based strategy-updating mechanism is proposed for each user to learn the equilibrium strategies. In the same Stackelberg-game framework, two different scenarios based on the continuous and discrete power profiles for the FUs are studied, respectively. The self-learning schemes in the two scenarios are designed based on the local best response. By studying the properties of the proposed game in the two situations, the convergence property of the learning schemes is provided. The simulation results are provided to support the theoretical finding in different situations of the proposed game, and the efficiency of the learning schemes is validated.

This paper proposes a distributed routing mechanism for the multi-hop Cognitive Radio Networks (CRNs) over multiple primary channels. The Secondary Users (SUs) attempts to utilize the channels and minimize their delay along the route while avoiding causing interference to the Primary Users (PUs). In order to address the problem of time-varying channel condition due to the PU dynamics, the route-selection process is modeled as a global Markov Decision Process (MDP). We show that such a global routing MDP can be decomposed into the layered MDPs, in which the interactions between neighbor SUs with their local next-hop selection are modeled as the local stochastic games. By applying reinforcement learning with utility based fictitious play, the best response of each SU can be learned from the local game with the only need for the information exchange from next-hop SUs. The proposed algorithm is evaluated through simulations and is shown to be effective in reducing the delays for multiple flows in the CRN.

This paper proposes a robust, spectrum-aware routing mechanism for the multi-hop, multi-channel Cognitive Radio Networks (CRNs) that are exposed to the Routing-toward-Primary-User Attack (RPUA). With RPUA, malicious Secondary Users (SUs) attempt to block the data transmission of the normal SUs by routing packets towards the SUs around the Primary Users (PUs). The proposed routing mechanism models the interaction between the non-malicious SUs and their one-hop neighbors as a stochastic game. With routing information back-propagation from the next-hop SUs, the local relay-channel-selection games along the routing path form a global, layered Markov Decision Process (MDP). To address the problem of the mixed attack with both RPUA and Sink-Hole-Attack (SHA), the trustworthiness of the neighbor nodes is evaluated as a single-buyer-multiple-seller price competition. For each SU, the best policy of relay-channel selection is learned in a reinforcement learning framework with fictitious play. Simulation results show that the proposed routing algorithm can avoid malicious relays and minimize the routing delay in the CRN under attacks.

In recent years, cross-layer Resource Management (RM) has been widely considered as an efficient approach for improving the network performance. However, due to the factors such as limited knowledge about the wireless environment and spontaneous characteristics of device behaviors, designing an efficient distributed RM scheme using the cross-layer paradigm becomes extremely difficult. In this paper, a distributed, cross-layer RM scheme for multi-users transmitting scalable video in an ad-hoc CDMA network is presented. For the cross-layer RM design, an end-to-end performance criterion at the APP layer is applied and the resource allocations in the PHY, LINK and APP layers are unified into a single decision process. Due to the dynamic property of the wireless channels and the uncoordinated user interactions, the distributed RM decision is modeled in the framework of stochastic game. In the game, each device gradually learns its policy using a distributed, Correlated Equilibrium (CE) based Q-Learning algorithm. In order to calculate the CE in a distributed way, the local devices apply the regret-matching based mechanism for policy and state-action value updating. Simulation experiments show that with the proposed resource management scheme, the system performance can be improved by about 15% compared to the layered RM scheme and 10% compared to the RM scheme purely based on the local-level learning.

A Cognitive Radio (CR) network has to adapt operations of all secondary users to meet performance goals while avoiding interfering the primary network (PN) beyond a set limit. For this, this paper considers a distributed cross-layer resource allocation CR algorithm. While the cross-layer approach notably improves performance in terms of average end-to-end distortion and network's congestion rate, it increases the number of iterations needed to find the resource allocation solution. In this paper, the extra complexity in the cross-layer approach is addressed through a novel cooperative cross-layer learning algorithm where peer nodes cooperate first distributing the learning tasks, followed by sharing of the complementary learned information. The algorithm does not rely on the availability of expert nodes that have already performed the learning process. Simulation results show that the cooperative learning technique reduces complexity by approximately 45% with a small and very acceptable sacrifice in performance.