UAVs are increasingly essential in wireless communication applications, such as internet of things (IoT) and sensor networks, due to their agile mobility. However, planning three-dimensional (3D) UAV trajectories over a continuous temporal-spatial domain remains challenging due to the computational complexity of non-convex optimization. This paper addresses UAV-assisted IoT data collection, aiming to minimize total energy consumption while considering UAV capabilities, heterogeneous IoT data demands, and 3D terrain. We propose a matrix-based differential evolution with constraint handling (MDE-CH), a computationally efficient algorithm for solving constrained non-convex optimization problems. Numerical results show that MDE-CH efficiently generates continuous 3D UAV trajectories, significantly reducing energy consumption and outperforming the conventional fly-hover-fly model for 2D and 3D trajectory planning.

In this work, we study learning-aided adaptive control of optical beam alignment to maintain a seamless connection with high communication performance in a point-to-point (P2P) underwater optical wireless communication (UOWC). To this end, we propose a two-step two-agent deep reinforcement learning (TSTA-DRL) algorithm that enables an underwater sensor (US) installed on the seabed to sequentially determine the beam orientation and beam divergence angles for transmitting its sensing data to an unmanned surface vehicle (USV) that may irregularly shake above the sea level. Specifically, the proposed TSTA-DRL algorithm includes two DRL agents: beam orientation (BO) and beam divergence (BD). The BO agent selects the beam orientation angle to point the optical beam of the US toward the USV to perform beam alignment between the US and USV. Moreover, given the beam orientation angle determined by the BO agent, the BD agent chooses the beam divergence angle to maximize the signal-to-noise ratio (SNR) while maintaining the seamless optical link between two nodes. For the practical application of the proposed algorithm, movement data of the USV measured in the South Sea of Korea are utilized for training the proposed algorithm. The simulation results demonstrate that the proposed TSTA-DRL algorithm achieves the highest SNR while maintaining a stable UOWC link compared with existing algorithms.

In this study, we first present a framework that jointly optimizes energy harvesting and information decoding for Internet of Things (IoT) devices, which are capable of simultaneous wireless information and power reception, in a smarty city. In particular, a generalized power-splitting receiver for IoT devices is designed, where each antenna in the receiver has an independent power splitter, unlike the existing works in which only one power splitter is employed regardless of the number of antennas in the receiver. Such a receiver design can provide a great degree of freedom to improve the network performance. Based on the presented framework, for each IoT device, we formulate an optimization problem whose objective is to maximize the harvested energy of each IoT device while satisfying its data rate requirement. To solve this problem, we propose a double-deep deterministic policy gradient-based online learning algorithm which enables each IoT device to jointly determine receive beamforming and power-splitting ratio vectors in real time. Furthermore, each IoT device can implement the proposed algorithm in a distributed manner using only its local channel state information. As such, cooperation and information exchange among the base stations and IoT devices are not necessary when performing the proposed algorithm at IoT devices. The extensive simulation results show the validity of the proposed algorithm.

Integrating underwater optical wireless communication (UOWC) with marine applications, such as underwater sensors, buoys, and marine surface vehicles (MSVs), requires the aligning and maintaining of the optical beam between the transmitter and receiver for point-to-point (P2P) UOWC during the data transmission. An additional issue is the difficulty in exchanging batteries for marine applications because of the relatively high costs and risks compared with battery exchanging in terrestrial applications. This study seeks to resolve these issues via joint optimization of the beam divergence angle and transmission power level in an underwater sensor (i.e., transmit node) to maintain a seamless connection with an MSV (i.e., receive node) while minimizing the battery consumption of the sensor. In this regard, we adopt a hybrid underwater acoustic-optical communication system, where acoustic and optical communications are used for low-rate control data transmission and high-rate sensing data transmission, respectively. Under this framework, we propose a two-phase deep reinforcement learning (TPDRL) algorithm considering two agents (inner and outer) that determine different actions using an underwater sensor. Specifically, the primary role of the outer agent is to choose a transmission power level based on the long-term signal-to-noise ratio (SNR) between the underwater sensor and MSV. Next, the inner agent finds the beam divergence angle for the given transmission power (selected from the outer agent) based on the short-term instantaneous SNR. Simulation results demonstrate that the proposed TPDRL algorithm enables seamless and energy-efficient P2P UOWC, performing better than the algorithm with only the inner agent and other existing algorithms.

This paper analyzes the achievable secrecy transmission rates (STRs) of Alamouti space–time block coding (STBC) with the decision feedback (DF) detection in wiretap time-selective fading channels and studies how to optimally design the Wyner’s codebook to maximize it. Specifically, we derive the STR with both connection outage probability and secrecy outage probability for an arbitrary temporal correlation and find the optimal codeword rate and the secrecy rate to maximize it.

As an Internet of Things (IoT) application in the smart grid, dynamic thermal rating (DTR) requires real-time measurement of conductor temperature from field sensors that are installed at remote transmission lines. For such IoT-based transmission line monitoring, we need a robust communication network. This article proposes to group sensors with similar measurement characteristics into a cluster such that each member can accurately represent the entire cluster. Then, communication robustness can be achieved by establishing multiple routes to the control center, where each route may originate from a different member in the cluster. First, we formulate a global constrained optimization for sensor clustering. We use the optimization solutions to train a set of artificial neural networks (ANNs), where each ANN is deployed to make a local decision at one sensor. Specifically, we propose the novel usage of ANN as a tool to transfer knowledge from a global decision maker to a set of local decision makers. Evaluation results show that there remains a noticeable knowledge gap between global and local decision makers, where in average, the global decision maker and an ANN make an identical decision in only about 70% of the times. Despite this less than ideal success probability, the number of clusters produced by the ANN-assisted local scheme is only 1.9 more than that by the global scheme.

In this article, we show the development and implementation of the Internet of Maritime Things (IoMT) platform that supports long-range and high-rate communication for remote and online marine water quality monitoring. We first develop the IoMT device and gateway that interwork with international standard Internet-of-Things (IoT) software platforms for universal utilization in the field of marine monitoring. The proposed IoMT device and gateway play a role similar to the IoT device and gateway on land, but it is optimized for marine use with consideration of the practical characteristics of using sensors in the ocean. Furthermore, to realize long-range and high-rate communication between marine sensors and a control center (CC) on land, an automatic beam adjustment system for directional antennas is developed. This enables the use of directional antennas, which can support long-range and high-rate communication by automatically adjusting the beam angle between the antennas while exchanging their control information, despite the antennas being shifted by undulation from the sea. Finally, we conduct real sea experiments to identify the performances of the proposed IoMT platform in various environments. The results show that the proposed IoMT platform can collect data from marine sensors as well as high definition camera video from distances of 25 and 40 km to the CC on land under one-hop and two-hop transmissions, respectively. These results indicate that the proposed IoMT platform provides at least twofold performance improvement in terms of communication coverage under a similar data rate condition, compared with existing commercial wireless networks (e.g., LTE in the ocean) not utilizing satellite communications.

In this paper, we consider the problem of handover decision making in the context of a dense heterogeneous network with a macro base station and multiple small base stations. We propose a distributed deep Q-learning based algorithm that minimizes the overall energy consumption by taking into account both the energy consumption from transmission and hand over overheads. The proposed algorithm is performed in a distributed and interactive manner in which a centralized training agent manages the replay buffer for training its deep Q-network, by gathering state, action, and reward information reported from distributed handover agents. We perform several numerical evaluations and demonstrate that the proposed algorithm provides 10% to 30% energy savings over other contemporary handover mechanisms depending on handover overhead costs.

In this paper, we consider a joint beam tracking and pattern optimization problem for massive multiple input multiple output (MIMO) systems in which the base station (BS) selects a beamforming codebook and performs adaptive beam tracking taking into account the user mobility. A joint adaptation scheme is developed in a two-phase reinforcement learning framework which utilizes practical signaling and feedback information. In particular, an inner agent adjusts the transmission beam index for a given beamforming codebook based on short-term instantaneous signal-to-noise ratio (SNR) rewards. In addition, an outer agent selects the beamforming codebook based on long-term SNR rewards. Simulation results demonstrate that the proposed online learning outperforms conventional codebook-based beamforming schemes using the same number of feedback information. It is further shown that joint beam tracking and beam pattern adaptation provides a significant SNR gain compared to the beam tracking only schemes, especially as the user mobility increases.

In this paper, we first present an analytical framework based on a queueing system that evaluates communication performances of underwater acoustic sensor networks (UASNs), where each underwater sensor distributed within a 3-dimensional (3D) space under sea surface performs fountain code (FC)-based automatic repeat request (ARQ) transmissions under the slotted ALOHA medium access control protocol. Under the proposed framework, we evaluate communication performances given in terms of successful FC-based ARQ transmission probability and the average queueing delay of an underwater sensor. Further, our analysis can formulate an optimization problem that aims at minimizing total cost occurred to install and operate 3D UASNs, without compromising two communication QoS requirements. To solve this problem, we propose a recursive algorithm to approach an optimal solution in reasonable time.

Community smart grid is formed by a group of neighboring households to share renewable generators and energy storage facilities. Within a community grid, all power grid entities are connected to power lines that are downstream to a common substation. In this paper, we use device-to-device communications to connect sensors that are installed at grid entities to their respective control units. The topology and node membership of such communication network depend on the topology of its power grid. As such, a communication node may not be assigned the best quality channel. This has imposed additional constraints in radio resource allocation. We exploit the feature that sensors and control units are static but cellular users are mobile, in proposing a two-stage radio resource allocation scheme. In the planning stage, we use the Hungarian method to find the optimal channel assignment for static sensors and control units. In the operation stage, we run recursively the Hungarian algorithm to allocate proportionally all remaining channels to cellular users. Given the channel assignment, the operation stage further performs transmit power allocation to all nodes. The power control problem is originally nonconvex. We transform the problem into a difference of convex structure, which can be solved successively. We have evaluated the proposed radio resource allocation scheme through extensive simulations. Results confirm that the scheme is indeed efficient in assigning channels and finding the minimum transmit power, to maximize sum-rate of cellular users while guaranteeing a minimum throughput to each sensor and control unit.

This study aims to optimize the design and performance of three-dimensional (3D) underwater acoustic sensor networks (UASNs). First, we develop an analytical model to quantify network performances in terms of the packet queueing delay and packet error probability for sensors in a 3D UASN. The model considers the retransmission delay due to packet transmission errors and packet collisions when calculating the packet queueing delay. The packet queueing delay depends on the choice of medium access control (MAC) protocol. We consider two different MAC protocols: slotted ALOHA and Request-to-Send and Clear-to-Send (RTS/CTS)-based MAC, which are the representative random access-based and handshake-based MAC protocols for UASNs, respectively. Based on the model, we further determine the smallest data sink density required to meet the desired packet error probability requirement for underwater sensor density, which can help network operators to design and build UASNs in a cost-effective way. To do this, we formulate a simple optimization problem that aims to minimize the data sink density while guaranteeing an upper bound for packet error probability. Using extensive numerical evaluations, network operation strategies are then presented to achieve the desired performance requirements in 3D UASNs.

This paper investigates the energy-efficiency of licensed-assisted access (LAA) which enable long-term evolution systems to operate in unlicensed spectrum. Since Wi-Fi with 802.11 n/ac is a typical radio access technology in the unlicensed spectrum, we consider a scenario that the LAA small-cell base stations (SCBSs) are deployed together with Wi-Fi access points. We propose three different listen-before-talk (LBT) schemes according to the channel sensing mechanisms for LAA SCBC, and establish Markov chain models for each scheme under non-saturated traffic condition. Using these analytical models, the energy-efficiencies of LAA SCBSs that perform the different LBT schemes are investigated. We also propose the algorithm to obtain the optimal contention window size of LAA SCBSs by which total energy-efficiency of networks is maximized while satisfying the required energy-efficiency of both LAA and Wi-Fi networks. Numerical results show that the LBT scheme which has an efficient channel sensing mechanism outperforms the other LBT schemes for the energy-efficiency perspective.

Carrier aggregation (CA) is an essential feature in the Long Term Evolution-Advanced (LTE-A) system, which can aggregate two and more component carriers (CCs) to form a much wider transmission bandwidth up to 100 MHz. By means of CA, a user equipment (UE) can be scheduled on multiple CCs simultaneously to transmit data traffic. Recently, the studies on the efficient operation of an LTE-A based CA system have received much attention under simultaneous access scenarios for multiple CCs. However, the existing literature lacks efforts on Quality of Experience (QoE)-aware CA, especially in the uplink. In this paper, we propose a joint resource allocation algorithm for an uplink LTE-A based system, focusing on QoE-aware CA. We first present a framework for the evaluation of UE's QoE satisfaction level by designing a link reward, resource cost, and link utility function. Then, a QoE-aware joint resource allocation algorithm is proposed for the efficient sharing of radio resource by UEs, so that uplink system utility is maximized. Numerical results demonstrate the validity of the proposed algorithm.

This paper proposes a joint radio resource allocation algorithm for uplink parallel multi-radio access (MRA)in heterogeneous networks, first introducing an economic link utility function, which quantifies Quality of Experience (QoE) satisfaction level of user, as a combination of link reward and resource cost function. Then, a QoE-based joint resource allocation algorithm is proposed for the efficient sharing of radio resources by multi-mode terminals, so that uplink system utility is maximized. In particular, this work studies two cases of parallel MRA systems that are distinguished by the existence of free wireless access points. We provide a sensitivity analysis on how cost in the joint resource allocation algorithm affects the performances of various parallel MRA systems.