The Satellite Internet of Things (S-IoT) enables global connectivity for remote sensing devices that must operate energy-efficiently over long time spans. We consider an S-IoT system consisting of a sender-receiver pair connected by a data channel and a feedback channel and capture its dynamics using a Markov Decision Process (MDP). To extend battery life, the sender has to decide on deep-sleep durations. Deep-sleep scheduling is the primary lever to reduce energy consumption, since sleeping devices consume only a fraction of their idle power. By choosing its deep-sleep duration online, the sender has to find a trade-off between energy consumption and data quality degradation at the receiver, captured by a weighted sum of costs. We quantify data quality degradation via the recently introduced Goal-Oriented Tensor (GoT) metric, which can take both age and content of delivered data into account. We assume a Markovian observed process and Markov channels with time-varying delay and erasure rates. The challenge is that content awareness of the GoT metric makes periodic transmissions inherently inefficient. Additionally, optimal sleep durations depends on the (unknown) future states of the observed process and the channels, both of which must be inferred online. We propose a novel algorithm using probabilistic simulation-based optimization (PSBO). With PSBO, the sensor forecasts future states based on estimated transition probabilities, and uses these forecasts to select the optimal deep-sleep duration. Extensive simulations demonstrate the strong performance of PSBO across diverse conditions. In S-IoT hardware experiments, PSBO reduces costs by 59% versus a threshold-based solution and by 89% versus Q-learning.

The rapid growth of space-based services has established Low Earth Orbit (LEO) satellite networks as a promising option for global broadband connectivity. Next-generation LEO networks leverage inter-satellite links (ISLs) to provide faster and more reliable communications compared to traditional bent-pipe architectures, even in remote regions. However, the high mobility of satellites, dynamic traffic patterns, and potential link failures pose significant challenges for efficient and resilient routing. To address these challenges, we model the LEO satellite network as a time-varying graph comprising a constellation of satellites and ground stations. Our objective is to minimize a weighted sum of average delay and packet drop rate. Each satellite independently decides how to distribute its incoming traffic to neighboring nodes in real time. Given the infeasibility of finding optimal solutions at scale, due to the exponential growth of routing options and uncertainties in link capacities, we propose SKYLINK, a novel fully distributed learning strategy for link management in LEO satellite networks. SKYLINK enables each satellite to adapt to the time-varying network conditions, ensuring real-time responsiveness, scalability to millions of users, and resilience to network failures, while maintaining low communication overhead and computational complexity. To support the evaluation of SKYLINK at global scale, we develop a new simulator for large-scale LEO satellite networks. For 25.4 million users, SKYLINK reduces the weighted sum of average delay and drop rate by 29% compared to the bent-pipe approach, and by 92% compared to Dijkstra. It lowers drop rates by 95% relative to k-shortest paths, 99% relative to Dijkstra, and 74% compared to the bent-pipe baseline, while achieving up to 46% higher throughput. At the same time, SKYLINK maintains constant computational complexity with respect to constellation size.

Status Update Systems (SUSs) are central components in applications like environmental sensing or smart cities. They consist of a sender monitoring a remote process and sending the sensed information to a receiver. The sender aims to deliver fresh information about the monitored process's state to allow the receiver to timely respond to the process's changes. In SUSs, the sender is usually battery operated. Therefore, to increase the available energy we consider Energy Harvesting (EH). Moreover, as at the receiver the information transmitted by the sender is only relevant when the process's state changes, we measure the information's freshness using Age of Incorrect Information (AoII). Finding the optimal transmission strategy at the sender that minimizes the AoII requires perfect system knowledge, i.e., the behavior of the monitored process, the channel quality, and the available energy. However, in real applications this knowledge is usually not available. To overcome this challenge, we first establish the optimality of threshold-based policies for AoII minimization in SUSs with EH capabilities by proving that there exists an AoII value depending on the observed state of the monitored process, the battery level and the receiver's estimation of the monitored process's state beyond which transmitting is preferable over idling. Next, we exploit the threshold-based policies' structure and deploy a learning algorithm based on Finite-Difference Policy Gradient (FDPG). Our proposed approach finds the AoII thresholds without requiring perfect system knowledge. Simulations show that our approach outperforms reference algorithms by at least 20% and efficiently learns near-optimal policies for AoII minimization.

Status Update System (SUS) are monitoring applications of Internet of Things (IoT). They are formed by a sender that monitors a remote process and sends status updates to a receiver over a wireless channel. For successful monitoring, the sender must keep the status updates at the receiver fresh. This freshness is generally measured using the Age of Information (AoI) metric. The aim of the sender is to find a monitoring and transmission strategy that minimizes the AoI. To find the optimal strategy, the sender needs to accurately track the AoI at the receiver, i.e., it needs to perfectly know whether a transmitted status update is correctly received or not. This knowledge can be achieved by using a feedback channel between receiver and sender to send acknowledge (ACK) or negative acknowledge (NACK) messages. However, in real applications, the feedback channel is not perfect, and the transmission of ACK/NACK messages might fail. This means, the monitoring and transmission decisions have to be made under uncertainty about the receiver's AoI. To overcome this challenge, we introduce the concept of a socalled belief distribution and propose a joint monitoring and transmission strategy at the sender based on reinforcement learning. Our approach, termed Belief Learning, exploits the belief distribution to minimize the AoI at the receiver. Through numerical simulations we show that Belief Learning enables the sender to achieve near-optimal performance with respect to the perfect feedback channel case.

Wireless backhauling at millimeter-wave frequencies (mmWave) in static scenarios is a well-established practice in cellular networks. However, highly directional and adaptive beamforming in today’s mmWave systems have opened new possibilities for self-backhauling. Tapping into this potential, 3GPP has standardized Integrated Access and Backhaul (IAB) allowing the same base station to serve both access and backhaul traffic. Although much more cost-effective and flexible, resource allocation and path selection in IAB mmWave networks is a formidable task. To date, prior works have addressed this challenge through a plethora of classic optimization and learning methods, generally optimizing Key Performance Indicators (KPIs) such as throughput, latency, and fairness, and little attention has been paid to the reliability of the KPI. We propose Safehaul, a risk-averse learning-based solution for IAB mmWave networks. In addition to optimizing the average performance, Safehaul ensures reliability by minimizing the losses in the tail of the performance distribution. We develop a novel simulator and show via extensive simulations that Safehaul not only reduces the latency by up to 43.2% compared to the benchmarks, but also exhibits significantly more reliable performance, e.g., 71.4% less variance in latency.

Based on the Age of Information as an optimization criterion, proposals for further age-based metrics have been made in recent years in the Internet of Things (IoT) domain. The research community's great interest in age-based metrics for point-to-point wireless communication has led to a multitude of different scenarios being investigated, including energy optimization, sensing, and risk-sensitivity. All these scenarios involve a sender-receiver pair and revolve around finding appropriate times for the sender to communicate status updates to the receiver. We propose a unified and modular framework that represents the aforementioned options in various combinations and enables transferring solutions developed for specific cases to a variety of scenarios. We generalize an existing optimization approach, which decides to transmit based on a threshold for the age-based metric, using this framework. We develop a unified and extended Q-learning-based algorithm with mechanisms to learn suitable solutions for all scenarios derived from our framework. These mechanisms accelerate the learning process and result in improved algorithmic performance compared to traditional Q-learning. Furthermore, we demonstrate the effectiveness of our solution in numerical simulations. Our unified solution outperforms several reference schemes in terms of age-based metrics, energy consumption, and risk. We present our findings as a starting point to investigate transmission strategies for more general settings with a more efficient approach.