This paper investigates the identification of the top-m user-scheduling sets in multi-user MIMO downlink, which is cast as a combinatorial pure-exploration problem in stochastic linear bandits. Because the action space grows exponentially, exhaustive search is infeasible. We therefore adopt a linear utility model to enable efficient exploration and reliable selection of promising user subsets. We introduce a gap-index framework that maintains a shortlist of current estimates of champion arms (top-m sets) and a rotating shortlist of challenger arms that pose the greatest threat to the champions. This design focuses on measurements that yield the most informative gap-index-based comparisons, resulting in significant reductions in runtime and computation compared to state-of-the-art linear bandit methods, with high identification accuracy. The method also exposes a tunable trade-off between speed and accuracy. Simulations on a realistic OFDM downlink show that shortlist-driven pure exploration makes online, measurement-efficient subcarrier selection practical for AI-enabled communication systems.

IoT networks often face conflicting routing goals such as maximizing packet delivery, minimizing delay, and conserving limited battery energy. These priorities can also change dynamically: for example, an emergency alert requires high reliability, while routine monitoring prioritizes energy efficiency to prolong network lifetime. Existing works, including many deep reinforcement learning approaches, are typically centralized and assume static objectives, making them slow to adapt when preferences shift. We propose a dynamic and fully distributed multi-objective Q-learning routing algorithm that learns multiple per-preference Q-tables in parallel and introduces a novel greedy interpolation policy to act near-optimally for unseen preferences. The algorithm learns to optimize for energy efficiency, packet delivery ratio, and the composite reward, adapting to changing trade-offs between these metrics without retraining or centralized control. A theoretical analysis further shows that the optimal value function is Lipschitz-continuous in the preference parameter, ensuring that proposed greedy interpolation policy yields provably near-optimal behavior. Simulation results show that our approach adapts in real time to shifting priorities and achieves up to 80–90% lower energy consumption and up to 5 × higher cumulative rewards and packet delivery compared to six baseline protocols, under dynamic and distributed settings. Sensitivity analysis across varying preference window lengths confirms that the proposed DPQ framework consistently achieves higher composite reward than all baseline methods, demonstrating robustness to changes in operating conditions.

Internet of Things (IoT) systems increasingly operate in environments where devices must respond in real time while managing fluctuating resource constraints, including energy and bandwidth. Yet, current approaches often fall short in addressing scenarios where operational constraints evolve over time. To address these limitations, we propose a novel Budgeted Multi-Armed Bandit framework tailored for IoT applications with dynamic operational limits. Our model introduces a decaying violation budget, which permits limited constraint violations early in the learning process and gradually enforces stricter compliance over time. We present the Budgeted Upper Confidence Bound (UCB) algorithm, which adaptively balances performance optimization and compliance with time-varying constraints. We provide theoretical guarantees showing that Budgeted UCB achieves sublinear regret and logarithmic constraint violations over the learning horizon. Extensive simulations in a wireless communication setting show that our approach achieves faster adaptation and better constraint satisfaction than standard online learning methods. These results highlight the framework’s potential for building adaptive, resource-aware IoT systems.

Federated learning (FL) has emerged as a popular distributed machine-learning paradigm. It involves many rounds of iterative communication between nodes to exchange model parameters. With the increasing complexity of ML tasks, the models can be large, having millions of parameters. Moreover, edge and IoT nodes often have limited energy resources and channel bandwidths. Thus, reducing the communication cost in FL is a bottleneck problem. This cost could be in terms of energy consumed, delay involved, or amount of data communicated. We propose a communication cost-adaptive model sparsification for FL with error compensation. The central idea is to adapt the sparsification level in run-time by optimizing the ratio between the impact of the communicated model parameters and communication cost. We carry out a detailed convergence analysis to establish the theoretical foundations of the proposed algorithm. We conduct extensive experiments to train both convex and nonconvex machine learning models on a standard dataset. We illustrate the efficiency of the proposed algorithm by comparing its performance with three baseline schemes. The performance of the proposed algorithm is validated for two communication models and three cost functions. Simulation results show that the proposed algorithm needs a substantially less amount of communication than the three baseline schemes while achieving the best accuracy and fastest convergence. The results are consistent for all the considered cost models, cost functions, and ML models. Thus, the proposedFL-CATEalgorithm can substantially improve the communication efficiency of FL, irrespective of the ML tasks, costs, and communication models.

Data similarity assumptions have traditionally been relied upon to understand the convergence behaviors of federated learning methods. Unfortunately, this approach often demands fine-tuning step sizes based on the level of data similarity. When data similarity is low, these small step sizes result in an unacceptably slow convergence speed for federated methods. In this paper, we present a novel and unified framework for analyzing the convergence of federated learning algorithms without the need for data similarity conditions. Our analysis centers on an inequality that captures the influence of step sizes on algorithmic convergence performance. By applying our theorems to well-known federated algorithms, we derive precise expressions for three widely used step size schedules: fixed, diminishing, and step-decay step sizes, which are independent of data similarity conditions. Finally, we conduct comprehensive evaluations of the performance of these federated learning algorithms, employing the proposed step size strategies to train deep neural network models on benchmark datasets under varying data similarity conditions. Our findings demonstrate significant improvements in convergence speed and overall performance, marking a substantial advancement in federated learning research.

Abstract Reinforcement learning (RL) is crucial for data science decision-making but suffers from sample inefficiency, particularly in real-world scenarios with costly physical interactions. This paper introduces a novel human-inspired framework to enhance the RL algorithm’s sample efficiency. It achieves this by initially exposing the learning agent to simpler tasks that progressively increase in complexity, ultimately leading to the main task. This method requires no pre-training and involves learning simpler tasks for just one episode. The resulting knowledge can facilitate various transfer learning approaches, such as value and policy transfer, without increasing computational complexity. It can be applied across different goals, environments, and RL algorithms, including value-based, policy-based, tabular, and deep RL methods. Experimental evaluations demonstrate the framework’s effectiveness in enhancing sample efficiency, especially in challenging main tasks, demonstrated through both a simple random walk and more complex optimal control problems with constraints.

Despite the huge success of reinforcement learning (RL) in solving many difficult problems, its Achilles heel has always been sample inefficiency. On the other hand, in RL, taking advantage of prior knowledge, intentionally or unintentionally, has usually been avoided, so that, training an agent from scratch is common. This not only causes sample inefficiency but also endangers safety –especially during exploration. In this paper, we help the agent learn from the environment by using the pre-existing (but not necessarily exact or complete) solution for a task. Our proposed method can be integrated with any RL algorithm developed based on policy gradient and actor-critic methods. The results on five tasks with different difficulty levels by using two well-known actor-critic-based methods as the backbone of our proposed method (SAC and TD3) show our success in greatly improving sample efficiency and final performance. We have gained these results alongside robustness to noisy environments at the cost of just a slight computational overhead, which is negligible.

Agile satellite imaging scheduling plays a vital role in improving emergency response, urban planning, national defense, and resource management. With the rise in the number of in-orbit satellites and observation windows, the need for diverse agile Earth observation satellite (AEOS) scheduling has surged. However, current research seldom addresses multiple optimization objectives, which are crucial in many engineering practices. This article tackles a multiobjective AEOS scheduling problem (MOAEOSSP) that aims to optimize total observation task profit, satellite energy consumption, and load balancing. To address this intricate problem, we propose a strategy-fused multiobjective dung beetle optimization (SFMODBO) algorithm. This novel algorithm harnesses the position update characteristics of various dung beetle populations and integrates multiple high-adaptability strategies. Consequently, it strikes a better balance between global search capability and local exploitation accuracy, making it more effective at exploring the solution space and avoiding local optima. The SFMODBO algorithm enhances global search capabilities through diverse strategies, ensuring thorough coverage of the search space. Simultaneously, it significantly improves local optimization precision by fine-tuning solutions in promising regions. This dual approach enables more robust and efficient problem-solving. Simulation experiments confirm the effectiveness and efficiency of the SFMODBO algorithm. Results indicate that it significantly outperforms competitors across multiple metrics, achieving superior scheduling schemes. In addition to these enhanced metrics, the proposed algorithm also exhibits advantages in computation time and resource utilization. This not only demonstrates the algorithm’s robustness but also underscores its efficiency and speed in solving the MOAEOSSP.

Some people with a motor disability that limits hand movements use technology to draw via their eyes. Free-eye drawing has been re-investigated recently and yielded state-of-the-art results via unimodal gaze control. However, limitations remain, including limited functions, conflicts between observation and drawing, and the brush tailing issue. We introduce a professional unimodal gaze control free-eye drawing application and improve upon free-eye drawing by extended gaze-based user interface functions, improved brush dynamics, and a double-blink gaze gesture. An experiment and a field study were conducted to assess the system’s usability compared to the mainstream gaze-control drawing method and hand drawing and the accessibility among users with motor disabilities. The results showed that the application provides efficient interaction and the ability to create hand-sketch-level graphics for people with motor disabilities. Herein, we contribute a robust and professional free-eye drawing application, detailing valuable design considerations for future developments in gaze interaction.

Federated learning is an emerging machine-learning technique that trains an algorithm across multiple decentralized edge devices or clients holding local data samples. It involves training local models on local data and uploading model parameters to a server node at regular intervals to generate a global model which is transmitted to all clients. However, edge nodes often have limited energy resources, and hence performing energy-efficient communication of model parameters is a bottleneck problem. We propose an energy-adaptive model sparsification for Federated Learning. The central idea is to adapt the sparsification level in run-time by optimizing the ratio between information content and energy cost. We illustrate the efficiency of the proposed algorithm by comparing its performance with three baseline schemes. We validate the performance of the proposed algorithm for two cost models. Simulation results show that the proposed algorithm needs exponentially less amount of communication and energy as compared to the three baseline schemes while achieving the best accuracy and fastest convergence.

We propose a delay-agnostic asynchronous coordinate update algorithm (DEGAS) for computing operator fixed points, with applications to asynchronous optimization. DEGAS includes novel asynchronous variants of ADMM and block-coordinate descent as special cases. We prove that DEGAS converges with both bounded and unbounded delays under delay-free parameter conditions. We also validate by theory and experiments that DEGAS adapts well to the actual delays. The effectiveness of DEGAS is demonstrated by numerical experiments on classification problems.

EyeCompass is a novel free-eye drawing system enabling high-fidelity and efficient free-eye drawing through unimodal gaze control, addressing the bottlenecks of gaze-control drawing. EyeCompass helps people to draw using only their eyes, which is of value to people with motor disabilities. Currently, there is no effective gaze-control drawing application due to multiple challenges including involuntary eye movements, conflicts between visuomotor transformation and ocular observation, gaze trajectory control, and inherent eye-tracking errors. EyeCompass addresses this using two initial gaze-control drawing mechanisms: brush damping dynamics and the gaze-oriented method. The user experiments compare the existing gaze-control drawing method and EyeCompass, showing significant improvements in the drawing performance of the mechanisms concerned. The field study conducted with motor-disabled people produced various creative graphics and indicates good usability of the system. Our studies indicate that EyeCompass is a high-fidelity, accurate, feasible free-eye drawing method for creating artistic works via unimodal gaze control.

Synthesizing realistic clouds is a complex and demanding task, as clouds are characterized by random shapes, complex scattering and turbulent appearances. Existing approaches either employ two-dimensional image matting or three-dimensional physical simulations. This paper proposes a novel sketch-to-image deep learning system using fast sketches to paint and edit volumetric clouds. We composed a dataset of 2000 real cloud images and translated simple strokes into authentic clouds based on a conditional generative adversarial network (cGAN). Compared to previous cloud simulation methods, our system demonstrates more efficient and straightforward processes to generate authentic clouds for computer graphics, providing a widely accessible sky scene design approach for use by novices, amateurs, and expert artists.

Federated learning is an emerging framework for collaborative machine-learning on devices which do not want to share local data. State-of-the art methods in federated learning reduce the communication frequency, but are not guaranteed to converge to the optimal model parameters. These methods also experience a communication bottleneck, especially when the devices are power-constrained and communicate over a shared medium. This paper presents ECO-FedSplit, an algorithm that increases the communication efficiency of federated learning without sacrificing solution accuracy. The key is to compress inter-device communication and to compensate for information losses in a theoretically justified manner. We prove strong convergence properties of ECO-FedSplit on strongly convex optimization problems and show that the algorithm yields a highly accurate solution with dramatically reduced communication. Extensive numerical experiments validate our theoretical result on real data sets.

In scalable machine learning systems, model training is often parallelized over multiple nodes that run without tight synchronization. Most analysis results for the related asynchronous algorithms use an upper bound on the information delays in the system to determine learning rates. Not only are such bounds hard to obtain in advance, but they also result in unnecessarily slow convergence. In this paper, we show that it is possible to use learning rates that depend on the actual time-varying delays in the system. We develop general convergence results for delay-adaptive asynchronous iterations and specialize these to proximal incremental gradient descent and block coordinate descent algorithms. For each of these methods, we demonstrate how delays can be measured on-line, present delay-adaptive step-size policies, and illustrate their theoretical and practical advantages over the state-of-the-art.

With the increasing scale of machine learning tasks, it has become essential to reduce the communication between computing nodes. Early work on gradient compression focused on the bottleneck between CPUs and GPUs, but communication-efficiency is now needed in a variety of different system architectures, from high-performance clusters to energy-constrained IoT devices. In the current practice, compression levels are typically chosen before training and settings that work well for one task may be vastly suboptimal for another dataset on another architecture. In this paper, we propose a flexible framework which adapts the compression level to the true gradient at each iteration, maximizing the improvement in the objective function that is achieved per communicated bit. Our framework is easy to adapt from one technology to the next by modeling how the communication cost depends on the compression level for the specific technology. Theoretical results and practical experiments indicate that the automatic tuning strategies significantly increase communication efficiency on several state-of-the-art compression schemes.

Noise is inherited in many optimization methods such as stochastic gradient methods, zeroth-order methods and compressed gradient methods. For such methods to converge toward a global optimum, it is intuitive to use large step-sizes in the initial iterations when the noise is typically small compared to the algorithm-steps, and reduce the step-sizes as the algorithm progresses. This intuition has been con-firmed in theory and practice for stochastic gradient methods, but similar results are lacking for other methods using approximate gradients. This paper shows that the diminishing step-size strategies can indeed be applied for a broad class of noisy gradient methods. Unlike previous works, our analysis framework shows that such step-size schedules enable these methods to enjoy an optimal $\mathcal{O}(1/k)$ rate. We exemplify our results on zeroth-order methods and stochastic compression methods. Our experiments validate fast convergence of these methods with the step decay schedules.

The convergence of stochastic gradient descent is highly dependent on the step-size, especially on non-convex problems such as neural network training. Step decay step-size schedules (constant and then cut) are widely used in practice because of their excellent convergence and generalization qualities, but their theoretical properties are not yet well understood. We provide convergence results for step decay in the non-convex regime, ensuring that the gradient norm vanishes at an $\mathcal{O}(\ln T/\sqrt{T})$ rate. We also provide near-optimal (and sometimes provably tight) convergence guarantees for general, possibly non-smooth, convex and strongly convex problems. The practical efficiency of the step decay step-size is demonstrated in several large-scale deep neural network training tasks.

One of the main advantages of second-order methods in a centralized setting is that they are insensitive to the condition number of the objective function's Hessian. For applications such as regression analysis, this means that less pre-processing of the data is required for the algorithm to work well, as the ill-conditioning caused by highly correlated variables will not be as problematic. Similar condition number independence has not yet been established for distributed methods. In this paper, we analyze the performance of a simple distributed second-order algorithm on quadratic problems and show that its convergence depends only logarithmically on the condition number. Our empirical results indicate that the use of second-order information can yield large efficiency improvements over first-order methods, both in terms of iterations and communications, when the condition number is of the same order of magnitude as the problem dimension.

The veritable scale of modern data necessitates information compression in parallel/distributed big-data optimization. Compression schemes using memory-based error compensation have displayed superior performance in practice, however, to date there are no theoretical explanations for these observed advantages. This paper provides the first theoretical support for why such compression schemes yields higher accuracy solutions in optimization. Our results cover both gradient and incremental gradient algorithms for quadratic optimization. Unlike previous works, our theoretical results explicitly quantify the accuracy gains from error compensation, especially for ill-conditioned problems. Finally, the numerical results on linear least-squares problems validate the benefit of error compensation and demonstrate tightness of our convergence guarantees.