Spatially distributed cavity (SDC) lasers are a promising technology for simultaneous light information and power transfer (SLIPT), offering benefits such as increased mobility and intrinsic safety, which are advantageous for various Internet of Things (IoT) devices. However, achieving beam transmission over meter-level long working distances presents significant challenges from cavity stability constraints, manufacturing/ assembly tolerances, and diffraction losses. This paper conducts a theoretical investigation of the fundamental restrictions limiting long-range resonant beam generation. We investigate cavity stability and beam characteristics, and propose a binary-search-based Monte Carlo simulation algorithm as well as a linear approximation algorithm to quantify the maximum acceptable tolerances for stable operation. Numerical results indicate that the stable region contracts sharply as distance increases. For fixed-component systems, an acceptable tolerance of 0.01 mm restricts the achievable transmission distance to less than 2 m. To address this limitation, we also prove the feasibility of long-range beam formation using precision adjustable elements, paving the way for advanced engineering applications. Experimental results verified this assumption, demonstrating that by tuning the stable region during assembly, the transmission distance could be extended to 2.8 m. This work provides essential theoretical insights and practical design guidelines for realizing stable, long-range SDC systems.

Sustainable energy supply and high-speed communications are two significant needs for the upcoming 6G applications. This paper introduces a self-aligning resonant beam system for simultaneous light information and power transfer (SLIPT), employing a novel coupled spatially distributed resonator (CSDR). The system utilizes a resonant beam for efficient power delivery and a second-harmonic beam for concurrent data transmission, inherently minimizing echo interference and enabling bidirectional communication. Through comprehensive analyses, we investigate the CSDR’s stable region, beam evolution, and power characteristics in relation to working distance and device parameters. Numerical simulations validate the CSDR-SLIPT system’s feasibility by identifying a stable beam waist location for achieving accurate mode-match coupling between two spatially distributed resonant cavities and demonstrating its operational range and efficient power delivery across varying distances. The research reveals the system’s benefits in terms of both safety and energy transmission efficiency. We also demonstrate the trade-off among the reflectivities of the cavity mirrors in the CSDR. Besides, an experiment was conducted to verified the feasibility of self-aligning beam generation and safety under the designed structure. These findings offer valuable design insights for resonant beam systems, advancing SLIPT with significant potential for remote device connectivity.

This article addresses the switching law design problem for switched nonlinear time-delay systems (SNTDSs). The existing switching laws, such as dwell time, average dwell time (ADT), and mode-dependent ADT (MDADT), depict the switching frequency by linear functions of switching interval length, which may insufficiently characterize the switching numbers and features of SNTDSs. To effectively ensure the system stability of SNTDSs and relax the conservatism of stability criteria, two novel switching laws, average switching density and mode-dependent average switching density (MDASD), are first proposed to illustrate the switching frequency of SNTDSs. Meanwhile, under the new switching laws, by constructing the proper multiple Lyapunov-Razumikhin functions, relaxed integral inequalities, and the trajectory-based approach, stability criteria are presented for SNTDSs, which can encompass and include certain aspects of prior research. Moreover, we apply the new switching laws and theoretical results to switched neural networks. Ultimately, we present two examples to confirm the effectiveness of the approaches we have developed.

Multiobjective optimization (MOO) is important for deep learning applications with multiple conflicting objectives. Pareto front learning (PFL) methods learn a single model conditioned on the preference of objectives and can be applied to any preference at inference time. However, existing PFL methods use predefined strategies (e.g., uniform sampling) to generate preferences, which could result in unevenly spaced solutions since the shape of Pareto front is largely ignored. In this article, we propose a lightweight and model-agnostic method to train a preference generator for a given PFL model, which learns to generate proper preferences from uniformly sampled ones, such that the resulting solutions are evenly spaced on the Pareto front. Compared to previous works, our method enables a more rational allocation of preferences, which can either be utilized to enhance a pretrained PFL model or be seamlessly integrated into the PFL training process to improve efficiency. We apply our method to state-of-the-art PFL methods with various backbones (e.g., multilayer perceptron, convolutional neural network, transformer) and validate the significance of preference generation across various tasks, from multitask supervised learning to multiobjective reinforcement learning-based neural combinatorial optimization. Experimental results show that our method improves the backbone algorithm in most settings, showing its effectiveness and general applicability.

Imitation learning (IL) has proven effective for enabling robots to acquire visuomotor skills through expert demonstrations. However, traditional IL methods are limited by their reliance on high-quality, often scarce, expert data, and suffer from covariate shift. To address these challenges, recent advances in offline IL have incorporated suboptimal, unlabeled datasets into the training. In this paper, we propose a novel approach to enhance policy learning from mixed-quality offline datasets by leveraging task-relevant trajectory fragments and rich environmental dynamics. Specifically, we introduce a state-based search framework that stitches state-action pairs from imperfect demonstrations, generating more diverse and informative training trajectories. Experimental results on standard IL benchmarks and real-world robotic tasks showcase that our proposed method significantly improves both generalization and performance. The code is available at https://github.com/BIT-KAUIS/SBR.

A critical bottleneck in deep reinforcement learning (DRL) is sample inefficiency, as training high-performance agents often demands extensive environmental interactions. Model-based reinforcement learning (MBRL) mitigates this by building world models that simulate environmental dynamics and generate synthetic experience, improving sample efficiency. However, conventional world models process observations holistically, failing to decouple dynamic objects and temporal features from static backgrounds. This approach is computationally inefficient, especially for visual tasks where dynamic objects significantly influence rewards and decision-making performance. To address this, we introduce DyMoDreamer, a novel MBRL algorithm that incorporates a dynamic modulation mechanism to improve the extraction of dynamic features and enrich the temporal information. DyMoDreamer employs differential observations derived from a novel inter-frame differencing mask, explicitly encoding object-level motion cues and temporal dynamics. Dynamic modulation is modeled as stochastic categorical distributions and integrated into a recurrent state-space model (RSSM), enhancing the model's focus on reward-relevant dynamics. Experiments demonstrate that DyMoDreamer sets a new state-of-the-art on the Atari $100$k benchmark with a $156.6$\% mean human-normalized score, establishes a new record of $832$ on the DeepMind Visual Control Suite, and gains a $9.5$\% performance improvement after $1$M steps on the Crafter benchmark.

Manufacturing industries play a pivotal role in global economies. It is a powerful driver of economic growth, creating a significant number of job opportunities and providing support for innovation and technological development. In the present world, where environmental concerns and resource conservation are paramount, reducing energy consumption while ensuring high processing quality is a significant challenge. During production, controlling the on/off of the machine reasonably can reduce energy consumption. This paper explores a method to analyze the performance of production lines containing buffers of finite capacity and machines that can be switched on and off when the specific conditions are met. We focus on finite production processes where the number of products is limited. In this paper, we establish a mathematical model for a three-machine production line and propose analytical methods for calculating production performance indicators. The analysis of the three-machine line employs the Markov method. Additionally, an aggregation method is introduced to extend the analysis to multi-machine systems, yielding promising results through numerical experiments. This research contributes to the advancement of production systems, improving a sustainable and competitive future for the manufacturing industry. Note to Practitioners—Small-batch production refers to the production of a small quantity of products, which can achieve customized order processing and improve resource utilization. Due to the dynamic nature of such processes, the traditional steady-state analysis method may not be applicable. This paper proposes an analytical method to predict the dynamic behavior of small batch production systems with Bernoulli machines that can be switch-on/off controlled and with finite capacity buffers. The algorithm developed in this paper can be used and support the production managers and engineers to predict the dynamic performance of the system with high accuracy. It can assist in decision-making in production control activities.

This paper investigates an air-ground cooperative monitoring problem for spatiotemporal fields, such as air pollution, forest fires, oil spills, etc, with unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs). To fully exploit complementarities of these heterogeneous vehicles and improve efficiency of the cooperative monitoring, we design a novel cooperation scheme: each UAV is assigned to loiter over and transmit its observations to a pre-allocated UGV, and the UGV provides guidance on informative trajectories for the UAV and aims to reach a target position as fast as possible. Such a scheme brings challenges to informative trajectory planning of the UGVs, lying in the delayed observations from the UAVs and the cumulative information constraint depending on the unknown field. To overcome them, this work proposes a model-free reinforcement learning (RL)-based trajectory planning method to learn continuous policies for the UGVs, where a field estimator is designed for each UGV to recover observability of the field. In addition, we derive model predictive control (MPC)-based trajectory planners for the UAVs with tailored reference positions, where the uncertain tracking errors can be handled by the RL-based method of the UGVs. Thus, a performance coupling problem of the heterogeneous vehicles is tackled. Simulations illustrate the effectiveness of the proposed trajectory planning methods and the efficiency of the air-ground cooperative monitoring scheme.Note to Practitioners—This article is motivated by cooperative monitoring tasks with UAVs and UGVs in practical applications, such as environmental monitoring, search and rescue after disasters, etc. Due to the complex dynamics of spatiotemporal fields in these tasks, trajectory planning for the cooperative monitoring system is challenging and requires much computations. To resolve the issues, we propose a novel cooperation scheme in this article, where the large computational capability of the UGVs is utilized to solve a minimum-time trajectory planning problem under a cumulative information constraint, and the UAVs only loiter over and transmit measurements about the field to the UGVs. To achieve this scheme, RL-based and MPC-based trajectory planning methods are proposed for the UGVs and the UAVs, respectively. Simulations have validated the effectiveness of the proposed trajectory planning methods and good performance of the cooperative monitoring system.

Robotic grasping is a crucial topic in robotics and computer vision, with broad applications in industrial production and intelligent manufacturing. Although some methods have begun addressing instance-level grasping, most remain limited to predefined instances and categories, lacking flexibility for open-vocabulary grasp prediction based on user-specified instructions. To address this, we propose RoG-SAM, a language-driven, instance-level grasp detection framework built on Segment Anything Model (SAM). RoG-SAM utilizes open-vocabulary prompts for object localization and grasp pose prediction, adapting SAM through transfer learning with encoder adapters and multi-head decoders to extend its segmentation capabilities to grasp pose estimation. Experimental results show that RoG-SAM achieves competitive performance on single-object datasets (Cornell and Jacquard) and cluttered datasets (GraspNet-1Billion and OCID), with instance-level accuracies of 91.2% and 90.1%, respectively, while using only 28.3% of SAM's trainable parameters. The effectiveness of RoG-SAM was also validated in real-world environments. A demonstration video is available athttps://www.youtube.com/playlist?list=PL7et4nGJAImLGytsJbglGbXl1hacA2dy_.

Distributed stochastic optimization (DSO) with local set constraints and coupled inequality constraints over a multiagent network is considered in this article. Usually, such problems are tackled by projected primal-dual methods, which require expensive projection operations when set constraints are complicated. In this context, this article focuses on the Frank-Wolfe (FW) framework, which provides computational simplicity by avoiding expensive projection operations, for solving DSO with local set and coupled inequality constraints. By combining recursive momentum and weighted averaging, this article proposes a distributed stochastic FW primal-dual algorithm (DSFWPD), which is the first stochastic FW solver for DSO problems with coupled constraints. The proposed algorithm achieves zero constraint violation on average with a sublinear decay of the optimality gap over a directed and time-varying network. The efficacy of DSFWPD is demonstrated by several numerical experiments.

This article studies the trajectory planning problem of an autonomous vehicle for exploring a spatiotemporal field subject to a constraint on cumulative information. Since the resulting problem depends on the signal strength distribution of the field, which is unknown in practice, we advocate the use of a model-free reinforcement learning (RL) method to find the solution. Given the vehicle's dynamical model, a critical (and open) question is how to judiciously merge the model-based optimality conditions into the model-free RL framework for improved efficiency and generalization, for which this work provides some positive results. Specifically, we discretize the continuous action space by leveraging analytic optimality conditions for the minimum-time optimization problem via Pontryagin's minimum principle (PMP). This allows us to develop a novel discrete PMP-based RL trajectory planning algorithm, which learns a planning policy faster than those based on a continuous action space. Simulation results: 1) validate the effectiveness of the PMP-based RL algorithm and 2) demonstrate its advantages, in terms of both learning efficiency and the vehicle's exploration time, over two baseline methods for continuous control inputs.

To meet the growing need for customized products, flexible production systems are gaining widespread application in modern factories. The inherent complexity of these flexible systems poses new challenges for production management. In this paper, we intend to contribute an operation-level analytical model for such systems, thus providing an effective evaluation tool for productivity and energy consumption analysis. Specifically, a multi-type serial production line with machines obeying the geometric reliability model and intermediate buffers having limited capacity is considered, in which K batches of parts are processed sequentially in each shift. Analytical solutions for key operation metrics are first developed for small systems using a Markov model. For multi-machine lines, a semi-analytical method based on aggregation is proposed. The effectiveness of the proposed analysis method is verified through simulation experiments. Besides, system properties are analyzed to provide insights for continuous improvement and optimization.

This article investigates the event-triggered switching control (ETSC) of switched nonlinear time-delay systems (SNTDSs) with asynchronous switching. First, we study the input-to-state stability (ISS) and integral ISS (iISS) for SNTDSs with asynchronous switching, where switching instants are generated based on the designed event-triggered mechanism. Among existing works on the ETSC, systems behavior at event-triggered instants is neglected. In fact, whenever an event is triggered, the systems mode will jump suddenly such that a switch is imposed to the systems, leading to the change of subsystems. Moreover, asynchronous switching behavior may occur between the actual subsystem and its corresponding controller. These facts bring great challenges for the event-triggered mechanism design and ISS analysis. To tackle these problems, a new Lyapunov-based event-triggered mechanism with adjustable parameters is designed to establish the relationship between systems switches and event triggers, and exclude the Zeno phenomenon. The analysis of the asynchronous switching behavior can be implemented and some ISS and iISS criteria of SNTDSs are derived utilizing the merging switching technique. Finally, two numerical examples, including a practical stirred tank reactor system, are presented to show the validity of the proposed methods.

This article investigates the input-to-state stability and integral input-to-state stability for the switched nonlinear neutral systems (SNNSs) with multiple time-varying delays (MTVDs) and asynchronous switching. According to the fact that the switching signals of the controllers and the subsystems are inconsistent, novel stability criteria on the input-to-state stability and integral input-to-state stability properties for SNNSs with MTVDs and the asynchronous switching phenomenon are presented by multiple Lyapunov-Krasovskii functionals, the merging switching signal, and the mode-dependent average dwell time technique. Compared with the existing related works, our results are less conservative. Meanwhile, our proposed works not only investigate the effects of switches and neutral terms on the stability property for SNNSs but also study the case of the systems with MTVDs and asynchronous switching. Finally, two practical examples, including the practical coupled mass-spring-damper-pendulum system, are presented to show the effectiveness of our results.

This article studies the informative trajectory planning problem of an autonomous vehicle for field exploration. In contrast to existing works concerned with maximizing the amount of information about spatial fields, this work considers efficient exploration of spatiotemporal fields with unknown distributions and seeks minimum-time trajectories of the vehicle while respecting a cumulative information constraint. In this work, upon adopting the observability constant as an information measure for expressing the cumulative information constraint, the existence of a minimum-time trajectory is proven under mild conditions. Given the spatiotemporal nature, the problem is modeled as a Markov decision process (MDP), for which a reinforcement learning (RL) algorithm is proposed to learn a continuous planning policy. To accelerate the policy learning, we design a new reward function by leveraging field approximations, which is demonstrated to yield dense rewards. Simulations show that the learned policy can steer the vehicle to achieve an efficient exploration, and it outperforms the commonly-used coverage planning method in terms of exploration time for sufficient cumulative information.

Flexible manufacturing has given rise to complex scheduling problems such as the flexible job shop scheduling problem (FJSP). In FJSP, operations can be processed on multiple machines, leading to intricate relationships between operations and machines. Recent works have employed deep reinforcement learning (DRL) to learn priority dispatching rules (PDRs) for solving FJSP. However, the quality of solutions still has room for improvement relative to that by the exact methods such as OR-Tools. To address this issue, this article presents a novel end-to-end learning framework that weds the merits of self-attention models for deep feature extraction and DRL for scalable decision-making. The complex relationships between operations and machines are represented precisely and concisely, for which a dual-attention network (DAN) comprising several interconnected operation message attention blocks and machine message attention blocks is proposed. The DAN exploits the complicated relationships to construct production-adaptive operation and machine features to support high-quality decision-making. Experimental results using synthetic data as well as public benchmarks corroborate that the proposed approach outperforms both traditional PDRs and the state-of-the-art DRL method. Moreover, it achieves results comparable to exact methods in certain cases and demonstrates favorable generalization ability to large-scale and real-world unseen FJSP tasks.

Recently, model-based reinforcement learning algorithms have demonstrated remarkable efficacy in visual input environments. These approaches begin by constructing a parameterized simulation world model of the real environment through self-supervised learning. By leveraging the imagination of the world model, the agent's policy is enhanced without the constraints of sampling from the real environment. The performance of these algorithms heavily relies on the sequence modeling and generation capabilities of the world model. However, constructing a perfectly accurate model of a complex unknown environment is nearly impossible. Discrepancies between the model and reality may cause the agent to pursue virtual goals, resulting in subpar performance in the real environment. Introducing random noise into model-based reinforcement learning has been proven beneficial. In this work, we introduce Stochastic Transformer-based wORld Model (STORM), an efficient world model architecture that combines the strong sequence modeling and generation capabilities of Transformers with the stochastic nature of variational autoencoders. STORM achieves a mean human performance of $126.7\%$ on the Atari $100$k benchmark, setting a new record among state-of-the-art methods that do not employ lookahead search techniques. Moreover, training an agent with $1.85$ hours of real-time interaction experience on a single NVIDIA GeForce RTX 3090 graphics card requires only $4.3$ hours, showcasing improved efficiency compared to previous methodologies.

The present paper considers the model-based and data-driven control of unknown discrete-time linear systems under event-triggering and self-triggering transmission schemes. To this end, we begin by presenting a dynamic event-triggering scheme (ETS) based on periodic sampling, and a discrete-time looped-functional approach, through which a model-based stability condition is derived. Combining the model-based condition with a recent data-based system representation, a data-driven stability criterion in the form of linear matrix inequalities (LMIs) is established, which also offers a way of co-designing the ETS matrix and the controller. To further alleviate the sampling burden of ETS due to its continuous/periodic detection, a self-triggering scheme (STS) is developed. Leveraging precollected input-state data, an algorithm for predicting the next transmission instant is given, while achieving system stability. Finally, numerical simulations showcase the efficacy of ETS and STS in reducing data transmissions as well as practicality of the proposed co-design methods.

This article studies the adaptive output-feedback consensus control problem of nonlinear multiagent systems (MASs) against denial-of-service (DoS) attacks. The attacks on the edges instead of nodes are considered, where we allow different attack intensities but at least one edge is connected in each attacking interval. Affected by output disturbance, the sensor feedback signal of every agent is inaccurate, which will reduce the approximation accuracy of the observer. Then, we design a signal to revise the sensor feedback signal subject to disturbance. Meanwhile, a prescribed performance function is used to ensure the transient and steady-state performance of error. Leveraging the Lyapunov stability theory and the backstepping technique, a distributed output-feedback control scheme subject to asymmetric saturation nonlinearity is designed. For the asymmetric input saturation, an auxiliary signal is designed to simplify the designed progress of controller input. To deal with the inherent problem of "explosion of complexity" emerging with backstepping, dynamic surface control is utilized. It is proved that the consensus errors converge to small neighborhoods of the origin, and all signals within the closed-loop system are bounded. Finally, simulation results are offered to demonstrate the effectiveness of the proposed method.

Deep neural networks (DNNs) have well-documented merits in learning nonlinear functions in high-dimensional spaces. Stochastic gradient descent (SGD)-type optimization algorithms are the ‘workhorse’ for training DNNs. Nonetheless, such algorithms often suffer from slow convergence, sizable fluctuations, and abundant local solutions, to name a few. In this context, the present paper draws ideas from adaptive control of dynamical systems, and develops an adaptive proportional-integral-derivative (AdaPID) solver for fast, stable, and effective training of DNNs. AdaPID relies on second-order moment estimates of gradients to adaptively adjust the PID coefficients. Numerical tests corroborate the merits of AdaPID on several tasks such as image generation using generative adversarial networks (GANs) and image classification using convolutional neural networks (CNNs) as well as long-short term memories (LSTMs).

In this article, the problem of distributed synchronization of networked systems with actuator bias faults is investigated. To effectively use the limited network bandwidth and avoid the requirement of global information, a novel adaptive event-triggered state feedback controller and a dynamic triggering law are designed jointly by employing a projection operator approach. The proposed synchronization scheme is different from existing ones that have focused on designing controllers and triggering laws independently. Besides, our scheme is extended to design an observer-based distributed adaptive event-triggered controller and corresponding dynamic triggering law when the system states are unmeasurable. Theoretical analysis shows that under the two different distributed event-triggered synchronization schemes, the following three results can be obtained: 1) fully distributed synchronization can be achieved without knowing global information associated with the underlying communication topology and node's scale; 2) continuous communication among adjacent nodes can be avoided for both designed controllers and dynamic triggering laws; and 3) exclusion of Zeno phenomenon is shown by contradiction. Finally, the effectiveness of the proposed algorithms is verified through three numerical examples.

For nonzero-sum (NZS) games of nonlinear systems, reinforcement learning (RL) or adaptive dynamic programming (ADP) has shown its capability of approximating the desired index performance and the optimal input policy iteratively. In this article, an event-triggered ADP is proposed for NZS games of continuous-time nonlinear systems with completely unknown system dynamics. To achieve the Nash equilibrium solution approximately, the critic neural networks and actor neural networks are utilized to estimate the value functions and the control policies, respectively. Compared with the traditional time-triggered mechanism, the proposed algorithm updates the neural network weights as well as the inputs of players only when a state-based event-triggered condition is violated. It is shown that the system stability and the weights' convergence are still guaranteed under mild assumptions, while occupation of communication and computation resources is considerably reduced. Meanwhile, the infamous Zeno behavior is excluded by proving the existence of a minimum inter-event time (MIET) to ensure the feasibility of the closed-loop event-triggered continuous-time system. Finally, a numerical example is simulated to illustrate the effectiveness of the proposed approach.

Sequential Recommendation aims to recommend items that a target user will interact with in the near future based on the historically interacted items. While modeling temporal dynamics is crucial for sequential recommendation, most of the existing studies concentrate solely on the user side while overlooking the sequential patterns existing in the counterpart, i.e., the item side. Although a few studies investigate the dynamics involved in the dual sides, the complex user-item interactions are not fully exploited from a global perspective to derive dynamic user and item representations. In this paper, we devise a novel Dynamic Representation Learning model for Sequential Recommendation (DRL-SRe). To better model the user-item interactions for characterizing the dynamics from both sides, the proposed model builds a global user-item interaction graph for each time slice and exploit time-sliced graph neural networks to learn user and item representations. Moreover, to enable the model to capture fine-grained temporal information, we propose an auxiliary temporal prediction task over consecutive time slices based on temporal point process. Comprehensive experiments on three public real-world datasets demonstrate DRL-SRe outperforms the state-of-the-art sequential recommendation models with a large margin.

Resonant beam communications (RBCom) is capable of providing wide bandwidth when using light as the carrier. Besides, the RBCom system possesses the characteristics of mobility, high signal-to-noise ratio (SNR), and multiplexing. Nevertheless, the channel of the RBCom system is distinct from other light communication technologies due to the echo interference issue. In this article, we reveal the mechanism of the echo interference and propose the method to eliminate the interference. Moreover, we present an exemplary design based on frequency shifting and optical filtering, along with its mathematic model and performance analysis. The numerical evaluation shows that the channel capacity is greater than 15 b/s/Hz.

Nonlinear couplings among units (nodes) are ubiquitous in engineering systems including, e.g., radar and sonar systems, which have been ignored in most works. In this article, the problem of distributed synchronization of nonlinear networked systems with nonlinear couplings is studied. Specifically, two kinds of nodes' communication couplings including nonlinear relative and nonlinear absolute state couplings are considered. To reduce the requirements of control and communication among nodes and avoid any global network information, two edge-based fully adaptive event-triggered control protocols based on nonlinear relative and absolute state couplings are proposed by using the projection operator technique, which is followed by design of corresponding dynamic event-triggered mechanisms. The advantages of our proposed dynamic event-triggered strategies show that it can boil down to existing static ones as special examples, and the minimal inter-execution time of the proposed dynamic triggering laws is larger than that of static ones. Theoretical analysis shows that the proposed algorithm not only guarantees fully adaptive Zeno-free synchronization of networked systems without requiring any global information, but also avoids continuous communications among nodes, and considerably reduce the frequency of controller updates. Finally, the practical merits of the proposed algorithms are corroborated using a Chua's circuit network.

The work analyzes dynamic responses of a healthy plant under optimal switching data-injection attacks on sensors and develops countermeasures from the vantage point of optimal control. This is approached in a cyber-physical system setting, where the attacker can inject false data into a selected subset of sensors to maximize the quadratic cost of states and the energy consumption of the controller at a minimal effort. A 0-1 integer program is formulated, through which the adversary finds an optimal sequence of sets of sensors to attack at optimal switching instants. Specifically, the number of compromised sensors per instant is kept fixed, yet their locations can be dynamic. Leveraging the embedded transformation and mathematical programming, an analytical solution is obtained, which includes an algebraic switching condition determining the optimal sequence of attack locations (compromised sensor sets), along with an optimal state-feedback-based data-injection law. To thwart the adversary, however, a resilient control approach is put forward for stabilizing the compromised system under arbitrary switching attacks constructed based on a set of state-feedback laws, each of which corresponds to a compromised sensor set. Finally, an application using power generators in a cyber-enabled smart grid is provided to corroborate the effectiveness of the resilient control scheme and the practical merits of the theory.

Motivated by the emerging use of multi-agent reinforcement learning (MARL) in engineering applications such as networked robotics, swarming drones, and sensor networks, we investigate the policy evaluation problem in a fully decentralized setting, using temporal-difference (TD) learning with linear function approximation to handle large state spaces in practice. The goal of the group of agents is to collaboratively learn the value function of a given policy from locally private rewards observed in a shared environment, through exchanging local estimates with neighbors. Despite their simplicity and widespread use, our theoretical understanding of such decentralized TD learning algorithms remains limited. Existing results were obtained based on i.i.d. data samples, or by imposing an ‘additional’ projection step to control the ‘gradient’ bias incurred by the Markovian observations. In this paper, we provide a finite-time analysis of the fully decentralized TD(0) learning under both i.i.d. as well as Markovian samples, and prove that all local estimates converge linearly to a small neighborhood of the optimum. The resultant error bounds are the first of its type—in the sense that they hold under the most practical assumptions—which is made possible by means of a novel multi-step Lyapunov approach.

Inspired by the widespread use of Q-learning algorithms in reinforcement learning (RL), this present paper studies a class of biased stochastic approximation (SA) procedures under an ‘ergodic-like’ assumption on the underlying stochastic noise sequence. Leveraging a \emph{multistep Lyapunov function} that looks ahead to several future updates to accommodate the gradient bias, we prove a general result on the convergence of the iterates, and use it to derive finite-time bounds on the mean-square error in the case of constant stepsizes. This novel viewpoint renders the finite-time analysis of \emph{biased SA} algorithms under a broad family of stochastic perturbations possible. For direct comparison with past works, we also demonstrate these bounds by applying them to Q-learning with linear function approximation, under the realistic Markov chain observation model. The resultant finite-time error bound for Q-learning is \emph{the first of its kind}, in the sense that it holds: i) for the unmodified version (i.e., without making any modifications to the updates), and ii), for Markov chains starting from any initial distribution, at least one of which has to be violated for existing results to be applicable.

Wireless and wireline networks, such as Internet, cellular, and content delivery networks are to serve end-user file requests proactively. To this aim, by storing anticipated highly popular files during off-peak periods, and fetching them to end-users during on-peak instances, these networks smoothen out the load fluctuations on the back-haul links. In this context, several practical networks comprise a parent caching node connected to multiple leaf nodes to serve end-user file requests. To model the two-way interactive influence between caching decisions at the parent and leaf nodes, a reinforcement learning formulation is put forth in this work. Furthermore, to endow with scalability so that the algorithm can effectively handle the curse of dimensionality, a deep reinforcement learning approach is also developed. Our novel caching policy relies on a deep Q-network to enforce the parent node with ability to learn-and-adapt to unknown policies of leaf nodes as well as spatio-temporal dynamic evolution of file requests, results in remarkable caching performance, as corroborated through numerical tests.

To perform any meaningful optimization task, distribution grid operators need to know the topology of their grids. Although power grid topology identification and verification has been recently studied, discovering instantaneous interplay among subsets of buses, also known as higher-order interactions in recent literature, has not yet been addressed. The system operator can benefit from having this knowledge when re-configuring the grid in real time, to minimize power losses, balance loads, alleviate faults, or for scheduled maintenance. Establishing a connection between the celebrated exact distribution flow equations and the so-called self-driven graph Volterra model, this paper puts forth a nonlinear topology identification algorithm, that is able to reveal both the edge connections as well as their higher-order interactions. Preliminary numerical tests using real data on a 47-bus distribution grid showcase the merits of the proposed scheme relative to existing alternatives.

The present contribution deals with decentralized policy evaluation in multi-agent Markov decision processes using temporal-difference (TD) methods with linear function approximation for scalability. The agents cooperate to estimate the value function of such a process by observing continual state transitions of a shared environment over the graph of interconnected nodes (agents), along with locally private rewards. Different from existing consensus-type TD algorithms, the approach here develops a simple decentralized TD tracker by wedding TD learning with gradient tracking techniques. The non-asymptotic properties of the novel TD tracker are established for both independent and identically distributed (i.i.d.) as well as Markovian transitions through a unifying multistep Lyapunov analysis. In contrast to the prior art, the novel algorithm forgoes the limiting error bounds on the number of agents, which endows it with performance comparable to that of centralized TD methods that are the sharpest known to date.

Wireless energy transfer (WET) has recently emerged as an appealing solution for power supplying mobile/Internet of Things (IoT) devices. As an enabling WET technology, resonant beam charging (RBC) is well documented for its long-range, high-power, and safe “WiFi-like” mobile power supply. To provide high-quality wireless charging services for multiple users in a given region, we formulate a deployment problem of multiple RBC transmitters for balancing the charging fairness and quality of charging service. Based on the RBC transmitter's coverage model and receiver's charging/discharging model, a genetic algorithm (GA)-based scheme and a particle swarm optimization (PSO)-based scheme are put forth to resolve the above issue. Moreover, we present a scheduling method to evaluate the performance of the proposed algorithms. The numerical results corroborate that the optimized deployment schemes outperform uniform and random deployment in 10%-20% charging efficiency improvement.

Multiview canonical correlation analysis (MCCA) looks for shared low-dimensional representations hidden in multiple transformations of common source signals. Existing MCCA approaches do not exploit the geometry of common sources, which can be either given a priori, or constructed from do- main knowledge. In this paper, a novel graph-regularized (G) MCCA is developed to account for such geometry-bearing in- formation via graph regularization in the classical maximum- variance MCCA model. GMCCA minimizes the distance between the sought canonical variables and the common sources, while incorporating the graph-induced prior of these sources. To capture nonlinear dependencies, GMCCA is fur- ther broadened to the graph-regularized kernel (GK) MCCA. Numerical tests using real datasets document the merits of G(K)MCCA in comparison with competing alternatives.

State forecasting plays a critical role in power system monitoring, by offering system awareness even ahead of the time horizon, enhancing system observability, and providing efficient identification of the grid topology and link parameter changes. However, available approaches relying on linear estimators or single-hidden-layer feed-forward neural networks (FNNs), cannot capture long-term nonlinear dependencies in the voltage time series, and lead to suboptimal performance. To bypass these hurdles, this paper advocates deep recurrent neural networks (RNNs) for power system state forecasting. Deep RNNs capture long-term dependencies, and are easy to implement. By also leveraging the physics behind power systems, a novel architecture based on prox-linear nets (RPLN) is further developed for state forecasting based on past measurements. Simulated tests show improved performance of the proposed RNN and RPLN predictors when compared to FNN and vector autoregression based alternatives.

Internet of Things (IoT) is powering up smart cities by connecting all kinds of electronic devices. The power supply problem of IoT devices constitutes a major challenge in current IoT development, due to the poor battery endurance as well as the troublesome cable deployment. The wireless power transfer (WPT) technology has recently emerged as a promising solution. Yet, existing WPT advances cannot support free and mobile charging like Wi-Fi communications. To this end, the concept of mobile energy transfer (MET) is proposed, which relies critically on a resonant beam charging (RBC) technology. The adaptive (A) RBC technology builds on RBC, but aims at improving the charging efficiency by charging devices at device preferred current and voltage levels adaptively. A mobile ARBC scheme is developed relying on an adaptive source power control. Extensive numerical simulations using a 1000-mAh Li-ion battery show that the mobile ARBC outperforms simple charging schemes, such as the constant power charging, the profile-adaptive charging, and the distance-adaptive charging in saving energy.

In the envisioned smart grid, high penetration of uncertain renewables, unpredictable participation of (industrial) customers, and purposeful manipulation of smart meter readings, all highlight the need for accurate, fast, and robust power system state estimation (PSSE). Nonetheless, most real-time data available in the current and upcoming transmission/distribution systems are nonlinear in power system states (i.e., nodal voltage phasors). Scalable approaches to dealing with PSSE tasks undergo a paradigm shift toward addressing the unique modeling and computational challenges associated with those nonlinear measurements. In this study, we provide a contemporary overview of PSSE and describe the current state of the art in the nonlinear weighted least-squares and least-absolutevalue PSSE. To benchmark the performance of unbiased estimators, the Cramér-Rao lower bound is developed. Accounting for cyber attacks, new corruption models are introduced, and robust PSSE approaches are outlined as well. Finally, distribution system state estimation is discussed along with its current challenges. Simulation tests corroborate the effectiveness of the developed algorithms as well as the practical merits of the theory.

Principal component analysis (PCA) has well-documented merits for data extraction and dimensionality reduction. PCA deals with a single dataset at a time, and it is challenged when it comes to analyzing multiple datasets. Yet in certain setups, one wishes to extract the most significant information of one dataset relative to other datasets. Specifically, the interest may be on identifying or extracting features that are specific to a single target dataset but not the others. This paper presents a novel approach for such so-termed discriminative data analysis, and establishes its optimality in the least-squares sense under suitable assumptions. The criterion reveals linear combinations of variables by maximizing the ratio of the variance of the target data to that of the remainders. The novel approach solves a generalized eigenvalue problem by performing SVD just once. Numerical tests using synthetic and real datasets showcase the merits of the proposed approach relative to its competing alternatives.

This paper presents a new algorithm, termed truncated amplitude flow (TAF), to recover an unknown vector x from a system of quadratic equations of the form yi= |〈ai, x〉|2, where ai's are given random measurement vectors. This problem is known to be NP-hard in general. We prove that as soon as the number of equations is on the order of the number of unknowns, TAF recovers the solution exactly (up to a global unimodular constant) with high probability and complexity growing linearly with both the number of unknowns and the number of equations. Our TAF approach adapts the amplitude based empirical loss function and proceeds in two stages. In the first stage, we introduce an orthogonality-promoting initialization that can be obtained with a few power iterations. Stage two refines the initial estimate by successive updates of scalable truncated generalized gradient iterations, which are able to handle the rather challenging nonconvex and nonsmooth amplitude based objective function. In particular, when vectors x and ai's are real valued, our gradient truncation rule provably eliminates erroneously estimated signs with high probability to markedly improve upon its untruncated version. Numerical tests using synthetic data and real images demonstrate that our initialization returns more accurate and robust estimates relative to spectral initializations. Furthermore, even under the same initialization, the proposed amplitude-based refinement outperforms existing Wirtinger flow variants, corroborating the superior performance of TAF over state-of-the-art algorithms.

A linear-time algorithm termed SPARse Truncated Amplitude flow (SPARTA) is developed for the phase retrieval (PR) of sparse signals. Upon formulating the sparse PR as a non-convex empirical loss minimization task, SPARTA emerges as an iterative solver consisting of two components: s1) a sparse orthogonality-promoting initialization leveraging support recovery and principal component analysis; and, s2) a series of refinements by hard thresholding based truncated gradient iterations. SPARTA is simple, scalable, and fast. It recovers any k-sparse n-dimensional signal (k ≪ n) of large enough minimum (in modulus) nonzero entries from about k2log n measurements with high probability; this is achieved at computational complexity of order k2n log n, improving upon the state-of-the-art by at least a factor of k. SPARTA is robust against bounded additive noise. Simulated tests corroborate the merits of SPARTA relative to existing alternatives.

This paper deals with finding an $n$-dimensional solution $\bm{x}$ to a system of quadratic equations $y_i=|\langle\bm{a}_i,\bm{x}\rangle|^2$, $1\le i \le m$, which in general is known to be NP-hard. We put forth a novel procedure, that starts with a \emph{weighted maximal correlation initialization} obtainable with a few power iterations, followed by successive refinements based on \emph{iteratively reweighted gradient-type iterations}. The novel techniques distinguish themselves from prior works by the inclusion of a fresh (re)weighting regularization. For certain random measurement models, the proposed procedure returns the true solution $\bm{x}$ with high probability in time proportional to reading the data $\{(\bm{a}_i;y_i)\}_{1\le i \le m}$, provided that the number $m$ of equations is some constant $c>0$ times the number $n$ of unknowns, that is, $m\ge cn$. Empirically, the upshots of this contribution are: i) perfect signal recovery in the high-dimensional regime given only an \emph{information-theoretic limit number} of equations; and, ii) (near-)optimal statistical accuracy in the presence of additive noise. Extensive numerical tests using both synthetic data and real images corroborate its improved signal recovery performance and computational efficiency relative to state-of-the-art approaches.

Variabilities of renewable energy sources critically challenge contemporary power distribution grids. Depending on grid conditions, solar energy may have to be curtailed to comply with network limitations. On the other hand, smart inverters installed with solar panels enable for reactive power support at fast response rates. Existing energy management schemes may not efficiently integrate intermittent generation. Inherent operational flexibilities, such as flexible voltage regulation margins and instantaneous inverter or distribution line overloading could be judiciously exploited. To that end, an ergodic energy management framework is put forth calling for joint control of active and reactive power using smart inverters. Although tighter operational constraints are enforced in an average sense, looser margins are satisfied at all times. A stochastic dual subgradient solver is devised using an approximate linearized grid model. The algorithm is distribution free, and enjoys provable convergence. Numerical tests on a 56-bus distribution feeder demonstrate that the novel scheme yields lower energy cost upon its deterministic counterpart.

This paper puts forth a novel algorithm, termed \emph{truncated generalized gradient flow} (TGGF), to solve for $\bm{x}\in\mathbb{R}^n/\mathbb{C}^n$ a system of $m$ quadratic equations $y_i=|\langle\bm{a}_i,\bm{x}\rangle|^2$, $i=1,2,\ldots,m$, which even for $\left\{\bm{a}_i\in\mathbb{R}^n/\mathbb{C}^n\right\}_{i=1}^m$ random is known to be \emph{NP-hard} in general. We prove that as soon as the number of equations $m$ is on the order of the number of unknowns $n$, TGGF recovers the solution exactly (up to a global unimodular constant) with high probability and complexity growing linearly with the time required to read the data $\left\{\left(\bm{a}_i;\,y_i\right)\right\}_{i=1}^m$. Specifically, TGGF proceeds in two stages: s1) A novel \emph{orthogonality-promoting} initialization that is obtained with simple power iterations; and, s2) a refinement of the initial estimate by successive updates of scalable \emph{truncated generalized gradient iterations}. The former is in sharp contrast to the existing spectral initializations, while the latter handles the rather challenging nonconvex and nonsmooth \emph{amplitude-based} cost function. Numerical tests demonstrate that: i) The novel orthogonality-promoting initialization method returns more accurate and robust estimates relative to its spectral counterparts; and ii) even with the same initialization, our refinement/truncation outperforms Wirtinger-based alternatives, all corroborating the superior performance of TGGF over state-of-the-art algorithms.

Albeit being in the big data era, a significant percentage of data accrued can be overlooked while maintaining reasonable quality of statistical inference at affordable complexity. By capitalizing on data redundancy, interval censoring is leveraged here to cope with the scarcity of resources needed for data exchanging, storing, and processing. By appropriately modifying least-squares regression, first- and second-order algorithms with complementary strengths that operate on censored data are developed for large-scale regressions. Theoretical analysis and simulated tests corroborate their efficacy relative to contemporary competing alternatives.

Power system state estimation (PSSE) constitutes a crucial prerequisite for reliable operation of the power grid. A key challenge for accurate PSSE is the inherent nonlinearity of SCADA measurements in the system states. Recent proposals for static PSSE tackle this issue by exploiting hidden convexity structure and solving a semidefinite programming (SDP) relaxation. In this work, an online PSSE algorithm based on SDP relaxation is proposed, which enjoys a similar convexity advantage, while capitalizing on past measurements as well for improved performance. An online convex optimization technique is adopted to derive an efficient algorithm with strong performance guarantees. Numerical tests verify the efficacy of the proposed approach.