Subgraph neural networks have recently gained prominence for various subgraph-level predictive tasks. However, existing methods either \emph{1)} apply simple standard pooling over graph convolutional networks, failing to capture essential subgraph properties, or \emph{2)} rely on rigid subgraph definitions, leading to suboptimal performance. Moreover, these approaches fail to model long-range dependencies both between and within subgraphs—a critical limitation, as many real-world networks contain subgraphs of varying sizes and connectivity patterns. In this paper, we propose a novel implicit subgraph neural network, the first of its kind, designed to capture dependencies across subgraphs. Our approach also integrates label-aware subgraph-level information. We formulate implicit subgraph learning as a bilevel optimization problem and develop a provably convergent algorithm that requires fewer gradient estimations than standard bilevel optimization methods. We evaluate our approach on real-world networks against state-of-the-art baselines, demonstrating its effectiveness and superiority.

For completely unknown affine nonlinear systems, in this article, a synergetic learning algorithm (SLA) is developed to learn an optimal control. Unlike the conventional Hamilton-Jacobi-Bellman equation (HJBE) with system dynamics, a model-free HJBE (MF-HJBE) is deduced by means of off-policy reinforcement learning (RL). Specifically, the equivalence between HJBE and MF-HJBE is first bridged from the perspective of the uniqueness of the solution of the HJBE. Furthermore, it is proven that once the solution of MF-HJBE exists, its corresponding control input renders the system asymptotically stable and optimizes the cost function. To solve the MF-HJBE, the two agents composing the synergetic learning (SL) system, the critic agent and the actor agent, can evolve in real-time using only the system state data. By building an experience reply (ER)-based learning rule, it is proven that when the critic agent evolves toward the optimal cost function, the actor agent not only evolves toward the optimal control, but also guarantees the asymptotic stability of the system. Finally, simulations of the F16 aircraft system and the Van der Pol oscillator are conducted and the results support the feasibility of the developed SLA.

We present experiments using mobile 28 GHz Phased Array Antenna Modules (PAAMs), demonstrating their ability to perform beam steering with high granularity. The mobile node contains a 64-element IBM 28 GHz PAAM along with a USRP software defined radio, allowing for configuration of the transmit/receive (TX/RX) parameters. These parameters include beam shape, beam steering, and duty cycling. We demonstrate the capabilities of the mobile PAAMs by forming a wireless OFDM link between two mobile PAAMs. We then showcase the beam steering capabilities of the PAAM by performing beam sweeping on the RX PAAM to find the angle of arrival from the TX PAAM. A simple graphical user interface is presented for configuring the PAAMs. A tutorial is available online for users interested in experimentation with 28 GHz PAAMs*.

In this article, a real-time online off-policy reinforcement learning (RL) method is developed for the optimal control problem of unknown continuous-time nonlinear systems. First, by applying the temporal difference technique to the iterative procedure of off-policy RL, the iterative value function and the iterative policy input can be learned in real-time online. It is proven that the fitting error of neural network (NN) weights is exponentially convergent in each iteration. Second, a model-free Hamilton–Jacobi–Bellman equation (MF-HJBE) is deduced by taking the limit of the iterative procedure of off-policy RL. In this manner, it not only eliminates system dynamics in the classical HJBE, but also vanishes the iteration index. By applying temporal difference to the MF-HJBE, a real-time online tuning rule is designed to learn the optimal value function and the optimal policy input. It is proven that the fitting error of NN weights caused by the real-time online tuning rule is exponentially convergent. Note that the two online tuning rules, the iterative one and the real-time one, use only current and previous state data extracted from system trajectories. Meanwhile, it is proven using the Lyapunov’s direct method that the system solution is uniformly ultimately bounded. Finally, simulation results demonstrate the validity of the proffered method.

For event-triggered optimal control problems of general nonlinear systems, it is very difficult to obtain the optimal analytical solution. In this paper, an adaptive near optimal output feedback control method is presented for discrete-time (DT) nonlinear systems. The event-triggered mechanism is introduced to significantly reduce the execution costs through aperiodic control updating intervals without affecting system responses. Furthermore, an integral term is presented in the performance index function to achieve the optimal constrained control. Consequently, iterative dual heuristic dynamic programming algorithm (DHP) is adopted to learn the optimal control law and the costate function. Finally, two examples are provided to illustrate the effectiveness of the proposed approach.

The approximate optimal control design for continuous-time nonlinear systems with partially unknown dynamics is studied in this paper. Based on the state-dependent coefficient parameterization, the dynamics of nonlinear systems are represented in a resemble linear manner. In this case, the Hamilton-Jacobi-Bellman equation can be recast in the form of the state-dependent Riccati equation (SDRE). Based on the integral reinforcement learning, an online policy iteration algorithm is developed to iteratively solve the SDRE using status and control data. In addition, it is proved that the proposed algorithm is equivalent to the traditional iterative solution of SDRE. A suboptimal control policy can be attained under proper conditions. The iterative feedback control policy has the ability to stabilize closed-loop system. The effectiveness of the presented algorithm is validated by simulation results.

In this article, an online adaptive optimal control algorithm based on adaptive dynamic programming is developed to solve the multiplayer nonzero-sum game (MP-NZSG) for discrete-time unknown nonlinear systems. First, a model-free coupled globalized dual-heuristic dynamic programming (GDHP) structure is designed to solve the MP-NZSG problem, in which there is no model network or identifier. Second, in order to relax the requirement of systems dynamics, an online adaptive learning algorithm is developed to solve the Hamilton-Jacobi equation using the system states of two adjacent time steps. Third, a series of critic networks and action networks are used to approximate value functions and optimal policies for all players. All the neural network (NN) weights are updated online based on real-time system states. Fourth, the uniformly ultimate boundedness analysis of the NN approximation errors is proved based on the Lyapunov approach. Finally, simulation results are given to demonstrate the effectiveness of the developed scheme.