Swarm intelligence results from a collective behaviour of swarm network, which harnesses distributed and simple rules of swarm systems to address complex problems without a central controller. One potential approach to transform such swarm networks into valuable and practical computational resources is by applying the reservoir computing framework. However, technical challenges, such as permutation symmetry and instability, could emerge in these networks during the process, which significantly hinder the computational performance. In this paper, we explore the potential of mobile swarm networks in a reservoir computing framework to perform machine learning tasks. We propose an observation layer using Gaussian kernel density estimation to be inserted into the reservoir computing framework. Our approach not only addresses permutation symmetry but also stabilises swarm behaviours, resulting in a scalable swarm network. We explore variations in computational capacity across different swarm sizes and combinations with four benchmark computations. We prove the effectiveness of our observation layer in addressing permutation symmetry and discovered the improvement in performance in combining different swarm networks in parallel. We found that the best ratio between ants and birds reservoir is 8:2. The performance achieves a covariance of approximately 0.20 with a swarm size of 20, comparable to that of echo-state-network (ESN) with 16 nodes. As the swarm size increases to 60, the covariance value reaches around 0.21, matching the performance of ESN with 18 nodes. This indicates that our swarm network has a reasonable amount of memory and nonlinearly capacity in performing computation tasks. We also validate our method's effectiveness on a handwriting classification task, further highlighting its practical applicability. Our findings delve into the impacts of the swarm networks' computational abilities, offering insights into mechanisms in this alternative means of swarm intelligence and application to AI.

Torque-restricted control remains a significant challenge in robotics, often necessitating precise modeling or large amounts of data for effective controller design. To address this problem, we introduce a novel training method that utilizes a Reservoir Computing (RC) framework to serve as a model-free controller that can effectively control a nonlinear robot using minimal training. This paper explores the application of the proposed framework to a torque-restricted single pendulum and achieves similar control performance to that of model-free reinforcement learning controllers while utilising just 0.5% of the data and a simple passive data collection method. We analyze 1,000 unique successful reservoir structures, examining their internal connectivity and memory properties, and identify key structural features that enhance control performance. Finally, this paper also explores our proposed controller’s robustness to changes in pendulum dimensionality and torque limit with successful control achieved for a large range of varying properties without any additional training.

Advances in robotics have been driving the development of human-robot interaction (HRI) technologies. However, accurately perceiving human actions and achieving adaptive control remains a challenge in facilitating seamless coordination between human and robotic movements. In this paper, we propose a hierarchical procedural framework to enable dynamic robot-assisted hand-object interaction (HOI). An open-loop hierarchy leverages the RGB-based 3D reconstruction of the human hand, based on which motion primitives have been designed to translate hand motions into robotic actions. The low-level coordination hierarchy fine-tunes the robot’s action by using the continuously updated 3D hand models. Experimental validation demonstrates the effectiveness of the hierarchical control architecture. The adaptive coordination between human and robot behavior has achieved a delay of ≤ 0.3 seconds in the tele-interaction scenario. A case study of ring-wearing tasks indicates the potential application of this work in assistive technologies such as healthcare and manufacturing.

Autonomous vehicles (AVs) are vehicles that traverse on the road without active human intervention. With a coordinator, AVs can be connected to provide high-efficiency transport services, such as AV-based public transport networks. The controller can manage the network by coordinating the transport request assignment, traveling, and charging/discharging schedule. On the other hand, AVs are likely to be electric and benefit the smart grid via vehicle-to-grid technology. A well-designed mobility network connecting electric AVs (EAVs) and smart grid can substantially reduce unnecessary travel and energy costs. In this article, we aim to maximize utilities in the AV-based public transport network and the power distribution network for the vehicle network containing EAVs, charging stations, and distributed power generations. We formulate the assignment and scheduling problem as a multiobjective mixed-integer program (MIP). To solve the optimization problem, we develop a hybrid heuristic approach based on nondominated sorting genetic algorithm II (NSGA-II) and branch-and-bound (BnB) algorithms. Experiments are conducted on a modified 15-bus distribution system and a simulated traffic network. The results show that the proposed strategy effectively minimizes the total travel and energy purchase cost by 21%. This study provides valuable insights on vehicle coordination for multiple tasks, offering visionary guidance for stakeholders engaged in multifaceted transportation endeavors.

Reconfigurable Intelligent Surface (RIS) is envisioned to revolutionize 6G wireless networks, particularly in complex environments like smart airports, by customizing analog beamforming with desired direction and magnitude. Through precise configuration refinement, the intelligent surface intends to achieve equivalent Quality of Service (QoS) with fewer antennas, thereby enhancing coverage and capacity in high-demand areas of airports. However, existing model-free algorithms struggle to obtain a stable policy gradient of intelligent surface configuration. Moreover, centralized channel estimation is inefficient to massive communication and more vulnerable to eavesdroppers. To address these challenges, a robust Proximal Policy Optimization-Huber (PPO-Huber) algorithm was developed to improve the efficiency and robustness of digital connectivity within airports. Concerning the privacy of channel models in massive communication, we proposed an optimal Differential Private Federated Learning (DPFL) with noise reduction, ensuring secure access to channel information. Comprehensive convergence analyses are conducted for each proposed algorithm to facilitate hyperparameter tuning and suggest potential research directions. Experimental results demonstrate that our algorithms not only offer flexible deployment of intelligent surface without accurate channel knowledge, but also substantially breaking the communication-privacy-utility trilemma in massive RIS-aided 6G wireless networks of smart airports.

Switching dynamics are prevalent in real-world systems, arising from either intrinsic changes or responses to external influences, which can be appropriately modeled by switched systems. Control synthesis for switched systems, especially integrating safety constraints, is recognized as a significant and challenging topic. This study focuses on devising a learning-based control strategy for switched nonlinear systems operating under arbitrary switching law. It aims to maintain stability and uphold safety constraints despite limited system data. To achieve these goals, we employ the control barrier function method and Lyapunov theory to synthesize a controller that delivers both safety and stability performance. To overcome the difficulties associated with constructing the specific control barrier and Lyapunov function and take advantage of switching characteristics, we create a neural control barrier function and a neural Lyapunov function separately for control policies through a state transition learning approach. These neural barrier and Lyapunov functions facilitate the design of the safe controller. The corresponding control policy is governed by learning from two components: policy loss and forward state estimation. The effectiveness of the developing scheme is verified through simulation examples.

This work addresses the output reachable set-based leader-following consensus problem, focusing on a group of positive agents over directed dwell-time switching networks. Two types of non-negative disturbances, namely, 1)$L_{1}$-norm bounded disturbances and 2)$L_{\infty,1}$-norm bounded disturbances are studied. Meanwhile, a class of directed dwell-time switching networks for modeling the communication protocol of positive agents is investigated. To deal with the presence of disturbances, an output-feedback control protocol is developed to achieve a robust consensus with positivity preserved based on the output reachable set. By exploiting the positive characteristics, switched linear copositive Lyapunov functions are adopted to establish output reachable set-based consensus conditions. These conditions can facilitate the control protocol design by solving a bilinear programming problem, and also generate hyperpyramidal regions to confine the output consensus error. A particle swarm optimization-based (PSO-based) algorithm is applied to compute the controller gains and optimize the volume of the hyperpyramids. The proposed methods are verified by the presented numerical case studies.

Cyber-physical systems, such as smart transportation, face security threats from both digital and physical realms. Recently, Mobility-as-a-Service (MaaS) has emerged as a novel transportation concept, offering passengers access to diverse mobility services via a unified platform. Central to this system is the smart MaaS coordinator, tasked with tailoring services to passengers based on their profiles and behaviors. However, the coordination of heterogeneous passengers introduces vulnerabilities, enabling malicious entities to exploit the system by impersonating priority passengers with falsified information. Effective detection mechanisms require a deep understanding of the spoofing process. This paper investigates threats to the smart MaaS coordinator, unveiling a new reinforcement learning-based attack named the passenger spoofing attack, which aims to mitigate the risk of inadvertently exposing MaaS vulnerabilities post-deployment. This attack leverages feedback from actions and experiences to manipulate system profitability and passenger satisfaction by generating false passenger information. Furthermore, our research reveals that multi-agent reinforcement learning, accounting for spatial distribution among malicious agents and passengers, strengthens the attack. Through simulations based on datasets from New York City and synthetic sources, we demonstrate that the attack can significantly reduce 70% of profit and 50% of passenger satisfaction. Spatial analysis indicates an effective distance of approximately two nodes from the origin or destination. This study enriches our comprehension of the vulnerabilities inherent in smart coordinators within MaaS, enabling the development of robust countermeasures against malicious actors.

Mobility-as-a-service (MaaS) is a new transport model that combines multiple transport modes in a single platform. Dynamic passenger behavior based on past experiences requires reinforcement-based optimization of MaaS services. Deep reinforcement learning (DRL) may improve passenger satisfaction by offering the most appropriate transport services based on individual passenger experiences and preferences. However, this produces a new privacy risk to the MaaS platform using the centralized DRL method. Information leakage will occur if the platform is not carefully designed with privacy-preserving mechanisms. In this paper, we propose a federated deep deterministic policy gradient (FDDPG) that maximizes passenger satisfaction and MaaS long-term profit while preserving privacy. We enforce an equally weighted experience sampling mechanism to prevent sampling bias such that the solution quality of FDDPG is statistically equivalent to the centralized algorithm. During the model training and inference, information is processed locally, and only the gradients are shared, which prevents information leakage to any semi-honest participants and eavesdroppers. Secure aggregation protocol in line with the dynamic property of the mobile agent is also used in the gradient sharing step to ensure that the algorithm is prevented from inference attacks. We perform experiments on New York City-based real-world and synthetic scenarios. The results show that the proposed FDDPG can improve the MaaS profit and passenger satisfaction by about 90% and 15%, respectively, and maintain stable training against agent dropout. Our approach and findings could enhance MaaS utility as well as facilitate passenger trust and participation in MaaS and other data-driven transportation systems.

Autonomous vehicles (AVs) traverse on the road without active human intervention. In an AV-based public transport system, AVs can be supported by an intelligent controller to provide high-efficiency transport services. The intelligent controller manages a fleet of AVs by determining their assignments to transport requests, sending instructions concerning the optimized plans, and managing the charging/discharging schedule and locations for the governed AVs. On the other hand, vehicle-to-grid is a promising technology to benefit the power system operation by using the battery storage of electric vehicles (EVs). A well-designed request assignment and travel schedule controller can substantially reduce the unnecessary travel and energy purchase cost for the transport system and power system, respectively. In this paper, we aim at coordinating the transport request assignment, traveling and charging/discharging schedule for a fleet of electric AVs (EAVs), and the reactive power support from distributed generations (DGs) to maximize utility in both the AV-based public transportation system and power distribution system. We formulate the assignment and scheduling problem as a mixed-integer program. Experiments on a modified 15-bus distribution system and a simulated traffic network are conducted. The results show that the formulated problem effectively minimizes the total travel and energy purchase cost by controlling the EAV behaviors and DG outputs. This study may promote the integration of transport and power technologies in smart grids as well as incentivize the utilization of EAVs.

Abstract Multimodal transportation systems require an effective journey planner to allocate multiple passengers to transport operators. One example is mobility-as-a-service, a new mobility service that integrates various transport modes through a single platform. In such a multimodal and diverse journey planning problem, accommodating heterogeneous passengers with different and dynamic preferences can be challenging. Furthermore, passengers may behave based on experiences and expectations, in the sense that the transport experience affects their state and decision of the next transport service. Current methods of treating each journey planning optimization as a non-time varying single experience problem cannot adequately model passenger experience and memories over many journeys over time. In this paper, we model passenger experience as a Markov model where prior experiences have a transient effect on future long-term satisfaction and retention rate. As such, we formulate a multi-objective journey planning problem that considers individual passenger preferences, experiences, and memories. The proposed approach dynamically determines utility weights to obtain an optimal journey plan for individual passengers based on their status. To balance the profit received by each transport operator, we present a variant-based proportional fairness. Our experiments using real-world and synthetic datasets show that our approach enhances passenger satisfaction, compared to baseline methods. We demonstrate that the overall profit is increased by 2.3 times, resulting in a higher retention rate caused by higher satisfaction levels. Our proposed approach can facilitate the participation of transport operators and promote passenger acceptance of MaaS.

An Autonomous Vehicle (AV) is believed to be the next generation transport that can enhance safety and efficiency for smart mobility. In an AV-based public transportation system, the full autonomy of AVs enables high-efficiency transport services and the potential ride-sharing feature of AV system enhances the utilization. The system manages a fleet of AVs, determines their assignments to transport requests, sends instructions concerning the optimized plans, and recommends the parking locations for the unoccupied AVs. The parking location of an empty AV is crucial in the sense that rebalancing AVs to areas with high potential service demand can curtail the unnecessary waiting time for passengers. As AVs are generally electric, proper parking locations can also facilitate vehicle-to-grid (V2G) support. In this paper, we propose a joint rebalancing and V2G coordination strategy for AV-based public transportation system. We formulate the problem as an integer linear program and propose a heuristic based on Genetic Algorithm and Model Predictive Control to solve the problem in low time complexity. Extensive experiments are performed with the real taxi service data from New York City. The formulated integer linear program is solved dynamically where each problem instance contains 3 to 5 AVs and 3 to 8 requests in 30s time interval. Compared with the transport system without rebalancing, the results show that the coordination strategy is efficient and effective in reducing unnecessary waiting time for passengers while satisfying V2G support. Compared to computational time with the standard solver, the proposed heuristic dramatically reduces the computational time.

An efficient transportation system can substantially benefit our society, but road intersections have always been among the major traffic bottlenecks leading to traffic congestion. Appropriate traffic signal timing adapted to real-time traffic may help mitigate such traffic congestion. However, most existing traffic signal control methods require a huge amount of road information, such as vehicle positions. In this paper, we focus on a particular road intersection and aim to minimize the average waiting time. We propose a traffic signal control (TSC) system based on an end-to-end off-policy deep reinforcement learning (deep RL) agent with background removal residual networks. The agent takes real-time images at the road intersection as input. Upon sufficient training, the agent can perform (near-) optimal traffic signaling based on real-time traffic conditions. We conduct experiments on different intersection scenarios and compare various TSC methods. The experimental results show that our end-to-end deep RL approach can adapt to the dynamic traffic based on the traffic images and outperforms other TSC methods.

Disturbance, which is generally unknown to the controller, is unavoidable in real-world systems and it may affect the expected system state and output. Existing control methods, like robust model predictive control, can produce robust solutions to maintain the system stability. However, these robust methods trade the solution optimality for stability. In this article, a method called generative adversarial control networks (GACNs) is proposed to train a controller via demonstrations of the optimal controller. By formulating the optimal control problem in the presence of disturbance, the controller trained by GACNs obtains neuro-optimal solutions without knowing the future disturbance and determines the objective function explicitly. A joint loss, composed of the adversarial loss and the least square loss, is designed to be used in the training of the generator. Experimental results on simulated systems with disturbance show that GACNs outperform other compared control methods.

An effective intelligent transportation system is a core part of modern smart city. The Internet of Things and vehicular communication technologies facilitate rapid development of connected and autonomous vehicles (CAVs). While most studies focus on standalone CAV technologies, collective CAV control has much potential. With the connectivity and automation of CAVs, we can employ dynamic lane reversal (DLR) to optimize the travel schedules of CAVs for performance enhancement. In this paper, we propose the dynamic lane reversal-traffic scheduling management (DLR-TSM) scheme for CAVs. The system collects the travel requests from CAVs and determines their optimal schedules and routes over dynamically reversible lanes. We formulate the routing and scheduling problem on DLR as an integer linear program. To address the scaling effect, an algorithm based on alternating direction method of multipliers is designed to solve the problem in a distributed manner. We extensively evaluate the DLR-TSM and the distributed algorithm with real-world transportation data. The simulation results show that the DLR-TSM can significantly improve the travel times of CAVs and the distributed algorithm can dramatically reduce the required computational time.

Advancements in sensing and the Internet of Things (IoT) technologies generate a huge amount of data. Mobility on demand (MoD) service benefits from the availability of big data in the intelligent transportation system. Given the future travel demand or origin-destination (OD) flows prediction, service providers can pre-allocate unoccupied vehicles to the customers' origins of service to reduce waiting time. Traditional approaches on future travel demand and the OD flows predictions rely on statistical or machine learning methods. Inspired by deep learning techniques for image and video processing, through regarding localized travel demands as image pixels, a novel deep learning model called multi-scale convolutional long short-term memory network (MultiConvLSTM) is developed in this paper. Rather than using the traditional OD matrix which may lead to loss of geographical information, we propose a new data structure, called OD tensor to represent OD flows, and a manipulation method, called OD tensor permutation and matricization, is introduced to handle the high dimensionality features of OD tensor. MultiConvLSTM considers both temporal and spatial correlations to predict the future travel demand and OD flows. Experiments on real-world New York taxi data of around 400 million records are performed. Our results show that the MultiConvLSTM achieves the highest accuracy in both one-step and multiple-step predictions and it outperforms the existing methods for travel demand and OD flow predictions.

Information exchange in a vehicular network between autonomous vehicles and the roadside infrastructure is important for improving road safety. These autonomous vehicles, equipped with a sensor suite, are capable of obtaining road map data that can be used to inform other vehicles and update the central road map repository through roadside units. The roadside infrastructure nodes act as local databases for distributing regional 3D road map data in form of point clouds to autonomous vehicles passing by. Since the vehicles might have various side information regarding the road network and traffic condition, minimizing the required number of transmissions to satisfy the demand of participating vehicles through network coding is an interesting research problem in road map data dissemination. In this paper, we propose the Road Map Data Encoding and Dissemination System (REDS) and evaluate its performance in a four-way junction scenario. It is based on index coding for broadcasting road map data from a centrally-managed roadside node to vehicles. REDS uses the data availability and demand knowledge for encoding and transmitting 3D point cloud road map data from different road segments. The data availability information helps prevent the transmission of duplicated road map data and provides the sets of side information in the index coding problem, while the data demand information further defines the message transmission priority based on the data demand of different road segments. Simulation results indicate that REDS reduces the average number of transmissions and transmitted point cloud data size by around 30% when the data availability probability is about 0.5 under random mobility in all simulated scenarios when compared to the traditional broadcasting approach.