Marine debris poses an alarming threat to ocean environments. Conventional methods of sea and ocean cleaning rely heavily on manual collection, a process that has repeatedly demonstrated its inefficiency and extensive demand for resources. This paper presents the SeaClear system, a novel multi-robot platform designed to autonomously detect and collect marine debris, thereby offering a more efficient solution to this environmental challenge. An overview of the system is presented, followed by a detailed description of each robot’s capabilities. Leveraging artificial intelligence, the system employs the deep-learning-based computer vision algorithm You Only Look Once (YOLO) for the detection of underwater litter, addressing the challenges of poor visibility and hydrodynamic disturbances of underwater environments. Additionally, the paper explores the implemented navigation and control methodologies, which are an essential part of the workflow of the system. The performance of the designed system is validated via field tests conducted in a real-world underwater environment. Finally, directions for future work are proposed. • The infrastructure of a multi-robot system for autonomous underwater debris detection, mapping, and collection is presented. • The suitability of YOLO-based deep learning for real-time debris detection in shallow waters is validated. • The sensing and control scheme that enables the operation of the multi-robotic platform is presented. • The practical performance of the system is confirmed with field experiments to validate the feasibility of AI-driven underwater operations.

Efficient energy management is essential for reliable and sustainable microgrid operation amid increasing renewable integration. In this paper, an imitation learning-based framework to approximate mixed-integer Economic Model Predictive Control (EMPC) is proposed for microgrid energy management, considering fuel generators, renewable energy resources, a unified energy storage unit, and curtailable loads. Within the proposed framework, a neural network is trained to imitate expert EMPC control actions from offline trajectories, thereby enabling fast real-time decision making without solving online mixed-integer optimization problems, which often exhibit highly variable solution times across instances and do not scale well to large problem sizes; in particular, worst-case solve times can be excessively large and therefore unsuitable for real-time deployment. In contrast, the learned policy provides predictable and consistently low computation times. To enhance robustness and generalization, the learning process incorporates noise injection during training to mitigate distribution shift and explicitly accounts for forecast uncertainty in renewable generation and demand. Furthermore, a constraint-tightening approach combined with a projection layer is proposed to ensure recursive feasibility and constraint satisfaction of the learned controller. Simulation results demonstrate that the learned policy achieves economic performance comparable to EMPC, while reducing computation time by approximately one order of magnitude relative to the optimization-based EMPC.

This paper presents a deep learning framework for analyzing on-board vibration response signals in infrastructure health monitoring. The proposed WaveletInception–BiGRU network uses a Learnable Wavelet Packet Transform (LWPT) for early spectral feature extraction, followed by one-dimensional Inception-Residual Network (1D Inception-ResNet) modules for multi-scale, high-level feature learning. Bidirectional Gated Recurrent Unit (BiGRU) modules then integrate temporal dependencies and incorporate operational conditions, such as the measurement speed. This approach enables effective analysis of vibration signals recorded at varying speeds, eliminating the need for explicit signal preprocessing. The sequential estimation head further leverages bidirectional temporal information to produce an accurate, localized assessment of infrastructure health. Ultimately, the framework generates high-resolution health profiles spatially mapped to the physical layout of the infrastructure. Case studies involving track stiffness regression and transition zone classification using real-world measurements demonstrate that the proposed framework significantly outperforms state-of-the-art methods, underscoring its potential for accurate, localized, and automated on-board infrastructure health monitoring. • WaveletInception-BiGRU proposed for on-board monitoring of railway infrastructure • WaveletInception extracts multi-scale local features from vibration signals • BiGRU captures temporal dependencies for localized health condition estimation • Late-stage fusion automates speed integration, eliminating manual feature engineering

In the backdrop of an increasingly pressing need for effective urban and highway transportation systems, this work explores the synergy between model-based and learning-based strategies to enhance traffic flow management by use of an innovative approach to the problem of ramp metering control that embeds Reinforcement Learning (RL) techniques within the Model Predictive Control (MPC) framework. The control problem is formulated as an RL task by crafting a suitable stage cost function that is representative of the traffic conditions, variability in the control action, and violations of the constraint on the maximum number of vehicles in queue. An MPC-based RL approach, which leverages the MPC optimal problem as a function approximation for the RL algorithm, is proposed to learn to efficiently control an on-ramp and satisfy its constraints despite uncertainties in the system model and variable demands. Simulations are performed on a benchmark small-scale highway network to compare the proposed methodology against other state-of-the-art control approaches. Results show that, starting from an MPC controller that has an imprecise model and is poorly tuned, the proposed methodology is able to effectively learn to improve the control policy such that congestion in the network is reduced and constraints are satisfied, yielding an improved performance that is superior to the other controllers.

Infinite-horizon optimal control of constrained piecewise affine (PWA) systems has been approximately addressed by hybrid model predictive control (MPC), which, however, has computational limitations, both in offline design and online implementation. In this article, we consider an alternative approach based on approximate dynamic programming (ADP), an important class of methods in reinforcement learning. We accommodate nonconvex union-of-polyhedra state constraints and linear input constraints into ADP by designing PWA penalty functions. PWA function approximation is used, which allows for a mixed-integer encoding to implement ADP. The main advantage of the proposed ADP method is its online computational efficiency. Particularly, we propose two control policies, which lead to solving a smaller-scale mixed-integer linear program than conventional hybrid MPC, or a single convex quadratic program, depending on whether the policy is implicitly determined online or explicitly computed offline. We characterize the stability and safety properties of the closed-loop systems, as well as the suboptimality of the proposed policies, by quantifying the approximation errors of value functions and policies. We also develop an offline mixed-integer-linear-programming-based method to certify the reliability of the proposed method. Simulation results on an inverted pendulum with elastic walls and on an adaptive cruise control problem validate the control performance in terms of constraint satisfaction and CPU time.

Parameterized model predictive control (PMPC) is one of the many approaches that have been developed to alleviate the high computational requirement of model predictive control (MPC), and it has been shown to significantly reduce the computational complexity while providing comparable control performance with conventional MPC. However, PMPC methods still require a sufficiently accurate model to guarantee the control performance. To deal with model mismatches caused by the changing environment and by disturbances, this paper first proposes a novel framework that uses reinforcement learning (RL) to adapt all components of the PMPC scheme in an online way. More specifically, the novel framework integrates various strategies to adjust different components of PMPC (e.g., objective function, state-feedback control function, optimization settings, and system model), which results in a synthesis framework for RL-based adaptive PMPC. We show that existing adaptive (P)MPC approaches can also be embedded in this synthesis framework. The resulting combined RL-PMPC framework provides a solution for an efficient MPC approach that can deal with model mismatches. A case study is performed in which the framework is applied to freeway traffic control. Simulation results show that for the given case study the RL-based adaptive PMPC approach reduces computational complexity by 98% on average compared to conventional MPC while achieving better control performance than the other controllers, in the presence of model mismatches and disturbances.

Real-time train scheduling is essential for passenger satisfaction in urban rail transit networks. This paper focuses on real-time train scheduling for urban rail transit networks considering uncertain time-dependent passenger origin-destination demands. First, a macroscopic passenger flow model we proposed before is extended to include rolling stock availability. Then, a distributed-knowledgeable-reduced-horizon (DKRH) algorithm is developed to deal with the computational burden and the communication restrictions of the train scheduling problem in urban rail transit networks. For the DKRH algorithm, a cost-to-go function is designed to reduce the prediction horizon of the original model predictive control approach while taking into account the control performance. By applying a scenario reduction approach, a scenario-based distributed-knowledgeable-reduced-horizon (S-DKRH) algorithm is proposed to handle the uncertain passenger flows with an acceptable increase in computation time. Numerical experiments are conducted to evaluate the effectiveness of the developed DKRH and S-DKRH algorithms based on real-life data from the Beijing urban rail transit network. The simulation results indicate that DKRH can be used to achieve real-time train scheduling for the urban rail transit network, while S-DKRH can handle the uncertainty in the passenger flows with an acceptable sacrifice in computation time.

Virtual coupling is regarded as an efficient way to improve the line capacity of rail transportation systems by reducing the spacing between consecutive trains. This paper is the first to compare and assess different distributed model predictive control (MPC) approaches, i.e., cooperative distributed MPC, serial distributed MPC, and decentralized MPC, for virtually coupled trains with a nonlinear train dynamic model. To make a balanced trade-off between computational complexity and efficiency, we also propose and assess convex approximations of the above control approaches. Furthermore, we are the first to introduce the relaxed dynamic programming approach to analyze the stability of the MPC-based nonlinear train control problem. By using the relaxed dynamic programming approach, a distributed stopping criterion with a stability guarantee is developed for the cooperative distributed MPC approach. In real life, masses of trains are different and can change at stations due to changes in passenger loads. This change in mass can significantly affect the dynamics and control of the virtually coupled trains when not taken into account in the control design. Therefore, we explicitly consider heterogeneous train masses when designing MPC approaches. We evaluate the different distributed MPC approaches through case studies based on the data of the Beijing Yizhuang Line. Simulation results indicate that the cooperative distributed MPC approach has the best tracking performance, while the serial distributed MPC approach can reduce communication requirements and computation capabilities with sacrifices of tracking performance.

Model predictive control (MPC) and deep reinforcement learning (DRL) have been developed extensively as two independent techniques for traffic management. Although the features of MPC and DRL complement each other very well, few of the current studies consider combining these two methods for application in the field of freeway traffic control. This paper proposes a novel framework for integrating MPC and DRL methods for freeway traffic control that is different from existing MPC-(D)RL methods. Specifically, the proposed framework adopts a hierarchical structure, where a high-level efficient MPC component works at a low frequency to provide a baseline control input, while the DRL component works at a high frequency to modify online the output generated by MPC. The control framework, therefore, needs only limited online computational resources and is able to handle uncertainties and external disturbances after proper learning with enough training data. The proposed framework is implemented on a benchmark freeway network in order to coordinate ramp metering and variable speed limits, and the performance is compared with standard MPC and DRL approaches. The simulation results show that the proposed framework outperforms standalone MPC and DRL methods in terms of total time spent (TTS) and constraint satisfaction, despite model uncertainties and external disturbances.

In this paper, the novel Distributed Bayesian (D-Bay) algorithm is presented for solving multi-agent problems within the Continuous Distributed Constraint Optimization Problem (C-DCOP) framework. This framework extends the classical DCOP framework towards utility functions with continuous domains. D-Bay solves a C-DCOP by utilizing Bayesian optimization for the adaptive sampling of variables. We theoretically show that D-Bay converges to the global optimum of the C-DCOP for Lipschitz continuous utility functions. The performance of the algorithm is evaluated empirically based on the sample efficiency. The proposed algorithm is compared to state-of-the-art DCOP and C-DCOP solvers. The algorithm generates better solutions while requiring fewer samples.

The vector field method was originally proposed to guide a single fixed-wing Unmanned Aerial Vehicle (UAV) towards a desired path. In this work, a non-uniform vector field method is proposed that changes in both magnitude and direction, for the purpose of achieving formations of UAVs. As compared to related work in the literature, the proposed formation control law does not need to assume absence of wind. That is, due to the effect of the wind on the UAV, one can handle the UAV air speed being different from its ground speed, and the UAV heading angle being different from its course angle. Stability of the proposed formation method is analyzed via Lyapunov stability theory, and validations are carried out in software-in-the-loop and hardware-in-the-loop comparative experiments. Note to Practitioners—The software-in-the-loop and hardware-in-the-loop experiments, which are done with PX4 autopilot software and hardware, show that the proposed method can be implemented on board of UAVs and integrated with the control architecture of existing autopilot suites. Comparisons with standard formation algorithms show that the proposed method is effective in achieving formation in different path scenarios.

This article attempts to realize zero-error constrained tracking for hypersonic flight vehicles (HFVs) subject to unknown control directions and asymmetric flight state constraints. The main challenges of reaching such goals consist in that addressing multiple unknown control directions requires novel conditional inequalities encompassing the summation of multiple Nussbaum integral terms, and in that the summation of conditional inequality may be bounded even when each term approaches infinity individually, but with opposite signs. To handle this challenge, novel Nussbaum functions that are designed in such a way that their signs keep the same on some periods of time are incorporated into the control design, which not only ensures the boundedness of multiple Nussbaum integral terms but preserves that velocity and altitude tracking errors eventually converge to zero. Fuzzy-logic systems (FLSs) are exploited to approximate model uncertainties. Asymmetric integral barrier Lyapunov functions (IBLFs) are adopted to handle the fact that the operating regions of flight state variables are asymmetric in practice, while ensuring the validity of fuzzy-logic approximators. Comparative simulations validate the effectiveness of our proposed methodology in guaranteeing convergence, smoothness, constraints satisfaction, and in handling unknown control directions.

This article proposes an output-feedback control protocol for hypersonic flight vehicle (HFV) swarms considering dynamic event-triggered communication. The peculiarities of the proposed method over existing ones consist in the following: 1) While carrying out scheduled maneuvers, the outputs of follower HFVs converge inside the convex hull spanned by leader HFVs whose task is to maintain a geometric space configuration; 2) a simple nonrecursive output-feedback design is established without involving any intermediate control laws or requiring full-state information; 3) an error-dependent monotonically decreasing exponential term is incorporated into the dynamic event-triggered threshold to reduce the communication bandwidth while preserving the desired track performance and excluding Zeno behavior. Comparative simulation results validate the effectiveness of the proposed methodology.

Real-time timetable scheduling is an effective way to improve passenger satisfaction and to reduce operational costs in urban rail transit networks. In this paper, a novel passenger-oriented network model is developed for real-time timetable scheduling that can model time-dependent passenger origin-destination demands with consideration of a balanced trade-off between model accuracy and computation speed. Then, a model predictive control (MPC) approach is proposed for the timetable scheduling problem based on the developed model. The resulting MPC optimization problem is a nonlinear non-convex problem. In this context, the online computational complexity becomes the main issue for the real-time feasibility of MPC. To reduce the online computational complexity, the MPC optimization problem is therefore reformulated into a mixed-integer linear programming (MILP) problem. The resulting MILP problem is exactly equivalent to the original MPC optimization problem and can be solved very efficiently by existing MILP solvers, so that we can obtain the solution very fast and realize real-time timetable scheduling. Numerical experiments based on a part of Beijing subway network show the effectiveness and efficiency of the developed model and the MILP-based MPC method.

Currently, with the development of driving technologies, driverless vehicles gradually are becoming more and more available. Therefore, there would be a long period of time during which self-driving vehicles and human-driven vehicles coexist. However, for a mixed platoon, it is hard to control the formation due to the existence of the manual vehicles resulting in weak robustness and slow consensus rate on this system of platoons because of uncertainties caused by human factors for manual vehicles. In order to solve this problem, we establish models of mixed platoons with mixed types of connected and automated vehicles (CAVs), human-driven vehicles (HDVs) and HDVs without the vehicle awareness device (HDVWs). We subsequently design$\mathcal {H}_{\infty} $controllers for the mixed platoons to realize the formation consensus. In addition, we use the$\mathcal {H}_{\infty} $norm of mixed platoons as the control objective investigating the robustness of the control algorithms in alleviating the platoon uncertainties. Furthermore, conditions are proved to maintain the stability of the mixed platoons, and the stability is analyzed based on the variation of the penetration rate of the manual vehicles. Finally, we formulate conditions for parameters according to the definition of string stability to avoid the collisions of vehicles. The results in this study are tested with simulations and suggest that the presented controllers can ensure the consensus of mixed platoons under uncertainties.

We propose a model-predictive control (MPC)-based approach to solve a human-in-the-loop control problem for a network system lacking sensors and actuators to allow for a fully automatic operation. The humans in the loop are, therefore, essential; they travel between the network nodes to provide the remote controller with measurements and to actuate the system according to the controller’s commands. Time instant optimization MPC is utilized to compute when the measurement and actuation actions are to take place to coordinate them with the network dynamics. The time instants also minimize the burden of human operators by tracking their energy levels and scheduling the necessary breaks. Fuel consumption related to the operators’ travel is also minimized. The results in a digital twin of the Dez Main Canal illustrate that the new algorithm outperforms previous methods in terms of meeting operational objectives and taking care of human well-being, but at the cost of higher computational requirements.

This paper focuses on the control of microgrids where both gas and electricity are provided to the final customer, i.e., multi-carrier microgrids. Hence, these microgrids include thermal and electrical loads, renewable energy sources, energy storage systems, heat pumps, and combined heat and power units. The parameters characterizing the multi-carrier microgrid are subject to several disturbances, such as fluctuations in the provision of renewable energy, variability in the electrical and thermal demand, and uncertainties in the electricity and gas pricing. With the aim of accounting for the data uncertainties in the microgrid, we propose a Robust Model Predictive Control (RMPC) approach whose goal is to minimize the total economical cost, while satisfying comfort and energy requests of the final users. In the related literature various RMPC approaches have been proposed, focusing either on electrical or on thermal microgrids. Only a few contributions have addressed the robust control of multi-carrier microgrids. Consequently, we propose an innovative RMPC algorithm that employs on an uncertainty set-based method and that can provide better performance compared with deterministic model predictive controllers applied to multi-carrier microgrids. With the aim of mitigating the conservativeness of the approach, we define suitable robustness factors and we investigate the effects of such factors on the robustness of the solution against variations of the uncertain parameters. We show the effectiveness of the proposed RMPC approach by applying it to a realistic residential multi-carrier microgrid and comparing the obtained results with the ones of a baseline robust method.Note to Practitioners—This work is motivated by the emerging need for effective energy management approaches in multi-carrier microgrids. The inherent difficulty of scheduling simultaneously the operations of various energy infrastructures (e.g., electricity, natural gas) is exacerbated by the inevitable presence of uncertainties that affect the inter-dependent dynamics of different energy resources and equipment. The proposed robust MPC-based control strategy allows the energy manager to effectively determine an optimal energy scheduling of multi-faceted system components, making a tradeoff between performance and protection against data uncertainty. The presented strategy is comprehensive and generic, as it can be applied to different microgrid frameworks integrating various types of system components and sources of uncertainty, while at the same time being implementable in any energy management system.

Railways are a well-recognized sustainable transportation mode that helps to satisfy the continuously growing mobility demand. However, the management of railway traffic in large-scale networks is a challenging task, especially when both a major disruption and various disturbances occur simultaneously. We propose an automatic rescheduling algorithm for real-time control of railway traffic that aims at minimizing the delays induced by the disruption and disturbances, as well as the resulting cancellations of train runs and turn-backs (or short-turns) and shuntings of trains in stations. The real-time control is based on the Model Predictive Control (MPC) scheme where the rescheduling problem is solved by mixed integer linear programming using macroscopic and mesoscopic models. The proposed resolution algorithm combines a distributed optimization method and bi-level heuristics to provide feasible control actions for the whole network in short computation time, without neglecting physical limitations nor operations at disrupted stations. A realistic simulation test is performed on the complete Dutch railway network. The results highlight the effectiveness of the method in properly minimizing the delays and rapidly providing feasible feedback control actions for the whole network.Note to Practitioners—This article aims at contributing to the enhancement of the core functionalities of Automatic Train Control (ATC) systems and, in particular, of the Automatic Train Supervision (ATS) module, which is included in ATC systems. In general, the ATS module allows to automate the train traffic supervision and consequently the rescheduling of the railway traffic in case of unexpected events. However, the implementation of an efficient rescheduling technique that automatically and rapidly provides the control actions necessary to restore the railway traffic operations to the nominal schedule is still an open issue. Most literature contributions fail in providing rescheduling methods that successfully determine high-quality solutions in less than one minute and include real-time information regarding the large-scale railway system state. This research proposes a semi-heuristic control algorithm based on MPC that, on the one hand, overcomes the limitations of manual rescheduling (i.e., suboptimal, stressful, and delayed decisions) and, on the other hand, offers the advantages of online and closed-loop control of railway traffic based on continuous monitoring of the traffic state to rapidly restore railway traffic operations to the nominal schedule. The semi-heuristic procedure permits to significantly reduce the computation time necessary to solve the rescheduling problem compared with an exact procedure; moreover, the use of a distributed optimization approach permits the application of the algorithm to large instances of the rescheduling problem, and the inclusion of both the traffic and rolling stock constraints related to the disrupted area. The method is tested on a realistic simulation environment, thus still requires further refinements for the integration into a real ATS system. Further developments will also consider the occurrence of various simultaneous disruptions in the network.

After a natural disaster, a quick inspection of all damaged components is crucial to recover the functionality of distribution networks. Unmanned aerial vehicles (UAVs) can perform inspection tasks, particularly for damages that are difficult to access for human repair crews. Additionally, UAVs can monitor the transmission lines to find potential dangers and early-stage damages, and to monitor the road infrastructure to provide real-time information about traffic conditions so that repair crews can select the best ways to reach damages. Besides, due to unpredictable events during restoration, the UAV routing strategy (UAVRS) needs to be updated in real time. Thus, the proposed UAVRS in this article determines the optimal routes for the UAVs allocated to inspect damages as well as the optimal routes for the UAVs to monitor transmission lines and roads in real time for distribution networks. To tackle the multi-time-scale characteristic of the proposed UAVRS, a two-layer decision-making architecture is proposed. A bilevel programming problem is solved in the first layer for the large-time-scale problem, and a mixed-integer linear programming problem is solved for the small-time-scale problem in the second layer. A case study based on the distribution network in Zaltbommel and its neighbor areas, in The Netherlands, illustrates the effectiveness of our real-time method compared to the offline methods. Furthermore, different solvers are studied and compared in view of the real-time requirement.

The development of assessment methods for the performance of Automated Vehicles (AVs) is essential to enable the deployment of automated driving technologies, due to the complex operational domain of AVs. One candidate is scenario-based assessment, in which test cases are derived from real-world road traffic scenarios obtained from driving data. Because of the high variety of the possible scenarios, using only observed scenarios for the assessment is not sufficient. Therefore, methods for generating additional scenarios are necessary. Our contribution is twofold. First, we propose a method to determine the parameters that describe the scenarios to a sufficient degree while relying less on strong assumptions on the parameters that characterize the scenarios. By estimating the probability density function (pdf) of these parameters, realistic parameter values can be generated. Second, we present the Scenario Representativeness (SR) metric based on the Wasserstein distance, which quantifies to what extent the scenarios with the generated parameter values are representative of real-world scenarios while covering the actual variety found in the real-world scenarios. A comparison of our proposed method with methods relying on assumptions of the scenario parameterization and pdf estimation shows that the proposed method can automatically determine the optimal scenario parameterization and pdf estimation. Furthermore, it is demonstrated that our SR metric can be used to choose the (number of) parameters that best describe a scenario. The presented method is promising, because the parameterization and pdf estimation can directly be applied to already available importance sampling strategies for accelerating the evaluation of AVs.

Traffic flow (TF) prediction is an important and yet a challenging task in transportation systems, since the TF involves high nonlinearities and is affected by many elements. Recently, neural networks have attracted much attention for TF prediction, but they are commonly black boxes with complex architectures and difficult to be interpreted, e.g., the contributions of specific traffic elements are not explicit, hardly providing informative guidance. In this paper, we aim at addressing more interpretable short-term TF prediction with joint consideration to high accuracy, and thus introduces a pragmatic method by applying the efficient hinging hyperplanes neural network (EHHNN) simply built upon sparse neuron connections. In the proposed method, different traffic factors are incorporated into the inputs, including their spatial-temporal information. Besides the pursuit of accuracy, we further extend the ANOVA decomposition of EHHNNs to the interpretation analysis with specifications to traffic data, in which the contributions concerning specific traffic variables are detected quantitatively. As such, the proposed method firstly applies the EHHNN to filter out more important traffic variables for dimensionality reduction while maintaining accurate prediction. Then, variable interpretation analysis is performed from different perspectives, e.g. to quantitatively investigate the influence of traffic factors and also their spatial-temporal impacts. Therefore, a predictor and an analyzing tool can both be attained for the TF by exerting the flexibility and extending the interpretability of EHHNNs, which is promising to provide informative guidance to future traffic control. Numerical experiments verify the effectiveness and potential of the proposed method in TF prediction and analysis.

The Model Predictive Control (MPC) framework has shown great potential for the control of Variable Speed Limits (VSLs) and Ramp Metering (RM) installations. However, the implementation to large freeway networks remains challenging. One major reason is that, by considering the VSLs to be discrete decision variables, an extremely difficult Mixed Integer Nonlinear Programming (MINLP) optimization problem has to be solved within every controller sampling interval. Consequently, many related papers relax the MINLP problems by considering the VSLs to be continuous variables. This paper proposes two novel MPC algorithms for coordinated control of discrete VSLs and continuous RM rates that do not make this relaxation. The proposed algorithms use a distributed control architecture and an alternating optimization scheme to relax the MINLP optimization problems but still consider the VSLs as discrete variables and, hence, offer a trade-off between computational complexity and system performance. The performance of the proposed algorithms is evaluated in a case study. The case study shows that relaxing the VSLs to be continuous variables with a distributed architecture results in a significant performance loss. Furthermore, both proposed algorithms have a lower computational complexity than the more conventional centralized approach and, as a result, they do manage to solve all optimization problems within the sampling intervals. Moreover, one of the proposed algorithms has a system performance that is remarkably similar to the optimal performance of the centralized approach.

Fault location algorithms for transmission lines use the parameters of the transmission line to locate faults after the faults have occurred along the line. Although these parameters can be estimated by the phasor measurement units (PMUs) at the terminal(s) of the transmission line continuously, the uncertainty in the measurements will give rise to stochastic errors in the measured values. Thus, the uncertainty in measurements definitely influences the estimations of the parameters of the transmission line, which, in turn, influences the results of fault location algorithms. Inaccurate results of fault location algorithms may lead to costly maintenance fees and prolonged outage time. Therefore, in this article, we estimate the parameters of the transmission line considering the uncertainty in the measurements so that a more accurate fault location can be derived. The uncertainty in the measurements will be modeled as a stochastic distribution, and the maximum likelihood estimation (MLE) method will be adopted to reduce the uncertainty in the measurements. In addition, as an illustration, the telegrapher's equations will be used to calculate the parameters of the transmission line, and the two-terminal positive sequence network fault location algorithm will be used to locate the fault. In a simulation, a case study of a real-life transmission line the influence of the uncertainty in the measurements on the transmission line parameter estimations and the effectiveness of the MLE method for estimations are simulated and analyzed. The results show that the influence of the uncertainty in the measurements on the positive sequence network fault location algorithm should not be neglected and that the proposed method is very effective in significantly reducing the influence of the uncertainty in the measurements.

This paper proposes a Logic-Based Traffic Flow Control algorithm (LB-TFC) for integrated control of Ramp Metering (RM) installations and Variable Speed Limits (VSLs) in order to reduce traffic jams created at bottlenecks. LB-TFC estimates, for each control time step, the number of vehicles that should be held back or released by the control measures (i.e. the VSLs and the RM rates) in order to avoid the capacity drop (maximizing the outflow of the bottleneck). Afterwards, based on the resulting estimated number of vehicles, the VSLs and/or the RM rates are increased or decreased in a pre-specified order. In order to avoid or reduce traffic breakdowns, the proposed controller (LB-TFC) anticipates the future evolution of the bottleneck density by using a feed-forward structure. As a result, the performance of the controller is very efficient and similar to the one obtained with an optimal controller while the implementation of the controller (with an almost instantaneous computation time) and the tuning of the parameters are easy. In the second part of this work, published in a separate paper (`Part 2: Simulation and Comparison'), LB-TFC is simulated, analyzed and compared for two freeways (one synthetic network and one stretch of the ring-road freeway SE-30 in Seville, Spain).

This paper simulates, analyzes, and compares, for two case studies (one synthetic freeway and one real-life freeway), the behavior of Logic-Based Traffic Flow Control (LB-TFC), an integrated control strategy for Ramp Metering (RM) installations and Variable Speed Limits (VSLs) that was proposed and derivated in the first part of the work (`Part 1: Controller'). For the first case study, which was presented in the first part of the work, the control performance of LB-TFC is compared with the ones obtained with the optimal solution and with the Mainstream Traffic Flow Control (MTFC) + PI-ALINEA algorithm. Moreover, the robustness of the considered controllers is analyzed for this case study concluding that LB-TFC is quite robust, specially when comparing with MTFC + PI-ALINEA. For the second study (a stretch of the ring-road freeway SE-30 in Seville, Spain), data from 10 different days have been used in order to simulate the performance of the considered controllers using real data for the afternoon peak period. In order to properly deal with a bottleneck with a dynamically changing number of lanes, the equations used for MTFC + PI-ALINEA have been slightly modified for the second case study. For both case studies, LB-TFC provides a robust performance that, in most cases, is close to the optimal one and that improves the reduction in the Total Time Spent (TTS) obtained with MTFC + PI-ALINEA. Moreover, this paper studies the tuning of the control parameters and the advantages and disadvantages of LB-TFC.

Effective traffic signal control strategies are critical for traffic management in urban traffic networks. Most existing optimization-based urban traffic control approaches update the traffic signal at regular time instants, where the length of the fixed update time interval is determined based on a trade-off between the computational efficiency and the control performance. Since event-triggered control (ETC) allows for more flexible and more efficient control than conventional time-triggered control by triggering the control action by events, and since it can refrain from redundant optimization while retaining a satisfactory behavior, we use an ETC scheme for traffic light control. In addition, based on the geographically distributed feature of traffic networks, a distributed paradigm is adopted to reduce the computational complexity for the optimization. We propose a distributed threshold-based event-triggered control strategy, where the independent triggering of agents leads to an asynchronous update of traffic signals in the system. The triggered agent then solves a mixed-integer linear programming problem and updates its traffic signals. The proposed approach is evaluated under various traffic demands by simulation, and is shown to yield the best trade-off between control performance and computational complexity compared to other control strategies.

This article addresses and solves the adaptive asymptotic tracking for a class of uncertain switched positive linear dynamics (also known in the literature as compartmental models) subject to dwell-time constraints. Compared to the state-of-the-art, the innovative feature of this method is to attain for the first time asymptotic set-point tracking, while guaranteeing non-negativity of the systems states. To achieve asymptotic tracking, an interpolated Lyapunov function is adopted, which is nonincreasing at the switching instants and decreasing in two consecutive switching instants. Such Lyapunov function results in a novel adaptive law with time-varying adaptive gains, as opposed to state-of-the-art laws with fixed positive adaptive gains. The developed design is applicable to classes of compartmental systems compatible with those proposed in the literature: an example involving the infusion of anesthesia is conducted to show that the proposed method can achieve better performance than existing methods.

This paper proposes a novel adaptive fuzzy dynamic surface control (DSC) method for an extended class of periodically disturbed strict-feedback nonlinear systems. The peculiarity of this extended class is that the control gain functions are not bounded a priori but simply taken to be continuous and with a known sign. In contrast with existing strategies, controllability must be guaranteed by constructing appropriate compact sets ensuring that all trajectories in the closed-loop system never leave these sets. We manage to do this by means of invariant set theory in combination with the Lyapunov theory. In other words, boundedness is achieved a posteriori as a result of stability analysis. The approximator composed of fuzzy logic systems and Fourier series expansion is constructed to deal with the unknown periodic disturbance terms.

The real-time rescheduling of railway traffic in case of unexpected events is a challenging task. This is mainly due to the complexity of the railway service, which has to ensure safety, punctuality, and efficiency to customers by respecting timetable, framework, and resources constraints. Most of the available researches focus on short delays (i.e., disturbances). Approaches typically rely on simplified macroscopic models for large-scale systems or detailed microscopic models for one or a few lines, due to the long computation time required for solving the rescheduling problem. Only a small number of works consider rescheduling in case of long delays (i.e., disruptions) and all of them are also based on either a macroscopic or a microscopic model. This research focuses on disruptions and aims at filling the gap between macroscopic and microscopic modelling by proposing an innovative bi-level rescheduling algorithm based on a mesoscopic Mixed Integer Linear Programming (MILP) model. The technique allows obtaining a feasible rescheduled timetable in a short computation time respecting not only timetable and safety constraints (typical of macroscopic models) but also capacity and ordering constraints for the disrupted stations (typical of microscopic models). The bi-level algorithm first solves the macroscopic MILP rescheduling problem and then, considering the cancellation and non-admissible platform assignments results, it solves a mesoscopic MILP rescheduling problem. This allows to significantly reduce the search space and consequently the computation time. The method is tested for the rescheduling of the Dutch railway traffic in case of a full blockade between two consecutive stations.

This paper proposes and simulates a speed limit controller for recurrent traffic jams (SPERT). SPERT is a simple yet efficient variable speed limit (VSL) control strategy based on the behavior of the optimal controller without any need for online optimization. The online implementation of SPERT is a simple rule-based controller that activates and deactivates the corresponding variable speed limit when the densities of the dominant bottlenecks (which are found offline) reach predefined thresholds. These thresholds are defined in order to activate and deactivate the speed limits at the same bottleneck density at which they would be activated or deactivated in the nominal case. The simulation results show that SPERT is able to approach the optimal behavior while eliminating online computational cost, increasing robustness, and outperforming previously proposed easy-to-implement VSL control algorithms.

This paper proposes a new ramp metering control algorithm, Feed-Foward ALINEA (FF-ALINEA), for bottlenecks located both nearby on an on-ramp and further away from it (i.e., more than just a few hundred meters). The formulation of the controller is based on a feed-forward modification of the well-known control algorithm for ramp metering, ALINEA. The feed-forward structure allows anticipating the future evolution of the bottleneck density in order to avoid or reduce traffic breakdowns. The proposed controller is tested, using the macroscopic traffic flow model METANET, for nine scenarios, and the results are compared with the ones obtained with ALINEA, PI-ALINEA, and with the optimal solution. The simulations show that the FF-ALINEA is able to approach the optimal behavior, thereby outperforming ALINEA and PI-ALINEA. Moreover, results indicate that the FF-ALINEA is quite robust in cases where different demands are considered, there are a limited number of available detectors, or there are errors in the estimation of the capacity and/or the critical density of the bottleneck.

The corrections given involveEquations (4),(5),(9)in the S-model and Equations (13)–(16) in the integrated flow and emission model introduced in[1]. For details and extensive proofs of the proposed corrections and modifications, we refer the readers to[2]and[3]. Moreover, we propose some extensions/modifications to[1, eq. (9)]that result in a simple formula that can be used to compute the integral in[1, eq. (9)]. Since the S-model was originally introduced in[4], the corrections and modifications given for the S-model in this paper also hold for[4].

Freeway congestion can reduce the freeway throughput due to the capacity drop or due to blocking caused by spillback to upstream ramps. Research has shown that congestion can be reduced by the application of ramp metering and variable speed limits. Model predictive control is a promising strategy for the optimization of the ramp metering rates and variable speed limits to improve the freeway throughput. However, several challenges have to be addressed before it can be applied for the control of freeway traffic. This paper focuses on the challenge of reducing the computation time of MPC strategies for the integration of variable speed limits and ramp metering. This is realized via a parameterized control strategy that optimizes the upstream and downstream boundaries of a speed-limited area and the parameters of the ALINEA ramp metering strategy. Due to the parameterization, the solution space reduces substantially, leading to an improved computation time. More specifically, the number of optimization variables for the variable speed limit strategy becomes independent of the number of variable message signs, and the number of optimization variables for the ramp metering strategy becomes independent of the prediction horizon. The control strategy is evaluated with a macroscopic model of a two-lane freeway with two ON-ramps and OFF-ramps. It is shown that parameterization realizes improved throughput when compared with a non-parameterized strategy when using the same amount of computation time.

Due to the lack of efficient control methods for a fleet of vehicles throughout a road network, the large-scale application of cybercars, which are fully automatic road vehicles providing on-demand and door-to-door transportation service, is still hindered. Although the fleet control problem for cybercars can be straightforwardly addressed in a centralized control setting, for reasons of scalability and fast computation, a centralized control method will not be tractable for the large-scale use of cybercars in the future. In this paper, we focus on the dynamic routing of a fleet of cybercars considering minimization of the combined system cost including the total time spent and the total energy consumption by all cybercars. We first propose a model of the dynamics and the energy consumption of a fleet of cybercars based on a description of the dynamics of every single cybercar and the states of the road network. After that, we propose several tractable and scalable multi-agent control methods including multi-agent model predictive control and parameterized control for the dynamic routing of cybercars. Finally, experiments by means of numerical simulations illustrate the performance of the proposed control methods.

Numerous prognostic methods have been developed, aiming at predicting future system reliability with the highest possible accuracy. It is striking that the relation with the subsequent maintenance optimization process is generally overlooked, while it is important in practice. Additionally, almost all existing methods are based on a single degradation measure, and focus on systems with only one degradation and failure mode. In practice, however, multiple degradation measures are often available and needed to adequately predict future system degradation. Moreover, systems may suffer from various kinds of faults, all resulting in different degradation behaviors. To accommodate these properties, we establish a link between failure prognosis and maintenance optimization, and accordingly propose a multivariate multiple-model approach to system reliability prediction. We conclude that in the presence of multiple degradation modes and provided they are correctly identified, a multiple-model approach outperforms a single-model approach with respect to the prediction accuracy. Moreover, in the presence of multiple degradation and failure modes, overall predictions of the remaining useful life as generated by common prognostic approaches are not directly suited for maintenance decision making, as different kinds of system failures and maintenance activities are associated with different costs. In contrast, our approach yields conditional predictions of future system reliability, which much better suit the maintenance optimization process.

The main aim of this paper is to use multi-class macroscopic traffic flow and emission models for Model Predictive Control (MPC) for traffic networks. In particular, we use and compare extended versions of multi-class METANET, FASTLANE, multi-class VT-macro, and multi-class VERSIT+. In addition, end-point penalties based on these multi-class traffic flow and emission models are also included in the objective function of MPC to account for the behavior of the traffic system beyond the prediction horizon. A simulation experiment is implemented to evaluate the multi-class models. The results show that the approaches based on multi-class METANET and the extended emission models (multi-class VT-macro or multi-class VERSIT+) can improve the control performance for the total time spent and the total emissions with respect to the non-control case, and they are more capable of dealing with the queue length constraints than the approaches based on FASTLANE. Including end-point penalties can further improve the control performance with a small sacrifice in the computational efficiency for the approaches based on multi-class METANET but not for the approaches based on FASTLANE.

The global growth in urbanisation increases the demand for services including road transport infrastructure, presenting challenges in terms of mobility. In this scenario, optimising the exploitation of urban road networks is a pivotal challenge. Existing urban traffic control approaches, based on complex mathematical models, can effectively deal with planned-ahead events, but are not able to cope with unexpected situations --such as roads blocked due to car accidents or weather-related events-- because of their huge computational requirements. Therefore, such unexpected situations are mainly dealt with manually, or by exploiting pre-computed policies. Our goal is to show the feasibility of using mixed discrete-continuous planning to deal with unexpected circumstances in urban traffic control. We present a PDDL+ formulation of urban traffic control, where continuous processes are used to model flows of cars, and show how planning can be used to efficiently reduce congestion of specified roads by controlling traffic light green phases. We present simulation results on two networks (one of them considers Manchester city centre) that demonstrate the effectiveness of the approach, compared with fixed-time and reactive techniques.

In this paper, we propose a deep convolutional neural network solution to the analysis of image data for the detection of rail surface defects. The images are obtained from many hours of automated video recordings. This huge amount of data makes it impossible to manually inspect the images and detect rail surface defects. Therefore, automated detection of rail defects can help to save time and costs, and to ensure rail transportation safety. However, one major challenge is that the extraction of suitable features for detection of rail surface defects is a non-trivial and difficult task. Therefore, we propose to use convolutional neural networks as a viable technique for feature learning. Deep convolutional neural networks have recently been applied to a number of similar domains with success. We compare the results of different network architectures characterized by different sizes and activation functions. In this way, we explore the efficiency of the proposed deep convolutional neural network for detection and classification. The experimental results are promising and demonstrate the capability of the proposed approach.

This paper presents a stochastic model predictive control problem for a class of discrete event systems, namely stochastic max-plus linear systems, which are of wide practical interest as they appear in many application domains for timing and synchronization studies. The objective of the control problem is to minimize a cost function under constraints on states, inputs and outputs of such a system in a receding horizon fashion. In contrast to the pessimistic view of the robust approach on uncertainty, the stochastic approach interprets the constraints probabilistically, allowing for a sufficiently small violation probability level. In order to address the resulting nonconvex chance-constrained optimization problem, we present two ideas in this paper. First, we employ a scenario-based approach to approximate the problem solution, which optimizes the control inputs over a receding horizon, subject to the constraint satisfaction under a finite number of scenarios of the uncertain parameters. Second, we show that this approximate optimization problem is convex with respect to the decision variables and we provide a-priori probabilistic guarantees for the desired level of constraint fulfillment. The proposed scheme improves the results in the literature in two distinct directions: we do not require any assumption on the underlying probability distribution of the system parameters; and the scheme is applicable to high dimensional problems, which makes it suitable for real industrial applications. The proposed framework is demonstrated on a two-dimensional production system and it is also applied to a subset of the Dutch railway network in order to show its scalability and study its limitations.

In this paper, the recently developed link transmission model (LTM) is utilized in an online hybrid model-based predictive control (MPC) framework. The model is extended to include the effects of ramp metering and variable speed limits. Next, an integrated freeway traffic control based on the new model is presented in order to minimize the total time spent in the network. The integrated scheme has the capability of controlling large-scale freeway networks in real time as the model is computationally efficient, and it is yet accurate enough for our control purposes. In addition, the extended model is reformulated as a system of linear inequalities with mixed binary and real variables. The reformulated model along with the linearized total travel time objective function establish a mixed-integer linear optimization problem that is more tractable and even faster than the original optimization problem integrated in the MPC scheme. Finally, to investigate the performance of the proposed approaches (nonlinear MPC and the mixed-integer linear counterpart), a freeway network layout based on the Leuven Corridor in Belgium is selected. The extended LTM is calibrated for this network using microsimulation data and then is used for prediction and control of the large network. Microsimulation results show that the proposed methods are able to efficiently improve the total travel time.

In the literature, several road pricing methods based on hierarchical Stackelberg games have been proposed to reduce congestion in traffic networks. We propose three novel schemes to apply the extended reverse Stackelberg game, through which traffic authorities can induce drivers to follow routes that are computed to reach a system-optimal distribution of traffic on the available routes of a freeway, e.g., to minimize the total time spent of traffic in the network and to reduce traffic emissions in urban traffic networks. In this game-theoretical approach, the leader player representing the traffic authority communicates with the followers (drivers) via an onboard computer, in which the main instrument of the leader is the so-called leader function. This function maps the follower's decision space into the leader's decision space, resulting in a leader decision that is directly dependent on the follower's decision variables. Compared with the original game, we can rely on solution methods developed for the general reverse Stackelberg game and show that a system-optimal behavior can be reached, while taking heterogeneous driver classes into account.

The nine papers in this special section were presented at the 16th IEEE International Conference on Intelligent Transportation Systems (ITSC’13), which was held in The Hague, The Netherlands, on October 6–9, 2013.

The real-time train scheduling problem for urban rail transit systems is considered with the aim of minimizing the total travel time of passengers and the energy consumption of the operation of trains. Based on the passenger demand in the urban rail transit system, the optimal departure times, running times, and dwell times are obtained by solving the scheduling problem. A new iterative convex programming (ICP) approach is proposed to solve the train scheduling problem. The performance of the ICP approach is compared with other alternative approaches, i.e., nonlinear programming approaches, a mixed-integer nonlinear programming (MINLP) approach, and a mixed-integer linear programming (MILP) approach. In addition, this paper formulates the real-time train scheduling problem with stop-skipping and shows how to solve it using an MINLP approach and an MILP approach. The ICP approach is shown, via a case study, to provide a better tradeoff between performance and computational complexity for the real-time train scheduling problem. Furthermore, for the train scheduling problem with stop-skipping, the MINLP approach turns out to have a good tradeoff between the control performance and the computational efficiency.

This paper discusses the applicability of Big Data techniques to facilitate maintenance decisions regarding railway tracks. Currently, in different countries, a huge amount of railway track condition-monitoring data is being collected from different sources. However, the data are not yet fully used because of the lack of suitable techniques to extract the relevant events and crucial historical information. Thus, valuable information is hidden behind a huge amount of terabytes from different sensors. In this paper, the conditions of the 5V's of Big Data (Volume, Velocity, Variety, Veracity and Value) in railway monitoring systems are discussed. Then, general methods that can be applied to facilitate the decision of efficient railway track maintenance are proposed for railway track condition monitoring. As a benchmark, axle box acceleration (ABA) measurements in the Dutch tracks are used, and generic reduction formulations to address new relevant information and handle failures are proposed.

The train scheduling problem for urban rail transit systems is considered with the aim of minimizing the total travel time of passengers and the energy consumption of the trains. We adopt a model-based approach, where the model includes the operation of trains at the terminus and at the stations. In order to adapt the train schedule to the origin-destination-dependent passenger demand in the urban rail transit system, a stop-skipping strategy is adopted to reduce the passenger travel time and the energy consumption. An efficient bilevel optimization approach is proposed to solve this train scheduling problem, which actually is a mixed-integer nonlinear programming problem. The performance of the new efficient bilevel approach is compared with the existing bilevel approach. In addition, we also compare the stop-skipping strategy with the all-stop strategy. The comparison is performed through a case study inspired by real data from the Beijing Yizhuang line. The simulation results show that the efficient bilevel approach and the existing bilevel approach have a similar performance but the computation time of the efficient bilevel approach is around one magnitude smaller than that of the bilevel approach.

We present a gait generation framework for multi-legged robots based on max-plus algebra that is endowed with intrinsically safe gait transitions. The time schedule of each foot liftoff and touchdown is modeled by sets of max-plus linear equations. The resulting discrete-event system is translated to continuous time via piecewise constant leg phase velocities; thus, it is compatible with traditional central pattern generator approaches. Different gaits and gait parameters are interleaved by utilizing different max-plus system matrices. We present various gait transition schemes and show that optimal transitions, in the sense of minimizing the stance time variation, allow for constant acceleration and deceleration on legged platforms. The framework presented in this paper relies on a compact representation of the gait space, provides guarantees regarding the transient and steady-state behavior, and results in simple implementations on legged robotic platforms.

In control of large-scale intelligent infrastructures, multilevel optimization can serve as a useful framework to deal with overall complex problems, especially in networks that already exhibit a natural hierarchy of decision makers with different objectives. In particular, solution methods from the hierarchical game theory can be adopted. Here, we focus on solving problems that belong to the specific class of reverse Stackelberg games. In this game, a follower player acts subsequent to the leader's revelation of her so-called leader function, which maps the leader decision space to the follower decision space. In general, the problem of finding a leader function such that the leader's objective function is optimized while taking into account a follower decision that is optimal for the follower, is difficult to solve. We provide a structured solution approach for the class of nonlinear leader functions and make a comparison with the evolutionary approaches proposed in the literature. In particular, a continuous multilevel optimization approach and a gridding approach are proposed to compute an optimal leader function based on basis functions. Also, leader functions derived by interpolation are discussed. All approaches are illustrated and compared in a worked example, in which the required computation times and deviation from the desired solution are considered.

Ant Colony Optimization (ACO) has proved to be a powerful metaheuristic for combinatorial optimization problems. From a theoretical point of view, the convergence of the ACO algorithm is an important issue. In this paper, we analyze the convergence properties of a recently introduced ACO algorithm, called ACO with stench pheromone (ACO-SP), which can be used to solve dynamic traffic routing problems through finding the minimum cost routes in a traffic network. This new algorithm has two different types of pheromone: the regular pheromone that is used to attract artificial ants to the arc in the network with the lowest cost, and the stench pheromone that is used to push ants away when too many ants converge to that arc. As a first step of a convergence proof for ACO-SP, we consider a network with two arcs. We show that the process of pheromone update will transit among different modes, and finally stay in a stable mode, thus proving convergence for this given case.

This paper addresses the computational intractability of traffic control when applying the integrated METANET freeway traffic model and the VT-macro emission model in a model-based predictive control (MPC) framework. To facilitate real-time implementation, a piecewise-affine (PWA) approximation of the nonlinear METANET model is proposed. While a direct MPC approach based on the full PWA model is intractable for online applications, a conversion to a mixed-logical dynamical (MLD) model description is made instead. The resulting MLD-MPC problem, which is written as a mixed-integer linear program (MILP), can be solved much more efficiently as it does not explicitly state all model equations for each particular region. As a benchmark, the computational efficiency and accuracy of the MLD-MPC approach is tested on a case study including variable speed limits and a metered on-ramp while optimizing the total time spent (TTS) and taking into account emissions and fuel consumption of the vehicles. The performance is evaluated against the original nonlinear and nonconvex MPC problem and shows an improved computational speed at the cost of some deviation in the cost function values.

Refining transportation mobility and improving the living environment are two important issues that need to be addressed in urban traffic. To reduce traffic delays and traffic emissions for urban traffic networks, this paper first proposes an integrated macroscopic traffic model that integrates a macroscopic urban traffic flow model with a microscopic traffic emission model for individual vehicles. This integrated model is able to predict the traffic flow states and the emissions released by every vehicle at different operational conditions, i.e., the speed and the acceleration. Then, model predictive control (MPC) is applied to control urban traffic networks based on this integrated traffic model, aiming at reducing both travel delays and traffic emissions of different gases. Finally, simulations are performed to assess this multiobjective control approach. The obtained simulation results illustrate the control effects of the model predictive controller.

We present an integrated traffic management and control approach for automated highway systems (AHS). The AHS consist of interacting roadside controllers and intelligent vehicles that are organized in platoons with short intraplatoon distances and larger distances between platoons. All vehicles are assumed to be fully automated, i.e., throttle, braking, and steering commands are determined by an automated onboard controller. The proposed control approach is based on a hierarchical traffic control architecture for AHS, and it also takes the connection and transition between the nonautomated part of the road network and the AHS into account. In particular, we combine dynamic speed limits and lane allocation for the platoons on the AHS highways with access control for the on-ramps using ramp metering, and we propose a model-based predictive control approach to determine optimal speed limits and lane allocations, as well as optimal release times for the platoons at the on-ramps. To illustrate the potential of the proposed traffic control method, we apply it to a simple simulation example.

We present a freeway-traffic control strategy that continuously adapts traffic control measures to prevailing traffic conditions and features faster computation speed than conventional model-based predictive control (MPC). The control approach is based on the principles of state feedback control and MPC. Instead of computing the control input sequence, the proposed controller optimizes the parameters of control laws that parametrize the control input sequences. This way, the computational burden of the controller is substantially reduced. We demonstrate the proposed control approach on a calibrated model of part of the Dutch A12 freeway using variable speed limits and ramp-metering rate.

Reinforcement learning (RL) allows agents to learn how to optimally interact with complex environments. Fueled by recent advances in approximation-based algorithms, RL has obtained impressive successes in robotics, artificial intelligence, control, operations research, etc. However, the scarcity of survey papers about approximate RL makes it difficult for newcomers to grasp this intricate field. With the present overview, we take a step toward alleviating this situation. We review methods for approximate RL, starting from their dynamic programming roots and organizing them into three major classes: approximate value iteration, policy iteration, and policy search. Each class is subdivided into representative categories, highlighting among others offline and online algorithms, policy gradient methods, and simulation-based techniques. We also compare the different categories of methods, and outline possible ways to enhance the reviewed algorithms.

We propose an online planning algorithm for finite-action, sparsely stochastic Markov decision processes, in which the random state transitions can only end up in a small number of possible next states. The algorithm builds a planning tree by iteratively expanding states, where each expansion exploits sparsity to add all possible successor states. Each state to expand is actively chosen to improve the knowledge about action quality, and this allows the algorithm to return a good action after a strictly limited number of expansions. More specifically, the active selection method is optimistic in that it chooses the most promising states first, so the novel algorithm is called optimistic planning for sparsely stochastic systems. We note that the new algorithm can also be seen as model-predictive (receding-horizon) control. The algorithm obtains promising numerical results, including the successful online control of a simulated HIV infection with stochastic drug effectiveness.

In this paper we compare four methods for decentralized Kalman filtering for distributed-parameter systems, which after spatial and temporal discretization, result in large-scale linear discrete-time systems. These methods are: parallel information filter, distributed information filter, distributed Kalman filter with consensus filter, and distributed Kalman filter with weighted averaging. These filters are suitable for sensor networks, where the sensor nodes perform not only sensing and computations, but also communicate estimates among each other. We consider an application of sensor networks to a heat conduction process. The performance of the decentralized filters is evaluated and compared to the centralized Kalman filter.

Switching gaits in many-legged robots can present challenges due to the combinatorial nature of the gait space. In this paper we present an intrinsically safe gait switching generator that minimizes the velocity variance of all the legs in stance, allowing for smooth acceleration in legged robots. The gait switching generator is modeled as a max-plus linear discrete event system which is translated to continuous time via a reference trajectory generator.

With the increasing application of distributed energy resources and novel information technologies in the electricity infrastructure, innovative possibilities to incorporate the demand side more actively in power system operation are enabled. A promising, controllable, residential distributed generation technology is a microcombined heat and power system (micro-CHP). Micro-CHP is an energy-efficient technology that simultaneously provides heat and electricity to households. In this paper, we investigate to what extent domestic energy costs could be reduced with intelligent, price-based control concepts (demand response). Hereby, first the performance of a standard, so-called heat-led micro-CHP system is analyzed. Then, a model-predictive control (MPC) strategy aimed at demand response is proposed for more intelligent control of micro-CHP systems. Simulation studies illustrate the added value of the proposed intelligent control approach over the standard approach in terms of reduced variable energy costs. Demand response with micro-CHP lowers variable costs for households by about 1%-14%. The cost reductions are highest with the most strongly fluctuating real-time pricing scheme.

In this paper, an advanced control strategy, i.e., model predictive control (MPC), is applied to control and coordinate urban traffic networks. However, due to the nonlinearity of the prediction model, the optimization of MPC is a nonlinear nonconvex optimization problem. In this case, the online computational complexity becomes a big challenge for the MPC controller if it is implemented in a real-life traffic network. To overcome this problem, the online optimization problem is reformulated into a mixed-integer linear programming (MILP) optimization problem to increase the real-time feasibility of the MPC control strategy. The new optimization problem can be very efficiently solved by existing MILP solvers, and the global optimum of the problem is guaranteed. Moreover, we propose an approach to reduce the complexity of the MILP optimization problem even further. The simulation results show that the MILP-based MPC controllers can reach the same performance, but the time taken to solve the optimization becomes only a few seconds, which is a significant reduction, compared with the time required by the original MPC controller.

This paper introduces an algorithm for direct search of control policies in continuous-state discrete-action Markov decision processes. The algorithm looks for the best closed-loop policy that can be represented using a given number of basis functions (BFs), where a discrete action is assigned to each BF. The type of the BFs and their number are specified in advance and determine the complexity of the representation. Considerable flexibility is achieved by optimizing the locations and shapes of the BFs, together with the action assignments. The optimization is carried out with the cross-entropy method and evaluates the policies by their empirical return from a representative set of initial states. The return for each representative state is estimated using Monte Carlo simulations. The resulting algorithm for cross-entropy policy search with adaptive BFs is extensively evaluated in problems with two to six state variables, for which it reliably obtains good policies with only a small number of BFs. In these experiments, cross-entropy policy search requires vastly fewer BFs than value-function techniques with equidistant BFs, and outperforms policy search with a competing optimization algorithm called DIRECT.

In this paper, we discuss the Ant Colony Learning (ACL) paradigm for non-linear systems with continuous state spaces. ACL is a novel control policy learning methodology, based on Ant Colony Optimization. In ACL, a collection of agents, called ants, jointly interact with the system at hand in order to find the optimal mapping between states and actions. Through the stigmergic interaction by pheromones, the ants are guided by each others experience towards better control policies. In order to deal with continuous state spaces, we generalize the concept of pheromones and the local and global pheromone update rules. As a result of this generalization, we can integrate both crisp and fuzzy partitioning of the state space into the ACL framework. We compare the performance of ACL with these two partitioning methods by applying it to the control problem of swinging-up and stabilizing an under-actuated pendulum.

State-of-the-art baggage handling systems transport luggage in an automated way using destination coded vehicles (DCVs). These vehicles transport the bags at high speeds on a network of tracks. Currently, the DCVs are routed through the system using routing schemes based on preferred routes. These routing schemes respond to the occurrence of predefined events. We do not consider such predefined preferred routes. Instead, we develop advanced control methods to determine the optimal routing in case of dynamic demand. In order to optimize the performance of the system, we first develop and compare efficient centralized, decentralized, and distributed predictive methods. Next, to reduce the computational requirements, we also propose some heuristic methods. Finally, to assess the performance of the proposed control approaches, the methods are compared for several scenarios on a benchmark case study.

This paper introduces a novel algorithm for approximate policy search in continuous-state, discrete-action Markov decision processes (MDPs). Previous policy search approaches have typically used ad-hoc parameterizations developed for specific MDPs. In contrast, the novel algorithm employs a flexible policy parameterization, suitable for solving general discrete-action MDPs. The algorithm looks for the best closed-loop policy that can be represented using a given number of basis functions, where a discrete action is assigned to each basis function. The locations and shapes of the basis functions are optimized, together with the action assignments. This allows a large class of policies to be represented. The optimization is carried out with the cross-entropy method and evaluates the policies by their empirical return from a representative set of initial states. We report simulation experiments in which the algorithm reliably obtains good policies with only a small number of basis functions, albeit at sizable computational costs.

Irrigation canals transport water from water sources (such as large rivers and lakes) to water users (such as farmers). Irrigation canals are typically very large in nature, covering vast geographical areas, and involving a significant number of control actuators, such as pumps, gates, and locks. The control of such canals is aimed at guaranteeing the adequate delivery of water with minimal water spillage and with minimal control structure usage. To take into account forecasts on, e.g., water consumption and weather, model predictive control (MPC) can be used to determine which actions to take. For large-scale systems, in which different parts of the canal are owned by different parties, distributed MPC control could then be employed. Although iterative distributed MPC approaches proposed earlier in the literature may yield overall optimal performance, the amount of iterations required before achieving this performance may be large, and thus require a significant amount of time. In this paper, the structure of systems consisting of serially interconnected subsystems is exploited to obtain an efficient non-iterative, cascaded MPC scheme. Simulation studies on a 7-reach irrigation canal illustrate the performance of this non-iterative scheme in comparison with an iterative scheme.

A large class of nonlinear systems can be well approximated by Takagi-Sugeno (TS) fuzzy models with linear or affine consequents. It is well known that the stability of these consequent models does not ensure the stability of the overall fuzzy system. Therefore, several stability conditions have been developed for TS fuzzy systems. We study a special class of nonlinear dynamic systems that can be decomposed into cascaded subsystems, which are represented as TS fuzzy models. We analyze the stability of the overall TS system based on the stability of the subsystems and prove that the stability of the subsystems implies the stability of the overall system. The main benefit of this approach is that it relaxes the conditions imposed when the system is globally analyzed, thereby solving some of the feasibility problems. Another benefit is that by using this approach, the dimension of the associated linear matrix inequality (LMI) problem can be reduced. For naturally distributed applications, such as multiagent systems, the construction and tuning of a centralized observer may not be feasible. Therefore, we also extend the cascaded approach to the observer design and use fuzzy observers to individually estimate the states of these subsystems. A theoretical proof of stability and simulation examples are presented. The results show that the distributed observer achieves the same performance as the centralized one, while leading to increased modularity, reduced complexity, lower computational costs, and easier tuning. Applications of such cascaded systems include multiagent systems, distributed process control, and hierarchical large-scale systems.

The 13 papers in this special section were originally presented at the 2008 IEEE Intelligent Vehicles Symposium (IV'08) held in Eindhoven, The Netherlands, on June 4-6, 2008.

Ant Colony Optimization (ACO) has proven to be a very powerful optimization heuristic for Combinatorial Optimization Problems (COPs). It has been demonstrated to work well when applied to various NP-complete problems, such as the traveling salesman problem. In this paper, an ACO approach to optimal control is proposed. This approach requires that a continuous-time, continuous-state model of the system, together with a finite action set, is formulated as a discrete, non-deterministic automaton. The control problem is then translated into a stochastic COP. This method is applied to the time-optimal swing-up and stabilization of a pendulum.

Swarms are characterized by the ability to generate complex behavior from the coupling of simple individuals. While the swarm approach to distributed systems of moving agents is gradually finding a way to engineering applications, a true successful demonstration of an engineered swarm is still missing. One of the reasons for this is the gap between the complexity of the swarms studied in fundamental research and the complexity needed for the application to interesting control problems. In the majority of the research on swarm intelligent systems, the moving agents in the swarm are modeled as simple reactive agents. This model comprises too little intelligence to fully exploit the potential of swarms. In this paper, a general comprehensive swarm framework is introduced and related to the established state of the art. Such a framework is novel and it is a first and important step in the development and analysis of more complex and intelligent swarms.

Reinforcement learning (RL) is a widely used paradigm for learning control. Computing exact RL solutions is generally only possible when process states and control actions take values in a small discrete set. In practice, approximate algorithms are necessary. In this paper, we propose an approximate, model-based Q-iteration algorithm that relies on a fuzzy partition of the state space, and on a discretization of the action space. Using assumptions on the continuity of the dynamics and of the reward function, we show that the resulting algorithm is consistent, i.e., that the optimal solution is obtained asymptotically as the approximation accuracy increases. An experimental study indicates that a continuous reward function is also important for a predictable improvement in performance as the approximation accuracy increases.

A large class of nonlinear systems can be well approximated by Takagi-Sugeno (TS) fuzzy models, with linear or affine consequents. It is well-known that the stability of these consequent models does not ensure the stability of the overall fuzzy system. Stability conditions developed for TS fuzzy systems in general rely on the feasibility of an associated system of linear matrix inequalities, whose complexity may grow exponentially with the number of rules. We study distributed systems, where the subsystems are represented as TS fuzzy models. For such systems, a centralized analysis is often unfeasible. We analyze the stability of the overall TS system based on the stability of the subsystems and the strength of the interconnection terms. For naturally distributed applications, such as multi-agent systems, when adding new subsystems “on-line”, the construction and tuning of a centralized observer is often intractable. Therefore, we also propose a decentralized approach to observer design. Applications of such systems include distributed process control, traffic networks, and economic systems.

Multiagent systems are rapidly finding applications in a variety of domains, including robotics, distributed control, telecommunications, and economics. The complexity of many tasks arising in these domains makes them difficult to solve with preprogrammed agent behaviors. The agents must, instead, discover a solution on their own, using learning. A significant part of the research on multiagent learning concerns reinforcement learning techniques. This paper provides a comprehensive survey of multiagent reinforcement learning (MARL). A central issue in the field is the formal statement of the multiagent learning goal. Different viewpoints on this issue have led to the proposal of many different goals, among which two focal points can be distinguished: stability of the agents' learning dynamics, and adaptation to the changing behavior of the other agents. The MARL algorithms described in the literature aim---either explicitly or implicitly---at one of these two goals or at a combination of both, in a fully cooperative, fully competitive, or more general setting. A representative selection of these algorithms is discussed in detail in this paper, together with the specific issues that arise in each category. Additionally, the benefits and challenges of MARL are described along with some of the problem domains where the MARL techniques have been applied. Finally, an outlook for the field is provided.

Reinforcement learning (RL) is a learning control paradigm that provides well-understood algorithms with good convergence and consistency properties. Unfortunately, these algorithms require that process states and control actions take only discrete values. Approximate solutions using fuzzy representations have been proposed in the literature for the case when the states and possibly the actions are continuous. However, the link between these mainly heuristic solutions and the larger body of work on approximate RL, including convergence results, has not been made explicit. In this paper, we propose a fuzzy approximation structure for the Q-value iteration algorithm, and show that the resulting algorithm is convergent. The proof is based on an extension of previous results in approximate RL. We then propose a modified, serial version of the algorithm that is guaranteed to converge at least as fast as the original algorithm. An illustrative simulation example is also provided.

A large class of nonlinear systems can be well approximated by Takagi-Sugeno (TS) fuzzy models, with local models often chosen linear or affine. It is well-known that the stability of these local models does not ensure the stability of the overall fuzzy system. Therefore, several stability conditions have been developed for TS fuzzy systems. We study a special class of nonlinear dynamic systems, that can be decomposed into cascaded subsystems. These subsystems are represented as TS fuzzy models. We analyze the stability of the overall TS system based on the stability of the subsystems. For a general nonlinear, cascaded system, global asymptotic stability of the individual subsystems is not sufficient for the stability of the cascade. However, for the case of TS fuzzy systems, we prove that the stability of the subsystems implies the stability of the overall system. The main benefit of this approach is that it relaxes the conditions imposed when the system is globally analyzed, therefore solving some of the feasibility problems. Another benefit is, that by using this approach, the dimension of the associated linear matrix inequality (LMI) problem can be reduced. Applications of such cascaded systems include multi-agent systems, distributed process control and hierarchical large-scale systems.

The ten papers in this special section are revised and extended versions of papers presented at the 8th IEEE International Conference on Intelligent Transportation Systems (ITSC'05) held in Vienna, Austria, on September 13-16, 2005. The papers in this section are briefly summarized here.

Multi-agent systems are rapidly finding applications in a variety of domains, including robotics, distributed control, telecommunications, economics. Many tasks arising in these domains require that the agents learn behaviors online. A significant part of the research on multi-agent learning concerns reinforcement learning techniques. However, due to different viewpoints on central issues, such as the formal statement of the learning goal, a large number of different methods and approaches have been introduced. In this paper we aim to present an integrated survey of the field. First, the issue of the multi-agent learning goal is discussed, after which a representative selection of algorithms is reviewed. Finally, open issues are identified and future research directions are outlined

Multi-agent systems are rapidly finding applications in a variety of domains, including robotics, distributed control, telecommunications, etc. Learning approaches to multi-agent control, many of them based on reinforcement learning (RL), are investigated in complex domains such as teams of mobile robots. However, the application of decentralized RL to low-level control tasks is not as intensively studied. In this paper, we investigate centralized and decentralized RL, emphasizing the challenges and potential advantages of the latter. These are then illustrated on an example: learning to control a two-link rigid manipulator. Some open issues and future research directions in decentralized RL are outlined

When freeway traffic is dense, shock waves may appear. These shock waves result in longer travel times and in sudden large variations in the speeds of the vehicles, which could lead to unsafe situations. Dynamic speed limits can be used to eliminate or at least to reduce the effects of shock waves. However, coordination of the variable speed limits is necessary in order to prevent the occurrence of new shock waves and/or a negative impact on the traffic flows in other locations. In this paper, we present a model predictive control approach to optimally coordinate variable speed limits for freeway traffic with the aim of suppressing shock waves. First, we optimize continuous valued speed limits, such that the total travel time is minimal. Next, we include a safety constraint that prevents drivers from encountering speed limit drops larger than, e.g., 10 km/h. Furthermore, to get a better correspondence between the computed and applied control signals, we also consider discrete speed limits. We illustrate our approach with a benchmark problem.