This study presents an innovative approach to optimal gait control for a soft quadruped robot enabled by four compressible tendon-driven soft actuators. Soft quadruped robots, compared to their rigid counterparts, are widely recognized for offering enhanced safety, lower weight, and simpler fabrication and control mechanisms. However, their highly deformable structure introduces nonlinear dynamics, making precise gait locomotion control complex. To solve this problem, we propose a novel model-based reinforcement learning (MBRL) method. The study employs a multi-stage approach, including state space restriction, data-driven surrogate model training, and MBRL development. Compared to benchmark methods, the proposed approach significantly improves the efficiency and performance of gait control policies. The developed policy is both robust and adaptable to the robot's deformable morphology. The study concludes by highlighting the practical applicability of these findings in real-world scenarios.

In the control of discrete-event systems for current-state opacity enforcement, it is difficult to synthesize a supervisor by supervisory control theory (SCT) without explicit formal models of the systems. This study utilizes the reinforcement learning (RL) method to obtain supervisory policies for opacity enforcement in the case when the automaton model of the system is unavailable. The state space of the environment in the RL is dynamically generated through system simulation. Actions are defined according to the control patterns of the SCT. A reward function is proposed to evaluate whether the secret is exposed or not. Then, a sequence of state-action-reward chains are obtained as system simulation goes on. The frameworks of Q-learning and State-Action-Reward-State-Action (SARSA) algorithms are adopted to implement the proposed approach. The goal of the training is to maximize the total accumulative reward by optimizing the action selection in the learning process. Then, an optimal supervisory policy is obtained when the training process converges. Experiments are performed to illustrate the effectiveness of the proposed approach. The contributions are two aspects. Firstly, a supervisor for opacity enforcement is learned by RL training without an explicit formal model of the system. Secondly, the ability of the proposed method in computing supervisory policies without formal models addresses a significant gap in the literature and offers a new direction for research in opacity enforcement in discrete event systems. Note to Practitioners—Supervisory Control Theory (SCT) supplies an effective way to synthesize supervisors, which traditionally handles tasks with explicit system models for current-state opacity enforcement by restricting behavior of systems. However, formal models of systems are often confidential or otherwise unavailable. This paper presents a method for supervisor synthesis via reinforcement learning in the case of lacking formal models of systems. The proposed method leverages the characteristics of control patterns in SCT and optimizes the action selection in the training process through a reward mechanism that evaluates the secrecy of states. The approach can be applied to model-free RL frameworks such as Q-learning and State-Action-Reward-State-Action (SARSA) algorithms. The training is performed as the system simulation goes on. When the training process converges, the optimal policy can be used to enforce opacity for the system. However, Q-table is used to save the Q-value in both Q-learning and SARSA algorithms. In the worst case, the size of the Q-table grows exponentially with the number of states and controllable events increasing. This can lead to memory exhaustion when the system’s scale is large. To make the approach scalable, we will attempt to use DRL to train control policies for opacity enforcement in the future.

We present the integration of a Non-Axiomatic Reasoning System (NARS) with mobile robots for planning and decision making. NARS enables robots to effectively handle uncertainty in real-time with complete sensor and actuator integration, thereby ensuring adaptability to evolving scenarios. We discuss essential parts of the logic, the architecture and working principles of NARS, and the integration of NARS as a ROS node. A case study is provided demonstrating the system’s proficiency to carry out a garbage collection task in an open-air environment by operating a mobile robot with manipulator arm, and we demonstrate its ability to learn about the place-dependent accumulation of garbage items. Case study also reveals that our approach performs more effectively on the overall task than the Belief-Desire-Intention model we compared with.

Human-centric industrial processes, such as logistics, inspection, maintenance, and complex assembly, heavily rely on human expertise and judgment. In today’s dynamic and complex manufacturing environments, enhancing operator perception is crucial for timely and accurate decision-making. To facilitate effective communication between human workers and the complex factory ecosystem, this research proposes a system framework leveraging Digital Twin (DT) and semantic technologies to manage industrial heterogeneous data and provide operators with real-time insights. The system architecture comprises three primary layers: the Field Layer, the Information and Service Layer, and the Application Layer. The Information Layer integrates four core engines: Knowledge Engine for managing process-specific knowledge, Data Engine for handling streaming data, Artificial Intelligence (AI) Engine for incorporating advanced machine learning models, and 3D Engine for virtual representation and simulation. This paper presents a detailed implementation of the proposed system framework and validates it through a practical in-plant logistics transport use case. Results demonstrate the framework’s effectiveness in enhancing operator perception and decision-making by providing intuitive interfaces and timely insights.

Modular supervisory control may lead to conflicts among the modular supervisors for large-scale discrete event systems. The existing methods for ensuring nonblocking control of modular supervisors either exploit favorable structures in the system model to guarantee the nonblocking property of modular supervisors or employ hierarchical model abstraction methods for reducing the computational complexity of designing a nonblocking coordinator. The nonblocking modular control problem is, in general, NP-hard. This study integrates supervisory control theory and a model-based deep reinforcement learning method to synthesize a nonblocking coordinator for the modular supervisors. The deep reinforcement learning method significantly reduces the computational complexity by avoiding the computation of synchronization of multiple modular supervisors and the plant models. The supervisory control function is approximated by the deep neural network instead of a large-sized finite automaton. Furthermore, the proposed model-based deep reinforcement learning method is more efficient than the standard deep Q network algorithm.

Using soft actuators as legs, soft quadruped robots have shown great potential in traversing unstructured and complex terrains and environments. However, unlike rigid robots whose gaits can be generated using foot pattern design and kinematic model of the rigid legs, the gait generation of soft quadruped robots remains challenging due to the high DoFs of the soft actuators and the uncertain deformations during their contact with the ground. This study is based on a quadruped robot using four Compressible Tendon-driven Soft Actuators (CTSAs) as the legs, with the actuator's compression motion being utilized to improve the walking performance of the robot. For the gait design, an inverse kinematics model considering the compression of the CTSA is developed and validated in simulation. Based on this model, walking gaits realizing different motion speeds and directions are generated. Closed loop direction and speed controllers are developed for increasing the robustness and precision of the robot walking. Simulation and experimental results show that omnidirectional locomotion and complex walking tasks can be realized by tuning the gait parameters and the motions are resistant to external disturbances.

Edge computing is projected to have profound implications in the coming decades, proposed to provide solutions for applications such as augmented reality, predictive functionalities, and collaborative Cyber-Physical Systems (CPS). For such applications, edge computing addresses the new computational needs, as well as privacy, availability, and real-time constraints, by providing local high-performance computing capabilities to deal with the limitations and constraints of cloud and embedded systems. Edge computing is today driven by strong market forces stemming from IT/cloud, telecom, and networking—with corresponding multiple interpretations of “edge computing” (e.g., device edge, network edge, distributed cloud). Considering the strong drivers for edge computing and the relative novelty of the field, it becomes important to understand the specific requirements and characteristics of edge-based CPS, and to ensure that research is guided adequately, e.g., avoiding specific gaps. Our interests lie in the applications of edge computing as part of CPS, where several properties (or attributes) of trustworthiness, including safety, security, and predictability/availability, are of particular concern, each facing challenges for the introduction of edge-based CPS. We present the results of a systematic mapping study, a kind of systematic literature survey, investigating the use of edge computing for CPS with a special emphasis on trustworthiness. The main contributions of this study are a detailed description of the current research efforts in edge-based CPS and the identification and discussion of trends and research gaps. The results show that the main body of research in edge-based CPS only to a very limited extent consider key attributes of system trustworthiness, despite many efforts referring to critical CPS and applications like intelligent transportation. More research and industrial efforts will be needed on aspects of trustworthiness of future edge-based CPS including their experimental evaluation. Such research needs to consider the multiple interrelated attributes of trustworthiness including safety, security, and predictability, and new methodologies and architectures to address them. It is further important to provide bridges and collaboration between edge computing and CPS disciplines.

Model-based systems engineering (MBSE) enables to verify the system performance using system behavior models, which can identify design faults that do not meet the stakeholders’ requirements as early as possible, thus reducing the R&D cost and error risks. Currently, different domain engineers make use of different modeling languages to create their own behavior models. Different behavior models are verified by different approaches. It is difficult to adopt a unified integrated platform to support the modeling and verification of heterogeneous behavior models during the conceptual design phase. This paper proposes a unified modeling and verification approach supporting system formalisms and verification. The KARMA language is used to support the unified formalisms across MBSE models and dynamic simulations for different domain specific models. In order to describe the behavior model more precisely and to facilitate verification, the syntax of hybrid automata is integrated into KARMA. We implemented behavior models and their verification in MetaGraph, a multi-architecture modeling tool. Finally, the effectiveness of the proposed approach is validated by two cases: 1) the scenario of booking railway tickets using BPMN models; 2) the behavior performance simulation of unmanned vehicles using a SysML state machine diagram.

The paper presents an approach to the control of black-box embedded systems by integrating automaton learning and supervisory control theory (SCT) of discrete-event systems (DES), where automaton models of both the system and requirements are unavailable or hard to obtain. First, the system is tested against the requirements. If all the requirements are satisfied, no supervisor is needed and the process terminates. Otherwise, a supervisor is synthesized to enforce the system to satisfy the requirements. To apply SCT and automaton learning technologies efficiently, the system is abstracted to be a finite-discrete model. Then, a C* learning algorithm is proposed based on the classical L* algorithm to infer a Moore automaton describing both the behavior of the system and the conjunctive behavior of the system and the requirements. Subsequently, a supervisor for the system is derived from the learned Moore automaton and patched on the system. Finally, the controlled system is tested again to check the correctness of the supervisor. If the requirements are still not satisfied, a larger Moore automaton is learned and a refined supervisor is synthesized. The whole process iterates until the requirements hold in the controlled system. The effectiveness of the proposed approach is manifested through two realistic case studies.

Autonomous vehicles apply pertinent planning and control algorithms under different driving conditions. The mode switch between these algorithms should also be autonomous. On top of the nominal planners, a safe fallback routine is needed to stop the vehicle at a safe position if nominal operational conditions are violated, such as for a system failure. This paper describes the design and formal verification of a supervisor to manage all requirements for mode switching between nominal planners, and additional requirements for switching to a safe stop trajectory planner that acts as the fallback routine. The supervisor is designed via a model-based approach and its abstraction is formally verified by model checking. The supervisor is implemented and integrated with the Research Concept Vehicle, an experimental research and demonstration vehicle developed at the KTH Royal Institute of Technology. Simulations and experiments show that the vehicle is able to autonomously drive in a safe manner between two parking lots and can successfully come to a safe stop upon GPS sensor failure.

Highly automated road vehicles need the capability of stopping safely in a situation that disrupts continued normal operation, e.g. due to internal system faults. Motion planning for safe stop differs from nominal motion planning, since there is not a specific goal location. Rather, the desired behavior is that the vehicle should reach a stopped state, preferably outside of active lanes. Also, the functionality to stop safely needs to be of high integrity. The first contribution of this paper is to formulate the safe stop problem as a benchmark optimal control problem, which can be solved by dynamic programming. However, this solution method cannot be used in real-time. The second contribution is to develop a real-time safe stop trajectory planning algorithm, based on selection from a precomputed set of trajectories. By exploiting the particular properties of the safe stop problem, the cardinality of the set is decreased, making the algorithm computationally efficient. Furthermore, a monitoring based architecture concept is proposed, that ensures dependability of the safe stop function. Finally, a proof of concept simulation using the proposed architecture and the safe stop trajectory planner is presented.

A fundamental requirement of the supervisory control theory (SCT) of discrete-event systems is a finite automaton model of the plant. The requirement does not hold for black-box systems whose source code and logical model are not accessible. To apply SCT to black-box systems, we integrate automaton learning technology with SCT and apply the new method to improve the requirements conformance of software reuse. If the reused software component does not satisfy a requirement, the method adds a supervisor component to prevent the black-box system from reaching “faulty sections.” The method employs learning-based testing (LBT) to verify whether the reused software meets all requirements in the new context. LBT generates a large number of test cases and iteratively constructs an automaton model of the system under test. If the system fails the test, the learned model is applied as the plant model for control synthesis using SCT. Then, the supervisor is implemented as an executable program to monitor and control the system to follow the requirement. Finally, the integrated system, including the supervisory program and the reused component, is tested by LBT to assure the satisfiability of the requirement. This paper makes two contributions. First, we innovatively integrate LBT and SCT for the control synthesis of black-box reactive systems. Second, software component reuse is still possible even if it does not satisfy user requirements at the outset.

In vehicular communication protocol stacks, received messages may not always be decoded successfully due to the complexity of the decoding functions, the uncertainty of the communication load and the limited computation resources. Even worse, an improper implementation of the protocol stack may cause an unfair data age distribution among all the communicating vehicles (the receiving bias problem). In such cases, some vehicles are almost locked out of the vehicular communication, causing potential safety risk in scenarios such as intersection passing. To our knowledge, this problem has not been systematically studied in the fields of vehicular communication and intelligent transport systems (ITS). This paper analyzes the root of the receiving bias problem and proposes architectural solutions to balance data age distribution. Simulation studies based on commercial devices demonstrate the effectiveness of these solutions. In addition, our system has been successfully applied during the Grand Cooperative Driving Challenge, where complicated scenarios involving platooning maneuvering and intersection coordination were conducted.

There are plentiful attempts for increasing the efficiency, generality and optimality of the Design Space Exploration (DSE) algorithms for resource allocation problems of distributed embedded systems. Most contemporary approaches formulate DSE as an optimization or SAT problem, based on a set of predefined constraints. In this way, the end users lose the flexibility to guide and customize the exploration based on specifics of their actual problem. Besides, during the design of the DSE algorithms, manual formulation is time consuming and error-prone. To solve these problems, 1) a formal representation is defined for capturing customized architectural constraints based on a combination of propositional logic and Pseudo-Boolean (PB) formulas; 2) A process is designed to automatically translate these architectural constrains into corresponding Integer Linear Programming (ILP) constraints, commonly used for DSE. The translation process is also optimized to create ILP formulation with less introduced variables so as to reduce computation time. The results show that the generated constraints correctly reflect the corresponding specification with decent efficiency.

Today, two of the main concerns in transportation industry are reducing fuel consumption and emissions, and tough regulations are put on the vehicle manufacturers in these regards. One of the main approaches towards reducing CO2emissions is hybridization of the powertrain system. Substantial R&D in this area over the last couple of years has resulted in rather optimal components and control strategies, and hence that further substantial improvements are difficult. This motivates research on other energy consuming vehicle subsystems, e.g. pneumatic and hydraulic systems. In this paper, the brake system of a hybrid city bus is studied. A complete electrification of the primary brake system would eliminate the use of low efficiency pneumatics for braking. It is therefore interesting to investigate how much energy can be saved by using electrically actuated and controlled primary brakes. The study is based on simulations in Autonomie which is a MATLAB/SIMULINK based vehicle simulation software package. Different representative driving cycles are studied. It is shown that fuel consumption can be reduced in the range of 0.5 to 1.5% by substituting the pneumatic brake system with a mechatronic one. This may seem limited, but can, combined with substitution of also other less efficient subsystems with their mechatronic counterparts, result in a substantial environmental and economic improvement.