Deep Learning (DL) techniques are at the heart of most future advanced software functions in Critical Autonomous AI-based Systems (CAIS), where they also represent a major competitive factor. Hence, the economic success of CAIS industries (e.g., automotive, space, railway) depends on their ability to design, implement, qualify, and certify DL-based software products under bounded effort/cost. However, there is a fundamental gap between Functional Safety (FUSA) requirements on CAIS and the nature of DL solutions. This gap stems from the development process of DL libraries and affects high-level safety concepts such as (1) explainability and traceability, (2) suitability for varying safety requirements, (3) FUSA-compliant implementations, and (4) real-time constraints. As a matter of fact, the data-dependent and stochastic nature of DL algorithms clashes with current FUSA practice, which instead builds on deterministic, verifiable, and pass/fail test-based software. The SAFEXPLAIN project tackles these challenges and targets by providing a flexible approach to allow the certification - hence adoption - of DL-based solutions in CAIS building on: (1) DL solutions that provide end-to-end traceability, with specific approaches to explain whether predictions can be trusted and strategies to reach (and prove) correct operation, in accordance to certification standards; (2) alternative and increasingly sophisticated design safety patterns for DL with varying criticality and fault tolerance requirements; (3) DL library implementations that adhere to safety requirements; and (4) computing platform configurations, to regain determinism, and probabilistic timing analyses, to handle the remaining non-determinism.

In this work, we present a new single-stage method for subject agnostic face swapping and identity transfer, named FaceDancer. We have two major contributions: Adaptive Feature Fusion Attention (AFFA) and Interpreted Feature Similarity Regularization (IFSR). The AFFA module is embedded in the decoder and adaptively learns to fuse attribute features and features conditioned on identity information without requiring any additional facial segmentation process. In IFSR, we leverage the intermediate features in an identity encoder to preserve important attributes such as head pose, facial expression, lighting, and occlusion in the target face, while still transferring the identity of the source face with high fidelity. We conduct extensive quantitative and qualitative experiments on various datasets and show that the proposed FaceDancer outperforms other state-of-the-art networks in terms of identity transfer, while having significantly better pose preservation than most of the previous methods. Code available at https://github.com/felixrosberg/FaceDance.

Platooning refers to a group of vehicles that—enabled by wireless vehicle-to-vehicle (V2V) communication and vehicle automation—drives with short inter-vehicular distances. Before its deployment on public roads, several challenging traffic situations need to be handled. Among the challenges are cut-in situations, where a conventional vehicle—a vehicle that has no automation or V2V communication—changes lane and ends up between vehicles in a platoon. This paper presents results from a simulation study of a scenario, where a conventional vehicle, approaching from an on-ramp, merges into a platoon of five cars on a highway. We created the scenario with four platooning gaps: 15, 22.5, 30, and 42.5 meters. During the study, the conventional vehicle was driven by 37 test persons, who experienced all the platooning gaps using a driving simulator. The participants’ opinions towards safety, comfort, and ease of driving between the platoon in each gap setting were also collected through a questionnaire. The results suggest that a 15-meter gap prevents most participants from cutting in, while causing potentially dangerous maneuvers and collisions when cut-in occurs. A platooning gap of at least 30 meters yield positive opinions from the participants, and facilitating more smooth cut-in maneuvers while less collisions were observed.

Measuring and comparing facial expression have several practical applications. One such application is to measure the facial expression embedding, and to compare distances between those expressions embeddings in order to determine the identity- and face swapping algorithms' capabilities in preserving the facial expression information. One useful aspect is to present how well the expressions are preserved while anonymizing facial data during privacy aware data collection. We show that a weighted supervised contrastive learning is a strong approach for learning facial expression representation embeddings and dealing with the class imbalance bias. By feeding a classifier-head with the learned embeddings we reach competitive state-of-the-art results. Furthermore, we demonstrate the use case of measuring the distance between the expressions of a target face, a source face and the anonymized target face in the facial anonymization context.

With the emerging connected automated vehicles, 5G and Internet of Things (IoT), vehicles and road infrastructure become connected and cooperative, enabling Cooperative Intelligent Transport Systems (C-ITS). C-ITS are transport system of systems that involves many stakeholders from different sectors. While running their own systems and providing services independently, stakeholders cooperate with each other for improving the overall transport performance such as safety, efficiency and sustainability. Massive information on road and traffic is already available and provided through standard services with different protocols. By reusing and composing the available heterogeneous services, novel value-added applications can be developed. This paper introduces a choreography-based service composition platform, i.e. the CHOReVOLUTION Integrated Development and Runtime Environment (IDRE), and it reports on how the IDRE has been successfully exploited to accelerate the reuse-based development of a choreography-based Urban Traffic Coordination (UTC) application. The UTC application takes the shape of eco-driving services that through real-time eco-route evaluation assist the drivers for the most eco-friendly and comfortable driving experience. The eco-driving services are realized through choreography and they are exploited through a mobile app for online navigation. From specification to deployment to execution, the CHOReVOLUTION IDRE has been exploited to support the realization of the UTC application by automatizing the generation of the distributed logic to properly bind, coordinate and adapt the interactions of the involved parties. The benefits brought by CHOReVOLUTION IDRE have been assessed through the evaluation of a set of Key Performance Indicators (KPIs).

This paper explores the process of designing an automatic multi-sensor drone detection system. Besides the common video and audio sensors, the system also includes a thermal infrared camera, which is shown to be a feasible solution to the drone detection task. Even with slightly lower resolution, the performance is just as good as a camera in visible range. The detector performance as a function of the sensor-to-target distance is also investigated. In addition, using sensor fusion, the system is made more robust than the individual sensors, helping to reduce false detections. To counteract the lack of public datasets, a novel video dataset containing 650 annotated infrared and visible videos of drones, birds, airplanes and helicopters is also presented 1.1. https://github.com/DroneDetectionThesis/Drone-detection-dataset. The database is complemented with an audio dataset of the classes drones, helicopters and background noise.

Several areas have been improved with Deep Learning during the past years. For non-safety related products adoption of AI and ML is not an issue, whereas in safety critical applications, robustness of such approaches is still an issue. A common challenge for Deep Neural Networks (DNN) occur when exposed to out-of-distribution samples that are previously unseen, where DNNs can yield high confidence predictions despite no prior knowledge of the input. In this paper we analyse two supervisors on two well-known DNNs with varied setups of training and find that the outlier detection performance improves with the quality of the training procedure. We analyse the performance of the supervisor after each epoch during the training cycle, to investigate supervisor performance as the accuracy converges. Understanding the relationship between training results and supervisor performance is valuable to improve robustness of the model and indicates where more work has to be done to create generalized models for safety critical applications.

Traffic situations leading up to accidents have been shown to be greatly affected by human errors. To reduce these errors, warning systems such as Driver Alert Control, Collision Warning and Lane Departure Warning have been introduced. However, there is still room for improvement, both regarding the timing of when a warning should be given as well as the time needed to detect a hazardous situation in advance. Two factors that affect when a warning should be given are the environment and the actions of the driver. This study proposes an artificial neural network-based approach consisting of a convolutional neural network and a recurrent neural network with long short-term memory to detect and predict different actions of a driver inside a vehicle. The network achieved an accuracy of 84% while predicting the actions of the driver in the next frame, and an accuracy of 58% 20 frames ahead with a sampling rate of approximately 30 frames per second.

At early market penetration, automated vehicles will share the road with legacy vehicles. For a safe transportation system, automated vehicle controllers therefore need to estimate the behavior of the legacy vehicles. However, mismatches between the estimated and real human behaviors can lead to inefficient control inputs, and even collisions in the worst case. In this paper, we propose a framework for evaluating the impact of model mismatch by interfacing a controller under test with a driving simulator. As a proof- of-concept, an algorithm based on Model Predictive Control (MPC) is evaluated in a braking scenario. We show how model mismatch between estimated and real human behavior can lead to a decrease in avoided collisions by almost 46%, and an increase in discomfort by almost 91%. Model mismatch is therefore non-negligible and the proposed framework is a unique method to evaluate them.

This paper is an experience report of team Halmstad from the participation in a competition organised by the i-GAME project, the Grand Cooperative Driving Challenge 2016. The competition was held in Helmond, The Netherlands, during the last weekend of May 2016. We give an overview of our car's control and communication system that was developed for the competition following the requirements and specifications of the i-GAME project. In particular, we describe our implementation of cooperative adaptive cruise control, our solution to the communication and logging requirements, as well as the high level decision making support. For the actual competition we did not manage to completely reach all of the goals set out by the organizers as well as ourselves. However, this did not prevent us from outperforming the competition. Moreover, the competition allowed us to collect data for further evaluation of our solutions to cooperative driving. Thus, we discuss what we believe were the strong points of our system, and discuss post-competition evaluation of the developments that were not fully integrated into our system during competition time.

Cooperative driving is based on wireless communications between vehicles and between vehicles and roadside infrastructure, aiming for increased traffic flow and traffic safety, while decreasing fuel consumption and emissions. To support and accelerate the introduction of cooperative vehicles in everyday traffic, in 2011, nine international teams joined the Grand Cooperative Driving Challenge (GCDC). The challenge was to perform platooning, in which vehicles drive in road trains with short intervehicle distances. The results were reported in a Special Issue of IEEE Transactions on Intelligent Transportation Systems, published in September 2012 [item 1 in the Appendix].

Cooperative adaptive cruise control and platooning are well-known applications in the field of cooperative automated driving. However, extension toward maneuvering is desired to accommodate common highway maneuvers, such as merging, and to enable urban applications. To this end, a layered control architecture is adopted. In this architecture, the tactical layer hosts the interaction protocols, describing the wireless information exchange to initiate the vehicle maneuvers, supported by a novel wireless message set, whereas the operational layer involves the vehicle controllers to realize the desired maneuvers. This hierarchical approach was the basis for the Grand Cooperative Driving Challenge (GCDC), which was held in May 2016 in The Netherlands. The GCDC provided the opportunity for participating teams to cooperatively execute a highway lane-reduction scenario and an urban intersection-crossing scenario. The GCDC was set up as a competition and, hence, also involving assessment of the teams' individual performance in a cooperative setting. As a result, the hierarchical architecture proved to be a viable approach, whereas the GCDC appeared to be an effective instrument to advance the field of cooperative automated driving.

Vehicle-to-vehicle communication is a key technology for achieving increased perception for automated vehicles, where the communication enables virtual sensing by means of sensors in other vehicles. In addition, this technology also allows detection and recognition of objects that are out-of-sight. This paper presents a trust system that allows a cooperative and automated vehicle to make more reliable and safe decisions. The system evaluates the current situation and generates a trust index indicating the level of trust in the environment, the ego vehicle, and the surrounding vehicles. This research goes beyond secure communication and concerns the verification of the received data on a system level. The results show that the proposed method is capable of correctly identifying various traffic situations and how the trust index is used while manoeuvring in a platoon merge scenario.

With the fifth generation (5G) communication technologies on the horizon, the society is rapidly transformed into a fully connected world. The Future Internet (FI) is foreseeable to consist of an infinite number of software components and things that coordinate with each other to enable different applications. Transport systems, as one of the most important systems in future smart cities, will embrace the connectivity, together with the fast development of cooperative and automated vehicles to enable smart traffic. To facilitate this transformation, a service choreography composition platform is under development to enable fast innovation and prototyping of choreography-based Internet of Things (IoT) applications by automatically synthesizing choreographies. Based on the method, a smart traffic application is developed and demonstrated.

A traffic situation estimator capable of analyzing driving behavior utilizing an image analysis-based tracking module is presented. The behavior is analyzed by using a state machine driven counter to estimate the traffic rhythm and determine if the detected vehicles are approaching, getting away, have been overtaken or have overtaken the ego-vehicle. Depending on the result, the traffic situation estimator suggest different reactions, either to drive faster, slower or optionally suggest to overtake vehicles ahead to help the driver to follow the traffic rhythm which in turn will improve safety and energy efficiency. The proposed approach is implemented in a smart-phone and has shown good performance while testing the application on a two-lane highway.

Intersection management is one of the most challenging problems within the transport system. Traffic light-based methods have been efficient but are not able to deal with the growing mobility and social challenges. On the other hand, the advancements of automation and communications have enabled cooperative intersection management, where road users, infrastructure, and traffic control centers are able to communicate and coordinate the traffic safely and efficiently. Major techniques and solutions for cooperative intersections are surveyed in this paper for both signalized and nonsignalized intersections, whereas focuses are put on the latter. Cooperative methods, including time slots and space reservation, trajectory planning, and virtual traffic lights, are discussed in detail. Vehicle collision warning and avoidance methods are discussed to deal with uncertainties. Concerning vulnerable road users, pedestrian collision avoidance methods are discussed. In addition, an introduction to major projects related to cooperative intersection management is presented. A further discussion of the presented works is given with highlights of future research topics. This paper serves as a comprehensive survey of the field, aiming at stimulating new methods and accelerating the advancement of automated and cooperative intersections.

Wireless technology supporting vehicle-to-vehicle (V2V), and vehicle-to-infrastructure (V2I) communication, allow vehicles and infrastructures to exchange information, and cooperate. Cooperation among the actors in an intelligent transport system (ITS) can introduce several benefits, for instance, increase safety, comfort, efficiency. Automation has also evolved in vehicle control and active safety functions. Combining cooperation and automation would enable more advanced functions such as automated highway merge and negotiating right-of-way in a cooperative intersection. However, the combination have influences on the structure of the overall transport systems as well as on its behaviour. In order to provide a common understanding of such systems, this paper presents an analysis of cooperative ITS (C-ITS) with regard to dimensions of cooperation. It also presents possible influence on driving behaviour and challenges in deployment and automation of C-ITS.

Despite the existing regulation efforts and measures, vehicles with dangerous goods still pose significant risks on public safety, especially in road tunnels. Solutions based on cooperative intelligent transportation system (C-ITS) are promising measures, however, they have received limited attention. We propose C-ITS applications that coordinate dangerous goods vehicles to minimize the risk by maintaining safe distances between them in road tunnels. Different mechanisms, including global centralized coordination, global distributed coordination, and local coordination, are proposed and investigated. A preliminary simulation is performed and demonstrates their effectiveness.