Autonomous vehicles require online learning capabilities to enable long-term, unattended operation. However, long-term online learning is accompanied by the problem of forgetting previously learned knowledge. This paper introduces an online learning framework that includes a catastrophic forgetting prevention mechanism, named Long-Short-Term Online Learning (LSTOL). The framework consists of a set of shortterm learners and a long-term controller, where the former is based on the concept of ensemble learning and aims to achieve rapid learning iterations, while the latter contains a simple yet efficient probabilistic decision-making mechanism combined with four control primitives to achieve effective knowledge maintenance. A novel feature of the proposed LSTOL is that it avoids forgetting while learning autonomously. In addition, LSTOL makes no assumptions about the model type of short-term learners and the continuity of the data. The effectiveness of the proposed framework is demonstrated through experiments across well-known datasets in autonomous driving, including KITTI and Waymo. The source code for the method implementation is publicly available at https://github.com/epan-utbm/lstol.

The automotive industry plays a critical role in the global economy, and particularly important is the expanding Chinese automobile market due to its immense scale and influence. However, existing automotive sector datasets are limited in their coverage, failing to adequately consider the growing demand for more and diverse variables. This paper aims to bridge this data gap by introducing a comprehensive dataset spanning the years from 2016 to 2022, encompassing sales data, online reviews, and a wealth of information related to the Chinese automotive industry. This dataset serves as a valuable resource, significantly expanding the available data. Its impact extends to various dimensions, including improving forecasting accuracy, expanding the scope of business applications, informing policy development and regulation, and advancing academic research within the automotive sector. To illustrate the dataset’s potential applications in both business and academic contexts, we present two application examples. Our developed dataset enhances our understanding of the Chinese automotive market and offers a valuable tool for researchers, policymakers, and industry stakeholders worldwide.

Exterior facelift has become an effective method for automakers to boost the consumers’ interest in an existing car model before it is redesigned. To support the automotive facelift design process, this study develops a novel computational framework – Generator, Evaluator, Optimiser (GEO) , which comprises three components: a StyleGAN2-based design generator that creates different facelift designs; a convolutional neural network (CNN) -based evaluator that assesses designs from the aesthetics perspective; and a recurrent neural network (RNN) -based decision optimiser that selects designs to maximise the predicted profit of the targeted car model over time. We validate the GEO framework in experiments with real-world datasets and describe some resulting managerial implications for automotive facelift. Our study makes both methodological and application contributions. First, the generator’s mapping network and projection methods are carefully tailored to facelift where only minor changes are performed without affecting the family signature of the automobile brands. Second, two evaluation metrics are proposed to assess the generated designs. Third, profit maximisation is taken into account in the design selection. From a high-level perspective, our study contributes to the recent use of machine learning and data mining in marketing and design studies. To the best of our knowledge, this is the first study that uses deep generative models for automotive regional design upgrading and that provides an end-to-end decision-support solution for automakers and designers.

As a critical sensor for high-level autonomous vehicles, LiDAR’s limitations in adverse weather (e.g. rain, fog, snow, etc.) impede the deployment of self-driving cars in all weather conditions. However, studies in literature on LiDAR’s performance in harsh conditions are insufficient. In this paper, based on a dataset collected with a popular Near-InfraRed (NIR) ToF LiDAR in a well-controlled artificial fog chamber, we statistically model the LiDAR ranging process in fog conditions through a data-driven approach. Specifically, giving an object at a known distance, our model is able to predict LiDAR measures (range and intensity) under various fog conditions. For a transmitted laser under fog, we first model and predict the minimum visibility required to detect its true range or not. Then, the noisy range and intensity measures are sampled from the probabilistic measurement distributions inferred from the dataset. The performance of the proposed method has been quantitatively and qualitatively evaluated. Experimental results show that our approach can provide a promising performance prediction of the utilized NIR ToF LiDAR under fog, which opens a new gate to the quantitative assessment of adverse weather and contributes to the specification of relevant Operational Domain Designs (ODDs). The developed ROS package is available at:https://github.com/cavayangtao/lanoising.

In this paper, we study the problem of autonomous and reliable positioning of a camera w.r.t. an object when only this latter is known but not the rest of the scene. We propose to combine the advantages and efficiency of a visual servoing scheme and the generalization ability of a generative adversarial network. The paper describes how to efficiently create a synthetic dataset in order to train a network that predicts an intermediate visual keyframe between two images. Subsequent predictions are used as visual features to autonomously converge towards the desired pose even for large displacements. We show that the proposed method can be used without any prior knowledge on the scene appearance except for the object itself, while being robust to various lighting conditions and specular surfaces. We provide experimental results, both in simulation and using a real service robot platform to validate and evaluate the effectiveness, robustness, and accuracy of our approach.

This paper presents a robust monocular visual teach-and-repeat (VT&R) navigation system for long-term operation in outdoor environments. The approach leverages deep-learned descriptors to deal with the high illumination variance of the real world. In particular, a tailored self-supervised descriptor, DarkPoint, is proposed for autonomous navigation in outdoor environments. We seamlessly integrate the localisation with control, in which proportional–integral control is used to eliminate the visual error with the pitfall of the unknown depth. Consequently, our approach achieves day-to-night navigation using a single-experience map and is able to repeat complex and fast manoeuvres. To verify our approach, we performed a vast array of navigation experiments in various outdoor environments, where both navigation accuracy and robustness of the proposed system are investigated. The experimental results show that our approach is superior to the baseline method with regards to accuracy and robustness.

This paper proposes a novel monocular teach-and-repeat navigation system with the capability of scale awareness, i.e. the absolute distance between observation and goal images. It decomposes the navigation task into a sequence of visual servoing sub-tasks to approach consecutive goal/node images in a topological map. To be specific, a novel hybrid model, named deep steering network is proposed to infer the navigation primitives according to the learned local feature and scale for each visual servoing sub-task. A novel architecture, Scale-Transformer, is developed to estimate the absolute scale between the observation and goal image pair from a set of matched deep representations to assist repeating navigation. The experiments demonstrate that our scale-aware teach-and-repeat method achieves satisfying navigation accuracy, and converges faster than the monocular methods without scale correction given an inaccurate initial pose. The proposed network is integrated into an onboard system deployed on a real robot to achieve real-time navigation in a real environment. A demonstration video can be found online: https://youtu.be/ctlwDaMKnHw

The field of autonomous driving has grown tremendously over the past few years, along with the rapid progress in sensor technology. One of the major purposes of using sensors is to provide environment perception for vehicle understanding, learning and reasoning, and ultimately interacting with the environment. In this paper, we first introduce a multisensor platform allowing vehicle to perceive its surroundings and locate itself in a more efficient and accurate way. The platform integrates eleven heterogeneous sensors including various cameras and lidars, a radar, an IMU (Inertial Measurement Unit), and a GPS-RTK (Global Positioning System / Real-Time Kinematic), while exploits a ROS (Robot Operating System) based software to process the sensory data. Then, we present a new dataset (https://epan-utbm.github.io/utbm_robocar_dataset/) for autonomous driving captured many new research challenges (e.g. highly dynamic environment), and especially for long-term autonomy (e.g. creating and maintaining maps), collected with our instrumented vehicle, publicly available to the community.

Models of human behaviour, such as pedestrian flows, are beneficial for safe and efficient operation of mobile robots. We present a new methodology for benchmarking of pedestrian flow models based on the afforded safety of robot navigation in human-populated environments. While previous evaluations of pedestrian flow models focused on their predictive capabilities, we assess their ability to support safe path planning and scheduling. Using real-world datasets gathered continuously over several weeks, we benchmark state-of-the-art pedestrian flow models, including both time-averaged and time-sensitive models. In the evaluation, we use the learned models to plan robot trajectories and then observe the number of times when the robot gets too close to humans, using a predefined social distance threshold. The experiments show that while traditional evaluation criteria based on model fidelity differ only marginally, the introduced criteria vary significantly depending on the model used, providing a natural interpretation of the expected safety of the system. For the time-averaged flow models, the number of encounters increases linearly with the percentage operating time of the robot, as might be reasonably expected. By contrast, for the time-sensitive models, the number of encounters grows sublinearly with the percentage operating time, by planning to avoid congested areas and times.

As a critical sensor for high-level autonomous vehicles, LiDAR's limitations in adverse weather (e.g. rain, fog, snow, etc.) impede the deployment of self-driving cars in all weather conditions. In this paper, we model the performance of a popular 903nm ToF LiDAR under various fog conditions based on a LiDAR dataset collected in a well-controlled artificial fog chamber. Specifically, a two-stage data-driven method, called LaNoising (la for laser), is proposed for generating LiDAR measurements under fog conditions. In the first stage, the Gaussian Process Regression (GPR) model is established to predict whether a laser can successfully output a true detection range or not, given certain fog visibility values. If not, then in the second stage, the Mixture Density Network (MDN) is used to provide a probability prediction of the noisy measurement range. The performance of the proposed method has been quantitatively and qualitatively evaluated. Experimental results show that our approach can provide a promising description of 903nm ToF LiDAR performance under fog.

We propose an efficient spatio-temporal model for mobile autonomous robots operating in human populated environments. Our method aims to model periodic temporal patterns of people presence, which are based on peoples' routines and habits. The core idea is to project the time onto a set of wrapped dimensions that represent the periodicities of people presence. Extending a 2D spatial model with this multidimensional representation of time results in a memory efficient spatio-temporal model. This model is capable of long-term predictions of human presence, allowing mobile robots to schedule their services better and to plan their paths. The experimental evaluation, performed over datasets gathered by a robot over a period of several weeks, indicates that the proposed method achieves more accurate predictions than the previous state of the art used in robotics.

This paper presents a novel 3DOF pedestrian trajectory prediction approach for autonomous mobile service robots. While most previously reported methods are based on learning of 2D positions in monocular camera images, our approach uses range-finder sensors to learn and predict 3DOF pose trajectories (i.e. 2D position plus 1D rotation within the world coordinate system). Our approach, T-Pose-LSTM (Temporal 3DOF-Pose Long-Short-Term Memory), is trained using long-term data from real-world robot deployments and aims to learn context-dependent (environment- and time-specific) human activities. Our approach incorporates long-term temporal information (i.e. date and time) with short-term pose observations as input. A sequence-to-sequence LSTM encoder-decoder is trained, which encodes observations into LSTM and then decodes the resulting predictions. On deployment, the approach can perform on-the-fly prediction in real-time. Instead of using manually annotated data, we rely on a robust human detection, tracking and SLAM system, providing us with examples in a global coordinate system. We validate the approach using more than 15 km of pedestrian trajectories recorded in a care home environment over a period of three months. The experiments show that the proposed T-Pose-LSTM model outperforms the state-of-the-art 2D-based method for human trajectory prediction in long-term mobile robot deployments.

Human detection and tracking is an essential task for service robots, where the combined use of multiple sensors has potential advantages that are yet to be fully exploited. In this paper, we introduce a framework allowing a robot to learn a new 3D LiDAR-based human classifier from other sensors over time, taking advantage of a multisensor tracking system. The main innovation is the use of different detectors for existing sensors (i.e. RGB-D camera, 2D LiDAR) to train, online, a new 3D LiDAR-based human classifier based on a new “trajectory probability”. Our framework uses this probability to check whether new detection belongs to a human trajectory, estimated by different sensors and/or detectors, and to learn a human classifier in a semi-supervised fashion. The framework has been implemented and tested on a real-world dataset collected by a mobile robot. We present experiments illustrating that our system is able to effectively learn from different sensors and from the environment, and that the performance of the 3D LiDAR-based human classification improves with the number of sensors/detectors used.

Human detection and tracking are essential aspects to be considered in service robotics, as the robot often shares its workspace and interacts closely with humans. This paper presents an online learning framework for human classification in 3D LiDAR scans, taking advantage of robust multi-target tracking to avoid the need for data annotation by a human expert. The system learns iteratively by retraining a classifier online with the samples collected by the robot over time. A novel aspect of our approach is that errors in training data can be corrected using the information provided by the 3D LiDAR-based tracking. In order to do this, an efficient 3D cluster detector of potential human targets has been implemented. We evaluate the framework using a new 3D LiDAR dataset of people moving in a large indoor public space, which is made available to the research community. The experiments analyse the real-time performance of the cluster detector and show that our online learned human classifier matches and in some cases outperforms its offline version.

Performance benchmarking has become an important topic within robotics. It is indeed, a critical way to compare different solutions under different conditions. In this paper, we focus on performance benchmarking of multi-robot systems which explore and map unknown terrains. We present a collection of metrics to objectively compare different algorithms that can be applied to collaborative multi-robot exploration. We also identify parameters that impact robotic fleet performances. By varying the parameters, we can identify strengths and limits of an algorithm. This work is also a first concrete step to address the general problem of objectively comparing different multi-robot coordination algorithms. We illustrate these contributions with realistic simulations of the frontier-based exploration strategy. The simulations were implemented in ROS, which enables to uncouple the control software from the drivers of the robot body. We can therefore use the same code on both simulation and real robots.

Abstract: This paper describes a sampling-based approach to multi-robot motion planning. The proposed approach is centralized, which aims to reduce interference between mobile robots such as collision, congestion and deadlock, by increasing the number of waypoints. The implementation based on occupancy grid map is decomposed into three steps: the first step is to identify primary waypoints by using the Voronoi diagram, the second step is to generate additional waypoints by sampling the Voronoi diagram, and the last step is to assign the waypoints to robots by using the Hungarian method. The approach has been implemented and tested in simulation and the experimental results show a good system performance for multi-robot motion planning. 1