Achieving efficient remote teleoperation is particularly challenging in unknown environments, as the teleoperator must rapidly build an understanding of the site’s layout. Online 3D mapping is a proven strategy to tackle this challenge, as it enables the teleoperator to progressively explore the site from multiple perspectives. However, traditional online map-based teleoperation systems struggle to generate visually accurate 3D maps in real-time due to the high computational cost involved, leading to poor teleoperation performances. In this work, we propose a solution to improve teleoperation efficiency in unknown environments. Our approach proposes a novel, modular and efficient GPU-based integration between recent advancement in gaussian splatting SLAM and existing online map-based teleoperation systems. We compare the proposed solution against state-of-the-art teleoperation systems and validate its performances through real-world experiments using an aerial vehicle. The results show significant improvements in decision-making speed and more accurate interaction with the environment, leading to greater teleoperation efficiency. In doing so, our system enhances remote teleoperation by seamlessly integrating photorealistic mapping generation with real-time performances, enabling effective teleoperation in unfamiliar environments.Video: https://www.youtube.com/watch?v=-Md49rKkV8I(Code: https://github.com/ian-pge/GS_SLAM_teleoperation.git

To deploy mobile robots in spaces that they will share with humans, mobile robots should have social navigation methods. One important feature to design such methods is the ability to follow a moving person. In this paper, we present how we detect and track a moving person that our mobile robot is following. A low level detection of the moving person followed, to detect the person independently of their position and orientation with respect to the mobile robot, is combined with a sliding window approach to track the moving person. Some experimental results show the robustness of the method on real scenarios.

This research introduces an innovative framework for the prediction of human hand motion in a shared workspace with a robot, fostering safe and efficient human-robot collab-oration. Due to the absence of benchmark datasets for this task, we created a custom dataset of human hand trajectories by orchestrating intentional collisions between humans and robots during data collection. To enable efficient processing and prediction, our framework leverages the quantization of sensitive human hand positions into small 3D cells. These cells are later modeled for learning the embedding for better human hand motion prediction. Notably, our framework outperforms the baseline model. Al-though the enhanced predictive power entails extra computation for finding the Nearest Neighbors (NN) during quantization, we efficiently manage this cost through the integration of off-the-shelf information retrieval frameworks like ANNOY. This strategic approach ensures real-time performance and maintains precise approximate NN results.

Localization is the ability for a mobile robot to know its position at all times. Most of the time, the localization process has to manage several possible positions that could correspond to the real one. The main drawback is that the cost of computational complexity could be high especially when the position of the mobile robot in its environment is unknown. In this paper, we present a new method to perform localization. The main interest of our method is that it is able to determine in a fast and robust way the best positions, using information retrieval techniques, that could correspond to the real one. Moreover, the number of positions is automatically adapted according to the confidence that we have in the localization of the robot. Experimental results show the benefits of the method.

Human-aware navigation and Social Navigation are growing fields of robotics, attempting to tackle challenging navigation problems in human environments. One difficult aspect is understanding how a mobile robot's navigation behavior impacts the way they are perceived by humans, and how they interact together. In our previous work, we proposed an adaptable navigation architecture. In this paper, we describe how we determine which motion variables should be controlled by our architecture. We take a bottom-up approach by determining which primitive notions of locomotion and appearance have the most impact on people's perceptions of a mobile robot. We present the results of two online and one in-person experiment showing that the manner in which a mobile robot navigates has a significant impact on people's attribution of attitudes and physical properties to the robot.

The advent of 3D LiDAR technology has revolutionized object detection in applications such as autonomous driving, robotics, and advanced driver assistance systems. However, existing methods often require substantial computational resources, limiting their practicality for real-time applications on devices with constrained hardware capabilities. This paper presents an efficient and lightweight 3D LiDAR object detection framework, 3D-PSH, that combines adaptive clustering with 3D Point Spatial Histograms (3D-PSH) and classical classification techniques to address these challenges. Our framework begins with an adaptive clustering algorithm that segments the point cloud data into distinct clusters, representing potential objects. 3D Point Spatial Histograms (3D-PSH) are then computed from these clusters and subsequently quantized into a Bag of Visual Words (BoVW) to create a compact and informative representation. These representations are then classified using robust classical classification methods to identify object types, such as pedestrians and vehicles. This multi-step approach ensures a balance between computational efficiency and detection accuracy, making it suitable for real-time deployment. Extensive experiments on the KITTI dataset and our live sensor data demonstrate the effectiveness and efficiency of our proposed framework. The results indicate that our method achieves competitive accuracy while significantly reducing computational requirements compared to traditional approaches. This framework offers a practical solution for deploying 3D object detection in a wide range of applications, particularly where computational resources are limited.

Social Navigation methods attempt to integrate knowledge from Human Sciences fields such as the notion of Proxemics into mobile robot navigation. They are often evaluated in simulations, or lab conditions with informed participants, and studies of the impact of the robot behavior on humans are rare. Humans communicate and interact through many vectors, among which are motion and positioning, which can be related to social hierarchy and the socio-physical context. If a robot is to be deployed among humans, the methods it uses should be designed with this in mind. This work acts as the first step in an ongoing project in which we explore how to design navigation methods for mobile robots destined to be deployed among humans. We aim to consider navigation as more than just a functionality of the robot, and to study the impact of robot motion on humans. In this paper, we focus on the person-following task. We selected a state of the art person-following method as the basis for our method, which we modified and extended in order for it to be more general and adaptable. We conducted pilot experiments using this method on a real mobile robot in ecological contexts. We used results from the experiments to study the Human-Robot Interaction as a whole by analysing both the person-following method and the human behavior. Our preliminary results show that the way in which the robot followed a person had an impact on the interaction that emerged between them.

With new, safer manipulator robots, the probability of serious injury due to collisions with humans remains low (5%), even at speeds as high as 2 m.s-1. Collisions would better be avoided nevertheless, because they disrupt the tasks of both the robot and the human. We propose in this paper to equip robots with exteroceptive sensors and online motion generation so that the robot is able to perceive and react to the motion of the human in order to reduce the occurrence of collisions. It's impossible to guarantee that no collision will ever take place in a partially unknown dynamic environment such as a shared workspace, but we can guarantee instead that, if a collision takes place, the robot is at rest at the time of collision, so that it doesn't inject its own kinetic energy in the collision. To do so, we adapt a Model Predictive Control scheme which has been demonstrated previously with two industrial manipulator robots avoiding collisions while sharing their workspace. The proposed control scheme is validated in simulation.

Localization is the ability for a mobile robot to know its position at all times. When the initial position is unknown, the localization process has to manage several possible positions that could correspond to the real one. The main drawback of this technique is the cost of computational complexity that could be high. In this paper, we present a new way to determine the set of initial possible positions that is fast (less than 3s) and enables to start the localization process with a small number of possible positions. The consequence is that our localization process determines the real position in a fast and robust way. Experimental results show the benefits of the method.

The accurate detection and classification of moving objects is a critical aspect of advanced driver assistance systems. We believe that by including the object classification from multiple sensor detections as a key component of the object's representation and the perception process, we can improve the perceived model of the environment. First, we define a composite object representation to include class information in the core object's description. Second, we propose a complete perception fusion architecture based on the evidential framework to solve the detection and tracking of moving objects problem by integrating the composite representation and uncertainty management. Finally, we integrate our fusion approach in a real-time application inside a vehicle demonstrator from the interactIVe IP European project, which includes three main sensors: radar, lidar, and camera. We test our fusion approach using real data from different driving scenarios and focusing on four objects of interest: pedestrian, bike, car, and truck.

In this paper, we present a layered approach for classification of moving objects from 3D range data based on supervised learning technique. Our approach combines the model based classification in 2D with boosting for classifying the objects into four classes of interest namely bus, car, bike and pedestrian. In contrast to most of the existing work on 3D classification which involves extensive feature extraction and description, this combination uses simple single-valued features and allows our system to perform efficiently. The proposed method can be used in conjunction with any type of range sensors, however, we have demonstrated its performance using the data acquired from a Velodyne HDL-64E laser scanner.

Intelligent vehicle perception involves the correct detection and tracking of moving objects. Taking into account all the possible information at early levels of the perception task can improve the final model of the environment. In this paper, we present an evidential fusion framework to represent and combine evidence from multiple lists of sensor detections. Our fusion framework considers the position, shape and appearance information to represent, associate and combine sensor detections. Although our approach takes place at detection level, we propose a general architecture to include it as a part of a whole perception solution. Several experiments were conducted using real data from a vehicle demonstrator equipped with three main sensors: lidar, radar and camera. The obtained results show improvements regarding the reduction of false detections and mis-classifications of moving objects.

The paper addresses the problem of object perception for intelligent vehicle applications with main tasks of detection, tracking and classification of obstacles where multiple sensors (i.e.: lidar, camera and radar) are used. New algorithms for raw sensor data processing and sensor data fusion are introduced making the most information from all sensors in order to provide a more reliable and accurate information about objects in the vehicle environment. The proposed object perception module is implemented and tested on a demonstrator car in real-life traffics and evaluation results are presented.

In this paper, we present a framework based on 3D range data to solve the problem of simultaneous localization and mapping (SLAM) with detection and tracking of moving objects (DATMO) in dynamic environments. The basic idea is to use an octree based Occupancy Grid representation to model dynamic environment surrounding the vehicle and to detect moving objects based on inconsistencies between scans. The proposed method for the discrimination between moving and stationary objects without a priori knowledge of the targets is the main contribution of this paper. Moreover, the detected moving objects are classified and tracked using Global Nearest Neighbor (GNN) technique. The proposed method can be used in conjunction with any type of range sensors however we have demonstrated it using the data acquired from a Velodyne HDL-64E LIDAR sensor. The merit of our approach is that it allows for an efficient three dimensional representation of a dynamic environment, keeping in view the enormous amount of information provided by 3D range sensors.

In this paper we have presented a fast algorithm to detect road borders from laser data. Two local search windows, one on right side of the host vehicle and the other on left, are moved right and left respectively from the current position of vehicle in map. A score function is evaluated to know the presence or absence of the road border in current search window. We have used the detected road border information to reduce false alarms in our previous work on DATMO (detection and tracking of moving objects). We also show how these information can be used to infer drivable area and the presence of intersections on the road. Results on data sets obtained from real demonstrator vehicles show that this technique can be successfully applied in real time.

In this paper, we detail a complete software architecture of a key task that an intelligent vehicle has to deal with: frontal object perception. This task is solved by processing raw data of a radar and a mono-camera to detect and track moving objects. Data sets obtained from highways, country roads and urban areas were used to test the proposed method. Several experiments were conducted to show that the proposed method obtains a better environment representation, i.e., reduces the false alarms and missed detections from individual sensor evidence.

In this paper, we present a generic architecture for perception of an intelligent vehicle in dynamic outdoor environment. This architecture is composed of two levels: a first level dedicated to real-time local simultaneous localization and mapping (SLAM) and a second one is dedicated to detection and tracking of moving objects (DATMO). The experimental results on datasets collected from different scenarios such as: urban streets, country roads and highways demonstrate the efficiency of the proposed algorithm on a Daimler Mercedes demonstrator in the framework of the European Project PReVENT-ProFusion2 and on a Volkswagen Demonstrator in the framework of the European Project Intersafe2.

The goal of mapping is to build a map of the environment using raw data provided by some sensors embedded on an intelligent vehicle. This map is used by an intelligent vehicle to have knowledge about its surrounding environment to better plan its future actions. In this paper, we present a method, based on occupancy grids [3], to map 3D environment. In this method, we discretize the environment in cells and the shape of each cell is approximated by one or several gaussians in order to achieve a balance between representational complexity and accuracy. Experimental results on 3D real outdoor data provided by a lidar are shown: a map of an urban environment is presented. Moreover a quantitative comparison of our method with state of the art methods is presented to show the interest of the method.

In this paper, we describe our approach for intersection safety developed in the scope of the European project INTERSAFE-2. A complete solution for the safety problem including the tasks of perception and risk assessment using on-board lidar and stereo-vision sensors will be presented and interesting results are shown.

Using multiple sensors in the context of environment perception for autonomous vehicles is quite common these days. Perceived data from these sensors can be fused at different levels like: before object detection, after object detection and finally after tracking the moving objects. In this paper we detail our object detection level fusion between laser and stereo vision sensors as opposed to pre-detection or track level fusion. We use the output of our laser processing to get a list of objects with position and dynamic properties for each object. Similarly we use the stereo vision output of another team which consists of a list of detected objects with position and classification properties for each object. We use Bayesian fusion technique on objects of these two lists to get a new list of fused objects. This fused list of objects is further used in tracking phase to track moving objects in an intersection like scenario. The results obtained on data sets of INTERSAFE-2 demonstrator vehicle show that this fusion has improved data association and track management steps.

In this paper we have developed a technique for low level data fusion between laser and monocular color camera using occupancy grid framework in the context of internal representation of external environment for object detection. Based on a small variant of background subtraction technique we construct an occupancy grid for camera and fuse it with the one constructed for laser to get a combined view. The results obtained using Cycab simulator prepared by INRIA show the effectiveness of our technique.

In this paper, we present an approach performing object behavior classification embedded in a complex and efficient perception method. This method, applied in dynamic outdoor environments using a moving vehicle equipped with a laser scanner, is composed of a local simultaneous localization and mapping (SLAM) with detection and tracking of moving objects (DATMO). While the SLAM is performed by an implementation of incremental scan matching method, the tracking if performed by a Multiple Hypothesis Tracker (MHT) coupled with an adaptive Interacting Multiple Models Filter (IMM). The classification process takes place in the filtering stage and is based on one of the key parameters of the IMM filter which is the Transition Probability Matrix (TPM) modeling objects motion transitions. It permits to automatically classify object behavior and to reuse the classification output to enhance the prediction step in the filtering process. The experimental results on datasets collected from a Daimler Mercedes demonstrator in the framework of the European Project PReVENT-ProFusion2 demonstrate the capacity of the proposed algorithm.

To address perception problems we must be able to track dynamic objects of the environment. An important issue of tracking is the association problem in which we have to associate each new observation with one existing object in the environment. This problem is complex: unfortunately, the number of observations generally does not correspond to the number of objects. Moreover, the number of objects is difficult to estimate since one object might be temporarily occluded or unobserved simply because objects can enter or go out of ranges of vehicle sensors. Moreover, the perception sensors or the object detection process might generate false alarm measurements. In this paper, we propose a new solution to solve the multiple objects tracking problem, using the Viterbi algorithm (VA) [2]. It is an established optimisation technique for discrete Markovian systems that has been extensively used in speech recognition. In this paper, we present an extension of VA to solve multiple objects tracking in clutter environment and show some experimental results on multiple pedestrian tracking and also some quantitative comparisons with MHT algorithms.

We present a method of simultaneous detection and tracking moving objects from a moving vehicle equipped with a single layer laser scanner. A model-based approach is introduced to interpret the laser measurement sequence by hypotheses of moving object trajectories over a sliding window of time. Knowledge of various aspects including object model, measurement model, motion model are integrated in one theoretically sound Bayesian framework. The data-driven Markov chain Monte Carlo (DDMCMC) technique is used to sample the solution space effectively to find the optimal solution. Experiments and results on real-life data of urban traffic show promising results.

In this paper, we present a vehicle safety application based on data gathered by a laser scanner and two short-range radars that recognize unavoidable collisions with stationary objects before they take place to trigger restraint systems. Two different software modules that perform the processing of raw data and deliver a description of the vehicle's environment are compared. A comprehensive experimental evaluation based on relevant crash and noncrash scenarios is presented.

We describe a general architecture of vehicle perception system developed in the framework of the European project PReVENT-ProFusion. Our system consists of two main parts: the first part where the vehicle environment is mapped and moving objects are detected; and the second part where previously detected moving objects are verified and tracked. In this paper, we focus on the first part, using occupancy grid to model the vehicle environment, perform sensor data fusion and detect moving objects. Experimental results on a Volvo Truck equipped with laser scanner and radars show the effectiveness of our approach.

In this paper, we detail the hardware and software perception system designed and developed to track pedestrians using a set of offboard cameras. It has been used in the context of vulnerable safety in a car park. This architecture is divided in two parts: a fusion part to fusion the data given by the set of offboard cameras and a tracking part to sequentially estimate the position of each pedestrian present in the environment and to determine the number of pedestrians. Finally, some experimental results are presented

To address perception problems we must be able to track dynamics targets of the environment. An important issue of tracking is filtering problem in which estimates of the target's state are computed while observations are progressively received. This paper presents an adaptive interacting multiple models (IMM) based filtering method. Interacting multiple models have been successfully applied to many applications as they allow, using several filters in parallel, to deal with the uncertainty on motion model, a critical component of filtering. Indeed targets can rapidly change their motion over a lapse of time. This is the case of pedestrians for which it is difficult to define an unique motion model which matches all their possible displacements. Nevertheless, the transition probability matrix (TPM) which models the interaction between different filters in an IMM is in currently defined a priori or needs an important amount of tuning to be used efficiently. In this paper, we put forward a method which automatically adapts online the TPM. The TPM adaptation using on-line data significantly improves the effectiveness of IMM filtering and so better target estimates are obtained. To validate our work we applied our method to pedestrian tracking in car parks on a real platform

Building occupancy grids (OGs) in order to model the surrounding environment of a vehicle implies to fusion occupancy information provided by the different embedded sensors in the same grid. The principal difficulty comes from the fact that each can have a different resolution, but also that the resolution of some sensors varies with the location in the field of view. In this article we present a new exact approach to this issue and we explain why the problem of switching coordinate systems is an instance of the texture mapping problem in computer graphics. Therefore we introduce a calculus architecture to build occupancy grids with a graphical processor unit (GPU). Thus we present computational time results that can allow to compute occupancy grids for 50 sensors at frame rate even for a very fine grid. To validate our method, the results with GPU are compared to results obtained through the exact approach

To reach a given goal, a mobile robot first computes a motion plan (i.e. a sequence of actions that takes it to its goal), and then executes it. Markov decision processes (MDPs) have been successfully used to solve these two problems. Their main advantage is that they provide a theoretical framework to deal with the uncertainties related to the robot's motor and perceptive actions during both planning and execution stages. While a previous paper addressed the motion planning stage, this paper deals with execution stage. It describes an approach based on Markov localization and focuses on experimental aspects, in particular, the learning of the transition function (that encodes the uncertainties related to the robot actions) and the sensor model. Experimental results carry out with a real robot demonstrate the robustness of the whole navigation approach.

To reach a given goal, a mobile robot first computes a motion plan (if a sequence of actions that will take it to its goal), and then executes it Markov decision processes (MDPs) have been successfully used to solve these two problems. Their main advantage is that they provide a theoretical framework to deal with the uncertainties related to the robot's motor and perceptive actions during both planning and execution stages. This paper describes a MDP-based planning method that uses a hierarchic representation of the robot's state space (based on a quadtree decomposition of the environment). Besides, the actions used better integrate the kinematic constraints of a wheeled mobile robot. These two features yield a motion planner more efficient and better suited to plan robust motion strategies.

A planetary rover must be able to identify states where it should stop or change its plan. With limited and infrequent communication from ground, the rover must recognize states accurately. However, the sensor data is inherently noisy, so identifying the temporal patterns of data that correspond to interesting or important states becomes a complex problem. We present an approach to state identification using second-order hidden Markov models. Models are trained automatically on a set of labeled training data; the rover uses those models to identify its state from the observed data. The approach is demonstrated on data from a planetary rover platform.

We present a new method to localize a mobile robot in dynamic environments. This method is based on place recognition, and a match between places recognized and the sequence of places that the mobile robot is able to see during a run from an initial place to an ending place. Our method gives a coarse idea of the robot's position and orientation. Moreover, the robot can determine the actual state of places (i.e. open doors, closed doors).

Second order hidden Markov models have been used for a long time in pattern recognition, especially in speech recognition. Their main advantages over other methods (neural networks etc.) are their capabilities of modeling noisy temporal signals of variable length. In a previous work, we proposed a new method based on second order hidden Markov models to learn and recognize places in an indoor environment by a mobile robot, and showed that this approach is well suited for learning and recognizing places. In this paper, we propose major modifications to increase the global rate of place recognition. Results of experiments on a real robot with distinctive places are given.

We present a multi agent architecture for controlling a mobile robot in an unpredictable environment. This architecture has been developed with the objective of coordinating the various competences of the robot (e.g., perception, navigation, planning). The architecture is made up of two agents: the first one specialized for cognitive tasks, the second one dedicated to control of the robot's physical devices. This two agent architecture guarantees a good robustness of the system, as the navigation modules can run independently of the cognitive ones.

In this paper, we propose a new method based on hidden Markov models to learn and recognize places in an indoor environment by a mobile robot. Hidden Markov models have been used for a long time in pattern recognition, especially in speech recognition. Their main advantages over other methods (e.g. neural networks) are their capabilities to modelize noisy temporal signals of variable length. We show in this paper that this approach is well adapted for learning and recognition of places by a mobile robot. Results of experiments on a real robot with five distinctive places are given.