In the classical reach-avoid problem, autonomous mobile robots are tasked to reach a goal while avoiding obstacles. However, it is difficult to provide guarantees on the robot's performance when the obstacles form a narrow gap and the robot is a black-box (i.e. the dynamics are not known analytically, but interacting with the system is cheap). To address this challenge, this paper presents NeuralPARC. The method extends the authors' prior Piecewise Affine Reach-avoid Computation (PARC) method to systems modeled by rectified linear unit (ReLU) neural networks, which are trained to represent parameterized trajectory data demonstrated by the robot. NeuralPARC computes the reachable set of the network while accounting for modeling error, and returns a set of states and parameters with which the black-box system is guaranteed to reach the goal and avoid obstacles. NeuralPARC is shown to outperform PARC, generating provably-safe extreme vehicle drift parking maneuvers in simulations and in real life on a model car, as well as enabling safety on an autonomous surface vehicle (ASV) subjected to large disturbances and controlled by a deep reinforcement learning (RL) policy.

Accurate segmentation of floating debris on water is often compromised by surface glare and changing outdoor illumination. Polarimetric imaging offers a single-sensor route to mitigate water-surface glare that disrupts semantic segmentation of floating objects. We benchmark state-of-the-art fusion networks on PoTATO, a public dataset of polarimetric images of plastic bottles in inland waterways, and compare their performance with single-image baselines using traditional models. Our results indicate that polarimetric cues help recover low-contrast objects and suppress reflection-induced false positives, raising mean IoU and lowering contour error relative to RGB inputs. These sharper masks come at a cost: the additional channels enlarge the models increasing the computational load and introducing the risk of new false positives. By providing a reproducible, diagnostic benchmark and publicly available code1, we hope to help researchers choose if polarized cameras are suitable for their applications and to accelerate related research.

Real-world robot system deployment is often performed in complex and unstructured environments. These complex environments coupled with multi-faceted global tasks often lead to complicated stakeholder structures, making designing for these environments extremely challenging. Magnifying this difficulty, tasks performed in these environments often cannot be accomplished by a single robot or even single robot type because of the broad range of needs and psychical constraints of the robots. In these cases, heterogeneous robot teams may need to be coupled to human team members to perform the global tasks. From a Human-Robot Interaction (HRI) perspective, this increases the complexity of designing and deploying the system significantly, as now complicated stakeholder structures are mixed with complex robot teams. This paper presents a novel real-world system and interface design leveraging multiple mediums to balance stakeholder needs. To this end, the UI presented here incorporates features that support shared mental models (SMMs), trust establishment and development, and utilizes a centralized data distribution architecture to improve team performance. In addition to the interface, this paper presents a detailed look at the design process and the lessons learned from the perspective of a multi-year, real-world deployed system, as part of a large European project consisting of 21 partners from varying countries and backgrounds.

3D LiDAR-based place recognition has been extensively researched in urban environments, yet it remains underexplored in agricultural settings. Unlike urban contexts, horticultural environments, characterized by their permeability to laser beams, result in sparse and overlapping LiDAR scans with suboptimal geometries. This phenomenon leads to intra-and inter-row descriptor ambiguity. In this work, we address this challenge by introducing SPVSoAP3D, a novel modeling approach that combines a voxel-based feature extraction network with an aggregation technique based on a second-order average pooling operator, complemented by a descriptor enhancement stage. Furthermore, we augment the existing HORTO-3DLM dataset by introducing two new sequences derived from horticultural environments. We evaluate the performance of SPVSoAP3D against state-of-the-art (SOTA) models, including OverlapTransformer, PointNetVLAD, and LOGG3D-Net, utilizing a cross-validation protocol on both the newly introduced sequences and the existing HORTO-3DLM dataset. The findings indicate that the average operator is more suitable for horticultural environments compared to the max operator and other first-order pooling techniques. Additionally, the results highlight the improvements brought by the descriptor enhancement stage. The code is publicly available at https://github.com/Cybonic/SPVSoAP3D.git

Despite the increasing adoption of Deep Reinforcement Learning (DRL) for Autonomous Surface Vehicles (ASVs), there still remain challenges limiting real-world deployment. In this paper, we first integrate buoyancy and hydrodynamics models into a modern Reinforcement Learning framework to reduce training time. Next, we show how system identification coupled with domain randomization improves the RL agent performance and narrows the sim-to-real gap. Real-world experiments for the task of capturing floating waste show that our approach lowers energy consumption by 13.1% while reducing task completion time by 7.4%. These findings, supported by sharing our open-source implementation, hold the potential to impact the efficiency and versatility of ASVs, contributing to environmental conservation efforts.

We propose a novel hybrid cable-based robot with manipulator and camera for high-accuracy, medium-throughput plant monitoring in a vertical hydroponic farm and, as an example application, demonstrate non-destructive plant mass estimation. Plant monitoring with high temporal and spatial resolution is important to both farmers and researchers to detect anomalies and develop predictive models for plant growth. The availability of high-quality, off-the-shelf structure-from-motion (SfM) and photogrammetry packages has enabled a vibrant community of roboticists to apply computer vision for non-destructive plant monitoring. While existing approaches tend to focus on either high-throughput (e.g. satellite, unmanned aerial vehicle (UAV), vehicle-mounted, conveyor-belt imagery) or high-accuracy/robustness to occlusions (e.g. turn-table scanner or robot arm), we propose a middle-ground that achieves high accuracy with a medium-throughput, highly automated robot. Our design pairs the workspace scalability of a cable-driven parallel robot (CDPR) with the dexterity of a 4 degree-of-freedom (DoF) robot arm to autonomously image many plants from a variety of viewpoints. We describe our robot design and demonstrate it experimentally by collecting daily photographs of 54 plants from 64 viewpoints each. We show that our approach can produce scientifically useful measurements, operate fully autonomously after initial calibration, and produce better reconstructions and plant property estimates than those of over-canopy methods (e.g. UAV). As example applications, we show that our system can successfully estimate plant mass with a Mean Absolute Error (MAE) of 0.586g and, when used to perform hypothesis testing on the relationship between mass and age, produces p-values comparable to ground-truth data (p=0.0020 and p=0.0016, respectively).

This paper presents a Graph-based Simultaneous Localization And Mapping (GraphSLAM) approach for a robotic system relying on the reflections of ultrasonic guided waves to enable long-range inspection tasks on plate-based metal structures. A measurement model that can leverage multi-order acoustic echoes is introduced for accurate localization, and beamforming is used for mapping the boundaries of individual metal panels. These two elements are subsequently integrated within a nonlinear least squares optimizer to solve the full offline SLAM problem. We experimentally evaluate the potential of this approach in a laboratory environment. We observe the improved localization accuracy of the multi-order echo model compared to a second model, from previous works, that relies solely on first-order echoes. We also show that the proposed approach can yield accurate SLAM results, hence showcasing the standalone capability of ultrasonic-based GraphSLAM for envisioned long-range inspection applications.

This paper discusses the problem of autonomously constructing a qualitative map of an unknown 3D environment using a 3D-Lidar. In this case, how can we effectively integrate the quality of the 3D-reconstruction into the selection of the Next-Best-View? Here, we address the challenge of estimating the quality of the currently reconstructed map in order to guide the exploration policy, in the absence of ground truth, which is typically the case in exploration scenarios. Our key contribution is a method to build a prior on the quality of the reconstruction from the data itself. Indeed, we not only prove that this quality depends on statistics from the observation viewpoints, but we also demonstrate that we can enhance the quality of the reconstruction by leveraging these statistics during the exploration. To do so, we propose to integrate them into Next-Best-View selection policies, in which the information gain is directly computed based on these statistics. Finally, we demonstrate the robustness of our approach, even in challenging environments, with noise in the robot localization, and we further validate it through a real-world experiment.

Trust is an essential component in HRI, yet it has been impossible to agree on a definition within the HRI context. Moreover, the volume of definitions with different conceptualizations has led trust research to be viewed by some as a quagmire. Instead of attempting to define trust, this paper takes a bottom-up approach, starting with the body of current literature and breaking it into conceptual components within a trust system. Applying concepts such as abstraction and encapsulation from computer science, this paper attempts to synthesize the current body of literature into a metamodel where the components can either represent models themselves or natural groupings of elements. By viewing trust as a system (represented by a metamodel), it is possible to hide some of the details and functionality in each component without losing semantic value. This helps to clarify the trust workspace and terminology, highlights areas of trust research that are actively being researched, shows how different areas of research are connected, and captures the current state of trust research in a single framework.From a practical perspective, using this model also provides trust researchers with a single reference point, simplifies scoping one’s research, and enhances homogeneity by making cross-referencing and comparing study methods easier. Finally, viewing trust as a system allows capturing nuances definitions cannot, and may help identify gaps in current research.

We introduce Object-Guided localization (OGuL) based on a novel method of local-feature matching. Direct matching of local features is sensitive to significant changes in illumination. In contrast, object detection often survives severe changes in lighting conditions. The proposed method first detects semantic objects and establishes correspondences of those objects between images. Object correspondences provide local coarse alignment of the images in the form of a planar homography. These homographies are consequently used to guide the matching of local features. Experiments on standard urban localization datasets (Aachen, RobotCar-Season) show that OGuL significantly improves localization results with as simple local features as SIFT, and its performance competes with the state-of-the-art CNN-based methods trained for day-to-night localization.

The ultrasonic mapping of plate-based facilities is an essential step towards the robotic inspection of large metal structures such as storage tanks or ship hulls. This work proposes a novel framework that exploits ultrasonic echoes to recover grid-based and feature-based spatial representations jointly. We aim to improve on a previous mapping method [1] subject to errors due to interference, and which provides plate geometry estimates without uncertainty assessment. The grid can represent, all along the mapping process, both areas identified as inside or outside the current plate and areas whose state is still unknown, making it is suitable e.g. for detecting a change of plate, or for use in a later active-sensing strategy. We also leverage the resulting spatial information to filter out candidate plate edges that are no longer relevant, mitigating the detrimental effect of interference. We test the approach in simulation, with acoustic data acquired manually and with a real robot. Results show that it is effective for building combined map representations and robust to echo misdetection, contrary to a more standard mapping approach.

Several applications of video rely on camera calibration, a key enabler towards the measurement of metric parameters from images. Existing calibration methods have not been adapted for use with cameras that are hard to reach such as security cameras. This paper presents a drone-based camera calibration technique for the calibration of cameras in a broader range of operating environments. The technique is enabled by the use of a drone that is equipped with location sensors. In the proposed approach, the drone is used to sample points in the 3D space and its detection on the images provides the 2D matching points enabling the calibration. To design a flight path that allows for an accurate calibration, the paper proposes a methodology to evaluate the impact of path parameters and drone localization and detection uncertainties on the calibration uncertainty. This methodology is applied to evaluate the performance of different sampling paths for the calibration of a large diversity of cameras for the purpose of recommending reliable path parameters.

This work presents the Manifold Invariant Extended Kalman Filter, a novel approach for better consistency and accuracy in state estimation on manifolds. The robustness of this filter allows for techniques with high noise potential like ultra-wideband localization to be used for a wider variety of applications like autonomous metal structure inspection. The filter is derived and its performance is evaluated by testing it on two different manifolds: a cylindrical one and a bivariate b-spline representation of a real vessel surface, showing its flexibility to being used on different types of surfaces. Its comparison with a standard EKF that uses virtual, noise-free measurements as manifold constraints proves that it outperforms standard approaches in consistency and accuracy. Further, an experiment using a real magnetic crawler robot on a curved metal surface with ultra-wideband localization shows that the proposed approach is viable in the real world application of autonomous metal structure inspection.

This paper presents a semi-supervised framework for multi-level description learning aiming for robust and accurate camera relocalization across large perception variations. Our proposed network, namely DLSSNet, simultaneously learns weakly-supervised semantic segmentation and local feature description in the hierarchy. Therefore, the augmented descriptors, trained in an end-to-end manner, provide a more stable high-level representation for local feature dis-ambiguity. To facilitate end-to-end semantic description learning, the descriptor segmentation module is proposed to jointly learn semantic descriptors and cluster centers using standard semantic segmentation loss. We show that our model can be easily fine-tuned for domain-specific usage without any further semantic annotations, instead, requiring only 2D-2D pixel correspondences. The learned descriptors, trained with our proposed pipeline, can boost the cross-season localization performance against other state-of-the-arts.

Most of the research effort on image-based place recognition is designed for urban environments. In bucolic environments such as natural scenes with low texture and little semantic content, the main challenge is to handle the variations in visual appearance across time such as illumination, weather, vegetation state or viewpoints. The nature of the variations is different and this leads to a different approach to describing a bucolic scene. We introduce a global image description computed from its semantic and topological information. It is built from the wavelet transforms of the image's semantic edges. Matching two images is then equivalent to matching their semantic edge transforms. This method reaches state-of-the-art image retrieval performance on two multi-season environment-monitoring datasets: the CMU-Seasons and the Symphony Lake dataset. It also generalizes to urban scenes on which it is on par with the current baselines NetVLAD and DELF.

This paper presents a proof-of-concept for a localization and mapping system for magnetic crawlers performing inspection tasks on structures made of large metal plates. By relying on ultrasonic guided waves reflected from the plate edges, we show that it is possible to recover the plate geometry and robot trajectory to a precision comparable to the signal wavelength. The approach is tested using real acoustic signals acquired on metal plates using lawn-mower paths and random-walks. To the contrary of related works, this paper focuses on the practical details of the localization and mapping algorithm.

This work describes a monocular visual odometry framework, which exploits the best attributes of edge features for illumination-robust camera tracking, while at the same time ameliorating the performance degradation of edge mapping. In the front-end, an ICP-based edge registration provides robust motion estimation and coarse data association under lighting changes. In the back-end, a novel edge-guided data association pipeline searches for the best photometrically matched points along geometrically possible edges through template matching, so that the matches can be further refined in later bundle adjustment. The core of our proposed data association strategy lies in a point-to-edge geometric uncertainty analysis, which analytically derives (1) a probabilistic search length formula that significantly reduces the search space and (2) a geometric confidence metric for mapping degradation detection based on the predicted depth uncertainty. Moreover, a match confidence based patch size adaption strategy is integrated into our pipeline to reduce matching ambiguity. We present extensive analysis and evaluation of our proposed system on synthetic and real- world benchmark datasets under the influence of illumination changes and large camera motions, where our proposed system outperforms current state-of-art algorithms.

This paper introduces a novel feature detector based only on information embedded inside a CNN trained on standard tasks (e.g. classification). While previous works already show that the features of a trained CNN are suitable descriptors, we show here how to extract the feature locations from the network to build a detector. This information is computed from the gradient of the feature map with respect to the input image. This provides a saliency map with local maxima on relevant keypoint locations. Contrary to recent CNN-based detectors, this method requires neither supervised training nor finetuning. We evaluate how repeatable and how `matchable' the detected keypoints are with the repeatability and matching scores. Matchability is measured with a simple descriptor introduced for the sake of the evaluation. This novel detector reaches similar performances on the standard evaluation HPatches dataset, as well as comparable robustness against illumination and viewpoint changes on Webcam and photo-tourism images. These results show that a CNN trained on a standard task embeds feature location information that is as relevant as when the CNN is specifically trained for feature detection.

Autonomous robotic weeding systems in precision farming have demonstrated their full potential to alleviate the current dependency on herbicides or pesticides by introducing selective spraying or mechanical weed removal modules, thus reducing the environmental pollution and improving the sustainability. However, most previous works require fast weed detection system to achieve real-time treatment. In this paper, a novel computer vision based weeding control system is presented, where a non-overlapping multi-camera system is introduced to compensate the indeterminate classification delays, thus allowing for more complicated and advanced detection algorithms, e.g. deep learning based methods. The suitable tracking and control strategies are developed to achieve accurate and robust in-row weed treatment, and the performance of the proposed system is evaluated in different terrain conditions in the presence of various delays.

Vision-based localization and mapping in outdoor environments is still a challenging issue, which requests significant robustness against various unpredictable illumination changes. In this paper, an illumination-robust direct monocular SLAM system that focuses on modeling outdoor scenery is presented. To deal with global and local lighting changes, such as solar flares, the state-of-art illumination invariant photometric costs for RGB-D and stereo SLAM systems are revisited in the context of their monocular counterpart, where the camera motion and scene structure are jointly optimized with a reasonably poor initialization. Based on our analysis, a combined cost is proposed to achieve a high-precision motion estimation with an improved convergence radius. The proposed system is extensively evaluated on the synthetic and real-world datasets regarding accuracy, robustness, and processing time, where our approach outperforms systems with other costs and state-of-art DSO and ORBSLAM2 systems.

This work demonstrates how to leverage previous network expert demonstrations of UAV deployment to automate the drones placement in civil applications. Optimal UAV placement is an NP-complete problem: it requires a closed-form utility function that defines the environment and the UAV constraints, it is not unique and must be defined for each new UAV mission. This complex and time-consuming process hinders the development of UAV-networks in civil applications. We propose a method that leverages previous network expert solutions of UAV-network deployment to learn the expert's untold utility function form demonstrations only. This is especially interesting as it may be difficult for the inspection expert to explicit his expertise into such a function as it is too complex. Once learned, our model generates a utility function which maxima match expert UAV locations. We test this method on a Wi-Fi UAV network application inside a crowd simulator and reach similar quality-of-service as the expert. We show that our method is not limited to this UAV application and can be extended to other missions such as building monitoring.

In this paper, we describe a multi-scale monocular direct sparse visual odometry (DSO) system to recover large-scale trajectories in unstructured natural environments in real time, while building a consistent metric map of the visited scenes. In contrast to the current state-of-the-art DSO system, the proposed method allows for more robust motion estimation and more accurate reconstruction in distant scenes by exploiting the characteristics of short- and long-range pixels, respectively. The long-range pixels, which are less sensitive to small camera translations, are used to initialize the camera rotation, so as to boost the tracking robustness in challenging natural environments. A multi-scale reconstruction framework is developed to recover short-range structure over successive frames, as well as the long-range structure over distant frames, hence allowing for a more consistent mapping precision. The reconstruction precision, the tracking accuracy, and the robustness of the proposed system are extensively evaluated with a publicly available vKITTI dataset, as well as the challenging Devon Island dataset, and Symphony Lake dataset. A detailed performance comparison between the proposed method and the state-of-the-art DSO system is presented.

Robotic sensors are promising instruments for monitoring spatial phenomena. Oftentimes, rather than aiming to achieve low prediction error everywhere, one is interested in determining whether the phenomenon exhibits certain critical behavior. In this paper, we consider the problem of focusing autonomous sampling to determine whether and where the sensed spatial field exceeds a given threshold value. We introduce a receding horizon path planner, LSE-DP, which plans efficient paths for sensing in order to reduce our uncertainty specifically around the threshold value. We report fully autonomous field experiments with an Autonomous Surface Vessel (ASV) in an aquatic monitoring setting, which demonstrate the effectiveness of the proposed method. LSE-DP is able to reduce the uncertainty around the threshold value of interest to 68% when compared to non-adaptive methods.

Articulation of 3D characters requires control over many degrees of freedom: a difficult task with standard 2D interfaces. We present a tangible input device composed of interchangeable, hot-pluggable parts. Embedded sensors measure the device's pose at rates suitable for real-time editing and animation. Splitter parts allow branching to accommodate any skeletal tree. During assembly, the device recognizes topological changes as individual parts or pre-assembled subtrees are plugged and unplugged. A novel semi-automatic registration approach helps the user quickly map the device's degrees of freedom to a virtual skeleton inside the character. User studies report favorable comparisons to mouse and keyboard interfaces for the tasks of target acquisition and pose replication. Our device provides input for character rigging and automatic weight computation, direct skeletal deformation, interaction with physical simulations, and handle-based variational geometric modeling.

This paper presents practical vision-based collision avoidance for objects approximating a single point feature. Using a spherical camera model, a visual predictive control scheme guides the aircraft around the object along a conical spiral trajectory. Visibility, state and control constraints are considered explicitly in the controller design by combining image and vehicle dynamics in the process model, and solving the nonlinear optimization problem over the resulting state space. Importantly, range is not required. Instead, the principles of conical spiral motion are used to design an objective function that simultaneously guides the aircraft along the avoidance trajectory, whilst providing an indication of the appropriate point to stop the spiral behaviour. Our approach is aimed at providing a potential solution to the See and Avoid problem for unmanned aircraft and is demonstrated through a series of experimental results using a small quadrotor platform.

This paper presents a novel underwater robot based on biological locomotion principle. A robotic platform imitating sea-turtle fin propulsion is described and tested. As fin locomotion is a novel and complex research area, basic control concepts are analyzed and implemented. Based on a simulation, a fin-trajectory morphing control strategy is developed in order to control the robots roll, pitch and yaw rates, thus allowing the robot to follow a given vector. Absolute position control or depth control, however, is not yet implemented. The paper concludes with the presentation of a working system that demonstrated motion capabilities in air as well as the first dive test in a swimming pool.

Future requirements for drastic reduction of CO2production and energy consumption will lead to significant changes in the way we see mobility in the years to come. However, the automotive industry has identified significant barriers to the adoption of electric vehicles, including reduced driving range and greatly increased refueling times. Automated cars have the potential to reduce the environmental impact of driving, and increase the safety of motor vehicle travel. The current state-of-the-art in vehicle automation requires a suite of expensive sensors. While the cost of these sensors is decreasing, integrating them into electric cars will increase the price and represent another barrier to adoption. The V-Charge Project, funded by the European Commission, seeks to address these problems simultaneously by developing an electric automated car, outfitted with close-to-market sensors, which is able to automate valet parking and recharging for integration into a future transportation system. The final goal is the demonstration of a fully operational system including automated navigation and parking. This paper presents an overview of the V-Charge system, from the platform setup to the mapping, perception, and planning sub-systems.

Visual homing enables a mobile robot to move to a reference position using only visual information. The approaches that we present in this paper utilize matched image key points (e.g., scale-invariant feature transform) that are extracted from an omnidirectional camera as inputs. First, we propose three visual homing methods that are based on feature scale, bearing, and the combination of both, under an image-based visual servoing framework. Second, considering computational cost, we propose a simplified homing method which takes an advantage of the scale information of key-point features to compute control commands. The observability and controllability of the algorithm are proved. An outlier rejection algorithm is also introduced and evaluated. The results of all these methods are compared both in simulations and experiments. We report the performance of all related methods on a series of commonly cited indoor datasets, showing the advantages of the proposed method. Furthermore, they are tested on a compact dataset of omnidirectional panoramic images, which is captured under dynamic conditions with ground truth for future research and comparison.

Visual Homing is the process by which a mobile robot moves to a Home position using only information extracted from visual data. The approach we present in this paper uses image keypoints (e.g. SIFT) extracted from omnidirectional images and matches the current set of keypoints with the set recorded at the Home location. In this paper, we first formulate three different visual homing problems using uncalibrated omnidirectional camera within the Image Based Visual Servoing (IBVS) framework; then we propose a novel simplified homing approach, which is inspired by IBVS, based only on the scale information of the SIFT features, with its computational cost linear to the number of features. This paper reports on the application of our method on a commonly cited indoor database where it outperforms other approaches. We also briefly present results on a real robot and allude on the integration into a topological navigation framework.

Visual homing has been widely studied in the past decade. It enables a mobile robot to move to a Home position using only information extracted from visual data. However, integration of homing algorithms into real applications is not widely studied and poses a number of significant challenges. Failures often occur due to moving people within the scene and variations in illumination. We present a novel integrated indoor topological navigation framework, which combines odometry motion with visual homing algorithms. We show robustness to scene variation and real-time performance through a series of tests conducted in four real apartments and several typical indoor scenes, including doorways, offices etc.

Pseudo-omnidirectional robots with independently steerable wheels require a method to synchronize the steering motion of the wheels in order to keep a unique instantaneous center of rotation (ICR). For standard wheels, the instantaneous center of rotation is defined as the intersection point of all wheel axes. We present a novel approach to deal with the problem of continuously shifting the center of rotation of a pseudo-omnidirectional rover from an initial to a demanded position in the Cartesian plane. The main contribution is the consideration of substantial velocity and acceleration limits on the steering units, as well as mechanical constraints and noise affected sensor measurements. We solve this problem by deriving a relationship between the steering accelerations of the single wheels and the acceleration of the center of rotation. We furthermore provide a contribution to the tracking of the ICR in the presence of significant sensor noise. Our results are evaluated by tests on the rover breadboard developed during the activities for the ExoMars mission.

Terrainability is mostly dependant on the suspension mechanism and the control of a space rover. For the six wheeled CRAB rover, this paper presents the composite control design with torque control and adaptive Kriging control to improve the terrainability, somewhat related to minimizing wheel slip. As CRAB is moving slowly, the torque control is processed by minimizing the variance of the required friction coefficient based on the static model. Adaptive Kriging control is used to track the commanded velocity. The system uncertainty is compensated by Kriging estimation based on the velocity dynamics. Experiment results with two different tires show the effectiveness of the control scheme.

This paper presents a semi-autonomous navigation strategy aimed at the control of assistive devices (e.g. an intelligent wheelchair) using low throughput interfaces. A mobile robot proposes the most probable action, as analyzed from the environment, to a human user who can either accept or reject the proposition. In case of rejection, the robot will propose another action, until both entities agree on what needs to be done. In a known environment, the system infers the intended goal destination based on the first executed actions. Furthermore, we endowed the system with learning capabilities, so as to learn the user habits depending on contextual information (e.g. time of the day or if a phone rings). This additional knowledge allows the robot to anticipate the user intention and propose appropriate actions, or goal destinations.

Homing is one of the fundamental functions for both the mobile robot and the flying robot. Furthermore, homing can be introduced into a topological navigation system by cyclically setting Home positions at the keypoints/nodes in a topological map. In this work, we describe a bearing-only homing method based on only few matching keypoints to grant the mobile robot the homing ability. Our method considers the homing problem as a visual servoing problem in 2D plane and even in 3D space, using an omnidirectional camera as the visual sensor. It doesn't require the distance information to the reference feature points. The proof of the convergence for the algorithm is also given. The simulation results confirm the feasibility and robustness of our method.

In search of underwater locomotion methods as alternatives to propellers, systems relying on the propagation of waves along a fin have already been designed and evaluated by several scientists. Considerable effort has been undertaken to optimise their efficiency both by fluid dynamic analysis and experiments on physical prototypes. One drawback of the systems hitherto has been their electro-mechanical complexity in that they required many actuators and refined control strategies to generate the desired fin undulation. Our approach has been to translate the result of these optimisations into a simpler, purely mechanical model relying on the principle of camshafts to achieve a similar undulatory fin motion. The goal was to evaluate whether this type of propulsion system is feasible and whether it was a viable alternative to propellers in Autonomous Underwater Vehicles. The prototype built during the project, CUTTLEFIN, reached comparable speeds to other undulating robot solutions. Force measurements also showed that the thrust produced is in qualitative accordance to a simplified fluid dynamics model. This makes the camshaft approach a promising option for generating an undulating wave in a membrane-based fin propulsion system, if one is willing to pay the price of lower flexibility compared to current dexterously actuated solutions.

The capability to overcome terrain irregularities or obstacles, named terrainability, is mostly dependant on the suspension mechanism of the rover and its control. For a given wheeled robot, the terrainability can be improved by using a sophisticated control, and is somewhat related to minimizing wheel slip. The proposed control method, named torque control, improves the rover terrainability by taking into account the whole mechanical structure. The rover model is based on the Newton-Euler equations and knowing the complete state of the mechanical structures allows us to compute the force distribution in the structure, and especially between the wheels and the ground. Thus, a set of torques maximizing the traction can be used to drive the rover. The torque control algorithm is presented in this paper, as well as tests showing its impact and improvement in terms of terrainability. Using the CRAB rover platform, we show that the torque control not only increases the climbing performance but also limits odometric errors and reduces the overall power consumption.

A method for detecting changes in the environment using only vision sensors is presented. We demonstrate that optical flow can be used to detect these changes at key locations in outdoor scenarios in difficult and varying lighting conditions. These key locations are used as nodes in a topological mapping and localization framework. To close the loop we employ a bag-of-words methodology. We show that bag-of-words methods can be used in real-time on a standard computer to detect loop closures in sparse topological maps. Experimental results from field trials using our quad-rotor UAV demonstrate the capability of the proposed scene change detection method.

This paper proposes an object classification framework based on a geometric grammar aimed for mobile robotic applications. The paper first discusses the geometric grammar as a compact representation form for object categories with primitive parts as its constituent elements. The paper then discusses the object classification implemented as parsing of primitive parts. In particular, two approaches are discussed that constrain the search space in order to render the parsing of the primitive parts practical. The two approaches are experimentally verified, first, for a generic object category of chair applied to real range images acquired with a range camera mounted on a mobile robot and, second, for multiple generic object categories applied to synthetic range images. The experimental results show the practicability of the framework.

Mobile robots rely on their ability of scene recognition to build a topological map of the environment and perform location-related tasks. In this paper, we describe a novel lightweight scene recognition method using an adaptive descriptor which is based on color features and geometric information for omnidirectional vision. Our method enables the robot to add nodes to a topological map automatically and solve the localization problem of mobile robot in realtime. The descriptor of a scene is extracted in the YUV color space and its dimension is adaptive depending on the segmentation result of the panoramic image. Furthermore, the descriptor is invariant to rotation and slight changes of illumination. The robustness of the scene matching and recognition is tested through real experiments in a dynamic indoor environment. The experiment is carried out on a mobile robot equipped with an omnidirectional camera. In our tests, the average processing time is 30 ms for each frame including feature extraction, matching, and the adding of new nodes.

In robotics, vertical lines have been always very useful for autonomous robot localization and navigation in structured environments. This paper presents a robust method for matching vertical lines in omnidirectional images. Matching robustness is achieved by creating a descriptor which is very distinctive and is invariant to rotation and slight changes of illumination. We characterize the performance of the descriptor on a large image dataset by taking into account the sensitiveness to the different parameters of the descriptor. The robustness of the approach is also validated through a real navigation experiment with a mobile robot equipped with an omnidirectional camera.

Trailer reversing is a problem frequently considered in the literature, usually with fairly complex non-linear control theory based approaches. In this paper, we present a simple method for stabilizing a tractor-trailer system to a trajectory based on the notion of controlling the hitch-angle of the trailer rather than the steering angle of the tractor. The method is intuitive, provably stable, and shown to be viable through various experimental results conducted on our test platform, the CSIRO autonomous tractor.

This paper reports work on the automation of a hot metal carrier, which is a 20 tonne forklift-type vehicle used to move molten metal in aluminium smelters. To achieve efficient vehicle operation, issues of autonomous navigation and materials handling must be addressed. We present our complete system and experiments demonstrating reliable operation. One of the most significant experiments was five-hours of continuous operation where the vehicle travelled over 8 km and conducted 60 load handling operations. Finally, an experiment where the vehicle and autonomous operation were supervised from the other side of the world via a satellite phone network are described.

This paper reports work involved with the automation of a hot metal carrier - a 20 tonne forklift-type vehicle used to move molten metal in aluminium smelters. To achieve efficient vehicle operation, issues of autonomous navigation and materials handling must be addressed. We present our complete system and experiments demonstrating reliable operation. One of the most significant experiments was five-hours of continuous operation where the vehicle travelled over 8 km and conducted 60 load handling operations. We also describe an experiment where the vehicle and autonomous operation were supervised from the other side of the world via a satellite phone network.

Hot metal carriers (HMCs) are large forklift-type vehicles used to move molten metal in aluminium smelters. The molten metal is contained in bucket-like crucibles, that the HMC picks up. In this paper we explore the feasibility of using active appearance models to recognise and localise the handle of a crucible, from a camera on board an autonomous HMC. A two-dimensional model is built that mimics the apparent perspective deformations of the three- dimensional handle. The model is fitted to the handle using efficient algorithms, that include M-estimators for improved robustness. The fitting algorithm also provides and estimate of the actual distance to the crucible. We evaluate the accuracy and robustness of the approach in different lighting conditions.

In this paper, we propose a real-time method to detect obstacles using theoretical models of the ground plane, first in a 3D point cloud given by a stereo camera, and then in an optical flow field given by one of the stereo pair's camera. The idea of our method is to combine two partial occupancy grids from both sensor modalities with an occupancy grid framework. The two methods do not have the same range, precision and resolution. For example, the stereo method is precise for close objects but cannot see further than 7 m (with our lenses), while the optical flow method can see considerably further but has lower accuracy. Experiments that have been carried on the CyCab mobile robot and on a tractor demonstrate that we can combine the advantages of both algorithms to build local occupancy grids from incomplete data (optical flow from a monocular camera cannot give depth information without time integration)

In this project, we took on the task of localizing an automatic vehicle and building a map of the car park in real time. This takes place within the car park of INRIA Rhone-Alpes on the CyCab vehicle with a Sick laser range scanner. Our method uses only laser scanners to retrieve the position and orientations of vehicles in the car park. With the detected vehicles as landmarks, CyCab performs a localization of itself and builds a map of the car park at the same time. Classical clustering and segmentation techniques to extract line segments from the laser scan data are applied. The key contribution of the paper is the extraction of vehicle poses from the line segments using Bayesian programming. The method of FastSLAM is used in localizing CyCab and estimating the pose of vehicles in the car park. A set of hypotheses is obtained as a result. The second contribution is a method of combining the set of hypotheses together to form a final map of the car park.

In this paper we address the problem of obstacles avoidance for car-like robots. We present a generic nonholonomic path deformation method that has been applied on two robots. The principle is to perturb the inputs of the system in order to move away from obstacles and to keep the nonholonomic constraints satisfied. We present an extension of the method to car-like robots. We have integrated the method on two robots (Dala and CyCab) and carried out experiments that show the portability and genericity of the approach.

Autonomous navigation of a mobile robot along a predefined trajectory is a widely studied problem in the robotics community. We propose a Bayesian architecture that aims at being able to replay any sensori-motor trajectory trajectory defined as a sequence of perceptions and actions - as long as the robot starts in its neighbourhood. In order to increase robustness, we also use this Bayesian framework to estimate system self-confidence while the robot is moving. This work has been validated both on a simulated robot and on a real robot: the CyCab.

The recent development of a new kind of public transportation system relies on a particular double-steering kinematic structure enhancing maneuverability in cluttered environments such as downtown areas. We call bi-steerable car a vehicle showing this kind of kinematics. Endowed with autonomy capacities, the bi-steerable car ought to combine suitably and safely a set of abilities: simultaneous localisation and environment modelling, motion planning and motion execution amidst moderately dynamic obstacles. In this paper we address the integration of these four essential autonomy abilities into a single application. Specifically, we aim at reactive execution of planned motion. We address the fusion of controls issued from the control law and the obstacle avoidance module using probabilistic techniques.

Evolve safely in an unchanged environment and possibly following an optimal trajectory is one big challenge presented by situated robotics research field. Collision avoidance is a basic security requirement and this paper proposes a solution based on a probabilistic approach called Bayesian Programming. This approach aims to deal with the uncertainty, imprecision and incompleteness of the information handled. Some examples illustrate the process of embodying the programmer preliminary knowledge into a Bayesian program and experimental results of these examples implementation in an electrical vehicle are described and commented. Some videos illustrating these experiments can be found at http://www-laplace.imag.fr.

More and more fields of applied computer science involve fusion of multiple data sources, such as sensor readings or model decision. However, incompleteness of the model prevents the programmer from having an absolute precision over their variables. Therefore Bayesian framework can be adequate fro such a process as it allows handling of uncertainty. We will be interested in the ability to express any fusion process as a product, for it can lead to reduction of complexity in time and space. We study in this paper various fusion schemes and propose to add consistency variable to justify the use of a product to compute distribution over the fused variable. We will then show application of this new fusion process to localization of a mobile robot and obstacle avoidance.

One of the common ways of localization in robotics is the triangulation using a system composed of a sensor and some landmarks (which can be artificial or natural). This paper presents a framework, namely the localization space, in order to deal with problems such as the landmark placement and motion planning including the localization constraint. Based on this framework, we present general approaches to the optimal distribution of the landmarks or to the computation of reliable trajectories. The case of a mobile robot equipped with an orientable sensor (such as a pan vision system) is presented to illustrate the formal concepts and to show the practical relevance of the proposed tools.

A common way of localization in robotics is using triangulation on a system composed of a sensor and some landmarks (which can be artificial or natural). First, when no identifying marks axe set on the landmarks, their identification by a robust algorithm is a complex problem which may be solved thanks to correspondence graphs. Second, when the localization system has no a priori information about its environment, it has to build its own map in parallel with estimating its position, a problem known as the simultaneous localization and mapping (SLAM). Recent works have proposed to solve this problem based on building a map made of invariant features. This paper describes the algorithms and data structure needed to deal with landmark matching, robot localization and map building in a single efficient process, unifying the previous approaches. Experimental results axe presented using an outdoor robot car equipped with a 2D scanning laser sensor.