State estimation in the context of dynamical systems is crucial for various applications, including control and monitoring. Moving Horizon Estimation (MHE) is an optimization-based state estimation algorithm that leverages a known dynamical model integrated over a moving horizon. The MHE optimization criterion corresponds to identify the initial state that best aligns the integrated trajectory with the system observation. In MHE setting, the state estimation performance increases with the considered length of the moving horizon but it can become computationally intensive which is a limiting factor for its applicability to fast-varying dynamical systems or on hardware with restricted computational power. Deep Learning (DL) methods can learn solutions to complex optimization problems without incurring any additional online computational cost beyond the inference of the considered architecture. In the context of state estimation we propose to study different type of DL architecture in order to provide full state estimation from partial and noisy system observations. The novel proposed method is based on an end-to-end differentiable formulation of the MHE optimization problem, enabling the offline training of a DL model to provide a state estimation that minimizes the MHE optimization criterion. Once training is completed, state estimations are generated through an explicit relationship learned by the DL model. The proposed method is compared to the online MHE formulation in various case studies, including scenarios with partially observed state and model discrepancies in the context of lateral vehicle dynamics. The results highlight improved state estimation performance both in terms of reduced computational time and accuracy with respect to the online MHE algorithm.

System identification is a fundamental task in understanding and modeling dynamical systems, with extensive applications in engineering. Traditional statistical estimators for system identification rely on loss functions based on single-step predictions of model state variables. However, these approaches often lack robustness and reliability in real-world scenarios characterized by noisy and imperfect data. Recent advancements have introduced multi-step loss functions for autoregressive neural network predictions, leading to significant improvements in system identification performance. These loss functions are optimized via gradient descent, leveraging backpropagation through the numerically integrated neural network architecture. Despite their potential, the statistical and mathematical properties of these gradient estimators, such as bias, variance and robustness, remain underexplored. This paper examines the statistical and mathematical characteristics of multi-step loss function estimators in the context of dynamical system identification. We provide a theoretical foundation for the bias-variance decomposition of these loss functions, enabling the separation of error contributions from disturbances and deterministic model parameterization. Theoretical insights are validated and extended through empirical analysis, allowing an exploration of the bias-variance decomposition dynamics across the training phase. Our results demonstrate both theoretically and practically the influence of the contractive properties of the underlying dynamical system and the autoregressive prediction horizon on training stability. By bridging the theoretical and practical gap in the exploration of multi-step loss functions, this work contributes to the understanding and development of more robust and reliable methods for dynamical system identification involving gradient descent.

Safety assurance using all perceptual information to predict the motion of dynamic agents is critical in urban environments and remains an open challenge. For Autonomous Vehicles (AV) operating around vulnerable road users, the risk assessment strategy often needs to address stochastic uncertainties in the multiple possible trajectories (or multimodal motion) of the surrounding traffic agents. However, this increases the complexity of the navigation problem using the existing planners. To address this issue, this paper presents a multi-level optimization strategy that combines sampling-based and direct optimization methods for decision-making and control with improved safety and trajectory smoothness. In the primary stage, a sampling-based optimization framework systematically identifies safe candidate trajectories by employing the Fusion of stochastic Predictive Inter-Distance Profile (F-sPIDP). F-sPIDP encapsulates the multimodal dynamics of traffic agents and explicitly computes the uncertainties in their estimated or tracked states. From the set of trajectories, a reference optimal trajectory and its F-sPIDP setpoints are selected, adhering to stringent safety constraints and motion smoothness. Subsequently, a secondary local control optimization refines the optimal trajectory to ensure compliance with the AV's kinematic and dynamic constraints while accounting for the quantified uncertainty within the F-sPIDP framework. The performance of the proposed method was assessed through simulations and statistical analyses, evaluating its robustness to diverse levels of uncertainty.

Trajectory forecasting in urban environments is a critical task that needs to be addressed for the safety of autonomous vehicles, particularly in urban road intersection scenarios, where agents exhibit diverse behaviors mainly due to complex interactions between agents and the environment and the diversity of paths available to the agents. Current state-of-the-art methods do not perform well in urban road intersection scenarios. To address this issue, the proposed novel framework CandidateGraph-Net (CG-Net), improves trajectory prediction in urban road intersection scenarios by encoding the available candidate centerlines at the current location of the target agent. The proposed interaction encoder in CG-Net is inspired by human behavior. It is modeled utilizing a bipartite graph attention network to predict the trajectory of the target agent. It estimates the trajectory in the same way as humans anticipate the trajectory of other vehicles and pedestrians in dynamic environments. The agent embeddings in the interaction encoder at each time step pay attention to nearby agents and surrounding scene elements simultaneously. This enables the model to learn how to prioritize interactions between nearby agents and the environment map. Further, CG-Net’s performance is evaluated using the Argoverse 2 motion forecasting dataset. The results demonstrate its effectiveness in urban road intersection scenarios, with an overall improvement in key metrics such as minFDE and minADE compared to baseline methods. These improvements highlight CG-Net’s ability to perform better motion forecasting in urban road intersection scenarios.

Ensuring connectivity and coordination in heterogeneous multi-robot systems (MRS) navigating complex environments is a critical challenge, especially when communication constraints and obstacles cause robots to become lost or disconnected. This paper presents a novel approach integrating Model Predictive Control (MPC) with Generalized Connectivity Maintenance (GCM) to enable real-time path adaptation while preserving connectivity. We introduce a decentralized decision-making framework that enables robots to recover lost members dynamically. When reconnection is infeasible, the system adapts the mission to continue while accounting for disconnected robots. Our method is evaluated through extensive simulations, showing its scalability and effectiveness in maintaining connectivity and ensuring mission success. Additionally, we propose a new evaluation metric that comprehensively assesses system performance, considering connectivity, coordination, and mission success in challenging environments.

Risk assessment and management in urban scenarios are difficult for automated vehicles due to perception uncertainties and the latent stochastic and high-dynamic behaviors of other agents e.g., Personal Light Electric Vehicles (PLEVs). Although the Predictive Inter-Distance Profile (PIDP) provides a continuous assessment of the risk between multiple agents, it fails when there are significant uncertainties in the estimated states of the agents. In this paper, we propose a Stochastic PIDP (sPIDP) to handle the uncertainties in the motion of the agents. sPIDP projects the uncertainties to the inter-distance between the agents. Furthermore, an Uncertainty-aware MPC is proposed to perform risk management. Statistical results considering multiple traffic scenarios show that our method is efficient for safe navigation.

Automated vehicles perform well in simple environments with clear rules, but urban traffic presents significant challenges due to the unpredictable behavior of road users, sometimes beyond traffic rules. Achieving full autonomy in such settings requires a systematic approach to modeling the possible actions of all agents. This paper presents a multi-rules reachability analysis framework that integrates graph-based maps with real-time perception data to dynamically characterize the surrounding space. By leveraging the semantic richness and modularity of Lanelet2 maps, our method provides a structured representation that enhances situational awareness. This allows for the extraction of navigation-relevant information, with the goal of supporting safer and more efficient decision-making in complex urban environments.

Stereo-matching is one of the most important low-level visual perception tasks. Currently, two-stage 2D-3D networks are the main solutions. These methods involve creating a cost volume using low-resolution stereo feature maps, which separate the network into a feature net and a matching net. However, two-stage methods may accumulate errors, and the use of a low-resolution cost volume may result in the loss of some of the matching information. To overcome these problems, we propose the first one-stage deep stereo network, named StereoOne. It has an efficient module that builds a cost volume at image resolution in real-time. The feature extraction and matching are learned in a single 3D network. Based on the experiments, the new network outperforms 2D-3D network baselines and achieves competitive performance with the state-of-the-art.

International audience

This paper presents a novel approach for Visual Servoing (VS) using a multispectral camera, where the number of data are more than three times that of a standard color camera. To meet real-time feasibility, the multispectral data captured by the camera are processed using dimensionality reduction techniques. Instead of relying on traditional approaches that select a subset of bands, the proposed method unlocks the full potential of a multispectral camera by pinpointing individual pixels that hold the richest information across all bands. While sacrificing spectral resolution for enhanced spatial resolution - crucial for precise robotic control in forested environments - this fusion process offers a powerful tool for robust and real-time VS in natural settings. Validated through simulations and real-world experiments, the proposed approach demonstrates its efficacy by leveraging the full spectral information of the camera while preserving spatial details.

This work focuses on localizing a single target robot with multi-robot formations in 2D space. The cooperative robots employ inter-robot range measurements to assess the target position. In the presence of noisy measurements, the choice of formation geometries significantly impacts the accuracy of the target robot’s pose estimation. While an infinite number of geometries exists to optimize localization accuracy, the current practice is to choose the final formation geometry based on convenience criteria such as simplicity or proximity to the initial position of the robots. The former leads to the selection of regular polygon-shaped formations, while the latter results in behaviour-based formations. Different from existing works, we conduct a complete robustness study of formation geometries in the presence of deviations from the desired formation and range measurement errors. In 2D scenarios, we establish necessary and sufficient conditions for formation geometries to be robust against robot positioning errors. This result substantiates the extensive use of regular polygon formations. However, our analysis reveals the lack of robustness of the commonly used square formation geometry, which stands as an exception. Simulation results illustrate the advantages of these robust geometries in enhancing target localization accuracy.

In the field of lidar odometry for autonomous navigation, the Iterative Closest Point (ICP) algorithm is a prevalent choice for estimating robot motion by comparing point clouds. However, ICP accuracy is strictly dependent on the nature of the features involved, but also on the directional choice of the extraction and matching, either from the current to the reference point cloud or vice-versa. Point-to-line or point-to-plane correspondences have been proven to provide the more accurate odometry results. The matching is generally done in a mono-directional framework: extract the features (lines or planes) in the current point cloud and match them to points in the reference point cloud. This paper introduces a novel formulation, named Balanced ICP, that performs feature extraction (lines or planes) in both point clouds and consequent matching in both the directions. Therefore, the cost function is designed to perform a simultaneous optimization of all available data balancing the noise and extraction errors. The experiments, conducted both on simulated and real data from the KITTI dataset, reveal that our method outperform the classical mono-directional formulations, in terms of robustness, accuracy and stability.

The navigation of autonomous vehicles around pedestrians is a key challenge when driving in urban environments. It is essential to test the proposed navigation system using simulation before moving to real-life implementation and testing. Evaluating the performance of the system requires the design of a diverse set of tests which spans the targeted working scenarios and conditions. These tests can then undergo a process of evaluation using a set of adapted performance metrics. This work addresses the problem of performance evaluation for an autonomous vehicle in a shared space with pedestrians. The methodology for designing the test simulations is discussed. Moreover, a group of performance metrics is proposed to evaluate the different aspects of the navigation: the motion safety, the quality of the generated trajectory and the comfort of the pedestrians surrounding the vehicle. Furthermore, the success/fail criterion for each metric is discussed. The implementation of the proposed evaluation method is illustrated by evaluating the performance of a pre-designed proactive navigation system using a shared space crowd simulator under Robot Operating System (ROS).

Autonomous mobile robot navigation in a human populated and encumbered environment is recognized as a hard problem to be solved in real-time. Most of the time, robots face the so-called ‘Freezing Robot Problem’, that occurs when the robot stops because no feasible and safe motion can be found. In order to provide to the robot the capability of proactive navigation, in this work we generalize the classical Social Force Model into a Universal Social Force Model (USFM) that attributes to any object surrounding the robot (humans, robots, obstacles) a social behavior. Nonlinear Model Predictive Control (MPC) can be used to solve the autonomous navigation problem since it can take into account all the possible constraints coming from the interaction model between the robot and the different surrounding objects. However, to be effective, MPC requires a sufficiently large prediction horizon, which generally implies a high computational cost. In order to considerably reduce the computational cost, we propose a new control parametrisation based on Thin Plate Spline Radial Basis Functions that allow us to have a large prediction horizon with fewer parameters. The global control framework is validated in simulation with virtual pedestrians, and in real world environments.

Robots are now widely used around humans, in homes and public places like the museums, all due to their many benefits. These autonomous robots are called social or service robots and they always find it difficult to navigate in crowded environments; largely because of the high level of uncertainty in observing and predicting human behaviours in a highly dynamic environment. Uncertainty is propagated during prediction and might grow to levels that renders the whole environment unsafe for the robot leading to the so-called Freezing Robot Problem - FRP. This work presents our proposed approach to proactively account for various uncertainties during the robot's motion planning using a stochastic Nonlinear Model Predictive Controller (SNMPC). Additionally, using numerical optimization methods enables the planner to compute new control commands in realtime.

Solving Direct Shooting Model Predictive Control (MPC) optimization problems online can be computationally expensive if a large horizon is used while also maintaining a dense time sampling. In these cases, it is accepted that tradeoffs between computational load and performances should be sought in order to meet real-time feasibility requirements. However, making the problem more tractable for the hardware should not necessarily imply a decrease in performances. One technique that has been proposed in the literature makes use of control input parameterizations to decrease the numerical complexity of nonlinear MPC problems without necessarily affecting the performances significantly. In this paper, we review the use of parameterizations and propose a simple Sequential Quadratic Programming algorithm for nonlinear MPC. We then benchmark the performances of the solver in simulation, showing that parameterizations allow to attain good performances with (significantly) lower computation times than state-of-the-art solvers.

Navigation in close proximity with pedestrians is a challenge on the way to fully automated vehicles. Pedestrian-friendly navigation requires an understanding of pedestrian reaction and intention. Merely safety based reactive systems can lead to sub-optimal navigation solutions resulting in the freezing of the vehicle in many scenarios. Moreover, a strictly reactive method can produce unnatural driving patterns which cannot guarantee the legibility or social acceptance of the automated vehicle. This work presents a proactive maneuvering method adapted to navigation in close interaction with pedestrians using a dynamic channel approach. The method allows to proactively explore the navigation options based on anticipating pedestrians cooperation. The navigation is tested in frontal and lateral crossing scenarios with variable space density. The system is implemented under ROS, and compared with the probabilistic Risk-RRT planning method. The results are evaluated based on the safety and comfort of the pedestrians, and the quality of the vehicle's trajectory.

Visual odometry is an important part of the perception module of autonomous robots. Recent advances in deep learning approaches have given rise to hybrid visual odometry approaches that combine both deep networks and traditional pose estimation methods. One limitation of deep learning approaches is the availability of ground truth data needed to train the neural networks. For example, it is extremely difficult, if not impossible, to obtain a ground truth dense depth map of the environment to be used for stereo visual odometry. Even if unsupervised training of networks has been investigated, supervised training remains more reliable and robust. In this paper, we propose a new hybrid dense stereo visual odometry approach in which a dense depth map is obtained with a network that is supervised using ground truth poses that can be more easily obtained than ground truth depths maps. The depth map obtained from the neural network is used to warp the current image into the reference frame and the optimal pose is obtained by minimizing a cost function that encodes the similarity between the warped image and the reference image. The experimental results show that the proposed approach, not only improves state-of-the-art depth maps estimation networks on some of the standard benchmark datasets, but also outperforms the state-of-the-art visual odometry methods.

This article formalizes, under a single common multisensor-based predictive control framework, five different types of parking maneuvers: perpendicular, diagonal for both forward and backward motions, and parallel for backward motions. Since, from a practical point of view, forward parallel parking is usually not advisable, it is not addressed in this work. By moving the effort from motion planning to control, the parking tasks can be completely defined solely from the detected empty parking spots. Additionally, the classical compromise between completeness and computational efficiency when compared to exploration-based path planning techniques is eliminated. The results of a few individual cases are presented and compared against a state-of-the-art path planning approach to illustrate the behavior and performance of the proposed framework as well as results from exhaustive simulations to assess its convergence. As shown in the convergence analyses, the presented approach allows us to park from virtually any sensible initial pose. Finally, real experimentation using a robotized Renault ZOE shows the validity and robustness in the convergence domain of the presented approach.

Visual servoing control schemes, such as Image-Based (IBVS), Pose Based (PBVS) or Hybrid-Based (HBVS) have been extensively developed over the last decades making possible their uses in a large number of applications. It is well-known that the main problems to be handled concern the presence of local minima or singularities, the visibility constraint, the joint limits, etc. Recently, Model Predictive Path Integral (MPPI) control algorithm has been developed for autonomous robot navigation tasks. In this paper, we propose a MPPI-VS framework applied for the control of a 6-DoF robot with 2D point, 3D point, and Pose Based Visual Servoing techniques. We performed intensive simulations under various operating conditions to show the potential advantages of the proposed control framework compared to the classical schemes. The effectiveness, the robustness and the capability in coping easily with the system constraints of the control framework are shown.

In this paper, a distributed observer-based approach is proposed to control the longitudinal motion of car-like vehicle platoon moving in an urban environment. To the best of our knowledge, this is the first work presenting an observer-based platoon controller that combines the advantages of high traffic capacity and a minimum number of communication links. To achieve a high traffic flow, a constant-spacing policy is used. However, for that policy, to make platoon string stable, the leader information must be broadcast to all the vehicles. Therefore, we propose a control law in which the predecessor position information is acquired by a sensor-based link while a communication-based link is used to obtain the leader information. Then, an observer is designed and integrated into the control law such that the velocity information of the predecessor can be estimated without the need to communicate with the preceding vehicle. For navigation in urban environments, we present a third order platoon model represented in the curvilinear coordinates. Conditions for asymptotic stability and string stability are given considering the vehicle actuator dynamics and the induced network/sensor time delay. Finally, we provide both simulation and real-time results to validate our approach feasibility and to corroborate our theoretical findings.

Recently, Model Predictive Path Integral (MPPI) control algorithm has been extensively applied to autonomous navigation tasks, where the cost map is mostly assumed to be known and the 2D navigation tasks are only performed. In this paper, we propose a generic MPPI control framework that can be used for 2D or 3D autonomous navigation tasks in either fully or partially observable environments, which are the most prevalent in robotics applications. This framework exploits directly the 3D-voxel grid acquired from an on-board sensing system for performing collision-free navigation. We test the framework, in realistic RotorS-based simulation, on goal-oriented quadrotor navigation tasks in a cluttered environment, for both fully and partially observable scenarios. Preliminary results demonstrate that the proposed framework works perfectly, under partial observability, in 2D and 3D cluttered environments.

This paper explores the feasibility of a Multi-Sensor-Based Predictive Control (MSBPC) approach in order to have constraint-based backward non-parallel (perpendicular and diagonal) parking maneuvers capable of dealing with moving pedestrians and, if necessary, performing multiple maneuvers. Our technique relies solely in sensor data expressed relative to the vehicle and therefore no localization is inherently required. Since the proposed approach does not plan any path and instead the controller maneuvers the vehicle directly, the classical path planning related issues are avoided. Real experimentation validates the effectiveness of our approach.

International audience

Developing autonomous vehicles capable of navigating safely and socially around pedestrians is a major challenge in intelligent transportation. This challenge cannot be met without understanding pedestrians' behavioral response to an autonomous vehicle, and the task of building a clear and quantitative description of the pedestrian to vehicle interaction remains a key milestone in autonomous navigation research. As a step towards safe proactive navigation in a space shared with pedestrians, this work introduces a pedestrian-vehicle interaction behavioral model. The model estimates the pedestrian's cooperation with the vehicle in an interaction scenario by a quantitative time-varying function. Using this cooperation estimation the pedestrian's trajectory is predicted by a cooperation-based trajectory planning model. Both parts of the model are tested and validated using real-life recorded scenarios of pedestrian-vehicle interaction. The model is capable of describing and predicting agents' behaviors when interacting with a vehicle in both lateral and frontal crossing scenarios.

Risk mitigation is an important element to consider in risk evaluation. Safety features have helped to decrease the death ratio over the years. However, to date, each driver assistance system works on a single domain of operation. The problem remains in how to use perception to contextualize the scene to fully minimize the collision severity in a complex emergency scenario. Up to now, works on cost maps have consider simple contextualized object in mitigation scenarios. For instance, the use of binary allowed/forbidden zones or, a fixed weight to each type of object in the scene. Our work employs the risk of injury issued by accidentology to each class of object present in the scene. Each class of object presents an injury probability with respect to the impact speed and ethical/economical/political factors. The method generates a cost map containing a collision probability along with to the risk of injury. It dynamically contextualizes the objects, since the risk of injury depends on the characteristics of the scene. Simulation and dataset results validate that changing the referred parameters alters the context and evaluation of the scene. Then, the proposed method allows a better assessment of the surroundings by creating a dynamic navigation cost map for complex scenarios.

Visual Servoing has been widely investigated in the last decades as it provides a powerful strategy for robot control. Thanks to the direct feed-back from a set of sensors, it allows to reduce the impact of some modeling errors and to perform tasks even in uncertain environments. The commonly exploited approach in this field is to use a model that expresses the rate of change of a set of features as a function of sensor twist. These schemes are commonly used to obtain a velocity command, which needs to be tracked by a low-level controller. Another approach that can be exploited consists in going one step further and to consider an acceleration model for the features. This strategy allows also to obtain a natural and direct link with the dynamic model of the controlled system. This study aims at comparing the use of velocity and acceleration-based models in feed-back linearization for Visual Servoing. We consider the case of a redundant manipulator and discuss what this implies for both control techniques. By means of simulations, we show that controllers based on features acceleration give better results than those based on velocity in presence of noisy feedback signals.

We present an experimental study for the generation of large 3D maps using our CoMapping framework. This framework considers a collaborative approach to efficiently manage, share, and merge maps between vehicles. The main objective of this work is to perform a cooperative mapping for urban and rural environments denied of continuous-GPS service. The study is split in to 2 stages: Pre-Local and Local. In the first stage, each vehicle builds a Pre-Local map of its surroundings in real-time using laser-based measurements, then relocates the map in a global coordinate system using just the low cost GPS data from the first instant of the map construction. In the second stage, vehicles share their pre-local maps, align and merge them in a decentralized way in order to generate more consistent and larger maps, named Local maps. To evaluate performance of all the cooperative system in terms of map alignments, tests are conducted using 3 cars equipped with LiDARs and GPS receiver devices in urban outdoor scenarios of the Ecole Centrale Nantes campus and rural environments.

In this research, a general control framework for platooning in urban environment is proposed. A consensus-based control law is described taking into account the nature of traveling in urban environment, that is the human driven leader travels with variable velocity. In addition, the proposed control law does not depend on the predecessor velocity, which in turn allows us to utilize a low cost limited bandwidth communication module by using a sensor-based link for predecessor distance and a communication-based link for leader's information. A constant-spacing policy is used to get a high capacity flow of vehicles. The control system is analyzed and conditions for both internal and string stability are set. The efficiency of the proposed framework and control law is verified via numerical analysis.

This paper explores the feasibility of a Multi-Sensor-Based Control (MSBC) approach for addressing forward nonparallel (perpendicular and diagonal) unparking problems of car-like vehicles as an alternative to classical approaches (e.g. path planning based, etc.). The results of individual cases are presented to illustrate the behavior and performance of the proposed approach as well as results from exhaustive simulations to evaluate the convergence and stability. The results presented in this work increase the versatility and validity of our MSBC approach towards a fully autonomous parking system.

Sampling-based planning algorithms have been extensively exploited to solve a wide variety of problems. In recent years, many efforts have been dedicated to extend these tools to solve problems involving constraints, such as geometric loop-closure, which lead the valid Configuration Space (CS) to collapse to a lower-dimensional manifold. One proposed solution considers an approximation of the constrained Configuration Space that is obtained by relaxing constraints up to a desired tolerance. The resulting set has then non-zero measure, allowing to exploit classical planning algorithms to search for a path connecting two given states. When constraints involve kinematic loops in the system, relaxation generally bears to undesired contact forces, which need to be compensated during execution by a proper control action. We propose a new tool that exploits relaxation to plan in presence of constraints. Local motions inside the approximated manifold are found as the result of an iterative scheme that uses Quadratic Optimization to proceed towards a new sample without falling outside the relaxed region. By properly guiding the exploration, paths are found with smaller relaxation factors and the need of a dedicated controller to compensate errors is reduced. We complete the analysis by showing the feasibility of the approach with experiments on a real platform.

The presence of Type 2 singularities in parallel robots severely affects their performances, mainly because the platform motion control is partially lost. It also leads to a size reduction of the operational workspace. Moreover, the dynamic model of the parallel mechanism degenerates and locally, the robot becomes underactuated in the singularity. It has been proven that it is possible to cross Type 2 singularities by respecting a dynamic criterion. Nevertheless, the controllers designed up to now require a pre-planned optimized trajectory including this criterion, and as a result, this strategy can only be used by qualified users. In order to avoid this drawback and to cross these types of singularities even if the trajectory is not pre-planned, this paper proposes a controller based on virtual constraints. Furthermore, the controller is integrated in a multi-control architecture in order to switch between a classical computed torque control far from the singularity and the virtual-constraint-based control law near to the singularity locus. Experimental results on a five-bar mechanism validated the automatic Type 2 singularity crossing.

An Automated Driving System (ADS) is subject to hazardous weather conditions and to failures, both of which can result in a partial or total loss of its ability to monitor the driving environment. Yet until high driving automation and full driving automation is achieved, a human driver is expected to respond appropriately to any malfunction or adverse on-road conditions preventing the ADS from reliably sustaining the dynamic driving task performance. However, automation causes drowsiness and hypo-vigilance, which can compromise a human driver's ability to respond to ADS-issued requests. Hence the necessity of defining dynamic driving task fallback strategies that can be performed by the ADS, if and when necessary. The proposed fallback strategy is aimed at level 4 ADS features designed to operate a vehicle on a road whose characteristics make any attempt at stopping hazardous. It naturally applies to level 5 ADS-operated vehicles and to ADS-dedicated vehicles as well. The transition stage, during which the strategy is triggered, consists in the replacement of missing vehicles and obstacles in the world model with ghost objects. An embedded visibility map is then used to retrieve the maximum distance at which the ADS-operated vehicle can be seen, when driving behind it. The speed profile underlying the fallback strategy meets a time to collision criterion of 4 s, which enables the avoidance and the mitigation of rear-end collisions. The behaviour of drivers in collision imminent situations cannot be observed in test track studies due to safety concerns. As a result, experiments were conducted in the driving simulation software SCANeR studio.

The determination of the singularity cases in visual servoing is a tricky problem, which is unsolved for most of the image-based approaches. In order to avoid singularities, redundant measurements may be used. However, they lead to the presence of local minima. Moreover, they do not always ensure that singularities can be avoided. Here, we show that a concept named the “hidden robot,” which was formerly used for understanding the singularities of a vision-based controller dedicated to parallel robots, can be used for interpreting the singularities in the visual servoing of image points. These singularity cases were already found in the case in which three points are observed, but we show that the hidden robot concept considerably simplifies the analysis by using geometric interpretations of the mapping degeneracy and tools provided by the mechanical engineering community. Moreover, to the best of our knowledge, for the first time, we provide the singularity conditions when more than three points are observed. We also discuss how these tools could be extended in order to find the singularity cases of other visual servoing techniques (e.g., when lines are observed).

In this paper, it is shown that the Enhanced Unified Camera Model, which is used to model fisheye cameras, projects straight lines into conic sections. Then a way to find equations of straight line projections is proposed. The method is applied to epipolar curves to adapt the Semi-Global Matching algorithm to fisheye stereo systems to compute a dense direct stereo correspondence without undistortion and rectification of fisheye images. It is done by efficient image sampling along an epipolar curve, using an algorithm for rasterization of implicit curves. A C++ implementation is available online.

Robots in industries are often used for repetitive tasks. Their motions rely on precise virtual model of their environment and they are not able to handle changes or unexpected events. This disqualifies them to perform tasks that would require precision in a non controlled environment such as an assembly task in real world. Visual servoing is a well known tool to control the robot using spatial sensors. It includes real world references at control law level. But, visual servoing and more generally sensor-based control schemes provide kinematic control law and do not consider robot dynamics. As consequences, tracking performances are poor and convergence behavior is hardly predictable. In this paper, we proposed a new control scheme considering second order sensor-based control law and robot dynamics. Our main goal is to enable full trajectory tracking in sensor-space. Additionally, the scheme is compatible with priority-ordered task sequencing and it can be also used within a hybrid control scheme where force control is considered. This new control scheme brings the possibility to make easier the robot task definition, dividing a complex positioning task into small easy-manageable ones. Multi-tasks operation has been validated in simulation by using MSC Adams software [1] and where the robot has to perform an engraving task on a surface.

Many ideas have been proposed to reduce traffic congestion. Driving a platoon of vehicles with constant spacing seems to be a promising idea as it increases traffic density. But keeping constant inter-vehicle spacing requires very reliable communication. Another control policy is to drive the platoon with a time headway between vehicles. It is a robust and well known policy but large inter-vehicle distances in addition to weak stability (unity error gain) near low frequencies make it less practical. We have proposed in [1], [2] a modification of the Constant Time Headway policy (CTH). This modification largely reduces the inter-vehicle distances using only one information shared between all vehicles. In this work, we propose an additional modification of our control law. This modification makes our control law similar, in form, to the classical constant spacing policy, but it requires to share only one information between the vehicles. This modification improves the stability of the platoon and removes the weak stability of the CTH near low frequencies. We prove the robustness of the control law in the presence of actuating lags, sensing and communication delays. This proof can also be used to prove the stability of the classical constant spacing policy in the presence of all previous delays, which makes our result more general than those established in the literature. Safety is also discussed and the maximum acceptable communication delay without losing safety is determined. Simulations have been done in many critical scenarios.

RPN (Robotic Programming Network) is an initiative to bring existing remote robot laboratories to a new dimension, by adding the flexibility and power of writing ROS code in an Internet browser and running it in the remote robot with a single click. The code is executed in the robot server at full speed, i.e. without any communication delay, and the output of the process is returned back. Built upon Robot Web Tools, RPN works out-of-the-box in any ROS-based robot or simulator. This paper presents the core functionality of RPN in the context of a web-enabled ROS system, its possibilities for remote education and training, and some experimentation with simulators and real robots in which we have integrated the tool in a Moodle environment, creating some programming courses and make it open to researchers and students (http://robotprogramming.uji.es).

Finding solutions to traffic congestion is an active area of research. Many ideas have been proposed to reduce this problem, among of this ideas is moving in platoon. The constant time headway policy (CTH) is a very important platoon control policy, but it is too conservative and induces large inter-vehicle distances. Recently, we have proposed a modification of CTH [1], [2]. This modification reduces inter-vehicle distances and makes CTH very practical. This paper focuses on the control of platoons in urban areas. To control the vehicles, we assume that the longitudinal and the lateral dynamics are decoupled. We take into account a simplified engine model. We linearize the two dynamics using exact linearisation technique. Then, we use the modified CTH control law, adapted to urban platoons, for the longitudinal control and the robust sliding mode control for lateral control. The stability and the safety of the platoon are also studied. The conditions of stability of homogeneous and nonhomogeneous platoons are established. The conditions to verify the safety of the platoon for the longitudinal control (assuming stable and accurate lateral control) are exhibited. The weaknesses (large inter-vehicle distance, weak stability near low frequencies) of the CTH are solved. The improved performance and the safety of the platoon are verified by simulation using TORCS (The Open Racing Car Simulator). A platoon consisting of ten vehicles is created and tested on a curved track, keeping a small desired intervehicle distance. The stability and safety of the longitudinal and lateral controls are tested in many scenarios. These scenarios include platoon creation, changing the speed and emergency stop on straight and curved tracks. The results demonstrate the effectiveness of the proposed approach.

Previous works on parallel robots have shown that their visual servoing using the observation of their leg directions was possible. There were, however, found two main results for which no answer was given. These results were: 1) the observed robot that is composed of n legs could be controlled in most cases using the observation of only m leg directions (m <; n), and 2) in some cases, the robot did not converge to the desired end-effector pose, even if the observed leg directions did (i.e., there was not a global diffeomorphism between the observation space and the robot space). Recently, it was shown that the visual servoing of the leg directions of the Gough-Stewart platform and the Adept Quattro was equivalent to controlling other virtual robots that have assembly modes and singular configurations different from those of the real ones. These hidden robot models are tangible visualizations of the mapping between the observation space and the real robots Cartesian space. Thanks to this concept, all the aforementioned points pertaining to the studied robots were answered. In this paper, the concept of the hidden robot model is generalized for any type of parallel robots controlled using visual servos based on the observation of elements other than the end-effector, such as the robot legs into motion. It is shown that the concept of the hidden robot model is a powerful tool that gives useful insights about the visual servoing of robots and that it helps define the necessary features to observe in order to ensure the controllability of the robot in its whole workspace. All theoretical concepts are validated through simulations with an Adams mockup linked to Simulink.

Optimizing inter-vehicle distances is very important to reduce traffic congestion on highways. A modification of the constant time headway policy (CTH) has been proposed in [1]. This modification significantly reduces inter-vehicle distances, but this reduction in the inter-distance may increase the risks of collisions. In this paper, the safety of the modified CTH applied to a homogeneous platoon is addressed. Many critical scenarios are discussed, including hard braking of the leader and followers in the event of communication loss. Safety conditions are presented. In addition, a method to find the maximum allowed delay to inform all the vehicles about communication loss, is also presented. Simulations have been run with 10 vehicles to check safety in the proposed scenarios.

In this paper, a force/vision control strategy is proposed for the robotic cutting of soft materials. The separation is performed by repeating a series of cuts, called passages, along a deformable three dimensional curve. An image based visual servoing system is proposed to control all six degrees of freedom of the cutting tool. This allows the system to change the cutting depth and angle in response to changes in the profile of the surface. The force is used to ensure the cut is performed cleanly without globally deforming the soft material. The proposed controller is validated experimentally by cutting soft foam material.

In order to increase the workspace size of parallel robots (largely reduced by the presence of singularities) several solutions have been proposed. One promising solution consists in the definition of optimal trajectories that ensure the non degeneracy of the dynamic model in the singularity and therefore are able to cross the Type 2 singularities. Those works are based on the computation of the optimal trajectories and assume that the robot can perfectly track the desired trajectory. Nevertheless, this assumption cannot be verified in reality due to modelling errors which largely impact the control law used to follow the desired trajectory. Therefore, if the optimal trajectory is not perfectly tracked, the dynamic model can degenerate near the Type 2 singularities and the robot might stay blocked. In order to solve that problem, this paper proposes a multimodel approach that allows parallel robots to cross the Type 2 singularities without any torque discontinuity. The main idea is to shift near singularities from the full robot dynamic model to another simplified one that can never degenerate. The proposed control law is then coupled with an optimal trajectory planning methodology that makes the singularity crossing more robust to modelling errors. The proposed approach is validated experimentally on a prototype of Five-bar planar parallel mechanism.

As the end-effector pose is an external property of a parallel robot, it is natural to use exteroceptive sensors to measure it in order to suppress inaccuracies coming from modelling errors. Cameras offer this possibility. So, it is possible to obtain higher accuracy than in the case of classic control schemes (based on geometrical model). In some cases, it is impossible to directly observe the end-effector, but the leg directions can instead be used. In this case, however, unusual results were recorded, namely: (i) the possibility of controlling the robot by observing a number of legs less than the total number of legs, and that (ii) in some cases, the robot does not converge to the desired end-effector pose, even if the observed leg directions did. These results can be explained through the use of the hidden robot concept, which is a tangible visualisation of the mapping between the observed leg direction space (internal property) and Cartesian space (external property). This hidden robot has different assembly modes and singular configurations from the real robot, and it is a powerful tool to simplify the analysis of the aforementioned mapping. In this paper, the concept of hidden robot model is generalised for any type of parallel robot controlled through visual servoing based on observation of the leg directions. Validation has been accomplished through experiments on a Quattro robot with 4 dof.

In this paper, a force/vision control strategy is proposed in order to separate soft deformable materials using cooperative robots. The separation is performed by repeating a series of cuts, called passages, along a curved trajectory. The vision control is used to locally update the robot trajectory in response to both on-line deformations and off-line modeling errors. The force controller is used to ensure that the cut is performed without global deformation or damage to the surrounding area. The second robot is used to facilitate the cutting by applying external forces to the object. The control scheme is validated experimentally by cutting soft foam material.

Heavy traffic on highways requires the optimization of inter-distances between vehicles in order to reach time performance and to provide safety solution in transport. Variable spacing and constant spacing are the two policies for the longitudinal control of platoon. Variable spacing doesn't require a lot of data (position, speed...) from other vehicles, and string stability using only on-board information is obtained. However, inter-vehicle distances are very large, and hence traffic density is low. Constant spacing can offer string stability with high traffic density, but it requires at least data from the leader. In this paper, a novel expression of the variable spacing policy has been proposed. It is effective to decrease the distance between the cars, to become nearly equal to the constant spacing policy. It also enables increasing the string stability and the robustness of the control regarding to unmodeled lags, and it can avoid control torque saturation. This novel approach doesn't require heavy communication between the cars. The new control law has been evaluated by simulation with perfect system using Matlab, and with imperfect system using TORCS. The good results have demonstrated the effectiveness of the novel approach.

International audience

This paper presents new insights about the sensor-based control of Gough-Stewart (GS) platforms. Previous works have shown that it was possible to control the GS platform by observing its legs directions instead of using the encoders values or the measurement of the platform pose. It was demonstrated that observing only three legs directions was enough for the control but no physical explanations were given. Moreover, sometimes, the GS platform was not converging to the desired pose and the reasons of these divergences were not disclosed. This paper aims at answering to this two opened problems. It is shown that observing three leg directions involves controlling the displacement of a hidden robot whose models differs from those of the usual GS platform. This robot has assembly modes and singular configurations different from those of the GS platform. This involves that the legs to observe should be chosen carefully in order to avoid inaccuracy problems. In this sense, the accuracy analysis of the new robot is performed to show the importance of the leg selection. All these results are validated on a GS platform simulator created using ADAMS/Controls and interfaced with Matlab/Simulink.

To control dynamics of a parallel robot, we should measure the state feedback accurately and fast. In this paper, we show how to estimate positions and velocities simultaneously (i.e., the state feedback) at a reasonable accuracy and speed. We did this using only the sequential visual contours of the legs. A single-iteration virtual visual servoing scheme regulates rapidly an error of these contours. We validated this theory, a step to control parallel robots at high speed by their leg kinematics, with simulations and experiments.

Previous works on the Gough-Stewart (GS) platform have shown that its visual servoing using the observation of its leg directions was possible by observing only three of its six legs but that the convergence to the desired pose was not guarantied. This can be explained by considering that the visual servoing of the leg direction of the GS platform was equivalent to controlling another robot, the 3-UPS that has assembly modes and singular configurations different from those of the GS platform. Considering this hidden robot model allowed the simplification of the singularity analysis of the mapping between the leg direction space and the Cartesian space. In this paper, the work on the definition of the hidden robot models involved in the visual servoing using the observation of the robot leg directions is extended to another robot, the Adept Quattro. It will be shown that the hidden robot model is completely different from the model involved in the control of the GS platform. Therefore, the results obtained for the GS platform are not valuable for this robot. The hidden robot has assembly modes and singular configurations different from those of the Quattro. An accuracy analysis is performed to show the importance of the leg selection. All these results are validated on a Quattro simulator created using ADAMS/Controls and interfaced with Matlab/Simulink.

This paper considers the navigation in formation of a mobile Multi-Robot System (MRS) in presence of obstacles. In such areas, the collision avoidance between the robots themselves and with other obstacles (static and dynamic) is a challenging issue. To deal with it, a reactive and a distributed control architecture is built. The navigation in formation of the MRS is ensured while tracking a global virtual structure (first controller). Limit-cycle principle is used to compute the setpoint of the obstacle avoidance task (second controller). In this paper, kinematic constraints of the robot are taken into account in order to generate an attainable set-point. The objective is to guarantee safety of the mobile robots with respect to their maximum velocities. Simulation and experimental results validate the proposed contributions.

When applying service robotic tasks using sensor based control, a classical exponential decrease of the error is usually used in the control laws which can reduces the performance of the executed task. In fact, due to this choice, the convergence time greatly increases especially at the end of the process. To ameliorate the performance of such tasks, we present in this paper two new error regulation strategies to accelerate the service tasks execution. These propositions are compared with the classical one in the case of performing autonomous object's manipulation tasks using real-time visual servoing. The Model Based Tracking method is used to apply head servoing and grasping of different objects using Nao humanoid robot.

Most of the existing appearance based topological mapping algorithms produce dense topological maps in which each image stands as a node in the topological graph. Sparser maps can be built by representing groups of visually similar images as nodes of a topological graph. In this paper, we present a sparse topological mapping framework which uses Image Sequence Partitioning (ISP) techniques to group visually similar images as topological graph nodes. We present four different ISP techniques and evaluate their performance. In order to take advantage of the afore mentioned maps, we make use of Hierarchical Inverted Files (HIF) which enable efficient hierarchical loop closure. Outdoor experimental results demonstrating the sparsity, efficiency and accuracy achieved by the combination of ISP and HIF in performing loop closure are presented.

This paper is dedicated to vision-based modeling and control of large-dimension parallel robots driven by inextensible cables of non-negligible mass. An instantaneous inverse kinematic model devoted to vision is introduced. This model relies on the specificities of a parabolic profile hefty cable modeling and on the resulting simplified static analysis. By means of a kinematic visual servoing method, computer vision is used in the feedback loop for easier control. According to the modeling derived in this paper, measurements that allow the implementation of this visual servoing method consist of the mobile platform pose, the directions of the tangents to the cable curves at their drawing points and the cable tensions. The proposed visual servoing scheme will be applied to the control of a large parallel robot driven by eight cables. To this end, in order to obtain the aforementioned desired measurements, we plan to use a multi-camera setup together with force sensors.

This paper deals with platooning navigation in the context of innovative solutions for urban transportation systems. More precisely, the case of a manually driven vehicle leading a convoy of automated ones is considered. Vehicle localization relies solely on monocular vision: a 3D map of the environment is built beforehand from reference video sequences, and then used to derive vehicle absolute location from the current camera image. The 3D vision map presents however distortions w.r.t. a metric world, but these latter can be shown to be locally homogeneous. They can then be accurately corrected via a 1-dim. function evaluated with a nonlinear observer relying on odometric data. Next, the platoon reference trajectory is built as a B-Spline curve extended on-line via local optimization from the successive locations of the lead vehicle, and a global decentralized control strategy, supported by intervehicle communication, is designed to achieve accurate platooning with no oscillation within the convoy. Experimental results, carried out with two urban vehicles, demonstrate the capabilities of the proposed approach.

This paper is focused on the design of a control strategy for the path tracking of off-road mobile robots acting at high speed. In order to achieve high accuracy in such a context, uncertain and fast dynamics have to be explicitly taken into account. Since these phenomena (grip conditions, delays due to inertial and low-level control properties) are hardly measurable directly, the proposed approach relies on predictive and observer-based adaptive control techniques. In particular, the adaptive part is based on an observer loop, taking advantage of both kinematic and dynamic vehicle models. This multi-model based adaptive approach permits to adapt on-line the grip conditions (represented by cornering stiffnesses), enabling highly reactive sideslip angles observation and then accurate path tracking. The relevance of this approach is investigated through full scale experiments.

This paper discusses variable selection for the efficient dynamic control of the Quattro parallel robot through an inverse dynamic model expressed by means of leg orientations. A selection is made within a group of variables where each can imply the state of the robot. Besides, in this work, steering a parallel robot dynamically using its self-projection onto the image plane (where the edges of the lower-legs are exploited in control) is proposed and validated for the first time. In the light of the realistic control simulation, the formative points of better control of the Quattro robot are figured out.

This paper deals with the vision-based control of cable-driven parallel robots. First, a 3D pose visual servoing is proposed, where the end-effector pose is indirectly measured and used for regulation. This method is illustrated and validated on a cable-driven parallel robot prototype. Second, to take into account the dynamics of the platform and using a Cartesian pose and velocity estimator, a vision-based computed torque control is developed and validated in simulation.

In mobile robot path tracking applications, an autonomous vehicle is steered to stay as close as possible to a desired path. If lateral wheel slip is an important variable, as it is the case at high speed and due to low tire-ground friction in off-road applications, limits of the steering actuators, the major input constraints of the system, have a major influence on the tracking control performance. This paper presents an algorithm to control the longitudinal velocity, a secondary control variable, of a mobile robot in order to respect the boundedness of the steering angle, and thus to improve the vehicle safety. The applicability of the algorithm has been verified through experiments with an off-road mobile robot.

Environment, sustainable development as well as new transportation service emergence in urban areas are major concerns. Consequently, studies are currently intended to automate electric vehicles designed for applications in free access. An additional functionality that appears very attractive is vehicle platooning. In order to avoid oscillations within the fleet when completing this task, a global control strategy, supported by inter-vehicle communications, is investigated. Vehicle absolute localization is then needed and is here derived from monocular vision. These data are however expressed in a virtual vision world, slightly distorted with respect to the actual metric one. It has previously been shown that such a distortion can accurately be corrected on-line in different ways, considering telemetric or odometric data. These strategies have here been refined in order to provide optimal corrections. A comparative study, supported by simulations and full-scale experiments, is reported to exhibit benefits and performances of proposed approaches.

Automation in outdoor applications (farming, surveillance, etc.) requires highly accurate control of mobile robots, at high speed, accounting for natural ground specificities (mainly sliding effects). In previous work, predictive control algorithms dedicated to All-Terrain Vehicle lateral stability was investigated. Satisfactory advanced simulation results have been reported but no experimental ones were presented. In this paper, the prevention of a real off-road mobile robot rollover is addressed. First, both rollover dynamic modeling and previous work on a Mixed observer designed to estimate on-line sliding phenomena for path tracking control are recalled. Then, this observer is here used to compute a rollover indicator accounting for sliding phenomena, from a low-cost perception system. Next, the maximum vehicle velocity, compatible with a safe motion over some horizon of prediction, is computed via Predictive Functional Control (PFC), and can then be applied, if needed, to the vehicle actuator to prevent from rollover. The capabilities of the proposed device are demonstrated and discussed thanks to real experimentation.

This paper addresses the problem of path generation and motion control for the autonomous maneuver of a farm vehicle with a trailed implement in headland. A reverse turn planner is firstly investigated, based on primitives connected together to easily generate the reference motion. Then, both steering and speed control algorithms are presented to accurately guide the vehicle-trailer system. They are based on a kinematic model extended with additional sliding parameters and on model predictive control approaches. Real world experiments have been carried out on a low friction terrain with an experimental mobile robot pulling a trailer. At the end of each row, the reverse turn is automatically generated to connect the next reference track, and the maneuver is autonomously performed by the vehicle-trailer system. Reported experiments demonstrate the capabilities of the proposed algorithms.

Vision has often been considered as not suitable for dynamic control of robots. The experimental results presented in this paper show that it is possible to perform better with a vision based dynamic control than with a model-based control. These results were obtained using a Cartesian computed torque control fed back, without any joint sensing, by a novel Cartesian pose and velocity estimator. The latter is designed as a virtual visual servoing scheme based on sequential acquisition of sub-images and a constant acceleration motion assumption.

The necessity of decreasing the environmental impact of agricultural activities, while preserving in the same time the level of production to satisfy the growing population demand, requires to investigate new production tools. Mobile robotic can constitute a promising solution, since autonomous devices may permit to increase production level, while reducing pollution thanks to a high accuracy. In this paper, the use of several mobile robots for field treatment is investigated. It is here considered that they can exchange data through wireless communication, and a formation control law, accurate despite typical off-road conditions (low grip, terrain irregularities, etc), is designed relying on nonlinear observer-based adaptive control. The algorithm proposed in this paper is tested through advanced simulations in order to study separately its capabilities, as well as experimentally validated.

One of the key steps in high-speed control of a parallel robot is to define an efficient dynamic model. It is usually not easy to have such a model for parallel robots, since many of them have complex structures. Here, we propose a vector-based approach, which employs the robot leg orientations, to obtain a simplified inverse dynamic model. At the least, this vector-based methodology is pioneering, when combined with the observation of orientations by a calibrated camera, in the sense of solving the entire control-oriented (hard) modeling problem, both kinematics and dynamics, in an almost algebraic manner through the knowledge of only a nominal set of image features: the edges of the robot legs and their time derivatives. Proposed method is verified on a simulator of the Quattro robot with a computed torque control where the leg orientations are steered.

Automated electric vehicles for public use constitute a promising very efficient and environment-friendly “urban transportation system”. An additional functionality that could enhance this transportation service is vehicle platooning. In order to avoid inter-distance oscillations within the platoon, a global control strategy, supported by inter-vehicle communications, is investigated. Vehicle localization in an absolute frame is needed and is derived here from monocular vision. The vision data is however expressed in a virtual world, slightly distorted with respect to the actual metric one. It is shown that such a distortion can accurately be corrected by designing a nonlinear observer that relies on odometric data. A global decentralized control strategy, relying on nonlinear control techniques, can then be designed to achieve accurate vehicle platooning. Simulations and full-scale experiments demonstrate the performance of the proposed approach.

In this paper, the control problem for a group of mobile robots keeping a geometric formation is considered. The proposed architecture of control allows to each robot to avoid obstacles and to rejoin the desired formation. To not complicate the control of such a system, it is proposed to divide the overall complex task into two basic tasks: attraction to a dynamical target, and obstacle avoidance. Thus, a desired geometric shape is defined and each robot has to track one node of this mobile shape. Each robot has to be autonomously able to avoid disturbing obstacles and to rejoin the formation in a reactive manner. Moreover, it chooses the optimal avoidance side thanks to limit-cycle method in order to reach as rapidly as possible its virtual target. The proposed control architecture is implemented in a distributed manner. In addition, this architecture uses the same control law (Lyapunov stable) for the two elementary tasks, and the switching from one task to another occurs only by changing the set-points. Experimental results validate the proposed control architecture.

This paper addresses the problem of sliding parameter estimation and lateral control of an off-road vehicle-trailer system. The aim is to accurately guide the position of the trailer with respect to a planned trajectory, whatever ground conditions and trajectory shape. Relevant sliding parameter estimation is first proposed, based on the kinematic model of the system extended with side slip angles. Then, a vehicle steering control algorithm is presented to move away the vehicle from the reference trajectory in order for the trailer to achieve accurate path tracking. Reported experiments demonstrate the capabilities of the proposed algorithms.

The active devices dedicated to on-road vehicle stability cannot be applied satisfactorily in an off-road context, since the variability and the non-linear features of grip conditions can no longer be neglected. Specific solutions have then to be investigated. In this paper, the prevention of light all-terrain vehicle (ATV) rollover is addressed. First, a backstepping observer is designed in order to estimate online a rollover indicator accounting for sliding phenomena, from a low-cost perception system. Next, the maximum vehicle velocity, compatible with a safe motion over some horizon of prediction, is computed via predictive functional control (PFC), and can then be applied, if needed, to the vehicle actuator to prevent from rollover. The capabilities of the proposed device are demonstrated and discussed thanks to an advanced simulation testbed that has proved to supply results very close to experimental ones.

This paper extends the recent work proposed in [21]. In this work, it has been noted that three visual features (to control three degrees of freedom) obtained from the spherical projection of 3D spheres allows nice decoupling properties and global stability. However, even if such an approach is theoretically attractive, it is limited by a major practical issue since spherical objects have to be observed while only three degrees of freedom can be controlled. In this paper, we show that similar properties can be obtained by observing a set of points. The basic idea is to build a virtual 3D sphere from two 3D points and to analyse its related spherical projection. Furthermore, to control the six degrees of freedom a 2D 1/2 control scheme is proposed which allows us to fully decouple rotational motions from translational motions.

Various "Urban Transportation Systems" are currently in developing, in order to put forward solutions to congestion and pollution in dense areas. Autonomous electric vehicles in free-access can be seen as an attractive approach, in view of the large flexibility that can be expected. One instrumental functionality linked to this solution is platoon motion: several autonomous vehicles accurately follow the trajectory of a manually driven first vehicle, with pre-specified inter-distances. A global decentralized platoon control strategy, supported by inter-vehicle communications and relying on nonlinear control techniques is here proposed. Each vehicle is controlled with respect to the same smooth reference trajectory, inferred on-line from the motion of the first vehicle via B-spline optimization. Experimental results, carried out with four urban vehicles, demonstrate the capabilities of the proposed approach.

This paper addresses the problem of path generation and motion control for the autonomous maneuvers of a farm vehicle in headland. A reverse turn planner is firstly investigated, based on primitives connected together to easily generate the reference motion. Then, both steering and speed control algorithms are presented to accurately guide the vehicle. They are based on a kinematic model extended with additional sliding parameters and on model predictive control approaches. Real world experiments have been carried out on a low adherent terrain with an experimental mobile robot. At the end of each row, the reverse turn is automatically generated to connect the next reference track, and the maneuver is autonomously performed by the vehicle. Reported experiments demonstrate the capabilities of the proposed algorithms.

This paper presents a vision-based navigation strategy for a vertical take-off and landing (VTOL) unmanned aerial vehicle (UAV) using a single embedded camera observing natural landmarks. In the proposed approach, images of the environment are first sampled and stored as a set of ordered key images (visual path) and organized providing a visual memory of the environment. The robot navigation task is then defined as a concatenation of visual path subsets (called visual route) linking the current observed image and a target image belonging to the visual memory. The UAV is controlled to reach each image of the visual route using a vision-based control law adapted to its dynamic model and without explicitly planning any trajectory. This framework is largely substantiated by experiments with a X4-flyer equipped with a fisheye camera.

This paper presents a high speed visual tracking method based on non simultaneous subimages acquisition. This method is formulated as a virtual visual servoing scheme. The sequential acquisition of regions of interest has a double benefit on visual servoing. The first one is that this acquisition method allows to increase the visual control sampling frequency by reducing the data amount to acquire and to transmit by the camera. The second one is that the associated image projection model depends on the observed object pose and velocity. Thanks to this property, a new vision-based control law can be defined. The particularity of this control law is that the control output consists of the kinematic and the dynamic twists. This allows to enhance the control performance in trajectory tracking applications. The experimental results in high speed visual tracking application show the effectiveness of this approach.

Accurate control of high-speed mobile robots moving off-road constitutes a challenging robotic issue: numerous time-varying dynamic phenomena (and first of all, sliding effects) are no longer negligible and must explicitly be taken into account in control design, in order to ensure high accuracy path tracking. Since these phenomena are hardly measurable at a reasonable cost, they have to be estimated on-line. A multi-model based observer is here proposed, in order to supply on-line tire cornering stiffnesses (i.e. grip conditions) as well as mobile robot sideslip angles. It takes part of the complementarity between kinematic and dynamic mobile robot models, in order to significantly decrease the number of required robot inertial parameters (since their values are sometimes difficult to obtain). Full scale experiments demonstrate that the proposed observer can supply reactive and reliable sideslip angle estimates, so that high accuracy path tracking can still be achieved, whatever grip conditions and vehicle velocity.

In this paper, we present a complete framework for autonomous vehicle navigation using a single camera and natural landmarks. When navigating in an unknown environment for the first time, usual behavior consists of memorizing some key views along the performed path to use these references as checkpoints for future navigation missions. The navigation framework for the wheeled vehicles presented in this paper is based on this assumption. During a human-guided learning step, the vehicle performs paths that are sampled and stored as a set of ordered key images, as acquired by an embedded camera. The visual paths are topologically organized, providing a visual memory of the environment. Given an image of the visual memory as a target, the vehicle navigation mission is defined as a concatenation of visual path subsets called visual routes. When autonomously running, the control guides the vehicle along the reference visual route without explicitly planning any trajectory. The control consists of a vision-based control law that is adapted to the nonholonomic constraint. Our navigation framework has been designed for a generic class of cameras (including conventional, catadioptric, and fisheye cameras). Experiments with an urban electric vehicle navigating in an outdoor environment have been carried out with a fisheye camera along a 750-m-long trajectory. Results validate our approach.

This paper deals with the visual control of the Gough-Stewart platform using a central catadioptric camera observing the platform's legs. This allows a large field of view to be obtained and avoids the occlusion problems observed when a classical perspective camera is used. An automatic and simple method to detect the projections of the leg in the image is also proposed. The control scheme presented here is shown to encompass the classical perspective camera case, as well as catadioptric ones. Finally, experimental results comparing two kinds of visual features (leg directions and leg edges) are described.

To address traffic saturation in cities, new ldquourban transportation systemsrdquo, based on electric vehicles in free-access, are in developing. One necessary functionality of such systems is their ability to move in a platoon fashion. A global decentralized platoon control strategy, supported by inter-vehicle communications, is addressed in this paper, relying on nonlinear control techniques. The main interest in a global approach is that servoing error accumulation can be avoided, whatever the platoon length. However, absolute vehicle localization is then required. In urban applications, cameras are realistic sensors, but localization is supplied in a 3D visual virtual world, slightly distorted w.r.t. the actual metric one. To enable accurate guidance, local corrections to the visual world are here computed from the data supplied by a laser rangefinder mounted on the second vehicle, and then shared with the whole platoon. Full-scale experiments demonstrate the performance of the proposed approach.

In this paper, we present a complete framework for autonomous indoor robot navigation. We show that autonomous navigation is possible in indoor situation using a single camera and natural landmarks. When navigating in an unknown environment for the first time, a natural behavior consists on memorizing some key views along the performed path, in order to use these references as checkpoints for a future navigation mission. The navigation framework for wheeled robots presented in this paper is based on this assumption. During a human-guided learning step, the robot performs paths which are sampled and stored as a set of ordered key images, acquired by an embedded camera. The set of these obtained visual paths is topologically organized and provides a visual memory of the environment. Given an image of one of the visual paths as a target, the robot navigation mission is defined as a concatenation of visual path subsets, called visual route. When running autonomously, the control guides the robot along the reference visual route without explicitly planning any trajectory. The control consists on a vision-based control law adapted to the nonholonomic constraint. The proposed framework has been designed for a generic class of cameras (including conventional, catadioptric and fish-eye cameras). Experiments with a AT3 Pioneer robot navigating in an indoor environment have been carried on with a fisheye camera. Results validate our approach.

An efficient method for global robot localization in a memory of omnidirectional images is presented. This method is valid for indoor and outdoor environments and not restricted to mobile robots. The proposed strategy is purely vision-based and uses as reference a set of prerecorded images (visual memory). The localization consists on finding in the visual memory the image which best fits the current image. We propose a hierarchical process combining global descriptors computed onto cubic interpolation of triangular mesh and patches correlation around Harris corners. To evaluate this method, three large images data sets have been used. Results of the proposed method are compared with those obtained from state-of-the-art techniques by means of 1) accuracy, 2) amount of memorized data required per image and 3) computational cost. The proposed method shows the best compromise in term of those criteria.

In this paper, a novel approach for parallel kinematic machine control relying on a fast exteroceptive measure is implemented and validated on the Orthoglide robot. This approach begins with rewriting the robot models as a function of the only end-effector pose. It is shown that such an operation reduces the model complexity. Then, this approach uses a classical Cartesian space computed torque control with a fast exteroceptive measure, reducing the control schemes complexity. Simulation results are given to show the expected performance improvements and experiments prove the practical feasibility of the approach.

Lateral rollover is the leading cause of fatal accidents in light all-terrain vehicles (e.g. quad bikes), especially in the agricultural area. The estimation and prediction of hazardous situations are preliminary steps in the design of active security devices. If numerous metrics have already been defined for on-road vehicles, few approaches are suitable for fast motions in a natural environment (mainly due to tire/ground contact specificity and variability). This paper proposes an algorithm dedicated to the estimation and prediction of one metric, namely lateral load transfer (LLT), in order to anticipate rollover situations on an irregular and natural ground. It is based on a vehicle dynamic model, used jointly with a backstepping observer. It allows to take into account tire/ground contact nonlinearities and variability, which impact the rollover tendency. The efficiency of the metric is investigated through advanced simulations and full scale experiments on a Kymco quad bike.

In this paper, automatic path tracking of a four-wheel-steering vehicle in presence of sliding is addressed. The attractive feature of such a steering system is that, despite of sliding phenomena, both lateral and angular deviations can be explicitly controlled. Indeed, previous research has demonstrated that high-precision path tracking on a low grip terrain can be achieved with two-wheel-steering vehicles. However, in this case, only the lateral deviation is kept satisfactorily close to zero, the angular deviation is non null in order to compensate for sliding effects. In this paper, previous adaptive control laws are extended to the case of four-wheel-steering mobile robots with the aim to servo both lateral and angular deviations. Relying on an extended kinematic model, a backstepping control approach, which considers successively front and rear steering control, has been designed. Real world experiments have been carried out on a low adherent terrain with a four-wheel-steering vehicle equipped with a single RTK-GPS. This demonstrates the capabilities of the proposed control law and its robustness in real all-terrain conditions.

In this paper, we consider the problem of controlling a 6 DOF holonomic robot and a nonholonomic mobile robot from the projection of 3-D straight lines in the image plane of central catadioptric systems. A generic central catadioptric interaction matrix for the projection of 3-D straight lines is derived using an unifying imaging model valid for an entire class of cameras. This result is exploited to design an image-based control law that allows us to control the 6 DOF of a robotic arm. Then, the projected lines are exploited to control a nonholonomic robot. We show that as when considering a robotic arm, the control objectives are mainly based on catadioptric image feature and that local asymptotic convergence is guaranteed. Simulation results and real experiments with a 6 DOF eye-to-hand system and a mobile robot illustrate the control strategy.

This paper proposes a method to control an I4R parallel robot by the observation of its legs with a calibrated camera. We show that the control law depends only on the edges of its forearms extracted from the image and that no proprioceptive sensors are used. Indeed, the variables needed for control (namely, the directions of the arms and forearms as well as the positions of the wrists) can be reconstructed from the forearms edges. Experimental validation of the reconstruction is given and simulation of the control with realistic noises is performed showing the validity of the approach.

This paper presents a hybrid decoupled vision-based control scheme valid for the entire class of central catadioptric sensors (including conventional perspective cameras). First, we consider the structure from motion problem using imaged 3D points. Geometrical relationships are exploited to enable a partial Euclidean reconstruction by decoupling the interaction between translation and rotation components of a homography matrix. The information extracted from the homography are then used to design a control law which allow us to fully decouple rotational motions from translational motions. Real time experimental results using an eye-to-hand robotic system with a paracatadioptric camera are presented and confirm the validity of our approach.

Most of active devices focused on vehicle stability concerns on-road cars and cannot be applied satisfactorily in an off-road context, since the variability and the non-linearities of the tire/ground contact are often neglected. In previous work, a rollover indicator devoted to light ATVs, accounting for these phenomena has been proposed. It is based on the prediction of the lateral load transfer. Such an indicator requires the online knowledge of the tire cornering stiffness, initially selected from a ground classes network. In this paper, an adapted backstepping observer, making only use of yaw rate measurement, is designed to improve specifically tire cornering stiffness estimation. Capabilities of such an observer are demonstrated and discussed through both advanced simulations and actual experiments.

Omnidirectional cameras have a wide field of view and are thus used in many robotic vision tasks. An omnidirectional view may be acquired by a fisheye camera which provides a full image compared to catadioptric visual sensors and do not increase the size and the weakness of the imaging system with respect to perspective cameras. We prove that the unified model for catadioptric systems can model fisheye cameras with distortions directly included in its parameters. This unified projection model consists on a projection onto a virtual unitary sphere, followed by a perspective projection onto an image plane. The validity of this assumption is discussed and compared with other existing models. Calibration and partial Euclidean reconstruction results help to confirm the validity of our approach. Finally, an application to the visual servoing of a mobile robot is presented and experimented.

In this paper, we present a novel approach for sensor-guided robotic execution of everyday tasks, which is amenable to be integrated in current mobile manipulators and humanoid robots. We consider a robot which is observing simultaneously his hand and the object to manipulate, by using an external camera (i.e. robot head). Task-oriented grasping algorithms are used in order to plan a suitable grasp on the object according to the task to perform. A new vision/force coupling approach [1] is used in order to, first, guide the robot hand towards the grasp position and, second, perform the task taking into account external forces. Experimental results on a real robot are presented which validate our approach.

This work deals with the control by vision of the Gough-Stewart platform. For that, a central catadioptric camera is used to observe the platform legs. This allows to obtain a large field of view, and then avoids the occlusion problems observed when a classical perspective camera is used. The leg projections onto the catadioptric plane are used to determine their orientation in the camera frame. Finally, the computed orientations will be used in a visual servoing scheme of the platform effector.

This paper proposes an original and novel vision sensing method to be used in vision-based dynamic identification of parallel robots. Indeed, it is shown that in the latter problem one requires to estimate (to the least) or measure (to the best) the end-effector pose and its time derivatives. The sensor we propose, based on a clever modelling of CMOS rolling shutter camera, measures simultaneously the end-effector pose of a calibrated visual pattern and its Cartesian velocity using a single view. Although motivated by parallel robot identification, this low cost sensor does not make any assumption on the kinematics of the robot and can thus be used for other applications. Experimental results with real data confirm the relevance of the approach and show the sensor good practical measurement accuracy

Motivated by the growing interest for omnidirectional sensors on robotic applications and particularly on vision-based control, we present a new framework to handle in a visual servoing scheme the projection of line features into the image plane of a central catadioptric camera. As it is well known, the projection of a 3D line in the image plane of a central catadioptric camera is a conic curve. We propose to use the polar line of the image center with respect to this conic curve to define the input of the vision-based control scheme. The visual observations obtained from the polar lines lead to a minimal representation of projected lines. An efficient control scheme based only on two image features can then be designed. Simulation and experimental results confirm the validity of our approach

Automatic devices dedicated to vehicle guidance in off-road conditions are necessarily confronted with sliding phenomenon, since it may considerably damage the accuracy of the following task. Control laws taking explicitly into account such a phenomenon have already been designed in previous work. They can actually improve the guidance accuracy. However their efficiency is highly dependent on the sliding parameters estimation (since these parameters cannot be provided by a direct measurement). In this paper, an observer-like estimator is designed, providing sideslip angles from a single exteroceptive sensor, namely a real time kinematic GPS (RTK-GPS). Improvements in guidance accuracy, with respect to previous estimation approaches, is demonstrated through full scale experiments, addressing agricultural applications

An original method for computing instantaneous 3D pose and velocity of fast moving objects using a single view is presented. It exploits image deformations induced by rolling shutter in CMOS image sensors. First of all, a general perspective projection model of a moving 3D point is presented. A solution for the pose and velocity recovery problem is then described. The method is based on bundle adjustment and uses point correspondences. The resulting algorithm enables to transform a CMOS low cost and low power camera into an original velocity sensor. Finally, experimental results with real data confirm the relevance of the approach.

In this paper, we show that visual feedback reduces the complexity of parallel robot Cartesian control. Namely, 3D pose visual servoing, where the end-effector pose is indirectly measured and used for regulation, is shown to be well suited to this task since it relieves the control from the difficult forward kinematic problem. Moreover, this complexity reduction is not coming with an increase of the implementation complexity since off-the-shelf hardware and software are now available for visual servoing. It is also shown that such a control gets rid of joint sensors. All this makes 3D pose visual servoing the most straightforward Cartesian control for parallel robots. Experimental results are provided using an open source visual servoing C++ library

This paper is related to the vision-based control of parallel robots. Indeed, a method is proposed to estimate the reduced set of kinematic parameters appearing in such a control. To do so, it extends a linear method, obtained for a perspective camera, to the case of an omnidirectional camera, using an existing unifying projection model. The proposed method remains linear, once adequate information is extracted from the omnidirectional images, and does not require any calibration pattern. It works both with perspective and omnidirectional cameras, which is underlined by the reported experiments

This paper presents a new hybrid decoupled vision-based control scheme valid for the entire class of central catadioptric sensors (including conventional perspective cameras). First, we consider the structure from motion problem using imaged 3D lines (conics). Polar lines of the principal point with respect to the conic curves are exploited to estimate a generic homography matrix from which a partial Euclidean reconstruction is obtained. The polar lines and the information extracted from the homography are then used to design a control law which allow us to fully decouple rotational motions from translational motions

Mineral fertilizers application is an agricultural task widely performed by centrifugal spreaders. These machines give satisfying results with regularly spaced parallel tractor trajectories but lead to over and under-applications when geometrical singularities occur (non-parallel paths, start and end of spreading,...). The application errors result then in watercourses pollution and important yield losses. In this study, in order to improve fertilizer application by centrifugal spreading, an optimization problem is considered. The optimal parameters are computed by taking into account the mechanical constraints of the machine so that they can be used as reference variables to control the spreader in the future. Faced with a large scale constrained problem, an augmented Lagrangian algorithm using a 1-bfgs technique is implemented. The improvements provided by this new method are exposed through simulation results for parallel and non parallel paths in the field

In this paper, it is shown that computer vision, used as an exteroceptive redundant metrology mean, simplifies the control of a Gough-Stewart parallel robot. Indeed, contrary to the usual methodology, where the robot is modeled independently from the control law which will be implemented, we take into account that vision will be used for control, from the early modeling stage. Hence, kinematic modeling and projective geometry are fused into a control-devoted projective kinematic model. Thus, a novel vision-based kinematic modeling of such a robot is proposed through the observation of its legs. Inspired by the geometry of lines, this model unifies and simplifies both identification and control. Indeed, it has a reduced parameter set, and allows us to propose a linear solution to its calibration. Using the same model, a visual servoing scheme is presented, where the attitudes of the nonrigidly linked legs are servoed, rather than the end-effector pose. Finally, theoretical results concerning the stability of this control law are provided

This paper presents a novel approach for vision-based control of the end-effector of parallel mechanisms. It is based on the metrological redundancy paradigm, which simplifies their kinematic models by introducing additional proprioceptive sensors. By observing the mechanism legs, vision replaces advantageously these sensors by delivering, in a Cartesian frame, an exteroceptive measurement of the internal state of the mechanism. Formally, the latter is expressed by an original concept of vision-based kinematics for parallel mechanisms. Based on it, control is derived that visually servoes the direction of the legs, rather than the end-effector pose. The method is illustrated and validated on a Gough-Stewart platform simulation.

When navigating in an unknown environment for the first time, a natural behavior consists in memorizing some key views along the performed path, in order to use these references as checkpoints for a future navigation mission taking a similar path. This assumption is used in this paper as the basis of a navigation framework for wheeled mobile robots in indoor environments. During a human-guided teleoperated learning step, the robot performs paths which are sampled and stored as a set of ordered key images, acquired by a standard embedded camera. The set of these obtained visual paths is topologically organized and provides a visual memory of the environment. Given an image of one of the visual paths as a target, the robot navigation mission is defined as a concatenation of visual path subsets, called visual route. When running autonomously, the robot is controlled by a visual servoing law adapted to its nonholonomic constraint. Based on the regulation of successive homographies, this control guides the robot along the reference visual route without explicitly planning any trajectory. Real experiment results illustrate the validity of the presented framework.

In order to solve problems of traffic saturation in cities, new alternative” Urban Transportation Systems” are based on electric vehicles in free-access. One necessary functionality of such systems is their ability to move in a platoon fashion. Platooning of these automatic guided vehicles, relying on RTK-GPS sensors and inter-vehicles communication, is addressed in this paper. More precisely, vehicles platoon is expected to follow a curved reference path. Relying on nonlinear control theory, lateral and longitudinal control are fully decoupled, and therefore addressed independently. To ensure passengers comfort, additional monitoring functions supervise our control system. Then, simulations followed by experiments carried out with urban vehicles, are presented.

High-precision autofarming is rapidly becoming a reality with the requirements of agricultural applications. Lots of research works have been focused on the automatic guidance control of farm vehicles, satisfactory results have been reported under the assumption that vehicles move without sliding. But unfortunately the pure rolling constraints are not always satisfied especially in agriculture applications where the working conditions are rough and not expectable. In this paper the problem of path following control of autonomous farm vehicles in presence of sliding is addressed. To take sliding effects into account, a vehicle-oriented kinematic model is constructed in which sliding effects are introduced as additive unknown parameters of the ideal kinematic model. Based on backstepping method a stepwise procedure is proposed to design an adaptive controller in which time-invariant sliding effects are learned and compensated by parameter adaptations. It is theoretically proven that for the farm vehicles subject to sliding, the lateral deviation can be stabilized near zero and the orientation errors converge into a neighborhood near the origin. To be more robust to disturbances including external noises and unmodeled time-varying sliding components, the adaptive controller is refined by integrating Variable Structure Controllers (VSC) or projection mappings. Simulation results show that the proposed robust adaptive controllers can reject sliding effects and guarantee high path-following accuracy.

To close the loop between motion and vision, tracked visual features must remain in the camera field of view (visibility constraint). To overcome the visibility constraint, visual servoing methods can benefit from panoramic sensors such as catadioptric cameras (combining both mirrors and lenses). In this paper, we present a vision-based framework to control a nonholonomic mobile robot using a catadioptric imaging system. We particularly focus on a suitable catadioptric image-based control strategy of a nonholonomic robot in order to follow a 3D straight line. Such strategy can be applied to navigate in indoor or urban environment since the extraction and the tracking of straight lines are natural. First the control objectives are formulated in the catadioptric image space. The control law is then designed according to a well suited chained system for a mobile robot state vector directly expressed in the image space using a generic camera model. Simulation results illustrate the control strategy in the case of hypercatadioptric and paracatadioptric cameras.

One of the major current developments in agricultural machinery aims at providing farm vehicles with automatic guidance capabilities. With respect to standard mobile robots applications, two additional difficulties have to be addressed: firstly, since farm vehicles operate on fields, sliding phenomena inevitably occurs. Secondly, due to large inertia of these vehicles, small delays introduced by low-level actuators may have noticeable effects. These two phenomena may lower considerably the accuracy of path following control laws. In this paper, a vehicle extended kinematic model is first built in order to account for sliding phenomena. These latter effects are then taken into account within guidance laws, relying upon nonlinear control techniques. Finally, a Model Predictive Control strategy is developed to reduce the effects induced by actuation delays and vehicle large inertia. Capabilities of this control scheme is demonstrated via full scale experiments carried out with a farm tractor, whose realtime localization is achieved relying uniquely upon a RTK GPS sensor.

To solve problems of traffic saturation in cities, new alternative ''urban transportation systems" are based on electric vehicles in free-access. One necessary functionality of such systems is their ability to move in a platoon fashion. Platooning of these automatic guided vehicles, relying on RTK-GPS sensors and inter-vehicles communication, is addressed in this paper. The developed control law is based on a global control strategy; actually, it can take into account all the platoon state, and not only the immediate previous vehicle state. Distance here is understood as difference of curvilinear abscissa along a reference trajectory. Relying on nonlinear control theory, lateral and longitudinal control are fully decoupled, and therefore addressed independently. To ensure passengers comfort, additional monitoring functions supervise our control system. Then, experiment, carried out with urban vehicles, and simulations of long platoon, are presented.

In automatic guidance of agriculture vehicles, lateral control is not the only requirement. Lots of research works have been focused on trajectory tracking control which can provide high longitudinal-lateral control accuracy. Satisfactory results have been reported as soon as vehicles move without sliding. But unfortunately pure rolling constraints are not always satisfied especially in agriculture applications where working conditions are rough and not expectable. In this paper the problem of trajectory tracking control of autonomous farm vehicles in presence of sliding is addressed. To take sliding effects into account, two variables which characterize sliding effects are introduced into the kinematic model based on geometric and velocity constrains in presence of sliding. With linearization approximation a refined kinematic model is obtained in which sliding appears as additive unknown parameters to the ideal kinematic model. By integrating parameter adaptation technique with backstepping method, a stepwise procedure is proposed to design a robust adaptive controller. It is theoretically proven that for the farm vehicles subjected to sliding, the longitudinal-lateral deviations can be stabilized near zero and the orientation errors converge into a neighborhood near the origin. To be more realistic for agriculture applications, an adaptive controller with projection mapping is also proposed. Simulation results show that the proposed (robust) adaptive controllers can guarantee high trajectory tracking accuracy regardless of sliding.

In this paper, we present how the 2 1/2 D visual servoing scheme can be used with omnidirectional cameras. Motivated by the growing interest for omnidirectional sensors on robotic applications and particularly on vision-based control, we extend this framework to the entire class of central catadioptric systems. Indeed, conventional cameras suffer from restricted field of view. Central catadioptric systems have larger fields of view thus overcoming the visibility problem encountered when using conventional cameras. The 2 1/2 D visual servoing is based on the estimation of the partial camera displacement between two views, given by the current and desired images. Geometrical relationships are exploited to enable a partial Euclidean reconstruction by decoupling the interaction between translation and rotation components of a homography matrix. First we describe how to obtain a generic homography matrix for central catadioptric cameras from the projection model of an entire class of camera. Then the information obtained from the homography is used to develop a 2 1/2 D visual servoing scheme.

In this paper, a novel approach is proposed for the kinematic calibration of parallel mechanisms with linear actuators at the base. The originality of the approach lies in the observation of the mechanism legs with a camera, without any mechanism modification. The calibration can hence be achieved online, as no calibration device is linked to the end-effector, on any mechanism since no additional proprioceptive sensor installation is necessary. Because of the conditions of leg observability, several camera locations may be needed during the experimentation. The associated calibration method does not however require any accurate knowledge of the successive camera positions. The experimental procedure is therefore easy to perform. The method is developed theoretically in the context of mechanisms with legs linearly actuated at the base, giving the necessary conditions of identifiability. Application to an I4 mechanism is achieved with experimental results.

Since Global Navigation Satellite systems are able to supply very accurate coordinates of a point (about 2 cm with a RTK GPS), such a sensor is very suitable to design vehicle guidance system. It is especially the case in agricultural tasks where a centimeter precision is often required (seeding, spraying,...). To answer to growing high precision agriculture principle demand, several control laws for automated vehicle guidance relying on this sensor have been developed. Such guidance systems are able to supply an acceptable steering accuracy as long as vehicle does not slide (path tracking on even ground with good adherence properties...), what alas inevitably occurs in agricultural tasks. Several principles are here presented to steer vehicle whatever properties of ground and path to be followed are. In this paper a new extended kinematic model with sliding accounted is presented which allows describing vehicle dynamics in all guidance conditions. Via this model a new nonlinear control law can be designed, which integrates sliding effects. Its capabilities are investigated through simulations and experimental tests.

In this paper, an original approach is proposed for the kinematic calibration of parallel mechanisms. The originality lies in the use of vision to get information on all parts of the mechanism, i.e. its end-effector as well as its legs. Metrological redundancy is therefore maximized to improve the calibration efficiency. The approach is implemented for the calibration of the I4 parallel mechanism, with the use of the Jacobian matrix. No accurate camera location is needed so that the experimental procedure is easy to achieve. The calibration algorithm is detailed and experimentally demonstrated more efficient than other calibration methods based on legs observation or end-effector observation.

When designing an accurate automated guidance vehicle system, a major problem is sliding and pseudo-sliding effects. It is especially the case in agricultural applications, where a five centimeters accuracy with respect to the desired trajectory is required, even if vehicles move on a slippery ground. Previous works have established that RTK GPS was a very suitable sensor to achieve automated guidance with such a high precision: several control laws have been designed for vehicles equipped with that sensor, and provide the expected guidance accuracy as long as vehicles do not slide. Further control developments have been previously proposed to take sliding into account: guidance accuracy in slippery environment has been shown to be preserved, except transiently at beginning/end of curves. In this paper, design of such a control law is first recalled and discussed. Model predictive control method is then applied in order to preserve guidance accuracy even during these transitions. Finally, the global control scheme is implemented, and improvements with respect to previous guidance laws are demonstrated through full-scale experiments.

In this paper we consider the problem of controlling a robotic system using the projection of 3D lines in the image plane of central catadioptric systems. Most of the efforts in visual servoing are devoted to points, only few works have investigated the use of lines in visual servoing with traditional cameras and none has explored the case of omnidirectional cameras. First a generic central catadioptric interaction matrix for the projection of 3D straight lines is derived from the projection model of an entire class of camera. Then an image-based control law is designed and validated through simulation results.

In this article, a robot formation control strategy based on a vision-based follow-the-leader scenario is proposed, with emphasize on its reliability. On the one hand, perception is enhanced by the control of a motorized zoom. On the other hand, bidirectional and non-oblivious (Suzuki, I and Yamashita, M, 1996) control is implemented, with an odometry-based fault detection of vision-based information and a leader path-planning strategy to improve its visibility. After introduction of the control strategy, extensive experimental results are presented.

Numerous agricultural applications require very accurate guidance of farm vehicles. Current works have established that RTK GPS was a very suitable sensor in order to meet the expected precision: several control laws have been designed for vehicles equipped with such a sensor, and satisfactory results have been achieved as long as vehicles do not slide. Nevertheless, in actual working conditions (sloping fields, entering into curves on a wet land, etc.), sliding inevitably occurs. In this paper, we design a nonlinear adaptive control law in order to preserve guidance precision in presence of sliding: realtime sliding estimation is used to correct vehicle evolution. Field experiments, demonstrating the capabilities of that control scheme are reported and discussed.

In this article, we present the kinematic calibration of H4 parallel robot using vision-based measuring device. Calibration is performed according to the inverse kinematic model method, using first the design model then a model developed for calibration purpose. With a precision of the order of magnitude of 0.2 mm and 0.03/spl deg/, our vision system allowed us to obtain a final positioning accuracy of the end-effector lower than 0.5 mm. Conclusions are given for the calibration of this class of mechanisms.

A very accurate vehicle guidance is required in numerous agricultural applications, as seeding, spraying, row cropping,... Accuracy in vehicle localization can be obtained in realtime from a RTK GPS sensor. Several control laws, relying on this sensor, have been previously designed and provide satisfactory results as long as vehicles do not slide. However, sliding has to occur in agricultural tasks (sloping fields, curves on a wet land, ...). The challenge addressed in this paper is to preserve vehicle guidance accuracy in such situations. A nonlinear adaptive control law is here designed. Simulation results and field experiments, demonstrating the capabilities of that control scheme, are reported and discussed.

The main goal of this paper, is the study of image-based stereo visual servoing. A pair of cameras is mounted on the end-effector of the manipulator arm. The visual features are the pair of images of an unknown object. The developed control laws use either the raw image points, or the estimated 3D coordinates. The experimental setup is challenging: large rotations are involved, images are noisy, and cameras are coarsely calibrated. In this setup, the trajectory of the end-effector differs notably, sometimes leading the arm near its joint range limits. Experimental results demonstrate that using pixel coordinates is disadvantageous, compared with 3D coordinates estimated from the same pixel data.

Visual servoing requires an object in the field of view of the camera, in order to control the robot evolution. Otherwise, the virtual link is broken and the control loop cannot continue to be closed. In this paper, a novel approach is presented in order to guarantee that the object remains in the field of view of the camera during the whole robot motion. It consists in tracking an iteratively computed trajectory. A position based modeling adapted to a moving target object is established, and is used to control the trajectory. A nonlinear decoupling approach is then used to control the robot. Experiments, demonstrating the capabilities of this approach, have been conducted on a Cartesian robot connected to a real time vision system, with a CCD camera mounted on the end effector of the robot.

In this article, an evaluation of a vision-based measuring system for parallel machine-tool calibration is performed. Simultaneous measurement of the 6 pose components enables one to perform calibration using the efficient inverse kinematic method. The system is composed of a single camera and a calibration board generated on a LCD monitor. Calibration board size can be adapted to the camera field of view, and specific points of interest can be generated, in order to improve pose measurement. Based on a single industrial camera, the measuring system is low-cost and easy-to-use. An experimental evaluation of the system is performed on a machine-tool axis. Measurement bias and precision are estimated by comparison with laser interferometry, and the influence of focal length and sensor resolution is examined.

In this paper, visual perception in car driving assistance is considered. The work deals with the development of a system combining a pan-tilt-zoom (PTZ) camera and a standard camera, in order to track the front vehicles. The standard camera has a small focal length, and is devoted to the analyse of the whole frontal scene. Here, the PTZ camera is used to track the closest vehicle. Camera rotations and zoom are controlled by visual servoing and by an efficient real time target tracking algorithm. The aim of this work is to keep the rear view image of target vehicle stable in scale and position. The methods presented were tested on real road sequences within the VELAC demonstration vehicle. Experimental results show the effectiveness of such an approach.

Most visual servoing applications are concerned with geometrically modeled objects. In this paper, the problem of controlling a motion by visual servoing around an unknown object with a stereovision system is addressed. The main goal is to move the end-effector around the object in order to observe several viewpoints of the object for other tasks, e.g. inspection or grasping. The present work uses the well-known image-based visual servoing approach with a point, but the importance of the relationship between the end-effector and camera frames is clarified and emphasized. This relationship is needed for properly stacking the Jacobians or interaction matrices of each camera. A comparison with a visual servoing approach with a direct stacking of the Jacobians is presented. The centroid of a region, obtained by color segmentation, is used to move around the observed object. Experiments are developed on a PA-10 robot connected to a real time stereovision system, with two cameras mounted on the end-effector. Experimental results demonstrate the importance of a proper definition of the stacked Jacobians, to avoid undesired motions in the servoing task. Particularly, when turning around an unknown object, undesired motions on roll angle of the stereovision system can be avoided.

Precision agriculture involves very accurate farm vehicle control along recorded paths, which are not necessarily straight lines. We investigate the possibility of achieving this task with CP-DGPS (carrier phase differential GPS) as the unique sensor. The vehicle heading is derived according to a Kalman state reconstructor, and a nonlinear velocity independent control law is designed, relying on chained systems properties. Field experiments, demonstrating the capabilities of our guidance system, are reported and discussed.

In this paper, the problem of controlling a motion by visual servoing around an unknown object is addressed. These works can be interpreted as an initial step towards a perception goal of an unmodeled object. The main purpose is to perform motion with regard to the object in order to discover several viewpoint of the object. The originality of our work is based on the choice and extraction of visual features in accordance with motions to be performed. The notion of invariant feature is introduced to control the navigational task around the unknown object. A real-time experimentation with a complex object is realized and shows the generality of the proposed ideas.

We propose several results about trajectory generation by visual servoing. The approach consists of defining a specific task function which allows one to take into account the time varying aspect of the reference feature and to synthesize a control law in the sensor space. This control law ensures the trajectory control in the image space and reduces the tracking error. Under specific conditions, the trajectory of the camera can be ensured in the robot workspace. The main goal of this work is to demonstrate the effectiveness of this approach through experimental results. In the experiments, we used a Cartesian robot and a real time vision system. A CCD camera was mounted on the end effector of the robot. We present two types of trajectory. The first one is a helical trajectory parallel to a cube side. The second one involves passing around a cube. This latter is built by linking several elementary trajectories (rotation and translation).

Neither of the classical visual servoing approaches, position-based or image-based methods, are completely satisfactory. This paper presents a different approach with some advantages of both, i.e., the trajectory of the camera motion is predictable and the image features remain in the field of view of the camera. Our new approach is based on the computation of the pose of the object from the image, thus an appropriate calibration of the camera and a geometric model of the object are required. Experimental results on a real robotic platform are presented.

Vision based control has retained attention of many authors during the last few years. We first have been interested in image based visual servoing approach and recently we have focused our attention in position based visual servoing approach. In this paper, our goal is to study how we can introduce 3D visual features in a closed robot control loop. We consider a camera mounted on the end effector of the manipulator robot to estimate the pose of the target object; The required positioning task is to reach a specific pose between the sensor frame and a target object frame. Knowing the target object model, we can localize the object in the 3D visual sensor frame and estimate the pose between the camera and the target object at each iteration. To perform the visual servoing task, we use a nonlinear state feedback. We propose a new exact model for parametrization of the pose (position and the orientation of the frame object in the sensor frame). The main advantage of this approach is that camera translation and camera rotation are separately controlled due to use of a particular choice of frames. Convergence and stability have been proved theoretically, and the tests in simulation and on our experimental site show good behaviour using this type of approach.

Describes an approach to the problem of trajectory generation in a workspace by visual servoing. Visual servoing is based on an array of measurements taken from a set of images and used each time as an error function to compute a control vector. This is applied to the system (robot and camera) and enables it to move in order to reach a desired situation, at the end of the task, directly depicted in the image. The originality of this work is based on the concept of a time varying reference feature. Classically, in visual servoing, the reference features are static and the task to be achieved is similar to a positioning task. We define a specific task function which allows us to take into account the time varying aspect and we synthesize a new control law in the sensor space. This control law ensure the trajectory control in the workspace. Considering that any trajectories in workspace can be depicted as a combination of rotation and translation, we have tested our approach using these two elementary trajectories.

Visual servoing is based on an array of measurements taken from a set of images and used each time as an error function to compute a control vector. This is applied to the system (robot and camera) and makes it move in order to reach a desired situation, at the end of the task, directly depicted in the image. The originality of this recent work consists in improving the visual servoing approach. To do this, we consider a signal sensor vector constructed of a geometrical feature ((x,y) point coordinates, line parameters, etc.) and its first derivative. In this paper, we show how to work out the interaction matrix. We have tested this new approach on a workstation in the case of the point feature. We have also implemented it on our robotic platform. The overall results show a great improvement due to the action of this new signal sensor. We are extending this approach to more complex features.

The work presented in this paper belongs to the realm of robotics and computer vision. The problem we seek to solve is the accomplishment of robotics tasks using visual features provided by a special sensor, mounted on a robot end effector. This sensor consists of two laser stripes fixed rigidly to a camera, projecting planar light on the scene. First, we briefly describe the classical visual servoing approach. We then generalize this approach to the case of our special sensor by considering its interaction with respect to a sphere. This interaction permits us to establish a kinematics relation between the sensor and the scene. Finally, both in simulation and in our experimental cell, the results are presented. They concern the positioning task with respect to a sphere, and show the robustness and the stability of the control scheme.