status: Published

status: Published

In this article, a six-step approach is proposed to simulate the grasp and evaluate the grasp quality for a versatile vacuum gripper by tracking the deformation and force-torque wrench of the gripping pad. Over 100 K synthetic grasps are generated for neural network training. Furthermore, a gripping attention convolutional neural network (GA-CNN) is developed to predict the grasp quality for real-world grasp, running by 15 Hz closed-loop control with the real-time robotic observation and force-torque feedback. Various experiments in both the simulation and physical grasps indicate that our GA-CNN can focus on the crucial region of the soft gripping pad to predict grasp qualities and perform a lower average error compared with a same-scale traditional CNN. In addition, the complexity of grasping clutters is defined from Level 1 to Level 9. The proposed grasping method achieves an average success rate of 90.2% for static clutters at Level 1 to Level 8 and an average success rate of >80.0% for dynamic grasping at Level 1 to Level 7, which outperforms state-of-the-art grasping methods.

Despite extensive efforts, it is still very challenging to correctly detect clouds automatically from RGB images. In this paper, an automated and effective cloud detection method is proposed based on a semi-supervised generative adversarial networks that was originally designed for anomaly detection in combination with structural similarity. By only training the networks on cloudless RGB images, the generator network is able to learn the distribution of normal input images and is able to generate realistic and contextually similar images. If an image with clouds is introduced, the network will fail to recreate a realistic and contextually similar image. Using this information combined with the structural similarity index, we are able to automatically and effectively segment anomalies, which in this case are clouds. The proposed method compares favourably to other commonly used cloud detection methods on RGB images.

Clock synchronisation between sensors plays a key role in applications such as in autonomous robot navigation and mobile robot mapping. Such robots are often equipped with cameras for gathering visual information. In this work, we address the problem of synchronizing visual data collected from a low-cost 2D camera, with IMU (Inertial Measurement Unit) data. Both sensors are assumed to be attached to the same rigid body; hence, their motion is correlated. We present a motion based approach using a particle filter to estimate the clock parameters of the camera with the IMU clock as a reference. We apply the Lucas-Kanade optical flow method to calculate the movements of the camera in its horizontal plane corresponding to its recorded images. These movements are correlated to the motion registered by the IMU. This match allows a particle filter to determine the camera clock parameters in the IMU's time frame and are used to calculate the timestamps of the images. We presume that only the IMU sensor provides timestamp data generated from its internal clock. Our experiments show that given enough features are present within the images, this approach has the ability to provide the image timestamps within the IMU's time frame.

Localization based on visible light is a novel technique for indoor positioning that provides a number of advantages over traditional radio frequency based approaches. An important step in the deployment of visible light positioning systems is the calibration procedure, during which environmental parameters such as the positions of light sources are determined. This work presents a proof-of-concept approach to obtain these parameters in an efficient manner by using a mobile robot. This robot builds a map of the environment, and adds the location and identifier of optical transmitters to this map. With this approach, light source modulation frequencies can be estimated with sufficient accuracy to uniquely identify each source. Additionally, the inter-LED distance has an average accuracy of less than 10 cm compared to the real distance.

This paper introduces a global, sampling-based motion planning approach to the tube following problem for redundant robot manipulators. Tube following is a motion planning problem where an under-defined end-effector path is given. We introduce a novel combination of existing task space and redundant configuration space sampling techniques. These techniques are applied to the tube following problem, comparing three different sampling techniques: uniform grid, uniform random and Halton sampling. In addition, an incremental sampling technique is proposed, specifically for tube following problems. Eventually, iterative grid refining is used to locally optimise path cost. Experimental results demonstrate the clear potential of these techniques to achieve fully automatic, collision-free motion planning for tube following with redundant robot manipulators. Our planning software is publicly available.

With the rise of solid state lighting, wireless positioning based on visible light is becoming more appealing. However, current visible light positioning systems require additional hardware at the transmitter end in order to modulate the light intensity. A receiver demultiplexes the combined signal from multiple sources into its components, which are then used by the positioning algorithm. This paper investigates the possibility of using unmodulated visible light for mobile robot positioning. Less hardware is required, consequently cost and complexity are much lower. Position estimation is achieved by modeling the received signal strength inside a room, which is used as input for an Iterated Extended Kalman filter. We show that the proposed approach can achieve decimeter level accuracy in a simulation environment. Even with imperfect calibration, the total positioning error usually remains below 0.5 m. Positioning errors due to blocking of the receiver can be mitigated by employing an innovation magnitude bounds test. We also show that by employing multiple receivers, accuracy and robustness can be further improved.

In collaborative human-robot navigation such as when driving semi-autonomous robotic wheelchairs, intuitive control of the mobile robot is only possible if the robot understands its user. This becomes especially important as users present varying levels of abilities and heterogeneous driving styles. Furthermore, the robot needs to consider the inherent uncertainty on its navigation task because the user may not be able to communicate his or her plans explicitly. In order to address these requirements, we have adopted a probabilistic framework to recognise navigation plans. A key component in this framework is a personalised driver model, which captures how a particular user transforms his or her mental navigation plan into inputs to the robot. In this work, we evaluate the use of Gaussian Processes to implement and calibrate this probabilistic, user-specific driver model, and this for use with haptic joysticks. Furthermore, special care was taken to obtain fast online evaluation of this user model through sparse approximation and parallel computation on a GPU. This resulted in an achievable user model evaluation frequency of 40 Hz, which is far above the navigation assistance frequency we aimed for, i.e. 5 Hz. We illustrate the validity of the approach by recognising the navigation plans of a spastic wheelchair user.

Many industrial robot applications require fewer task constraints than the robot's degrees of freedom. For welding robots, for example, rotations of the welding torch around its axis do not negatively impact welding quality. Furthermore, the tool center point's Cartesian position and desired orientation as a function of time is often determined by the (manufacturing) process. Nevertheless, programming these robots can be time consuming. Reducing or eliminating this programming cost will allow robots to be used for producing small series. Recently, a promising software package for Cartesian path planning with the name Descartes was released by the ROS-Industrial community. To the authors' knowledge, an in-depth description of this algorithm and an experimental evaluation is lacking in literature. This paper describes the path planning approach used by the Descartes package. Moreover, the software's performance is evaluated for several key robot welding tasks and the encountered limitations are discussed. In addition, we show that the planner's performance can be improved by changing the cost function that the planner's graph search algorithm minimises.

Over the last few years, the use of medium density fibreboard (MDF), oriented strand board (OSB) and other particleboard (grade B wood) has increased dramatically in the timber industry. This represents a major challenge for the wood recycling industry, which grinds this wood into chips to sell it to the wood chip industry. For this reason, the demand for an accurate, reliable, cheap and fast system that separates MDF, OSB and other particleboard material from quality wood in a mixed wood waste stream is huge, as this is still performed manually in many places. This paper presents a vision-based solution for such a wood waste sorting system. A main contribution of our work is an industry-ready, performance integration of state-of-the-art computer vision and machine learning techniques into a robust wood waste sorting setup. This system has been implemented in a Belgian company, where it runs 16 hours/day and processes 7.5 tons of wood waste per hour (reaching a Technology Readiness Level of 9). The return on investment was less than one year. From an input stream with 45% - 55% of grade B wood, two output streams are generated, one which contains less than 2% quality wood and one which contains less than 5% grade B wood.

Navigating an electrical wheelchair can be very challenging due to its large size and limited maneuverability. Additionally, target users often suffer from cognitive or physical disabilities, which interfere with safe navigation. Therefore, a robotic wheelchair that helps to drive can prove invaluable. Such a wheelchair shares the control with its human operator. Typically, robots excel in fine-motion control whereas users want to remain in charge. Hence, the robot should focus its help locally and let the user decide about global behavior. Further, an effective robot should understand the navigation plans of its user. It needs to consider the user's abilities to avoid frustrating the user with wrong assistance. In order to address these requirements, we propose a probabilistic framework to recognize local navigation plans in a user-specific way. The framework infers navigation plans online and provides a method to calibrate all model parameters from real driving data. It fuses past local information with a user-specific model to reason about how and where the user intends to navigate. We illustrate the validity of our approach by recognizing the local navigation plans of a spastic user driving in a daily environment.

This article introduces a set of novel haptic guidance algorithms intended to provide intuitive and reliable assistance for electric wheelchair navigation through narrow or crowded spaces. The proposed schemes take hereto the non-holonomic nature and a detailed geometry of the wheelchair into consideration. The methods encode the environment as a set of collision-free circular paths and, making use of a model-free impedance controller, `haptically' guide the user along collision-free paths or away from obstructed paths or paths that simply do not coincide with the motion intended by the user. The haptic feedback plays a central role as it establishes a fast bilateral communication channel between user and wheelchair controller and allows a direct negotiation about wheelchair motion. If found unsatisfactory, suggested trajectories can always be overruled by the user. Relying on inputs from user modeling and intention recognition schemes, the system can reduce forces needed to move along intended directions, thereby avoiding unnecessary fatigue of the user. A commercial powered wheelchair was upgraded and feasability tests were conducted to validate the proposed methods. The potential of the proposed approaches was hereby demonstrated.

Removing the safety fences that separate humans and robots, to allow for an effective human-robot interaction, requires innovative safety control systems. An advanced functionality of a safety controller might be to detect the presence of humans entering the robotic cell and to estimate their intention, in order to enforce an effective safety reaction. This paper proposes advanced algorithms for cognitive vision, empowered by a dynamic model of human walking, for detection and tracking of humans. Intention estimation is then addressed as the problem of predicting online the trajectory of the human, given a set of trajectories of walking people learnt offline using an unsupervised classification algorithm. Results of the application of the presented approach to a large number of experiments on volunteers are also reported.

For people with very severe motor dysfunctions, Brain-Computer Interfaces (BCIs) may provide the solution to regain mobility and manipulation capabilities. Unfortunately, BCIs are characterized by a limited bandwidth and uncertainty on the BCI output. In the past, we have developed a Bayesian plan recognition framework that estimates from uncertain human-robot interface signals the task a robot should execute. This paper extends our plan recognition framework to incorporate uncertain BCI signals. A benchmark test is proposed and adopted to evaluate both the plan recognition framework and the performance of the BCI user, for the concrete application of wheelchair driving.

Many elderly and disabled people experience difficulties when maneuvering an electric wheelchair. In order to make wheelchair driving a safer and more comfortable experience, there has long been the claim to equip wheelchairs with some form of intelligent controller assisting in difficult or unsafe situations. It has been observed that every user presents different symptoms causing a specific driving pattern. Therefore, if the user is to be helped and not frustrated, his/her particular driving behavior should be taken into account when assisting him/her. In this paper we present a general user modeling technique for our Bayesian framework for plan recognition and shared wheelchair control. Plan recognition corresponds to estimating the plan a user has in mind. Assistive actions can then be taken based on the estimated user plan. A user modeling technique based on Gaussian processes has been selected, which can be adapted online to any type of driving style. The potential of Gaussian processes for user modeling is illustrated on a case study with a disabled patient suffering from spastic quadriplegia.

The last years have witnessed a significant increase in the percentage of old and disabled people. Members of this population group very often require extensive help for performing daily tasks like moving around or grasping objects. Unfortunately, assistive technology is not always available to people needing it. For instance, steering a wheelchair can represent an extremely fatiguing or simply impossible task to many elderly or disabled users. Most of the existing assistance platforms try to help users without considering their specific needs. However, driving performance may vary considerably across users due to different pathologies or just due to temporary effects like fatigue. Therefore, we propose in this paper a user adapted shared control approach aimed at helping users in driving a power wheelchair. Adaption to the user is achieved by estimating the user's true intent out of potentially noisy steering signals before assisting him/her. The user's driving performance is explicitly modeled in order to recognize the user's intention or plan together with the uncertainty on it. Safe navigation is achieved by merging the potentially noisy input of the user with fine motion trajectories computed online by a 3D planner. Encouraging results on assisting a user who cannot steer to the left are reported on K.U.Leuven's intelligent wheelchair Sharioto.

Many elderly and disabled people today experience difficulties when manoeuvring an electric wheelchair. In order to help these people, several robotic assistance platforms have been devised in the past. In most cases, these platforms consist of separate assistance modes, and heuristic rules are used to automatically decide which assistance mode should be selected in each time step. As these decision rules are often hard-coded and do not take uncertainty regarding the user's intent into account, assistive actions may lead to confusion or even irritation if the user's actual plans do not correspond to the assistive system's behavior. In contrast to previous approaches, this paper presents a more user-centered approach for recognizing the intent of wheelchair drivers, which explicitly estimates the uncertainty on the user's intent. The paper shows the benefit of estimating this uncertainty using experimental results with our wheelchair platform Sharioto

This paper describes a novel adaptive filter approach to reduce the handicap a patient may experience when navigating an electric wheelchair. The filter automatically adapts to the specific handicap the patient has by training a connectionist structure that converts the joystick signal of the patient to the signal a reference user would give in the same context. Experimental results show that for various handicaps the filter improves the driving performance significantly

This paper presents an extension of current global dynamic window approaches to holonomic and nonholonomic mobile robots with an arbitrary cross-section. The algorithm proceeds in two stages. In order to account for an arbitrary robot footprint, the first stage takes the robot's orientation explicitly into account by constructing a navigation function in the (x, y, /spl theta/) configuration space. In a second stage, an admissible velocity is chosen from a window around the robot's current velocity, which contains all velocities that can be reached under the acceleration constraints. Fast computation over large areas is achieved by adopting multi-resolution (x, y) and (x, y, /spl theta/) planning. Several measures are taken to obtain safe and robust robot behaviour. Experimental results on our wheelchair test platform show the feasibility of the approach.

Many elderly and disabled people today experience difficulties when maneuvering an electric wheelchair. In order to help these people, several robotic assistance platforms have been devised in the past. These platforms' architectures usually consist of separate assistance modes that each realise a specific navigation behaviour, such as "avoid-obstacles", or "drive-through-door". In most cases, heuristic rules are used to decide automatically which assistance mode should be selected in each time step. These decision rules are often hard coded and therefore not very adaptable to different user's actual plans do not correspond to the assistive system's behaviour. Moreover, navigation algorithms are used that take the wheelchair's kinematic and dynamic constraints only approximately into account. Consequently, these robotic wheelchairs may and do fail in executing the very same maneuvers with which elderly and disabled people have problems. In contrast with previous approaches, this paper presents a user-centered architecture for shared wheelchair control. The framework continuously estimates the user's intention explicitly before trying to assist him or her. The actual navigation assistance is performed by a fine motion planner that takes the kinematic and dynamic constraints into account. The paper presents experimental results and an evaluation of the architecture.