This paper introduces an extension of the reverse priority framework for the kinematic control of redundant robots. It integrates, in a unified framework, the treatment of multiple tasks, multiple kinematic chains, different joint priorities and hard constraints. The management of multiple tasks is based on the reverse priority method, that has been modified so that it makes possible the assignment of different priorities to each joint in order to accomplish the tasks. This framework is also suitable for robotic systems with multiple kinematic chains, which could share several joints. Moreover, it can deal with bilateral and unilateral constraints, that can be defined either at joint or cartesian space. Hard constraints are considered at each priority level, instead of treating them separately at the highest priority level. The proposed framework has been evaluated in simulation and in real experiments with a redundant underwater vehicle-manipulator system at sea.

As far as we know, currently all the underwater interventions, requiring robotic manipulation, are carried out by using a well-known technology, based on the commercial work class ROVs (Remotely Operated Vehicles). These systems require a lot of support, including a vessel in the surface, where a pilot (i.e. user expert) is able of teleoperating the actions of the ROV by means of an umbilical cable. On the other hand, from the last ten years new research has been developed, promoting in the market the so called Autonomous Underwater Vehicles (AUVs), enabling interventions. Obviously the AUVs have some main advantages over ROVs: no vessel, umbilical or pilots are needed now, but presenting a critical drawback: any kind of potential manipulation skills are impossible. Thus, the present situation becomes in a new vehicle: the Hybrid-ROV (HROV), trying to join the best of both systems, ROVs and AUVs. However, new problems arise concerning the HROV control. Now the system can operate in two different ways: autonomous or teleoperated, and so, a new control approach should be developed. This paper presents a control architecture for an HROV, discussing details of this approach from the human-robot interaction viewpoint.

This article presents research on the subject of autonomous underwater robot manipulation. Ongoing research in underwater robotics intends to increase the autonomy of intervention operations that require physical interaction in order to achieve social benefits in fields such as archaeology or biology that cannot afford the expenses of costly underwater operations using remote operated vehicles. Autonomous grasping is still a very challenging skill, especially in underwater environments, with highly unstructured scenarios, limited availability of sensors and adverse conditions that affect the robot perception and control systems. To tackle these issues, we propose the use of vision and segmentation techniques that aim to improve the specification of grasping operations on underwater primitive shaped objects. Several sources of stereo information are used to gather 3D information in order to obtain a model of the object. Using a RANSAC segmentation algorithm, the model parameters are estimated and a set of feasible grasps are computed. This approach is validated in both simulated and real underwater scenarios.

While commercially available autonomous underwater vehicles (AUVs) are routinely used in survey missions, a new set of applications exist which clearly demand intervention capabilities: the maintenance of permanent underwater structures as well as the recovery of benthic stations or black-boxes are a few of them. These tasks are addressed nowadays using manned submersibles or work-class remotely operated vehicles (ROVs), equipped with teleoperated arms under human supervision. In the context of the TRITON Spanish funded project, a subsea panel docking and an intervention procedure are proposed. The light-weight intervention AUV (I-AUV) Girona 500 is used to autonomously dock into a subsea panel using a funnel-based docking method for passive accommodation. Once docked, an autonomous fixed-based manipulation system, which uses feedback from a digital camera, is used to turn a valve and plug/unplug a connector. The paper presents the techniques used for the autonomous docking and manipulation as well as how the adapted subsea panel has been designed to facilitate such operations.

This paper presents the progress that has been made recently in the TRITON project. The TRITON project is an on going research project being carried out in Spain which has as principal objective the production of an AUV capable of autonomous underwater interventions. The GRASPER sub-project focuses on developing the necessary manipulation skills. Currently, a lot of research in the underwater robotics context is developing increasing levels of autonomy for all kinds of intervention operations, which always require some kind of physical interaction. However, if autonomous robotic manipulation on land remains a relatively undeveloped field, the situations is at an even more primitive stage in underwater scenarios where currently the systems are tele-operated by an expert user from a surface vessel. Only very few underwater systems have the capacity to carry out manipulation without any kind of umbilical cables teleoperating these actions. In particular, this work introduces a new approach for increasing the autonomy levels of an underwater manipulation system, discussing also preliminary results. In order to test this concept, different objects, without predefined models, are approached and recovered from the bottom in water tank conditions. To achieve this purpose, a scan of the scene is performed using a structured laser beam attached to the forearm of the manipulator. At the same time, a digital video camera is used to capture the scene with the laser beam projected onto the object. The laser stripes are triangulated to obtain a 3D point cloud. Moreover, the underwater robot gripper is provided with strain gauge tactile sensors, which enable the execution of a more reliable grasp. On the other hand, the process is shown inside an underwater simulator previously developed, named UWSim, acting in this case as a virtual representation of the real environment. This virtual representation allows the user to specify the grasp, highlighting how the virtual grasp will be defined for the selected target. The feasibility and reliability of the underwater manipulation system is demonstrated though the experimental results.

Autonomous grasping of unknown objects by a robot is a highly challenging skill that is receiving increasing attention in the last years. This problem becomes still more challenging (and less explored) in underwater environments, with highly unstructured scenarios, limited availability of sensors and, in general, adverse conditions that affect in different degree the robot perception and control systems. This paper describes an approach for semi-autonomous grasping and recovery on underwater unknown objects from floating vehicles. A laser stripe emitter is attached to a robot forearm that performs a scan of a target of interest. This scan is captured by a camera that also estimates the motion of the floating vehicle while doing the scan. The scanned points are triangulated and transformed according to the motion estimation, thus recovering partial 3D structure of the scene with respect to a fixed frame. A user then indicates the part where to grab the object, and the final grasp is automatically planned on that area. The approach herein presented is tested and validated in water tank conditions.

This paper presents UWSim: a new software tool for visualization and simulation of underwater robotic missions. The software visualizes an underwater virtual scenario that can be configured using standard modeling software. Controllable underwater vehicles, surface vessels and robotic manipulators, as well as simulated sensors, can be added to the scene and accessed externally through network interfaces. This allows to easily integrate the simulation and visualization tool with existing control architectures, thus allowing hardware-in-the-loop simulations (HIL). UWSim has been successfully used for simulating the logics of underwater intervention missions and for reproducing real missions from the captured logs. The software is offered as open source, thus filling a gap in the underwater robotics community, where commercial simulators oriented to ROV pilot training predominate.

This paper presents an approach for integrating vision, tactile and force sensors in a robotic manipulation framework. Having an initial estimation of the object pose in the environment, a position-based visual servoing loop controls the hand for task execution, based on the input received from a model-based articular pose estimator following the Virtual Visual Servoing approach. The visual control is combined with another control signal obtained from tactile feedback, through a set of selection matrices that can be modified at runtime in order to select the best modality for a given cartesian degree of freedom. The result of the preliminary integration is modified by an impedance force controller, in charge of performing the task motion along the task direction, at the same time that forces are regulated on the rest of directions. The design of the controller allows to perform the task even if a sensor is not available or provides inaccurate data. Several experiments are performed, first by considering only force feedback, and then adding vision and, finally, tactile information. Errors in the estimation of the object initial position are manually introduced in the experiments, and results show how the vision-tactile-force combination is able to deal with them, performing much better than the vision-force and force-alone approaches.

In this video we present a work in progress in the UJI (i.e. the acronym for University Jaume I) robotics telelaboratory. This telelaboratory uses a remote control system based on networked robots and FPGAs technology. The main devices included in this cell are: a SCARA manipulator (AdeptOne), a robot arm with six degrees of freedom (Motoman), an industrial belt, several sensors and cameras, an FPGA that takes care of the computer vision algorithms (i.e. including grasping determination), and a distributed architecture that allows any user to control remotely via Internet a specific manufacturing task. The different components of this system are connected by a 100BaseT Ethernet network and follow the SNRP architecture (i.e. Simple Network Robot Protocol), which permits the integration of network robots and sensors within an e-learning platform in a simple and reliable manner. The whole telelaboratory is connected to the Internet through a router that permits the user to control the networked devices according to security constraints. This distributed architecture allows any user to control remotely via Internet a specific manufacturing task.

This paper addresses the problem of grasp planning of unmodelled 3D objects with the aide of vision. The approach proposes the extension of fast 2D contour-based grasp planning algorithms, and the use of images of the target object from different points of view to recover critical 3D information like the size, location and the pose. A complete method is described, and experimental results are presented showing its feasibility.

In robotics, the manipulation ofa prioriunknown objects involves several steps and problems that must be carefully considered and solved by proper planning and control algorithms. For example, once suitable contact points have been computed, the control system should be able to track them in the approach phase, i.e., while the relative position/orientation of the object and the gripper of the robotic system change due to the approaching movement of the robot toward the object. This correspondence paper proposes a practical method for the tracking of grasp points in image space that is based on transferring previously computed grasp points from an initial image to subsequent ones and on the analysis of the new grasp configuration. Three different options are proposed for this transference. Experimental results show the interesting practical performance of the general procedure.

In this paper we present a robot that is able to perform daily manipulation tasks in a home environment, such as opening doors and drawers. Taking as input a simplified object model and the task to perform, the robot automatically finds a grasp suitable for the task and performs it. For this, we identify a set of hand preshapes and classify them according to the grasp wrench space they generate. Given a task, the robot selects the most suitable hand preshape and automatically plans a set of actions in order to reach the object and to perform the task, taking continuously into account the task forces. The concept of hand preshape is extended for the inclusion of a task frame, which is a concept from task planning, thus filling the gap between the grasp and the task.

This work deals with the problem of dependable physical interaction through manipulation in partially-known everyday human environments. We present a modular software architecture that allows the definition and compliant execution of manipulation tasks under the Task Frame Formalism. Several software modules implemented within this architecture, enable higher levels of adaptability and robustness, as well as the incremental incorporation of more complex skills in a modular fashion. The whole system is validated making a real robot arm with a three-finger hand perform two different manipulation tasks: opening a doorhandle and taking a book out of a bookshelf.

This paper deals with the problem of dependable physical interaction through manipulation in partially-known everyday human environments. We present a modular software architecture that allows the definition and compliant execution of manipulation tasks under the task frame formalism. We show the details of several software modules implemented within this architecture, that enable higher levels of adaptability and robustness, as well as the incremental incorporation of more complex skills in a modular fashion. The whole system is validated making a real robot arm with a three-finger hand perform a complex manipulation task: taking a book out of a bookshelf. Results show how the presented framework is suitable for easily defining and performing a great variety of manipulation tasks

This paper describes design and implementation aspects of an autonomous robot that is intended to perform service tasks in a library. We first address problems concerning the arrangement of its components into a compact mobile manipulator suitable for our needs. Then, software issues are also discussed, related to the techniques that are necessary in order to navigate and to reliably manipulate books in a library. In this context, two methods are presented, concerning visual tracking and force/vision control. The first one is aimed at locating a book in a bookshelf by means of vision. For this, a tracking algorithm is implemented, which takes into account the spatio-temporal constraints of the problem. Book tags are located in the image and processed using open source optical character recognition (OCR) software, until the desired book is found. The second of the methods addresses the problem of extracting the book from the shelf. A hybrid force/vision control algorithm has been implemented, which uses vision for guiding the gripper to the correct book, and force for detecting contacts.

For those situations in which the user wants to interact with the system by using, for example, voice commands, it would be convenient to refer to the objects by their names (e.g., "cube") instead of other types of interactions (e.g., "grasp object 1"). Thus, automatic object recognition is the first step in order to acquire a higher level of interaction between the user and the robot. Nevertheless, applying object recognition techniques when the camera images are being transmitted through the web is not an easy task. In this situation, images cannot have a very high resolution, which affects enormously the recognition process due to the inclusion of more errors while digitalizing the real image. Some experiments with the Universitat Jaume I Online Robot evaluate the performance of different neural-network implementations, comparing it to that of some distance-based object recognition algorithms. Results will show which combination of object features, and algorithms (both statistical and neural networks) is more appropriate to our purpose in terms of both effectiveness and computing time.

The visual tracking of grasp points is an essential operation for the execution of an approaching movement of a robot arm to an object: the grasp points are used as features for the definition of the control law. This work describes a strategy for tracking grasps on planar objects based on the use of a homography. In particular, the homography is used for transferring (translating) a grasp from a view of an object to a second one, providing in this way a correspondence in the second view. The grasp tracking procedure combines the search of a new grasp with the translation and the evaluation of that grasp between object views. Results of the proposed grasp tracking strategy are shown in case of grasps computed for a two-finger and a three-finger gripper.

The positioning of a gripper with respect to an object is one of the basic steps involved in the manipulation of that object. In particular, in a grasping task the fingers of the gripper should be placed at a set of selected grasp points belonging to the object. In this paper, a two-finger gripper and a stereo pair of cameras mounted in an eye-in-hand configuration are considered for this task. A visual servoing control system is proposed for the guidance of the positioning movement in which the grasp points are tracked in image space and used as input features for the control law. This tracking is based on the use of an invariant description of the grasp with respect to the object. The control law is restricted to control the degrees of freedom that produce movements that respect the invariance required by the grasp tracking.

The design of online robots is a difficult task that must take into account multiple aspects like for example: (1) the system architecture, (2) the user interface, (3) communications, (4) time delay, (5) limited bandwidth, etc. Besides this, taking into account that the Web permits multiple people to easily access a single system, methods for managing the control of a unique robot must be designed (e.g. off-line virtual interfaces, connecting several robots to the system, etc.). In this paper we present the system architecture that have been used in our laboratory to let users program remotely two educational and two industrial robots through the same Web based system. Experimental results show different alternatives to organize the architecture and focus on the system performance aspects.

In this paper, we present the system architecture that has been used in order to implement a whole Internet-based tele-laboratory, which allows researchers and students to program remotely an online robot. In fact, the challenge has been demonstrating that remote programming combined with an advanced multimedia user interface for remote control is very convenient, flexible and profitable for the design of the tele-laboratory. In fact, the contribution consist of designing a system architecture that permits to any external program (i.e. remote experiment, speech recognition module, etc.) to have access to almost every feature of the already existing "UJI online robot" (i.e. cameras, object recognition, robot control, etc.) [R. Martin, et al., May 2002].

This paper presents work in progress towards a complete system working to assist users in a library. With this aim, the system must be capable to looking for a specific book in a shelf, asked by any user, and whether it is found, deliver it as soon as possible to the user. To get its objectives the system integrates automatic object recognition, visually guided grasping, and force feedback, among other advanced capabilities. Implementation details about the main modules developed presently are shown. Finally, after success in preliminary results obtained in our campus, we are encouraged to fallow working in this way to obtain the complete prototype.

Within the context of online robots, a considerable amount of research has traditionally focused on the global system functionality, including the way of interaction between the user and the robot. Recent results in different robotics areas have demonstrated the potential of a number of techniques from the Pattern Recognition and Machine Learning domains, although very few work has been specifically addressed to online robots, where the object recognition is directly performed by the user. In this paper, we investigate the feasibility of using a neural network approach to object recognition in the context of online robots, and discuss the main advantages over the application of statistical learning methods. Some experiments with the UJI (Universitat Jaume I) online robot evaluate the performance of different neural network implementations, comparing it to that of some distance-based object recognition algorithms.

This paper describes a web-based telerobotic system with two basic capabilities: object recognition and incremental or continual learning. Within this context, the present paper investigates the feasibility of applying several distance-based classifiers to remote object recognition tasks. On the other hand, incremental learning enables the system to maintain a representative set of past training examples that are used together with new data to appropriately modify the currently held knowledge. This can constitute a very significant advantage because it allows a system to continually increase its knowledge and accordingly, to enhance its performance in recognition as it learns. And also, because this capability allows to detect an incorrect or anomalous employment of this particular remote system. The performance of the system is evaluated by means of an empirical analysis addressed to assess two competing goals: computing time and recognition accuracy.

The system consists of a multirobot architecture that gives access to an educational and an industrial robot, through the Internet, by using advanced multimedia and distributed programming tools like Java, Java3D and CORBA. The main effort is focused on the user interface that allows specification of a task into a 3D model and sending commands to the real robot once the operator approves them. This kind of interaction has the ability to consume very little bandwidth in order to perform a remote manipulation via the Internet. The user interface offers augmented and virtual reality possibilities to the users, as well as object recognition in order to specify commands to the robot in a natural way. This high-level task specification would not be possible without the grasping determination module, which suggests manipulation characteristics for every scene's object.

A very interesting robotics area is the high-level tasks' specification, and the way a user-robot interface can facilitate programming complex actions to be executed on real environments. This paper shows a novel contribution in the user-robot interaction domain, and particularly in the web robotics field. The user, who is in the control loop, is able to specify the robot actions over a simulated scenario (running off-line), assisted with 3D virtual and augmented reality. Then, once the task is defined, it can be easily executed over the real scenario (running online). To get its main goal (i.e. task specification) working in real life scenarios, the system incorporates automatic object recognition and visually-guided grasping among others capabilities.

This article shows the way a speech recognition module has been implemented that provides the translation between voice input and the convenient text based commands to be executed on an already existing web robot. The novel contribution is the way the procedure is defined to be run over the Internet, and the interface implemented to connect any kind of external speech recognition program to the robot controller. First section gives an overview to the whole project, from a user point of view. Secondly, the software architecture is presented, giving low level details on the connectivity procedure and the interface defined between the remote controller and the speech recognition and synthesizer program. Moreover, it introduces some of the technologies (e.g. Microsoft Speech SDK) used to implement this capability. Finally, results are shown by means of both, accuracy details of the speech recognition procedure, and computing time.

This paper presents an implemented vision-based strategy for computing three-finger stable grasps on unknown planar objects. Its only input is an image of a real unknown curved object instead of synthetic polygonal models. Grasp regions are identified for achieving tolerance around contact points so that errors in finger positioning as well as a finite finger size are considered. It can find expanding and squeezing grasps involving internal and external regions, including holes. Theoretical constraints for force-closure are met while assuring robustness and real-time performance under real-world conditions.

Research in grasping and manipulation has mainly focused on the theoretical framework of the grasp analysis and synthesis, however there are few works in the practical implementation of these approaches. We propose several methods and criteria for constraining visually-selected planar grasps to the particular kinematics of a three finger gripper, the Barrett Hand. The method relies on the features of an object provided by a vision system, and uses several criteria for classifying the possible grasps according to the configurations of the gripper that fit them. It has been implemented on the UMass Torso and the results are discussed.

In this paper, a strategy that combines the tracking of an object and the selected grasp on it is presented. This strategy is intended to provide enough information to a control system for executing the grasp. The tracking of the object uses the ICondensation and AARM prediction algorithms to predict the current object position and shape in the image space. Using a set of quality criteria, stable grasps are searched for on the object contour and the best one is selected. The selected grasp is tracked together with the object contour in the sequence of images.

A key issue in robotics is the development of the ability to grasp unknown objects. This ability requires a grasp determination mechanism that, based on the analysis of the description of the object, determines how it can be stably grasped. In this paper, a grasp determination method is presented that computes a set of grasps that comply with the force-closure condition. Its input is a set of contours, extracted from the vision data, describing the shape of the object. The internal holes of the object are taken into account, so the algorithm can find grasps on them. The algorithm also finds expansion and squeezing grasps, which are executed by opening and closing the gripper fingers.

We present an integrated vision-guided grasping system for service robots. Our system integrates computer vision to capture the shape of the objects, online grasp determination based on that shape, and image-based control for grasp execution. Novel techniques are presented to solve the problems concerned under the imposed resource constraints, namely, for information reduction and segmentation in image processing, strategies for grasp determination as well as vision-guided control for grasp execution. A preshaping heuristic strategy is combined with symmetry and local shape analysis. Also, the surface of contact between fingers and object is considered and a novel technique for sampling the visuomotor Jacobian is introduced. Experimental validation results are provided showing how the robot arm can efficiently and stably grasp unknown everyday objects.