Omnidirectional cameras are a suitable and cost-effective choice for Visual Place Recognition (VPR), as they provide comprehensive information from the scene regardless of the robot orientation. However, vision sensors are vulnerable to environmental appearance changes (e.g., illumination, weather, season or moving objects). While multi-modal sensing approaches can overcome these challenges, they introduce significant cost and system complexity. This paper introduces PDPR (Panoramic-Depth Place Recognition), a novel fusion framework that enhances the robustness of VPR methods by integrating visual data with geometric features derived from monocular depth estimation techniques, while using a single-camera setup. In the ablation study, both early and late fusion strategies are evaluated to optimally combine appearance-based and depth-derived features. The extensive evaluation on challenging, indoor and outdoor datasets demonstrates that PDPR consistently boosts retrieval performance across multiple state-of-the-art VPR models. Furthermore, this improvement is achieved without requiring any fine tuning, allowing our method to function as a pluggable module for pretrained models. Consequently, this work presents a powerful, practical and low-cost solution for robust VPR, with high potential to scale as monocular depth estimation and VPR models continue to improve. The project website can be found at https://marcosalfaro.github.io/projects-PDPR/ . • Monocular depth estimation is used to enhance place recognition. • A thorough evaluation of preprocessing techniques to enhance the depth maps. • Fusion techniques are designed to leverage visual and geometric data. • A model-agnostic approach that improves the performance even with no fine tuning. • A robust method across different scenarios and lighting conditions.

Abstract Full spherical views provide advantages in many applications that use visual information. Dual back-to-back fisheye cameras are receiving much attention to obtain this type of view. However, obtaining a high-quality full spherical view is very challenging. In this paper, we propose a correction step that models the relation between the pixels of the pair of fisheye images in polar coordinates. This correction is implemented during the mapping from the unit sphere to the fisheye image using the equidistant fisheye projection. The objective is that the projections of the same point in the pair of images have the same position on the unit sphere after the correction. In this way, they will also have the same position on the equirectangular coordinate system. Consequently, the discontinuity between the spherical views for blending is minimized. Throughout the manuscript, we show that the angular polar coordinates of the same scene point in the fisheye images are related by a sine function and the radial distance coordinates by a linear function. Also, we propose employing a polynomial as a geometric transformation between the pair of spherical views during the image alignment since the relationship between the matching points of pairs of spherical views is not linear, especially in the top/bottom regions. Quantitative evaluations demonstrate that using the correction step improves the quality of the full spherical view, i.e. IQ MS-SSIM, up to 7%. Similarly, using a polynomial improves the IQ MS-SSIM up to 6.29% with respect to using an affine matrix.

Redundant manipulators offer several advantages, including improved manipulability, singularity avoidance, and obstacle evasion. However, kinematic redundancy also introduces additional challenges, such as the need to solve an underdetermined inverse kinematic problem to control the manipulator. This paper introduces a novel approach for motion planning of redundant manipulators, based on the exploration of feasibility maps. The proposed method is an extension of the RRT algorithm, modified to explore the redundant space in order to find a suboptimal feasible path in the joint space, sacrificing optimality for scalability to higher degrees of redundancy. The method is able to follow a given task trajectory while considering other constraints, such as joint limits, self-collisions, and obstacles.

Abstract This work presents a framework to create a visual model of the environment which can be used to estimate the position of a mobile robot by means of artificial intelligence techniques. The proposed framework retrieves the structure of the environment from a dataset composed of omnidirectional images captured along it. These images are described by means of global-appearance approaches. The information is arranged in two layers, with different levels of granularity. The first layer is obtained by means of classifiers and the second layer is composed of a set of data fitting neural networks. Subsequently, the model is used to estimate the position of the robot, in a hierarchical fashion, by comparing the image captured from the unknown position with the information in the model. Throughout this work, five classifiers are evaluated (Naïve Bayes, SVM, random forest, linear discriminant classifier and a classifier based on a shallow neural network) along with three different global-appearance descriptors (HOG, gist, and a descriptor calculated from an intermediate layer of a pre-trained CNN). The experiments have been tackled with some publicly available datasets of omnidirectional images captured indoors with the presence of dynamic changes. Several parameters are used to assess the efficiency of the proposal: the ability of the algorithm to estimate coarsely the position (hit ratio), the average error (cm) and the necessary computing time. The results prove the efficiency of the framework to model the environment and localize the robot from the knowledge extracted from a set of omnidirectional images with the proposed artificial intelligence techniques.

Feature matching is a key technique for a wide variety of computer vision and image processing applications such as visual localization. It permits finding correspondences of significant points within the environment that eventually determine the localization of a mobile agent. In this context, this work evaluates an Adaptive Probability-Oriented Feature Matching (APOFM) method that dynamically models the visual knowledge of the environment in terms of the probability of existence of features. Several improvements are proposed to achieve a more robust matching in a visual odometry framework: a study on the classification of the matching candidates, enhanced by a nearest neighbour search policy; a dynamic weighted matching that exploits the probability of feature existence in order to tune the matching thresholds; and an automatic false positive detector. Additionally, a comparison of performance is carried out, considering a publicly available dataset composed of two kinds of wide field-of-view images: catadioptric and fisheye. Overall, the results validate the appropriateness of these contributions, which outperform other well-recognized implementations within this framework, such as the standard visual odometry, a visual odometry method based on RANSAC, as well as the basic APOFM. The analysis shows that fisheye images provide more visual information of the scene, with more feature candidates. Contrarily, omnidirectional images produce fewer feature candidates, but with higher ratios of feature acceptance. Finally, it is concluded that improved precision is obtained when the location problem is solved by this method.

Singularities produce important changes in the kinetostatic properties of parallel robots, such as the ability to resist external forces with zero actuation torques (or the inability to resist them at all), or losses of control or dexterity. These kinetostatic effects of singularities can be graphically visualized by means of four velocity and force ellipsoids that degenerate when the robot crosses a singularity. This paper presents an educational simulation tool to help students to understand these effects by means of the visualization of the aforementioned ellipses, at the same time that the PID control of the robot is simulated under external forces, in order to demonstrate how singularities affect the control of the robot.

Nowadays education programs in engineering degrees have evolved towards advanced learning models and methodologies, which are either partially or entirely sustained by ICT (Information, Communication, and Technology) resources, and blended approaches. In this sense, electronics courses have become of paramount importance in most education plans within engineering degrees at university. Therefore the adaption to such novel methodologies is increasingly demanded. According to this, we propose an improved teaching program concentrated on the use of an online simulation tool, amongst other digital resources. The program is addressed to students in first levels of engineering degrees, within the framework of the Spanish public university system. In particular, the methodology has been devised through the use of an online circuit simulation applet in Java, which does not require any software installation. The main purpose is to enhance the general achievement of the students, particularizing on their practical competences, digital skills, engagement and motivation towards the learning of electronics, sustained by digital resources such as simulation. A population of 258 students enrolled during the academic year 2017/2018 has been established as a sample for presenting achievement results, surveys data and comparison statistics with other digital resources. Additionally, test groups of roughly 50% out of the total population of students have been established in order to confirm the success of the approach, in contrast to the former teaching methodology. As a result, the approach proves to be an active model which allows the students to develop long-term and autonomous skills in electronics and simulation.

In this work, a clustering approach to obtain compact topological models of an environment is developed and evaluated. The usefulness of these models is tested by studying their utility to solve the robot localization problem subsequently. Omnidirectional visual information and global appearance descriptors are used both to create and compress the models and to estimate the position of the robot. Comparing to the methods based on the extraction and description of landmarks, global appearance approaches permit building models that can be handled and interpreted more intuitively and using relatively straightforward algorithms to estimate the position of the robot. The proposed algorithms are tested with a set of panoramic images captured with a catadioptric vision sensor in a large environment under real working conditions. The results show that it is possible to compress substantially the visual information contained in topological models to arrive to a balance between the computational cost and the accuracy of the localization process.

In this paper a platform to create different kinds of data sets from virtual environments is presented. These data sets contain some information about the visual appearance of the environment and the distance from some reference positions to all the objects. Robot localization and mapping using images are two active fields of research and new algorithms are continuously proposed. These algorithms have to be tested with several sets of images to validate them. This task can be made using actual images; however, sometimes when a change in the parameters of the vision system is needed to optimize the algorithms, this system must be replaced and new data sets must be captured. This supposes a high cost and slowing down the first stages of the development. The objective of this work is to develop a versatile tool that permits generating data sets to test efficiently mapping and localization algorithms with mobile robots. Another advantage of this platform is that the images can be generated from any position of the environment and with any rotation. Besides, the images generated have not noise; this is an advantage since it allows carrying out a preliminary test of the algorithms under ideal conditions. The virtual environment can be created easily and modified depending on the desired characteristics. At last, the platform permits carrying out another advanced tasks using the images and the virtual environment.

This paper presents the workspace analysis of a redundant serial-parallel robot. Due to the complexity of the robot, the complex constraints (joint limits and no-interference between the legs of the robot), and the globally serial structure of the robot, a discretization method based on the forward kinematics would be most appropriate to compute the workspace. However, this widely used method can only obtain the external boundaries of the workspace, missing the internal barriers that hinder the motion of the robot, which may exist inside the boundaries. To avoid missing these barriers, we use a discretization method that uses the solution of the inverse kinematic problem of the robot. By studying the feasibility of attaining a desired position and orientation by the different branches of the solution to the inverse kinematics, the proposed discretization method is able to obtain both the external boundaries and the internal barriers of the workspace. Some examples are presented to show the importance of these internal barriers in the motions of the robot inside the workspace.

This work presents a height estimation method that uses visual information. This method is based on the global appearance of the scenes. Every omnidirectional scene is described with a global appearance descriptor without any other transformation. This approach is tested with our own image database. This database is generated synthetically based on two different virtual rooms. One of the advantages of generating the images synthetically is that noise or occlusions can be added to test the robustness of the algorithms. This database is formed by a set of omnidirectional images captured from different points of these rooms and at different heights. With these scenes we build the descriptor of each image and we use our method to estimate the relative height of the robot. The experimental results show the effectiveness and the robustness of the method.

This paper presents a novel climbing robot that explores 3-D truss structures for maintenance and inspection tasks. The robot is biped and has a hybrid serial-parallel architecture since each leg is composed of two parallel mechanisms connected in series. First, the forward kinematic problem of the complete robot is solved, obtaining the relative position and orientation between the feet in terms of the ten joint coordinates of the robot. The inverse kinematics is more complex due to the redundancy of the robot. Hence, a simplified inverse kinematic problem that assumes planar and symmetric movements is analyzed. Then, a tool to simulate the kinematics of the robot is presented, and it is used to demonstrate that the robot can completely explore 3-D structures, even when some movements are restricted to be planar and symmetric.

This work presents a purely visual topologic odometry system for robot navigation. Our system is based on a Multi-Scale analysis that allows us to estimate the relative displacement between consecutive omnidirectional images. This analysis uses global appearance techniques to describe the scenes. The visual odometry system also makes use of global appearance descriptors of panoramic images to estimate the phase lag between consecutive images and to detect loop closures. When a previous mapped area is recognized during the navigation, the system re-estimates the pose of the scenes included in the map, reducing the error of the path. The algorithm is validated using our own database captured in an indoor environment under real dynamic conditions. The results demonstrate that our system permits estimating the path followed by the robot with accuracy comparing to the real route.

Visual-based navigation has been a source of numerous researches in the field of mobile robotics. In this paper we present a topological map building and localization algorithm using wide-angle scenes. Global-appearance descriptors are used in order to optimally represent the visual information. First, we build a topological graph that represents the navigation environment. Each node of the graph is a different position within the area, and it is composed of a collection of images that covers the complete field of view. We use the information provided by a camera that is mounted on the mobile robot when it travels along some routes between the nodes in the graph. With this aim, we estimate the relative position of each node using the visual information stored. Once the map is built, we propose a localization system that is able to estimate the location of the mobile not only in the nodes but also on intermediate positions using the visual information. The approach has been evaluated and shows good performance in real indoor scenarios under realistic illumination conditions.

In this paper we compare two methods to carry out topological mapping using only visual information captured by a robot. This map should contain enough information so that the robot can estimate its position and orientation and redundant information should be removed to get an acceptable computational cost during the localization process. Apart from this, it is also important to know the topology of the map created since it will make possible a high-level planification of the path to move to the target points. We propose to build this topological map only using the panoramic images taken by an omnidirectional vision system and using appearance-based methods. We have carried out an exhaustive experimentation to study the validity of the proposed methods and to perform an objective comparison between them. Also, we have tested the processing time to create the topological map.

In a multi-robot system, in which each of the robots constructs its own local map, it is necessary to perform the fusion of these maps into a global one. This task is normally performed in two different steps: by aligning the maps and then merging the data. This paper focusses on the first step: Map Alignment, which consists in obtaining the transformation between the local maps built independently. In this way, these local maps will have a common reference frame. In this paper, a collection of algorithms for solving the map alignment are analyzed under different conditions of noise in the data and intersection between local maps. This study is performed in a visual SLAM context, in which the robots construct landmark-based maps. The landmarks consist in 3D points captured from the environment and characterized by a visual descriptor.

This paper presents an approach to vision-based simultaneous localization and mapping (SLAM). Our approach uses the scale invariant feature transform (SIFT) as features and applies a rejection technique to concentrate on a reduced set of distinguishable, stable features. We track detected SIFT features over consecutive frames obtained by a stereo camera and select only those features that appear to be stable from different views. Whenever a feature is selected, we compute a representative feature given the previous observations. This approach is applied within a Rao-Blackwellized particle filter to make the data association easier and furthermore to reduce the number of landmarks that need to be maintained in the map. Our system has been implemented and tested on data gathered with a mobile robot in a typical office environment. Experiments presented in this paper demonstrate that our method improves the data association and in this way leads to more accurate maps

This paper concerns the continuity of vision based robot control law. Recently, we have proposed a way of allowing the temporary disappearance of image features during the control task based on weighted features. Futher more, we have redefined the camera invariant visual servoing approach in order to take into account the change of image features when zooming in or out during a positioning task. In this paper, we study ho to select some parameters of the weight function and we propose a local stability analysis of the invariant visual servoing approach with weighted features. Finally, experimental results demonstrate the improvements that can be achieved in the performance of vision-based control task.

A redundancy resolution technique devoted to grasp synthesis is presented. Given a set of contact points and a certain robot arm and gripper, the goal is to select both the best assignment of gripper fingers to contact points and the best joint values that allow the fingers to reach such contact points. The system proposed is based on the generation of an inverse kinematics tree where fast searches can be performed in order to find the optimum configuration. Optimality is defined as similarity to previously stored examples over a hierarchical structure of configuration data, which includes finger assignments and robot joints.

This paper presents a study of the kinematics and dynamics performance of a climbing parallel robot (CPR) to avoid nodes on structural frames. To avoid a structural node, a CPR can acquire some determined postures. A series of postures can be combined to generate the convenient movements to climb along the structural node. The postures of a CPR must be studied to detect its feasibility, because the postures can drive the robot near its singular configurations. Also, the forces originated in the actuators to evade the structural nodes are evaluated. Therefore, the goal of this paper is to show that a Stewart-Gough (S-G) parallel platform can be used as a climbing robot, because a CPR can avoid structural nodes easily and elegantly, in contrast with other types of robots. To support the simulation results presented in the first part of this paper, an experimental testbed has been developed to study the dynamic performance of the CPR prototype around a structural node. The results obtained are very interesting, and show that an industrial potential to use the parallel S-G robot as a climbing robot exists.

This paper presents a visual tracking system developed for a teleoperation project. The goal of this project is a both hands master-slave system in which the operator needs to see the robot hand (gripper) every time. For this reason, a new and robust algorithm to track the robot hand during its movement is developed. This is a model based tracking so the knowledge of the robot hand CAD model is needed (pose is obtained). This information is used to move a pan-tilt camera and keep the gripper centered in the image using an adaptive fuzzy logic controller. Due to the continuous gripper movement, we need a position predictor to reduce the error. In our case, the extended Kalman filter - EKF is used to do it. Vision based systems have a lot of empirically adjustable parameters for a good working. With the algorithm proposed in this paper, some parameters are auto-adjustable, so the system is easier to use and the robustness is increased.

This work presents a method for identifying real three-dimensional objects in non-controlled backgrounds using independent component analysis to eliminate redundant image information present in each object image. The proposed method is a two-step process that allows a coarse color-based detection and an exact localization using shape information. The paper describes an efficient implementation, making this approach suitable for real-time applications.

Proposes an application of the Stewart-Gough parallel platform as a climbing robot and its kinematics control to climb through long structures describing unknown spatial trajectories, such as palm trunks, tubes, etc. First, the description and design of the climbing parallel robot is presented. Second, the inverse and forward kinematics analysis of a mobile six-degrees-of-freedom parallel robot is described, based on spatial constraint formulation. Finally, the gait pattern and the climbing strategy of the parallel robot is described. The information from this research is being used in an actual climbing parallel robot design at Miguel Hernandez University of Elche (Alicante), Spain.

Automatic quality control examination of banknotes constitutes an important research challenge for the main printing organizations in the world. It is worth to note that presently there is no commercial product with an acceptable performance, in spite of the efforts spent during last ten years. The product complexity lies in the combination of several printings with the purpose of achieving hard to counterfeit banknotes. The printing system that is necessary for banknotes manufacturing, causes significant differences between several printed sheets in similar conditions. However, in spite of these differences, while respecting certain tolerances, the banknotes are accepted. The required quality of the product demands an exhaustive inspection of the bill sheets. Nowadays the quality control is carried out by manual turn over of the bill sheet, reaching a speed of 400 sheets/hour. It has been developed a system that works out an automatic inspection at a rate of 3600 sheets/hour.