This paper presents CrossPlace, an innovative method for cross-modal place recognition between heterogeneous sensor modalities, particularly between fisheye cameras and LiDAR. Place recognition is the fundamental capability of mobile robots to determine their most likely location within a database, based on sensory input queries. In cross-modal place recognition, the goal is to localize using a different sensor from the one originally used to construct the database. The core contribution of this paper is a unified feature space that integrates intensity, depth and semantic information. Both the database entries and the queries are obtained by embedding sensor readings through the same CrossPlace model, ensuring a consistent representation across modalities. Consequently, a database constructed from LiDAR can be queried with fisheye images, and vice versa, using a single shared architecture. Furthermore, a comprehensive data transformation and preprocessing pipeline is presented. Specifically, CrossPlace is constituted by three independent branches, each one for processing intensity, depth and semantic information. Each branch consists of a CosPlace model for image embedding with shared weights across sensor modalities. Late fusion through concatenation of the intensity, depth and semantic embeddings provides optimal global performance. We conduct an exhaustive evaluation on the KITTI-360 dataset, where CrossPlace surpasses state-of-the-art techniques across all metrics, establishing a new standard for cross-modal place recognition in urban and highway environments. The results demonstrate the effectiveness of our unified approach for place recognition across different sensor modalities while maintaining a robust performance under various operating environments.

Omnidirectional cameras are a suitable and cost-effective choice for Visual Place Recognition (VPR), as they provide comprehensive information from the scene regardless of the robot orientation. However, vision sensors are vulnerable to environmental appearance changes (e.g., illumination, weather, season or moving objects). While multi-modal sensing approaches can overcome these challenges, they introduce significant cost and system complexity. This paper introduces PDPR (Panoramic-Depth Place Recognition), a novel fusion framework that enhances the robustness of VPR methods by integrating visual data with geometric features derived from monocular depth estimation techniques, while using a single-camera setup. In the ablation study, both early and late fusion strategies are evaluated to optimally combine appearance-based and depth-derived features. The extensive evaluation on challenging, indoor and outdoor datasets demonstrates that PDPR consistently boosts retrieval performance across multiple state-of-the-art VPR models. Furthermore, this improvement is achieved without requiring any fine tuning, allowing our method to function as a pluggable module for pretrained models. Consequently, this work presents a powerful, practical and low-cost solution for robust VPR, with high potential to scale as monocular depth estimation and VPR models continue to improve. The project website can be found at https://marcosalfaro.github.io/projects-PDPR/ . • Monocular depth estimation is used to enhance place recognition. • A thorough evaluation of preprocessing techniques to enhance the depth maps. • Fusion techniques are designed to leverage visual and geometric data. • A model-agnostic approach that improves the performance even with no fine tuning. • A robust method across different scenarios and lighting conditions.

An optimal solution to the localization problem is essential for developing autonomous robotic systems. Apart from autonomous vehicles, precision agriculture is one of the fields that can benefit most from these systems. Although LiDAR Place Recognition (LPR) is a widely used technique in recent years to achieve accurate localization, it is mostly used in urban settings. However, the lack of distinctive features and the unstructured nature of agricultural environments make place recognition challenging. This work presents a comprehensive review of state-of-the-art deep learning applications for agricultural environments and LPR techniques. We focus on the challenges that arise in these environments. We analyze the existing approaches, datasets, and metrics used to evaluate LPR system performance and discuss the limitations and future directions of research in this field. This is the first survey that focuses on LiDAR-based localization in agricultural settings, with the aim of providing a thorough understanding and fostering further research in this specialized domain.

Abstract Full spherical views provide advantages in many applications that use visual information. Dual back-to-back fisheye cameras are receiving much attention to obtain this type of view. However, obtaining a high-quality full spherical view is very challenging. In this paper, we propose a correction step that models the relation between the pixels of the pair of fisheye images in polar coordinates. This correction is implemented during the mapping from the unit sphere to the fisheye image using the equidistant fisheye projection. The objective is that the projections of the same point in the pair of images have the same position on the unit sphere after the correction. In this way, they will also have the same position on the equirectangular coordinate system. Consequently, the discontinuity between the spherical views for blending is minimized. Throughout the manuscript, we show that the angular polar coordinates of the same scene point in the fisheye images are related by a sine function and the radial distance coordinates by a linear function. Also, we propose employing a polynomial as a geometric transformation between the pair of spherical views during the image alignment since the relationship between the matching points of pairs of spherical views is not linear, especially in the top/bottom regions. Quantitative evaluations demonstrate that using the correction step improves the quality of the full spherical view, i.e. IQ MS-SSIM, up to 7%. Similarly, using a polynomial improves the IQ MS-SSIM up to 6.29% with respect to using an affine matrix.

Abstract This work presents a framework to create a visual model of the environment which can be used to estimate the position of a mobile robot by means of artificial intelligence techniques. The proposed framework retrieves the structure of the environment from a dataset composed of omnidirectional images captured along it. These images are described by means of global-appearance approaches. The information is arranged in two layers, with different levels of granularity. The first layer is obtained by means of classifiers and the second layer is composed of a set of data fitting neural networks. Subsequently, the model is used to estimate the position of the robot, in a hierarchical fashion, by comparing the image captured from the unknown position with the information in the model. Throughout this work, five classifiers are evaluated (Naïve Bayes, SVM, random forest, linear discriminant classifier and a classifier based on a shallow neural network) along with three different global-appearance descriptors (HOG, gist, and a descriptor calculated from an intermediate layer of a pre-trained CNN). The experiments have been tackled with some publicly available datasets of omnidirectional images captured indoors with the presence of dynamic changes. Several parameters are used to assess the efficiency of the proposal: the ability of the algorithm to estimate coarsely the position (hit ratio), the average error (cm) and the necessary computing time. The results prove the efficiency of the framework to model the environment and localize the robot from the knowledge extracted from a set of omnidirectional images with the proposed artificial intelligence techniques.

Feature matching is a key technique for a wide variety of computer vision and image processing applications such as visual localization. It permits finding correspondences of significant points within the environment that eventually determine the localization of a mobile agent. In this context, this work evaluates an Adaptive Probability-Oriented Feature Matching (APOFM) method that dynamically models the visual knowledge of the environment in terms of the probability of existence of features. Several improvements are proposed to achieve a more robust matching in a visual odometry framework: a study on the classification of the matching candidates, enhanced by a nearest neighbour search policy; a dynamic weighted matching that exploits the probability of feature existence in order to tune the matching thresholds; and an automatic false positive detector. Additionally, a comparison of performance is carried out, considering a publicly available dataset composed of two kinds of wide field-of-view images: catadioptric and fisheye. Overall, the results validate the appropriateness of these contributions, which outperform other well-recognized implementations within this framework, such as the standard visual odometry, a visual odometry method based on RANSAC, as well as the basic APOFM. The analysis shows that fisheye images provide more visual information of the scene, with more feature candidates. Contrarily, omnidirectional images produce fewer feature candidates, but with higher ratios of feature acceptance. Finally, it is concluded that improved precision is obtained when the location problem is solved by this method.

Singularities produce important changes in the kinetostatic properties of parallel robots, such as the ability to resist external forces with zero actuation torques (or the inability to resist them at all), or losses of control or dexterity. These kinetostatic effects of singularities can be graphically visualized by means of four velocity and force ellipsoids that degenerate when the robot crosses a singularity. This paper presents an educational simulation tool to help students to understand these effects by means of the visualization of the aforementioned ellipses, at the same time that the PID control of the robot is simulated under external forces, in order to demonstrate how singularities affect the control of the robot.

Nowadays education programs in engineering degrees have evolved towards advanced learning models and methodologies, which are either partially or entirely sustained by ICT (Information, Communication, and Technology) resources, and blended approaches. In this sense, electronics courses have become of paramount importance in most education plans within engineering degrees at university. Therefore the adaption to such novel methodologies is increasingly demanded. According to this, we propose an improved teaching program concentrated on the use of an online simulation tool, amongst other digital resources. The program is addressed to students in first levels of engineering degrees, within the framework of the Spanish public university system. In particular, the methodology has been devised through the use of an online circuit simulation applet in Java, which does not require any software installation. The main purpose is to enhance the general achievement of the students, particularizing on their practical competences, digital skills, engagement and motivation towards the learning of electronics, sustained by digital resources such as simulation. A population of 258 students enrolled during the academic year 2017/2018 has been established as a sample for presenting achievement results, surveys data and comparison statistics with other digital resources. Additionally, test groups of roughly 50% out of the total population of students have been established in order to confirm the success of the approach, in contrast to the former teaching methodology. As a result, the approach proves to be an active model which allows the students to develop long-term and autonomous skills in electronics and simulation.

In this work, a clustering approach to obtain compact topological models of an environment is developed and evaluated. The usefulness of these models is tested by studying their utility to solve the robot localization problem subsequently. Omnidirectional visual information and global appearance descriptors are used both to create and compress the models and to estimate the position of the robot. Comparing to the methods based on the extraction and description of landmarks, global appearance approaches permit building models that can be handled and interpreted more intuitively and using relatively straightforward algorithms to estimate the position of the robot. The proposed algorithms are tested with a set of panoramic images captured with a catadioptric vision sensor in a large environment under real working conditions. The results show that it is possible to compress substantially the visual information contained in topological models to arrive to a balance between the computational cost and the accuracy of the localization process.

In this paper a platform to create different kinds of data sets from virtual environments is presented. These data sets contain some information about the visual appearance of the environment and the distance from some reference positions to all the objects. Robot localization and mapping using images are two active fields of research and new algorithms are continuously proposed. These algorithms have to be tested with several sets of images to validate them. This task can be made using actual images; however, sometimes when a change in the parameters of the vision system is needed to optimize the algorithms, this system must be replaced and new data sets must be captured. This supposes a high cost and slowing down the first stages of the development. The objective of this work is to develop a versatile tool that permits generating data sets to test efficiently mapping and localization algorithms with mobile robots. Another advantage of this platform is that the images can be generated from any position of the environment and with any rotation. Besides, the images generated have not noise; this is an advantage since it allows carrying out a preliminary test of the algorithms under ideal conditions. The virtual environment can be created easily and modified depending on the desired characteristics. At last, the platform permits carrying out another advanced tasks using the images and the virtual environment.

This paper presents the workspace analysis of a redundant serial-parallel robot. Due to the complexity of the robot, the complex constraints (joint limits and no-interference between the legs of the robot), and the globally serial structure of the robot, a discretization method based on the forward kinematics would be most appropriate to compute the workspace. However, this widely used method can only obtain the external boundaries of the workspace, missing the internal barriers that hinder the motion of the robot, which may exist inside the boundaries. To avoid missing these barriers, we use a discretization method that uses the solution of the inverse kinematic problem of the robot. By studying the feasibility of attaining a desired position and orientation by the different branches of the solution to the inverse kinematics, the proposed discretization method is able to obtain both the external boundaries and the internal barriers of the workspace. Some examples are presented to show the importance of these internal barriers in the motions of the robot inside the workspace.

This work presents a height estimation method that uses visual information. This method is based on the global appearance of the scenes. Every omnidirectional scene is described with a global appearance descriptor without any other transformation. This approach is tested with our own image database. This database is generated synthetically based on two different virtual rooms. One of the advantages of generating the images synthetically is that noise or occlusions can be added to test the robustness of the algorithms. This database is formed by a set of omnidirectional images captured from different points of these rooms and at different heights. With these scenes we build the descriptor of each image and we use our method to estimate the relative height of the robot. The experimental results show the effectiveness and the robustness of the method.

This work presents a purely visual topologic odometry system for robot navigation. Our system is based on a Multi-Scale analysis that allows us to estimate the relative displacement between consecutive omnidirectional images. This analysis uses global appearance techniques to describe the scenes. The visual odometry system also makes use of global appearance descriptors of panoramic images to estimate the phase lag between consecutive images and to detect loop closures. When a previous mapped area is recognized during the navigation, the system re-estimates the pose of the scenes included in the map, reducing the error of the path. The algorithm is validated using our own database captured in an indoor environment under real dynamic conditions. The results demonstrate that our system permits estimating the path followed by the robot with accuracy comparing to the real route.

Visual-based navigation has been a source of numerous researches in the field of mobile robotics. In this paper we present a topological map building and localization algorithm using wide-angle scenes. Global-appearance descriptors are used in order to optimally represent the visual information. First, we build a topological graph that represents the navigation environment. Each node of the graph is a different position within the area, and it is composed of a collection of images that covers the complete field of view. We use the information provided by a camera that is mounted on the mobile robot when it travels along some routes between the nodes in the graph. With this aim, we estimate the relative position of each node using the visual information stored. Once the map is built, we propose a localization system that is able to estimate the location of the mobile not only in the nodes but also on intermediate positions using the visual information. The approach has been evaluated and shows good performance in real indoor scenarios under realistic illumination conditions.

In this paper we compare two methods to carry out topological mapping using only visual information captured by a robot. This map should contain enough information so that the robot can estimate its position and orientation and redundant information should be removed to get an acceptable computational cost during the localization process. Apart from this, it is also important to know the topology of the map created since it will make possible a high-level planification of the path to move to the target points. We propose to build this topological map only using the panoramic images taken by an omnidirectional vision system and using appearance-based methods. We have carried out an exhaustive experimentation to study the validity of the proposed methods and to perform an objective comparison between them. Also, we have tested the processing time to create the topological map.

In a multi-robot system, in which each of the robots constructs its own local map, it is necessary to perform the fusion of these maps into a global one. This task is normally performed in two different steps: by aligning the maps and then merging the data. This paper focusses on the first step: Map Alignment, which consists in obtaining the transformation between the local maps built independently. In this way, these local maps will have a common reference frame. In this paper, a collection of algorithms for solving the map alignment are analyzed under different conditions of noise in the data and intersection between local maps. This study is performed in a visual SLAM context, in which the robots construct landmark-based maps. The landmarks consist in 3D points captured from the environment and characterized by a visual descriptor.