Reliable and robust navigation of autonomous mobile robots in indoor environments faces significant challenges due to the absence of GPS, visual degradation, repetitive structures, illumination variations, and low texture. These factors adversely affect localization systems. Current robots often use a uniform navigation approach, regardless of the varying localization uncertainties within different indoor environments. In this paper, we propose a holistic, active vision-based path planning method that produces efficient trajectories, aiming to minimize localization error and enhance navigation performance. Specifically, we utilize a 3D model of an indoor environment to derive an Artificial Potential Field (APF) with its associated localizability scores that encapsulate both visual features’ richness and fiducial markers’ placement. APF is employed to direct a Kinematically Constrained Bi-directional Rapidly Exploring Random Tree (KB-RRT) planner towards the calculation of optimal paths, prioritizing high localization areas. Subsequently, we use an online weight-adaptive MPC-based approach that, apart from robust path planning and obstacle avoidance, guides the robot towards areas with the most robust visual features in order to further refine the localization error. The proposed framework has been extensively tested in both simulation and real-world experiments with a mobile robot in a visually challenging indoor environment.

Automation of vineyards cultivation necessitates for mobile robots to retain accurate localization system. The paper introduces a stereo vision-based Graph-Simultaneous Localization and Mapping (Graph-SLAM) pipeline custom-tailored to the specificities of vineyard fields. Graph-SLAM is reinforced with a Loop Closure Detection (LCD) based on semantic segmentation of the vine trees. The Mask R-CNN network is applied to segment the trunk regions of images, on which unique visual features are extracted. These features are used to populate the bag of visual words (BoVW s) retained on the formulated graph. A nearest neighbor search is applied to each query trunk-image to associate each unique feature descriptor with the corresponding node in the graph using a voting procedure. We apply a probabilistic method to select the most suitable loop closing pair and, upon an LCD appearance, the 3D points of the trunks are employed to estimate the loop closure constraint to the graph. The traceable features on trunk segments drastically reduce the number of retained BoVWs, which in turn expedites significantly the loop closure and graph optimization, rendering our method suitable for large scale mapping in vineyards. The pipeline has been evaluated on several data sequences gathered from real vineyards, in different seasons, when the appearance of vine trees vary significantly, and exhibited robust mapping in long distances.

During the last decades, the research endeavours on autonomous driving found great resonance in Advanced Driver-Assistance Solutions that equipped the contemporary civilian vehicles and significantly boosted their driver-less mobility. The existing applications are mostly focused on urban scenarios where signs, road lanes and markers are well defined and ordered favouring the motion of the vehicles whilst, less attention has been paid to the semi-structured and rural environments where traffic infrastructure is scarce. The paper at hand introduces a holistic framework for autonomous vehicles navigation in semi-structured environments. Semantic cues fused with geometrical information of LiDAR data are used for road detection and tracking. OpenStreetMaps are employed as a rough route planner, the waypoints of which are rectified via a probability distribution function over the visible area of vehicle’s vicinity. Thus, vehicle’s localization is obtained by Normal Distribution Transform (NDT) SLAM, where the covariance of egomotion estimation is obtained by processing short-term 3D maps, fused with GPS measurements by means of an extended Kalman filter. Local planning and execution of vehicle’s motion is applied on local cost maps formulated by the union of 2D laser readings and the detected road boundaries fitted through Bézier curves. The complete framework has been evaluated with the aid of a real Autonomous Guided Vehicle in a constrained semi-structured urban area, exhibiting robust navigation performance.

Automated guided vehicles operation in human populated factory environments is a challenging task, especially when there is a demand to operate without following fixed paths defined by guide wires, magnetic tape, magnets, or transponders embedded in the floor. The paper at hand introduces a vision-based method enabling safe and autonomous operation of pallet moving vehicles that accommodate pallet detection, pose estimation, docking control and pallet pick up in such industrial environments. A dedicated perception topology relying on monocular vision and laser-based measurements has been applied and installed on-board a novel robotic pallet truck. Pallet detection and pose estimation are performed in two steps. Firstly, a deep neural network is used for the fast isolation of pallets' regions of interest and, secondly, model-based geometrical pattern matching on point cloud data is applied to extract the pallet pose. Robot alignment with candidate pallet is performed with a dedicated visual servoing controller. The developed method has been extensively evaluated both in simulated and real industrial environments with the pallet truck and proved to have real-time performance achieving increased accuracy in navigation, pallet detection and pick-up.

As autonomous collaborative robots are more widely used in work environments alongside humans it is of great importance to facilitate the communication between people and robotic systems, in a way that promotes safety and productivity. To this end, we propose an Augmented Reality (AR) based system that allows workers in a human-robot collaborative environment to interact with a robot while also receiving information regarding the robot state and plans that relate to the human’s safety and trust, such as the intended movement of the robotic arm or the navigation plan of the mobile platform. To evaluate the effectiveness of the proposed system we conducted experiments with 13 participants, where two users had to work in the same workspace while being assisted by a mobile manipulator. We measured the task completion time as well as the robot idle time using our AR-based human-robot interaction system and compared them to a conventional setup without the use of augmented reality. Additional, subjective evaluations related to user satisfaction, system usability, perceived safety and trust showed that users assessed the system in a positive way and preferred AR visualization over more traditional interfaces.

The contemporary SLAM mapping systems assume a static environment and build a map that is then used for mobile robot navigation disregarding the dynamic changes in this environment. The paper at hand presents a novel solution for the problem of life-long mapping that continually updates a metric map represented as a 2D occupancy grid in large scale indoor environments with movable objects such as people, robots, objects etc. suitable for industrial applications. We formalize each cell's occupancy as a failure analysis problem and contribute temporal persistence modeling (TPM), an algorithm for probabilistic prediction of the time that a cell in an observed location is expected to be “occupied” or “empty” given sparse prior observations from a task specific mobile robot. Our work is evaluated in Gazebo simulation environment against the nominal occupancy of cells and the estimated obstacles persistence. We also show that robot navigation with life-long mapping demands less replans and leads to more efficient navigation in highly dynamic environments.

Automatic subsurface mapping is essential in the construction services, as it is anticipated to become the main operational environment of the future robots to be realized in the respective domain. Towards this direction, the paper at hand, introduces for the first time herein, an integrated framework for subsurface mapping by exploiting a surface operating mobile robot with a Ground Penetrating Radar (GPR). The mobile robot tows the GPR antenna, which is mounted on a specifically designed trailer, and is utilized as the mean to cover the surface area, while at the same time the antenna scans the subsurface by emitting electromagnetic pulses. The gathered data are processed for the construction of a subsurface 3D map. Specifically, image processing techniques, that involve background segmentation, HOG [1] feature extraction, hypothesis verification and matching are applied on the 2D radargram (B-Scan) for the detection of the salient points that correspond to buried utilities. By employing the pulse propagation velocity into the subsurface and the soil utilities, the salient points are expressed in world coordinates and used for the composition of the 3D subsurface map. Our method has been evaluated on a real test site, accompanied by ground-truth annotation data of experts and revealed remarkable performance, exhibiting not only the feasibility of underground mapping but also the capacity to obtain exploitable results for underground robotic applications.

This video features RAMCIP, a new service robot developed to provide proactive and discreet assistance to elderly with Mild Cognitive Impairments (MCI), supporting their daily activities at home. Starting with a thorough analysis of needs and requirements of the target population, the RAMCIP robot was developed as an integrated ensemble of advanced H/W and S/W components, realizing the robot skills of perception, cognition, safe navigation, grasping, manipulation, and human-robot communication, ample to operate in real, rather challenging domestic environments. The RAMCIP use-cases include proactive assistance provision to user's cooking, eating and medication activities, through discreet user monitoring and robot interventions by reminders and robotic manipulations., RAMCIP can bring the medicine, recognize fallen objects and electric appliance that has been forgotten turned on. It also recognizes the user walking in low-light conditions and turns on the light, as well as detects cases of emergency such as a fall. The robot provides also the user with cognitive training games and stimulates the user to contact with relatives through video-calls. Pilot trials of the RAMCIP robot have been performed in real homes of more than ten different users, in Barcelona, Spain; the video at hand exhibits the robot performing the target use cases.

For a proactive and user-centered robotic assistance and communication, an assistive robot must make decisions about the level of assistance to be provided. Therefore, the robot must be aware of the preferences and the capabilities of the elderly. At the same time, relying on a sensing setup which is totally embedded in the assistive robot would increase its usability. In the framework of the RAMCIP project, a novel skills evaluation methodology has been developed to make the robot aware of the user's perceptual, cognitive and motor skills. This paper presents such a methodology and its preliminary evaluation. Based on a task analysis of the activities for which the robot provides assistance, the user's skills are given a score which is updated at different time scales based on the source of information. Highly reliable information is gathered from caregivers at a low rate by means of a graphical interface hosted by the robot. This information refers to standard medical examinations. Based on the modules for motion tracking, object and activity recognition, specific actions of ADL are selected to update motor skills score at a higher rate, which is typically twice per day. The two sources of information are then fused in a Kalman filter. Preliminary results on the illustrative example of arm precision show that the robot's sensing and cognitive capabilities suffice to obtain a state-of-the-art evaluation of the arm precision skill.

This paper presents a multi-robot system for autonomous vehicle extraction and transportation based on the “a-robot-for-a-wheel” concept. The developed prototype is able to extract vehicles from confined spaces with delicate handling, swiftly and in any direction. The novel lifting robots are capable of omnidirectional movement, thus they can under-ride the desired vehicle and dock to its wheels for a synchronized lifting and extraction. The overall developed system applies reasoning about available trajectory paths, wheel identification, local and undercarriage obstacle detection, in order to fully automate the process. The validity and efficiency of the AVERT robotic system is illustrated via experiments in an indoor parking lot, demonstrating successful autonomous navigation, docking, lifting and transportation of a conventional vehicle.

The congregation of crowd undoubtedly constitutes an important risk factor, which may endanger the safety of the gathered people. The solution reported against this significant threat to citizens safety is to consider careful planning and measures. Thereupon, in this paper, we address the crowd evacuation problem by suggesting an innovative technological solution, namely, the use of mobile robot agents. The contribution of the proposed evacuation system is twofold: (i) it proposes an accurate Cellular Automaton simulation model capable of assessing the human behavior during emergency situations and (ii) it takes advantage of the simulation output to provide sufficient information to the mobile robotic guide, which in turn approaches and redirects a group of people towards a less congestive exit at a time. A custom-made mobile robotic platform was accordingly designed and developed. Last, the performance of the proposed robot guided evacuation model has been examined in real-world scenarios exhibiting significant performance improvement during the crucial first response time window.