Visual Simultaneous Localization and Mapping (vSLAM)is the method of employing an optical sensor to map the robot’s observable surroundings while also identifying the robot’s pose in relation to that map. The accuracy and speed of vSLAM calculations can have a very significant impact on the performance and effectiveness of subsequent tasks that need to be executed by the robot, making it a key building component for current robotic designs. The application of vSLAM in the area of humanoid robotics is particularly difficult due to the robot’s unsteady locomotion. This paper introduces a pose graph optimization module based on RGB (ORB) features, as an extension of the KinectFusion pipeline (a well-known vSLAM algorithm), to assist in recovering the robot’s stance during unstable gait patterns when the KinectFusion tracking system fails. We develop and test a wide range of embedded MPSoC FPGA designs, and we investigate numerous architectural improvements, both precise and approximation, to study their impact on performance and accuracy. Extensive design space exploration reveals that properly designed approximations, which exploit domain knowledge and efficient management of CPU and FPGA fabric resources, enable real-time vSLAM at more than 30 fps in humanoid robots with high energy-efficiency and without compromising robot tracking and map construction. This is the first FPGA design to achieve robust, real-time dense SLAM operation targeting specifically humanoid robots. An open source release of our implementations and data can be found in [ 1 ].

Visual Simultaneous Localization and Mapping (vSLAM) is the process of mapping the robot's observed environment using an optical sensor, while concurrently determining the robot's pose with respect to that map. For humanoid robots, the implementation of vSLAM is particularly challenging, due to the intricate motions of the robot. In this work, we present a pose graph optimization module based on RGB features, as an extension on the KinectFusion pipeline (a well-known vSLAM algorithm), to help recover the robot's pose during unstable gait patterns where the KinectFusion tracking system fails. We implement and evaluate a plethora of embedded MPSoC FPGA designs and we explore several architectural optimizations, both precise and approximate, highlighting their effect on performance and accuracy. Properly designed approximations, which exploit domain knowledge and efficient management of CPU and FPGA fabric resources, enable real-time vSLAM (at more than 30 fps) in humanoid robots without compromising robot tracking and map construction. We show that a combination of precise and approximate optimizations and tuning of algorithmic parameters provide a speedup of up to 15.7X and 22.5X compared with the precise FPGA and ARM-only implementations, respectively, without violating the tight accuracy constraints.

In the next few years, new demanding applications will be supported on mobile platforms by reconciling two conflicting requirements: high performance (often with real-time limitations) and low power consumption. The objective of the vipGPU project is to develop hardware and software technology to provide efficient support for two such application scenarios, namely (a) simultaneous localization and mapping (SLAM) in mobile robotics systems, and (b) virtual reality (VR) in portable devices to simulate serious games with emphasis on simulating surgical interventions and medical training in general. In this project, we aim at developing a new heterogeneous platform consisting of hardware accelerators for low power embedded systems optimized (at the hardware and software level) for the implementation of the two applications mentioned above.

In the current paper we investigate the challenges of localizing walking humanoid robots using Visual SLAM (VSLAM). We propose a novel dense RGB-D SLAM framework that seamlessly integrates with the dynamic state of a humanoid, to provide real-time localization and dense mapping of its surroundings. Following the path of recent research in humanoid localization, in the current work we explore the integration between a VSLAM system and the humanoid state, by considering the gait cycle and the feet contacts. We analyze how these effects undermine the quality of data acquisition and association for VSLAM, by capturing the unilateral ground forces at the robot’s feet, and design a system that mitigates their impact.We evaluate our framework on both open and closed-loop bipedal gaits, using a low-cost humanoid platform, and demonstrate that it outperforms kinematic odometry and state-of-the-art dense RGB-D VSLAM methods, by continuously localizing the robot, even in the face of highly irregular and unstable motions.

The capacity of a system to automatically analyze and predict the performance of a human in a particular task can provide important information in Human-Robot Interaction. Despite its usefulness, the above topic has received rather limited attention in the literature. In the current work, we introduce a method for performance prediction and profiling of human activities. Using little information about a task, our method is able to extract the characteristic motion patterns of an agent, analyze them and predict his/her performance in a given activity. We demonstrate the robustness of the method in several different activities, that involve both periodic and oscillatory primitive motions. In addition, we evaluate it thoroughly on data obtained from public datasets and discuss its usefulness for contemporary robotic applications.

The ability to predict the duration of an activity can enable a robot to plan its behaviors ahead, interact seamlessly with other humans, by coordinating its actions, and allocate effort and resources to tasks that are time-constrained or critical. Despite its usefulness, models that examine the temporal properties of an activity remain relatively unexplored. In the current paper we present, to the best of our knowledge, the first method that can estimate temporal properties of an activity by observation. We evaluate it on three use-cases (i) wiping a table, (ii) chopping vegetables and (iii) cleaning the floor, using ground truth data from real demonstrations, and show that it can make predictions with high accuracy and little training. In addition, we investigate different methods to approximate the progress of each task, and demonstrate how a model can generalize, by reusing part of it in different activities.

Visual Odometry (VO) is a proven technology for planetary exploration rovers, facilitating their localization with a small error over medium-sized trajectories. However, due to VO's incremental mode of operation, its estimation error accumulates over time, resulting in considerable drift for long trajectories. This paper proposes a global localization method that counters VO drift by matching boulders extracted from overhead and ground images and using them periodically to relocalize the rover and refine VO estimates. The performance of the proposed method is evaluated with the aid of overhead imagery of different resolutions. Experimental results demonstrate that a very terse representation, consisting of approximate boulder locations only, suffices for significantly improving the accuracy of VO over long traverses.

Computational modeling of natural systems can be used for interdisciplinary applications, such as the configuration of robotic systems or the validation of biological ones. Up to date there has been a little progress on suggesting a framework for automating the process of creating a computational model for biological processes. Instead researchers focus on the implementations of systems that are intended to replicate a tight set of biological behaviors. Such framework should be able to construct any system based on the appropriate level of abstraction chosen by the designer, as well as be able to enforce the appropriate biological consistency without compromising on performance or scalability of the generated models. In this paper we propose a framework that can automate the construction of computational models using genetic algorithms and demonstrate how this framework can construct a model of the parieto-frontal and premotor regions involved in grasping.

The control of the hand in primate species is characterized by a high dimensionality, due to the large number of joints in the fingers. In this study we present how its manipulation can be simplified without compromising its usage, through a constraint methodology that is inspired from recent neurobiological findings. We further develop a computational model, consisting of several brain areas related to hand control, using a co-evolutionary architecture. Due to its neurobiological basis the methodology gives rise to a number of emergent properties that have been shown to occur in primate species during reach-to-grasp tasks.

Recent brain imaging studies on primates revealed that a network of brain areas is activated both during observation and during execution of movements. The present work aims at modeling this group of areas, implementing a distributed computational system. The modeling process follows the agent-based coevolutionary framework that is very effective in terms of designing complex distributed systems addressing successfully the multi-modality of the interacting regions. The implemented model is successfully embedded in a simulated humanoid robot, replicating existing biological findings.

Neural networks have been proposed as an ideal cognitive modeling methodology to deal with the symbol grounding problem. More recently, such neural network approaches have been incorporated in studies based on cognitive agents and robots. In this paper we present a new model of symbol grounding transfer in cognitive robots. Language learning simulations demonstrate that robots are able to acquire new action concepts via linguistic instructions. This is achieved by autonomously transferring the grounding from directly grounded action names to new higher-order composite actions. The robot's neural network controller permits such a grounding transfer. The implications for such a modeling approach in cognitive science and autonomous robotics are discussed.