As legged robots continue to evolve, new control methods are being developed to provide fast, robust, accurate and computationally efficient algorithms for traversing challenging environments. This paper presents a realtime adaptive locomotion controller for quadrupeds, designed to maintain stability and controllability on various surfaces, including highly slippery terrains. The proposed approach optimizes control effort distribution based on the probability of slippage by utilizing a surface-independent adaptation layer. By balancing the robot's redundant kinematic system through rank relaxation-similar to loosening constraints in optimization problems-this method demonstrates significant performance improvements. Unlike Reinforcement Learning (RL) approaches, which depend on pre-trained policies and may struggle to adapt velocity tracking control across different terrains, our method rapidly adjusts to changing conditions, as validated by extensive simulation experiments.

In this work, a control scheme for human-robot collaborative object transportation is proposed, considering a quadruped robot equipped with the MIGHTY suction cup that serves both as a gripper for holding the object and a force/torque sensor. The proposed control scheme is based on the notion of admittance control, and incorporates a variable damping term aiming towards increasing the controllability of the human and, at the same time, decreasing her/his effort. Furthermore, to ensure that the object is not detached from the suction cup during the collaboration, an additional control signal is proposed, which is based on a barrier artificial potential. The proposed control scheme is proven to be passive and its performance is demonstrated through experimental evaluations conducted using the Unitree Go1 robot equipped with the MIGHTY suction cup.

Anomaly detection is a fundamental task for time-series analytics with important implications for the downstream performance of many applications. Despite increasing academic interest and the large number of methods proposed in the literature, recent benchmark and evaluation studies demonstrated that there exists no single best anomaly detection method when applied to heterogeneous time series datasets. Therefore, the only scalable and viable solution to solve anomaly detection over very different time series collected from diverse domains is to propose a model selection method that will choose, based on time series characteristics, the best anomaly detection method to run. This paper describes ADecimo, a modular and extensible web application that helps users understand the performance of time series classification algorithms used as model selection methods for time series anomaly detection. Overall, our system enables users to compare 17 different classifiers over 1980 time series, and decide on the most suitable time series classification method for their own time series and use cases.

Nowadays, robotic arms are ubiquitously employed for object manipulation across a spectrum of applications, spanning from production lines to warehouses, and encompassing both stationary and mobile robotic systems. Among the most prevalent end-effectors, used for the majority of these applications, are suction cups. The rudimentary act of grasping an object and relocating it, devoid of a cognizant awareness of the forces stemming from the object’s motion and grip, can result in suboptimal and inefficient robot movements. In more dire circumstances, such negligent handling may precipitate detachment of the object from the end-effector, potentially incurring damage to either the object or the arm.In this paper, we build upon the advanced sensing and attaching capabilities of our suction cup MIGHTY, and introduce GEM-C, a novel Gravity, External forces and Motion Compensation controller, that constantly adapts the orientation of the suction cup so as to enhance the quality of attachment. Throughout all examined scenarios and experiments, our approach remarkably improved the robot’s performance by providing the optimal end-effector pose while also reducing the stress on the motors and the overall power consumption. The derived results, clearly demonstrate the MIGHTY and GEM-C schema’s potential for a wide range of demanding robotic manipulation tasks.

Legged robot navigation in unstructured and slippery terrains depends heavily on the ability to accurately identify the quality of contact between the robot's feet and the ground. Contact state estimation is regarded as a challenging problem and is typically addressed by exploiting force measurements, joint encoders and/or robot kinematics and dynamics. In contrast to most state of the art approaches, the current work introduces a novel probabilistic method for estimating the contact state based solely on proprioceptive sensing, as it is readily available by Inertial Measurement Units (IMUs) mounted on the robot's end effectors. Capitalizing on the uncertainty of IMU measurements, our method estimates the probability of stable contact. This is accomplished by approximating the multimodal probability density function over a batch of data points for each axis of the IMU with Kernel Density Estimation. The proposed method has been extensively assessed against both real and simulated scenarios on bipedal and quadrupedal robotic platforms such as ATLAS, TALOS and Unitree's GO1.

The spectrum of applications of robotic systems is constantly being expanded in the research, industrial and even the defense sectors, ranging from manipulation and assembly to critical infrastructure monitoring and post-disaster response. Nevertheless, contemporary robotic capabilities are significantly hindered when traversing through or interacting with complex, unstructured and dynamic environments. To alleviate for that, we introduce in the current work MIGHTY (Multi-functional Intelligent Gripping with High Tolerance), a novel, lightweight, sensor-enhanced vacuum suction cup providing not only enhanced attachment capabilities on a plethora of surfaces of varying roughness, but also robust and accurate contact and force estimation. Its unique design facilitates multiple functionalities, from acting as a gripper for object manipulation to operating as a foot for stable walking and steep surface climbing. The proposed suction cup was extensively assessed under varying experimental setups, in order to validate its capacity to sense the applied force and torque and the torque's axis, as well as its ability to attach on a variety of surfaces. In all cases remarkable results were demonstrated, attesting for its effectiveness and robustness.

Anomaly detection is a fundamental task for time-series analytics with important implications for the downstream performance of many applications. Despite increasing academic interest and the large number of methods proposed in the literature, recent benchmark and evaluation studies demonstrated that no overall best anomaly detection methods exist when applied to very heterogeneous time series datasets. Therefore, the only scalable and viable solution to solve anomaly detection over very different time series collected from diverse domains is to propose a model selection method that will select, based on time series characteristics, the best anomaly detection method to run. Existing AutoML solutions are, unfortunately, not directly applicable to time series anomaly detection, and no evaluation of time series-based approaches for model selection exists. Towards that direction, this paper studies the performance of time series classification methods used as model selection for anomaly detection. Overall, we compare 17 different classifiers over 1800 time series, and we propose the first extensive experimental evaluation of time series classification as model selection for anomaly detection. Our results demonstrate that model selection methods outperform every single anomaly detection method while being in the same order of magnitude regarding execution time. This evaluation is the first step to demonstrate the accuracy and efficiency of time series classification algorithms for anomaly detection, and represents a strong baseline that can then be used to guide the model selection step in general AutoML pipelines.

Visual Simultaneous Localization and Mapping (vSLAM)is the method of employing an optical sensor to map the robot’s observable surroundings while also identifying the robot’s pose in relation to that map. The accuracy and speed of vSLAM calculations can have a very significant impact on the performance and effectiveness of subsequent tasks that need to be executed by the robot, making it a key building component for current robotic designs. The application of vSLAM in the area of humanoid robotics is particularly difficult due to the robot’s unsteady locomotion. This paper introduces a pose graph optimization module based on RGB (ORB) features, as an extension of the KinectFusion pipeline (a well-known vSLAM algorithm), to assist in recovering the robot’s stance during unstable gait patterns when the KinectFusion tracking system fails. We develop and test a wide range of embedded MPSoC FPGA designs, and we investigate numerous architectural improvements, both precise and approximation, to study their impact on performance and accuracy. Extensive design space exploration reveals that properly designed approximations, which exploit domain knowledge and efficient management of CPU and FPGA fabric resources, enable real-time vSLAM at more than 30 fps in humanoid robots with high energy-efficiency and without compromising robot tracking and map construction. This is the first FPGA design to achieve robust, real-time dense SLAM operation targeting specifically humanoid robots. An open source release of our implementations and data can be found in [ 1 ].

Visual Simultaneous Localization and Mapping (vSLAM) is the process of mapping the robot's observed environment using an optical sensor, while concurrently determining the robot's pose with respect to that map. For humanoid robots, the implementation of vSLAM is particularly challenging, due to the intricate motions of the robot. In this work, we present a pose graph optimization module based on RGB features, as an extension on the KinectFusion pipeline (a well-known vSLAM algorithm), to help recover the robot's pose during unstable gait patterns where the KinectFusion tracking system fails. We implement and evaluate a plethora of embedded MPSoC FPGA designs and we explore several architectural optimizations, both precise and approximate, highlighting their effect on performance and accuracy. Properly designed approximations, which exploit domain knowledge and efficient management of CPU and FPGA fabric resources, enable real-time vSLAM (at more than 30 fps) in humanoid robots without compromising robot tracking and map construction. We show that a combination of precise and approximate optimizations and tuning of algorithmic parameters provide a speedup of up to 15.7X and 22.5X compared with the precise FPGA and ARM-only implementations, respectively, without violating the tight accuracy constraints.

In this article, we propose a deep learning frame-work that provides a unified approach to the problem of leg contact detection in humanoid robot walking gaits. Our formulation accomplishes to accurately and robustly estimate the contact state probability for each leg (i.e., stable or slip/no contact). The proposed framework employs solely propriocep-tive sensing and although it relies on simulated ground-truth contact data for the classification process, we demonstrate that it generalizes across varying friction surfaces and different legged robotic platforms and, at the same time, is readily transferred from simulation to practice. The framework is quantitatively and qualitatively assessed in simulation via the use of ground-truth contact data and is contrasted against state-of-the-art methods with an ATLAS, a NAO, and a TALOS humanoid robot. Furthermore, its efficacy is demonstrated in base estimation with a real TALOS humanoid. To reinforce further research endeavors, our implementation is offered as an open-source ROS/Python package, coined Legged Contact Detection (LCD).

In the next few years, new demanding applications will be supported on mobile platforms by reconciling two conflicting requirements: high performance (often with real-time limitations) and low power consumption. The objective of the vipGPU project is to develop hardware and software technology to provide efficient support for two such application scenarios, namely (a) simultaneous localization and mapping (SLAM) in mobile robotics systems, and (b) virtual reality (VR) in portable devices to simulate serious games with emphasis on simulating surgical interventions and medical training in general. In this project, we aim at developing a new heterogeneous platform consisting of hardware accelerators for low power embedded systems optimized (at the hardware and software level) for the implementation of the two applications mentioned above.

In the current paper we investigate the challenges of localizing walking humanoid robots using Visual SLAM (VSLAM). We propose a novel dense RGB-D SLAM framework that seamlessly integrates with the dynamic state of a humanoid, to provide real-time localization and dense mapping of its surroundings. Following the path of recent research in humanoid localization, in the current work we explore the integration between a VSLAM system and the humanoid state, by considering the gait cycle and the feet contacts. We analyze how these effects undermine the quality of data acquisition and association for VSLAM, by capturing the unilateral ground forces at the robot’s feet, and design a system that mitigates their impact.We evaluate our framework on both open and closed-loop bipedal gaits, using a low-cost humanoid platform, and demonstrate that it outperforms kinematic odometry and state-of-the-art dense RGB-D VSLAM methods, by continuously localizing the robot, even in the face of highly irregular and unstable motions.

Contact detection is an important topic in contemporary humanoid robotic research. Up to date control and state estimation schemes readily assume that feet contact status is known in advance. In this work, we elaborate on a broader question: in which gait phase is the robot currently in? We introduce an unsupervised learning framework for gait phase estimation based solely on proprioceptive sensing, namely joint encoder, inertial measurement unit and force/torque data. Initially, a meaningful physical explanation on data acquisition is presented. Subsequently, dimensionality reduction is performed to obtain a compact low-dimensional feature representation followed by clustering into three groups, one for each gait phase. The proposed framework is qualitatively and quantitatively assessed in simulation with ground-truth data of uneven/rough terrain walking gaits and insights about the latent gait phase dynamics are drawn. Additionally, its efficacy and robustness is demonstrated when incorporated in leg odometry computation. Since our implementation is based on sensing that is commonly available on humanoids today, we release an open-source ROS/Python package to reinforce further research endeavors.

The capacity of a system to automatically analyze and predict the performance of a human in a particular task can provide important information in Human-Robot Interaction. Despite its usefulness, the above topic has received rather limited attention in the literature. In the current work, we introduce a method for performance prediction and profiling of human activities. Using little information about a task, our method is able to extract the characteristic motion patterns of an agent, analyze them and predict his/her performance in a given activity. We demonstrate the robustness of the method in several different activities, that involve both periodic and oscillatory primitive motions. In addition, we evaluate it thoroughly on data obtained from public datasets and discuss its usefulness for contemporary robotic applications.

Contemporary humanoids are equipped with visual and LiDAR sensors that are effectively utilized for Visual Odometry (VO) and LiDAR Odometry (LO). Unfortunately, such measurements commonly suffer from outliers in a dynamic environment, since frequently it is assumed that only the robot is in motion and the world is static. To this end, robust state estimation schemes are mandatory in order for humanoids to symbiotically co-exist with humans in their daily dynamic environments. In this article, the robust Gaussian Error-State Kalman Filter for humanoid robot locomotion is presented. The introduced method automatically detects and rejects outliers without relying on any prior knowledge on measurement distributions or finely tuned thresholds. Subsequently, the proposed method is quantitatively and qualitatively assessed in realistic conditions with the full-size humanoid robot WALK-MAN v2.0 and the mini-size humanoid robot NAO to demonstrate its accuracy and robustness when outlier VOLO measurements are present. Finally, in order to reinforce further research endeavours, our implementation is released as an open-source ROS/C++package.

The ability to predict the duration of an activity can enable a robot to plan its behaviors ahead, interact seamlessly with other humans, by coordinating its actions, and allocate effort and resources to tasks that are time-constrained or critical. Despite its usefulness, models that examine the temporal properties of an activity remain relatively unexplored. In the current paper we present, to the best of our knowledge, the first method that can estimate temporal properties of an activity by observation. We evaluate it on three use-cases (i) wiping a table, (ii) chopping vegetables and (iii) cleaning the floor, using ground truth data from real demonstrations, and show that it can make predictions with high accuracy and little training. In addition, we investigate different methods to approximate the progress of each task, and demonstrate how a model can generalize, by reusing part of it in different activities.

Efficient storing and managing of robot's experiences is of utmost importance in long-term recurring HumanRobot Interaction scenarios, where the volume of information increases constantly. To address these issues, a novel entity-based episodic memory is introduced in this work. Knowledge is represented by hierarchical multigraphs enabling for fast information retrieval. Consisting entities are asynchronously updated, while a time-correlated importance factor modulates the merging, forgetting or refreshing of memories, in order to facilitate search and management of the stored information. An HMM-based probabilistic inference is employed to infer or predict the HRI state or to identify abnormal scenario unfolding and, thus, guide future robot activities. The performance of the employed memory schema is assessed on both simulated and real “breakfast preparation” scenarios. The results indicate that the proposed memory model is able to efficiently store and manage the acquired data without any loss of critical information. Moreover, our approach was also shown capable of successfully inferring user's hidden preferences and thus guiding robot behavior accordingly in order to improve user's HRI experience.

Learning from Demonstration (LfD) is addressed in this work in order to establish a novel framework for Human-Robot Collaborative (HRC) task execution. In this context, a robotic system is trained to perform various actions by observing a human demonstrator. We formulate a latent representation of observed behaviors and associate this representation with the corresponding one for target robotic behaviors. Effectively, a mapping of observed to performed actions is defined, that abstracts action variations and differences between the human and robotic manipulators, and facilitates execution of newly-observed actions. The learned action-behaviors are then employed to accomplish task execution in an HRC scenario. Experimental results obtained regard the successful training of a robotic arm with various action behaviors and its subsequent deployment in HRC task accomplishment. The latter demonstrate the validity and efficacy of the proposed approach in human-robot collaborative setups.

We present a novel neural episodic memory architecture that utilizes reservoir computing to extract and recall information gleaned over time from a multilayer perceptron that receives sensory input. Reservoir computing models project input data into a high-dimensional dynamical space and also serve as a fading memory that holds on to past inputs thereby enabling the direct association of the current input with the past. The architecture presented utilizes these capabilities via an abstract feedback mechanism and in doing so creates attractor-like states within the reservoir that are associated with each discrete memory and associates these states and therefore memories over time into episodes. In addition, the feedback mechanism provides stabilization to an otherwise chaotic complex dynamical system.

Recent introduction of low-cost depth cameras triggered a number of interesting works, pushing forward the state-of-the-art in human body pose extraction and tracking. However, despite the remarkable progress, many of the contemporary methods cope inadequately with complex scenarios, involving multiple interacting users, under the presence of severe inter- and intra-occlusions. In this work, we present a model-based approach for markerless articulated full body pose extraction and tracking in RGB-D sequences. A cylinder-based model is employed to represent the human body. For each body part a set of hypotheses is generated and tracked over time by a Particle Filter. To evaluate each hypothesis, we employ a novel metric that considers the reprojected Top View of the corresponding body part. The latter, in conjunction with depth information, effectively copes with difficult and ambiguous cases, such as severe occlusions. For evaluation purposes, we conducted several series of experiments using data from a public human action database, as well as own-collected data involving varying number of interacting users. The performance of the proposed method has been further compared against that of the Microsoft's Kinect SDK and NiTE (TM) using ground truth information. The results obtained attest for the effectiveness of our approach.

Time perception is a fundamental component of intelligence that structures the way humans act in various contexts. As action evolves over time, timing is necessary to appreciate environmental contingencies, estimate relations between events and predict the effects of our actions at future moments. Despite the fundamental role of time in human cognition it remains largely unexplored in the field of artificial cognitive systems. The present work makes concrete steps towards making artificial systems aware that the notion of time as a unique entity that can be processed on its own right. To this end, we evolve artificial neural networks to perceive the flow of time and to be able to accomplish three different duration processing tasks. Subsequently we study the internal dynamics of neural networks to obtain insight on the representation and processing mechanisms of time. The self-organized neural network solutions exhibit important brain-like properties and suggests directions for extending existing theories in timing neuro-psychology.

Articulated human body tracking is one of the most thoroughly examined, yet still challenging, tasks in Human Robot Interaction. The emergence of low-cost real-time depth cameras has greatly pushed forward the state of the art in the field. Nevertheless, the overall performance in complex, real life scenarios is an open-ended problem, mainly due to the high-dimensionality of the problem, the common presence of severe occlusions in the observed scene data, and errors in the segmentation and pose initialization processes. In this paper we propose a novel model-based approach for markerless pose detection and tracking of the articulated upper body of multiple users in RGB-D sequences. The main contribution of our work lies in the introduction and further development of a virtual User Top View, a hypothesized view aligned to the main torso axis of each user, to robustly estimate the 3D torso pose even under severe intra- and inter-personal occlusions, exempting at the same time the requirement of arbitrary initialization. The extracted 3D torso pose, along with a human arm kinematic model, gives rise to the generation of arms hypotheses, tracked via Particle Filters, and for which ordered rendering is used to detect possible occlusions and collisions. Experimental results in realistic scenarios, as well as comparative tests against the NiTETM user generator middleware using ground truth data, validate the effectiveness of the proposed method.

In this paper we address an important issue in human-robot interaction, that of accurately deriving pointing information from a corresponding gesture. Based on the fact that in most applications it is the pointed object rather than the actual pointing direction which is important, we formulate a novel approach which takes into account prior information about the location of possible pointed targets. To decide about the pointed object, the proposed approach uses the Dempster-Shafer theory of evidence to fuse information from two different input streams: head pose, estimated by visually tracking the off-plane rotations of the face, and hand pointing orientation. Detailed experimental results are presented that validate the effectiveness of the method in realistic application setups.

Executive control incorporates cognitive functions involved in the control and management of other cognitive processes. Such high-level skills are hard to be explored with brain imaging studies because they require complex and persistent experimental procedures. Alternatively, computational modeling may provide a new way to indirectly explore executive control mechanisms. The current work adopts this latter approach to explore possible characteristics of executive control, focusing particularly on behavioral rule switching and confidence neurodynamics in artificial agents. To this end, our study explores a robotic version of the classical Wisconsin Card Sorting Test, incorporating also the option of betting. Our ability to perform multiple and statistically independent computational experiments together with the in-depth study of the mechanisms created in the artificial cognitive systems, provides suggestions for the executive control aspects of the human brain.

In this paper we present a novel approach for hand gesture recognition. The proposed system utilizes upper body part tracking in a 9-dimensional configuration space and two Multi-Layer Perceptron/Radial Basis Function (MLP/RBF) neural network classifiers, one for each arm. Classification is achieved by buffering the trajectory of each arm and feeding it to the MLP Neural Network which is trained to recognize between five gesturing states. The RBF neural network is trained as a predictor for the future gesturing state of the system. By feeding the output of the RBF back to the MLP classifier, we achieve temporal consistency and robustness to the classification results. The proposed approach has been assessed using several video sequences and the results obtained are presented in this paper.

Computational modeling of natural systems can be used for interdisciplinary applications, such as the configuration of robotic systems or the validation of biological ones. Up to date there has been a little progress on suggesting a framework for automating the process of creating a computational model for biological processes. Instead researchers focus on the implementations of systems that are intended to replicate a tight set of biological behaviors. Such framework should be able to construct any system based on the appropriate level of abstraction chosen by the designer, as well as be able to enforce the appropriate biological consistency without compromising on performance or scalability of the generated models. In this paper we propose a framework that can automate the construction of computational models using genetic algorithms and demonstrate how this framework can construct a model of the parieto-frontal and premotor regions involved in grasping.

In this paper we present a novel methodology for detection and tracking of facial features like eyes, nose and mouth in image sequences. The proposed methodology is intended to support natural interaction with autonomously navigating robots that guide visitors in museums and exhibition centers and, more specifically, to provide input for the analysis of facial expressions that humans utilize while engaged in various conversational states. For face and facial feature region detection and tracking, we propose a methodology that combines appearance-based and feature-based methods for recognition and tracking, respectively. For the stage of face tracking the introduced method is based on least squares matching (LSM), a matching technique able to model effectively radiometric and geometric differences between image patches in different images. Thus, compared with previous research, the LSM approach can overcome the problems of variable scene illumination and head in-plane rotation. Another significant characteristic of the proposed approach is that tracking is performed on the image plane only wherever laser range information suggests so. The increased computational efficiency meets the real time demands of human-robot interaction applications and hence facilitates the development of relevant systems.

Recent research in cognitive systems aims to uncover important aspects of biological cognitive processes and additionally formulate design principles for implementing artificially intelligent systems. Despite the increasing amount of research efforts addressing cognitive phenomena, the issue of time perception and how it is linked to other cognitive processes remains largely unexplored. In the current paper, we make a first attempt for studying artificial time perception by means of simulated robotic experiments. Specifically, we investigate a behavioral rule switching task consisting of repeating trials with dynamic temporal duration. An evolutionary process is used to search for neuronal mechanisms accomplishing the rule switching task taking also into account its particular temporal characteristics. Our repeated simulation experiments showed that (i) time perception and ordinary cognitive processes may co-exist in the system sharing the same neural resources, and (ii) time perception dynamics bias the functionality of neural mechanisms with other cognitive responsibilities. Finally, in the current paper we make contact of the obtained results with previous brain imaging studies on time perception, and we make predictions for possible time-related dynamics in the real brain.

This paper addresses the problem of real-time moving-object detection, classification and tracking in populated and dynamic environments. In this scenario, a mobile robot uses 2D laser range data to recognize, track and avoid moving targets. Most previous approaches either rely on pre-defined data features or off-line training of a classifier for specific data sets, thus eliminating the possibility to detect and track different-shaped moving objects. We propose a novel and adaptive technique where potential moving objects are classified and learned in real-time using a fuzzy ART neural network algorithm. Experimental results indicate that our method can effectively distinguish and track moving targets in cluttered indoor environments, while at the same time learning their shape.

Tracking of the upper human body is one of the most interesting and challenging research fields in computer vision and comprises an important component used in gesture recognition applications. In this paper a probabilistic approach towards arm and hand tracking is presented. We propose the use of a kinematics model together with a segmentation of the parameter space to cope with the space dimensionality problem. Moreover, the combination of particle filters with hidden Markov models enables the simultaneous tracking of several hypotheses for the body orientation and the configuration of each of the arms.

We address the development of brain-inspired models that will be embedded in robotic systems to support their cognitive abilities. We introduce a novel agent-based coevolutionary computational framework for modeling assemblies of brain areas. Specifically, self-organized agent structures are employed to represent brain areas. In order to support the design of agents, we introduce a hierarchical cooperative coevolutionary (HCCE) scheme that effectively specifies the structural details of autonomous, yet cooperating system components. The design process is facilitated by the capability of the HCCE-based design mechanism to investigate the performance of the model in lesion conditions. Interestingly, HCCE also provides a consistent mechanism to reconfigure (if necessary) the structure of agents, facilitating follow-up modeling efforts. Implemented models are embedded in a simulated robot to support its behavioral capabilities, also demonstrating the validity of the proposed computational framework.

A major issue in the field of mobile robotics today is the detection and tracking of moving objects (DATMO) from a moving observer. In dynamic and highly populated environments, this problem presents a complex and computationally demanding task. It can be divided in sub-problems such as robotpsilas relative motion compensation, feature extraction, measurement clustering, data association and targetspsila state vector estimation. In this paper we present an innovative approach that addresses all these issues exploiting various probabilistic and deterministic techniques. The algorithm utilizes real laser-scanner data to dynamically extract moving objects from their background environment, using a time-fading grid map method, and tracks the identified targets employing a joint probabilistic data association with interacting multiple model (JPDA-IMM) algorithm. The resulting technique presents a computationally efficient approach to already existing target-tracking research for real time application scenarios.

The control of the hand in primate species is characterized by a high dimensionality, due to the large number of joints in the fingers. In this study we present how its manipulation can be simplified without compromising its usage, through a constraint methodology that is inspired from recent neurobiological findings. We further develop a computational model, consisting of several brain areas related to hand control, using a co-evolutionary architecture. Due to its neurobiological basis the methodology gives rise to a number of emergent properties that have been shown to occur in primate species during reach-to-grasp tasks.

Recently, many research efforts are directed towards co-evolutionary algorithms. The present work aims at the assessment of Hierarchical Cooperative CoEvolution (HCCE) being properly formulated to address hierarchical problems where simple components having separate design objectives, are parts of other more complex ones. HCCE is able to highlight the specialties of each component and additionally enforce their successful integration in a composite structure. Here we present HCCE describing also the internal dynamics that provide its effectiveness in addressing difficult distributed design problems. Additionally, the results described in the present work attest to its validity and superior performance against ordinary Unimodal evolution, and Enforced SubPopulation coevolution.

Recent brain imaging studies on primates revealed that a network of brain areas is activated both during observation and during execution of movements. The present work aims at modeling this group of areas, implementing a distributed computational system. The modeling process follows the agent-based coevolutionary framework that is very effective in terms of designing complex distributed systems addressing successfully the multi-modality of the interacting regions. The implemented model is successfully embedded in a simulated humanoid robot, replicating existing biological findings.

This paper describes an off-line, iterative algorithm for simultaneous localization and mapping within large indoor environments. The proposed approach is based on the idea of separately tracking multiple robot pose hypotheses that are generated each time the robot closes a loop by revisiting an already mapped area. Each tracked pose hypothesis corresponds to a separate possible robot path and maintains a separate map. Loop identification is inherently pursued by a hybrid localization algorithm with global localization capabilities. During the first step of the algorithm, all recorded sensor information is sequentially processed, in order to create a set of possible robot paths and their corresponding maps. After all sensor input is processed, the history of all tracked hypotheses is validated in order to select the most probable robot path and the corresponding map. The second step of the algorithm utilizes an EM-like iterative procedure in order to close the loops identified by the first step, compute a correct robot pose history and produce the final map. Experimental results demonstrate that the proposed algorithm facilitates computation of correct maps regardless of the number of loops in the robot's path

The current work addresses the development of cognitive abilities in artificial organisms. In the proposed approach, neural network-based agent structures are employed to represent distinct brain areas. We introduce a hierarchical collaborative coevolutionary (HCCE) approach to design autonomous, yet cooperating agents. Thus, partial brain models consisting of many substructures can be designed. Replication of lesion studies is used as a means to increase reliability of brain model, highlighting the distinct roles of agents. The HCCE is appropriately designed to support systematic modelling of brain structures, able to reproduce biological lesion data. The proposed approach effectively designs cooperating agents by considering the desired pre and post-lesion performance of the model. In order to verify and assess the implemented model, the latter is embedded in a robotic platform to facilitate its behavioral capabilities.

The current work addresses the development of cognitive utilities in artificial organisms, a topic that has attracted many research efforts recently. In our approach, neural network-based agent structures are employed to represent distinct brain areas. We introduce a hierarchical collaborative coevolutionary (HCCE) approach to design autonomous, yet cooperating agents. Thus, partial brain models consisting of many substructures can be designed. Replication of lesion studies is used as a means to increase reliability of brain model, highlighting the distinct roles of agents. The HCCE is appropriately designed to support systematic modelling of brain structures, able to reproduce biological lesion data. The proposed approach designs cooperating agents properly, by considering the desired pre- and post- lesion performance of the model. The effectiveness of the proposed approach is illustrated on the design of a computational model of primary motor cortex and premotor cortex interactions in the mammalian brain. The model is successfully tested in driving a simulated robot, with different pre- and post- lesion performance.

In this paper we propose an offline feature mapping algorithm capable of identifying and correctly closing multiple loops in cyclic environments. The proposed algorithm iteratively alternates between a Kalman smoother based localization step and a map features recalculation step. The identification of loops is done during the localization step by a hybrid localization algorithm that generates and tracks hypotheses generated each time the robot visits an already mapped area. The main contribution of this paper lies on the ability of the proposed algorithm to exploit information contained within the hypotheses histories in order to calculate correct maps, regardless of the complexity of the environment and the number of loops in the robot's path.

This paper introduces a methodology for avoiding obstacles by controlling the robot's velocity. Contemporary approaches to obstacle avoidance usually dictate a detour from the originally planned trajectory to its goal position. In our previous work, we presented a method for predicting the motion of obstacles, and how to make use of this prediction when planning the robot trajectory to its goal position. This is extended in the current paper by also using this prediction to decide if the robot should change its speed to avoid an obstacle more effectively. The robot can choose to move at three different speeds: slow, normal and fast. A hierarchical partially observable Markov decision process (POMDP) controls the robot movement. The POMDP formulation is not altered to accommodate for the three different speeds, to avoid the increase of the size of the state space. Instead, a modified solution of POMDPs is used.

In this paper we focus on one of the most important issues for autonomous mobile robots: their ability to localize themselves safely and reliably within their environments. We propose a probabilistic framework for modelling the robot's state and sensory information based on a switching state-space model. The proposed framework generalizes two of the most successful probabilistic model families currently used for this purpose: the Kalman filter linear models and hidden Markov models. The proposed model combines the advantages of both models, relaxing at the same time inherent assumptions made individually in each of these existing models.

In this paper we deal with one of the fundamental problems for mobile robots, namely their ability to produce accurate representations of their environment. We propose an off-line mapping algorithm that iteratively alternates between a Kalman smoother based localization step and a map features recalculation step. Moreover, a hybrid algorithm with global localization capabilities is employed in the first step, enabling correct identification of already mapped areas, and, thus, ensuring map correctness in cyclic environments.

This paper considers the problem of a robot navigating in a crowded or congested environment. A robot operating in such an environment can get easily blocked by moving humans and other objects. To deal with this problem it is proposed to attempt to predict the motion trajectory of humans and obstacles. Two kinds of prediction are considered: short-term and long-term. The short-term prediction refers to the one-step ahead prediction and the long-term to the prediction of the final destination point of the obstacle's movement. The robot movement is controlled by a partially observable Markov decision process (POMDP). POMDPs are utilized because of their ability to model information about the robot's location and sensory information in a probabilistic manner. The solution of a POMDP is computationally expensive and thus a hierarchical representation of POMDPs is used.

Abstract Knowledge of the position and orientation of 3D planes that exist in a scene is very important for many machine vision-based tasks, such as navigation, self-localization and docking. In this paper we propose two iterative methods for robustly computing the plane parameters out of given image sequences. Both methods include the (limited) ability to determine unknown parameters of the 3D-camera motion.

The structure in digitized images resides within two scales, the inner and outer scale. The inner scale is defined by the sampling resolution, and the outer scale is given by the image size. However, some images contain almost no fine scale structure, and these may be down-sampled without essential loss of image detail. Likewise some images may be reduced in size by removing borders with no structure. Hence we define essential inner and outer scales. Such considerations are the essence of local size estimation: a textured patch in an image has an essential inner scale related to the structure of the primitive textons, and an essential outer scale given by the size of the patch. In this paper, several functionals are examined that automatically find both the essential inner and outer scales in local neighborhoods of an image. In this preliminary work we present a general formulation for local scale selection, that is shown to be a generalization of Lindeberg's (see International Journal of Computer Vision, vol.30, no.2, p.117-56, 1998) Blob-detector and its morphological equivalent, and we present promising results using locally orderless images.

We address the issue of environment representation for navigational tasks by using reference scene patterns, the so-called landmarks, to adequately describe the robot's workspace. Mathematical tools from projective geometry are employed for landmark identification. A complete framework is presented for landmark extraction and recognition based on projective and point-permutation invariant vectors.

Landmark-based navigation usually relies on the identification and subsequent recognition of a number of environment objects, that are deemed adequate in describing the workspace structure. This process is inherently difficult in practice, due to the many different poses of an object that may be encountered in navigational trials. To alleviate for that, we propose an approach that employs projective invariants computed on quintuples of points as worldspace landmarks. Such quantities remain invariant under different camera positions and provide for effective description of the workspace structure. In order to identify potential corresponding quintuples in image frames, we introduce a simple test based on the covariance matrix estimate of each quintuple. With this test, we effectively by-pass the calculation of point correspondences. Since the above test indicates correspondence between quintuples, and not between their individual points, we subsequently employ a permutation projective invariant for quintuple recognition. Our approach has been extensively evaluated using synthetic as well as real environments. The results obtained verify its robustness along with its applicability in robotic navigation.

An approach towards providing advanced navigational support to robotic wheelchair platforms is presented. In order to avoid any modifications to the environment, we propose an approach that employs computer vision techniques which facilitate space perception and navigation. Computer vision has not been introduced to date in rehabilitation robotics, since the former is not mature enough to meet the needs of this sensitive application. However, in the proposed approach, stable techniques are exploited that facilitate reliable, automatic navigation to any point in the visible environment. Preliminary results obtained from its implementation on a laboratory robotic platform indicate its usefulness and flexibility.

The robot navigation using visual landmark approach is described. The landmarks are not preselected or otherwise defined a priori, but rather, they are extracted automatically during a learning phase. To facilitate this, a saliency map is constructed which highlights potential landmarks. This is used in conjunction with a qualitative segregation of the workspace, to further delineate the search areas for environment landmarks. For the sake of robustness, no semantic information is attached to the landmarks; they are stored as raw patterns along with information readily available from the workspace segregation, that facilitates their accurate rate recognition at a later, navigation session. During such a session, similar steps with the learning phase are employed to locate landmarks. The stored information is used to transform a previously leaned landmark pattern, according to the current position of the observer, achieving thus accurate landmark recognition. Results obtained from our approach demonstrate its validity and applicability in indoor workspaces.

Recent works in multispectral image processing advocate the employment of vector approaches for this class of signals. Vector processing operators that involve the minimization of a suitable error criterion have been proposed and shown appropriate for this task. In this framework, two main classes of vector processing filters have been reported in the literature. Astola et al. (1990) introduce the well-known class of vector median filters (VMF), which are derived as maximum likelihood (ML) estimates from exponential distributions. Trahanias et al. (see ibid., vol.2, no.4, p.528-34, 1993 and vol.5, no.6, p.868-80, 1996) study the processing of color image data using directional information, considering the class of vector directional filters (VDF). We introduce a new filter structure, the directional-distance filters (DDF), which combine both VDF and VMF in a novel way. We show that DDF are robust signal estimators under various noise distributions, they have the property of chromaticity preservation and, finally, compare favorably to other multichannel image processing filters.

This paper addresses the implementation of image processing and computer vision techniques to automate tracer flow extraction in images obtained by the photochromic dye technique. This task is important in modeled arterial blood flow studies. Currently, it is performed via manual application of B-spline curve fitting. However, this is a tedious and error-prone procedure and its results are nonreproducible. In the proposed approach, active contours, snakes, are employed in a new curve-fitting method for tracer flow extraction in photochromic images. An algorithm implementing snakes is introduced to automate extraction. Utilizing correlation matching, the algorithm quickly locates and localizes all flow traces in the images. The feasibility of the method for tracer flow extraction is demonstrated. Moreover, results regarding the automation algorithm are presented showing its accuracy and effectiveness. The proposed approach for tracer flow extraction has potential for real-system application.

This paper presents a methodology for the detection of objects that move independently of the observer in a 3D dynamic environment. Independent 3D motion detection is formulated as a problem of robust regression applied to visual input acquired by a binocular, rigidly moving observer. The qualitative analysis of images acquired by a parallel stereo configuration yields a segmentation of a scene into depth layers. A depth layer consists of points of the 3D space with almost constant depth from the observer. Robust regression in the form of Least Median of Squares estimation is applied within each depth layer in order to segment the latter into coherently moving regions. Finally, a combination stage is applied across all layers in order to come up with an integrated view of independent motion in the whole 3D scene. In contrast to other existing approaches for independent motion detection which are based on the ill-posed problem of optical flow computation, the proposed methodology relies...

This paper presents a method for the detection of objects that move independently of the observer in a 3D dynamic scene, Independent motion detection is achieved through processing of stereoscopic image sequences acquired by a binocular, rigidly moving observer. A weak assumption is made about the observer's motion (egomotion), namely that the direction of the translational and rotational components of egomotion are constant in small image patches. This assumption facilitates the extraction of qualitative information about depth from motion, while additional qualitative depth information is independently computed from image stereo pairs acquired by the binocular vision system. Robust regression in the form of least median of squares estimation is applied within each image patch to test for consistency between the depth functions computed from motion and stereo. Possible inconsistencies signal the presence of independently moving objects. In contrast to other existing approaches for independent motion detection, which are based on the ill-posed problem of optical flow computation, the proposed method relies on normal flow fields for both stereo and motion processing. By exploiting local constraints of qualitative nature, the problem of independent motion detection is approached directly, without relying on a solution to the general structure from motion problem. Experimental results indicate that the proposed method is both effective and robust.

Robot motion planning is being approached in this paper by estimating the uncertainty of its configuration that is computed by the robot sensors. Since mobile robotic platforms are usually equipped with a variety of range sensors, measurements returned from all sensors are employed for the above estimation. The notion of sensory uncertainty fields (SUFs), recently proposed, is being extended to incorporate all the available sources of external sensory data. The multisensory uncertainty field (MSUF) is introduced which results in more accurate configuration estimation. Moreover, in order to cope with unexpected objects (obstacles) encountered at execution time, the navigation algorithm is augmented with an obstacle avoidance and navigation resuming technique. The introduction of multiple sensors and obstacle avoidance facilitates accurate navigation in indoor environments and in the presence of unexpected objects. This is demonstrated by navigation results obtained from an implementation of this method.

Multiple input devices are increasingly used in user interfaces to make human-computer communication more efficient and effective. Interface designers have not only to decide on which input modes should be supported, but also how to fuse them into a single representation format that can be processed by the underlying application system. Drawing appropriate decisions requires, however, a sufficient understanding of the properties of fusion itself. While others have informally characterized input fusion as a transformation between information types, the purpose of the paper is to explore fusion by means of formal process modelling. That is, fusion processes are defined in a formal framework which supports proof of the existence of necessary properties following directly from the process definitions. The presented approach can be applied to analyse and compare fusion processes in existing systems, as well as an aid for interface designers, who have to verify the behaviour of their systems.

The processing of color image data using directional information is studied. The class of vector directional filters (VDF), which was introduced by the authors in a previous work, is further considered. The analogy of VDF to the spherical median is shown, and their relation to the spatial median is examined. Moreover, their statistical and deterministic properties are studied, which demonstrate their appropriateness in image processing. VDF result in optimal estimates of the image vectors in the directional sense; this is very important in the case of color images, where the vectors' direction signifies the chromaticity of a given color. Issues regarding the practical implementation of VDF are also considered. In addition, efficient filtering schemes based on VDF are proposed, which include adaptive and/or double-window structures. Experimental and comparative results in image filtering show very good performance measures when the error is measured in the L*a*b* space. L*a*b* is known as a space where equal color differences result in equal distances, and therefore, it is very close to the human perception of colors. Moreover, an indication of the chromaticity error is obtained by measuring the error on the Maxwell triangle; the results demonstrate that VDF are very accurate chromaticity estimators.

Color edge detection is approached in this paper using vector order statistics. Based on the R-ordering method, a class of color edge detectors is defined. These detectors function as vector operators as opposed to component-wise operators. Specific edge detectors can be obtained as special cases of this class. Various such detectors are defined and analyzed. Experimental results show the noise robustness of the vector order statistics operators. A quantitative evaluation and comparison to other color edge detectors favors our approach. Edge detection results obtained from real color images demonstrate the effectiveness of the proposed approach in real applications.

It is generally accepted that multispectral image processing should be employed using a vector approach. Two are the main classes of vector processing filters that have been reported in the literature. Astola et al. (1990) introduce the well known class of vector median filters (VMF), which are derived as MLE estimates from exponential distributions. Trahanias et al. (see IEEE Trans. Image Process., vol.2, p.528-34, Oct. 1993) study the processing of color image data using directional information, considering the class of vector directional filters (VDF). We introduce a new class of filters, the directional-distance filters (DDF), which combine both VDF and VMF in a novel way. We show that DDF can eliminate the noise much more effectively than the VMF (even the impulsive noise), and that they have the property of chromaticity preservation.

A method is proposed whereby a color image is treated as a vector field and the edge information carried directly by the vectors is exploited. A class of color edge detectors is defined as the minimum over the magnitudes of linear combinations of the sorted vector samples. From this class, a specific edge detector is obtained and its performance characteristics studied. Results of a quantitative evaluation and comparison to other color edge detectors, using Pratt's (1991) figure of merit and an artificially generated test image, are presented. Edge detection results obtained for real color images demonstrate the efficiency of the detector.

Vector directional filters (VDF) for multichannel image processing are introduced and studied. These filters separate the processing of vector-valued signals into directional processing and magnitude processing. This provides a link between single-channel image processing where only magnitude processing is essentially performed, and multichannel image processing where both the direction and the magnitude of the image vectors play an important role in the resulting (processed) image. VDF find applications in satellite image data processing, color image processing, and multispectral biomedical image processing. Results are presented here for the case of color images, as an important example of multichannel image processing. It is shown that VDF can achieve very good filtering results for various noise source models.

The method of histogram equalization for grey-level image enhancement is extended to color images in the paper. A method of direct 3-D histogram equalization is proposed which results in a uniform histogram of the RGB values. Due to the correlation between the bands, the principle on which grey-level (1-D) histogram equalization is based is not valid in the case of color images (3-D). This problem is alleviated by employing a histogram specification method where a uniform histogram is specified.>