Nowadays, robotic arms are ubiquitously employed for object manipulation across a spectrum of applications, spanning from production lines to warehouses, and encompassing both stationary and mobile robotic systems. Among the most prevalent end-effectors, used for the majority of these applications, are suction cups. The rudimentary act of grasping an object and relocating it, devoid of a cognizant awareness of the forces stemming from the object’s motion and grip, can result in suboptimal and inefficient robot movements. In more dire circumstances, such negligent handling may precipitate detachment of the object from the end-effector, potentially incurring damage to either the object or the arm.In this paper, we build upon the advanced sensing and attaching capabilities of our suction cup MIGHTY, and introduce GEM-C, a novel Gravity, External forces and Motion Compensation controller, that constantly adapts the orientation of the suction cup so as to enhance the quality of attachment. Throughout all examined scenarios and experiments, our approach remarkably improved the robot’s performance by providing the optimal end-effector pose while also reducing the stress on the motors and the overall power consumption. The derived results, clearly demonstrate the MIGHTY and GEM-C schema’s potential for a wide range of demanding robotic manipulation tasks.

The spectrum of applications of robotic systems is constantly being expanded in the research, industrial and even the defense sectors, ranging from manipulation and assembly to critical infrastructure monitoring and post-disaster response. Nevertheless, contemporary robotic capabilities are significantly hindered when traversing through or interacting with complex, unstructured and dynamic environments. To alleviate for that, we introduce in the current work MIGHTY (Multi-functional Intelligent Gripping with High Tolerance), a novel, lightweight, sensor-enhanced vacuum suction cup providing not only enhanced attachment capabilities on a plethora of surfaces of varying roughness, but also robust and accurate contact and force estimation. Its unique design facilitates multiple functionalities, from acting as a gripper for object manipulation to operating as a foot for stable walking and steep surface climbing. The proposed suction cup was extensively assessed under varying experimental setups, in order to validate its capacity to sense the applied force and torque and the torque's axis, as well as its ability to attach on a variety of surfaces. In all cases remarkable results were demonstrated, attesting for its effectiveness and robustness.

Efficient storing and managing of robot's experiences is of utmost importance in long-term recurring HumanRobot Interaction scenarios, where the volume of information increases constantly. To address these issues, a novel entity-based episodic memory is introduced in this work. Knowledge is represented by hierarchical multigraphs enabling for fast information retrieval. Consisting entities are asynchronously updated, while a time-correlated importance factor modulates the merging, forgetting or refreshing of memories, in order to facilitate search and management of the stored information. An HMM-based probabilistic inference is employed to infer or predict the HRI state or to identify abnormal scenario unfolding and, thus, guide future robot activities. The performance of the employed memory schema is assessed on both simulated and real “breakfast preparation” scenarios. The results indicate that the proposed memory model is able to efficiently store and manage the acquired data without any loss of critical information. Moreover, our approach was also shown capable of successfully inferring user's hidden preferences and thus guiding robot behavior accordingly in order to improve user's HRI experience.

Recent introduction of low-cost depth cameras triggered a number of interesting works, pushing forward the state-of-the-art in human body pose extraction and tracking. However, despite the remarkable progress, many of the contemporary methods cope inadequately with complex scenarios, involving multiple interacting users, under the presence of severe inter- and intra-occlusions. In this work, we present a model-based approach for markerless articulated full body pose extraction and tracking in RGB-D sequences. A cylinder-based model is employed to represent the human body. For each body part a set of hypotheses is generated and tracked over time by a Particle Filter. To evaluate each hypothesis, we employ a novel metric that considers the reprojected Top View of the corresponding body part. The latter, in conjunction with depth information, effectively copes with difficult and ambiguous cases, such as severe occlusions. For evaluation purposes, we conducted several series of experiments using data from a public human action database, as well as own-collected data involving varying number of interacting users. The performance of the proposed method has been further compared against that of the Microsoft's Kinect SDK and NiTE (TM) using ground truth information. The results obtained attest for the effectiveness of our approach.

Articulated human body tracking is one of the most thoroughly examined, yet still challenging, tasks in Human Robot Interaction. The emergence of low-cost real-time depth cameras has greatly pushed forward the state of the art in the field. Nevertheless, the overall performance in complex, real life scenarios is an open-ended problem, mainly due to the high-dimensionality of the problem, the common presence of severe occlusions in the observed scene data, and errors in the segmentation and pose initialization processes. In this paper we propose a novel model-based approach for markerless pose detection and tracking of the articulated upper body of multiple users in RGB-D sequences. The main contribution of our work lies in the introduction and further development of a virtual User Top View, a hypothesized view aligned to the main torso axis of each user, to robustly estimate the 3D torso pose even under severe intra- and inter-personal occlusions, exempting at the same time the requirement of arbitrary initialization. The extracted 3D torso pose, along with a human arm kinematic model, gives rise to the generation of arms hypotheses, tracked via Particle Filters, and for which ordered rendering is used to detect possible occlusions and collisions. Experimental results in realistic scenarios, as well as comparative tests against the NiTETM user generator middleware using ground truth data, validate the effectiveness of the proposed method.

We address the question of whether service robots that interact with humans in public spaces must express socially appropriate behaviour. To do so, we implemented a robot bartender which is able to take drink orders from humans and serve drinks to them. By using a high-level automated planner, we explore two different robot interaction styles: in the task only setting, the robot simply fulfils its goal of asking customers for drink orders and serving them drinks; in the socially intelligent setting, the robot additionally acts in a manner socially appropriate to the bartender scenario, based on the behaviour of humans observed in natural bar interactions. The results of a user study show that the interactions with the socially intelligent robot were somewhat more efficient, but the two implemented behaviour settings had only a small influence on the subjective ratings. However, there were objective factors that influenced participant ratings: the overall duration of the interaction had a positive influence on the ratings, while the number of system order requests had a negative influence. We also found a cultural difference: German participants gave the system higher pre-test ratings than participants who interacted in English, although the post-test scores were similar.

In this paper we present a novel approach for hand gesture recognition. The proposed system utilizes upper body part tracking in a 9-dimensional configuration space and two Multi-Layer Perceptron/Radial Basis Function (MLP/RBF) neural network classifiers, one for each arm. Classification is achieved by buffering the trajectory of each arm and feeding it to the MLP Neural Network which is trained to recognize between five gesturing states. The RBF neural network is trained as a predictor for the future gesturing state of the system. By feeding the output of the RBF back to the MLP classifier, we achieve temporal consistency and robustness to the classification results. The proposed approach has been assessed using several video sequences and the results obtained are presented in this paper.

Tracking of the upper human body is one of the most interesting and challenging research fields in computer vision and comprises an important component used in gesture recognition applications. In this paper a probabilistic approach towards arm and hand tracking is presented. We propose the use of a kinematics model together with a segmentation of the parameter space to cope with the space dimensionality problem. Moreover, the combination of particle filters with hidden Markov models enables the simultaneous tracking of several hypotheses for the body orientation and the configuration of each of the arms.