Predicting arm movements is a key issue in physical human robots interactions. It allows robots to prepare for action and meet human requirements and needs on time. Different to human action recognition, the prediction of human movements relies on few samples, namely the first ones. In this paper, we explore the use of LSTM (Long Short Term Memory) networks in deriving the final position and the time of the human hand when performing a high five game with robots. For such a context, the synchrony of human and robot movements should be achieved at early stages of the human to meet the constraints of both real-time robot control and the realism of the robot movement. The results we obtained are very encouraging and opening new questions as well. Our solution predicts acceptable final position and contact time regardless the morphology of people and their positioning.

Remote control with force feedback is one of the key features of tele-robotics systems. In this paper we investigate about the mechanical properties of a joystick based on a 12-6 Stewart platform. In most of classical 6 DOF parallel mechanisms, the forward kinematics cannot be described with closed-form solutions, which prevents from real-time control and straightforward implementations. In this contribution we address a 12-6 Stewart redundant parallel mechanism. We present the forward kinematics of this mechanism and we analyze the corresponding compatibility equations.1We introduce intermediate variables, which allow revealing scale relationships among feature points. Accordingly, we derive 15 quadratic isomorphic compatible equations, and then we convert the forward kinematics equations into 12 linear compatible equations. Based on matrix algebra, we determine the final and unique solution of the redundant system. Furthermore, we study the influence of the initial values and the structural parameters. We give simulation results showing both the efficiency and the precision of the proposed compared to classical approaches. Our method outperforms previous works, allowing a better design and a faster control for accurate and effective 6 DOF joysticks.

In this paper a system for analyzing human body movements with tremors is proposed. We developed a latent forces based method to evaluate quantitatively tremors, as they are important indicators for some diseases, such as Parkinson, Essential tremor and Strokes. Our contribution is a data driven method. It relies on a mechanistic model, from which latent mechanistic features are derived to explain human body movements in general and tremors in particular. To this end, we define two groups of forces: one group drives the normal body movements and the other one governs tremors. Moreover, in order to quantify tremors, we created an index based on distributions distances indicating the severity of symptoms. We applied this formalism to tremors in human body and tested it on a collection of realistic data.

Object detection and tracking is a basic but a key feature in many robotics tasks, including for human-robots interactions. Handing over objects for instance rely almost exclusively on such a capability: robots should be able to detect the object of interest, localize and track its movements in order to synchronize temporally and spatially the transition phase. In the past few years, a variety of architectures based on convolutional neural networks, especially that of the R-CNN and YOLO (You Only Look Once) models, largely contribute to the improvement of detection accuracy and efficiency. In this paper, we design a visual detection system to detect some objects the Nao robot should detect and track based on the YOLO architecture. We adopt the tiny YOLO network as a pretrained model and modify the last fully connected layer for three-class object detection. The model is trained on 4322 training images and achieves the mean average precision (mAP) of 44.3% on our test set. This model is then applied to a well-known function of the Nao vision system: detecting and localizing landmarks. It is also applied to daily objects. The results demonstrate that it could help localizing and tracking accurately objects of interest in HRI (human-robot interaction).

Object recognition and localization technology based on robot vision plays an important role in robot motion, obstacle avoidance, object grabbing and multi robot cooperation. The object recognition and localization algorithm provided by NAO robot official documents is not robust to different light conditions. It often causes misjudged or low accuracy. This paper uses the method of adaptive luminance RGB, combined with median filter, Hough circle detection, according to the color difference image segmentation technology, greatly reduce the misjudged and improve the accuracy. Then we use the transformation between image coordinate system and three-dimensional space coordinate system to get more precise object location. By analyzing the error of experimental data, a linear error compensation algorithm is added for different area errors, which further improves the positioning accuracy. The average error is reduced to 0.37 centimeters.

This paper presents a novel video descriptor, referred to as Histogram of Oriented Track lets, for recognizing abnormal situation in crowded scenes. Unlike standard approaches that use optical flow, which estimates motion vectors only from two successive frames, we built our descriptor over long-range motion trajectories which is called track lets in the literature. Following the standard procedure, we divided video sequences in spatio-temporal cuboids within which we collected statistics on the track lets passing through them. In particular, we quantized orientation and magnitude in a 2-dimensional histogram which encodes the motion patterns expected in each cuboid. We classify frames as normal and abnormal by using Latent Dirichlet Allocation and Support Vector Machines. We evaluated the effectiveness of the proposed descriptors on three datasets: UCSD, Violence in Crowds and UMN. The experiments demonstrated (i) very promising results in abnormality detection, (ii) setting new state-of-the-art on two of them, and (iii) outperforming former descriptors based on the optical flow, dense trajectories and the social force model.

In this contribution, we describe an unified controller we implemented in order to simplify creating the common interactive behaviors that the robot may performs with human users. This controller is based on a multi-objective optimization technique allowing simultaneous achievement of different constraints and goals. The core of the developed algorithm is based on the particle swarm optimization technique (PSO). Through some examples, namely, a wheeled manipulator following human in populated and cluttered environments, and a humanoid robot reaching and tracking objects. we show how the PSO is effective in solving the given problems in flexible and generic ways. We detail in this paper the algorithm and give some results obtained from simulations and field trials.

This work investigates the temporal dynamics between electro dermal activity (EDA) and gestures during real-life public speaking. Wrist movements and EDA were recorded from speakers during a high-stakes public speaking event. The extreme values theory, more typically used for detecting relatively rare events in financial markets or flood calculations, was adapted and combined with continuous wavelet transform to automatically detect significant temporal events from the derivatives of the movement and EDA time series. The literature describing motion artifacts in EDA predicts that detected movement events should precede or co-occur with EDA. In contrast, we also observed a second case in which EDA changes preceded movement (approximately 40% of cases). These results suggest that humans may move or gesture in response to affective changes, which has clear implications for anyone studying arousal and gesture 'in the wild.

Using gestures for natural interactions in virtual environments require robust and smart recognition systems. In these contexts, gestures and gesticulations are part of the a continuous information stream: the first are sufficient to convey meaningful information such as commands and indications. On the contrary, gesticulations are unconscious body movements performed mainly, to support speech. In the majority of gestures recognition systems, the implicit assumption of "isolated patterns" is made. Indeed, following the Kendon's morpho-kinetics model, a gesture is the part of the armmovement contained between the pre-stroke and the post-stroke. This strong assumption shifts the recognition problem toward a clustering issue, e.g., recognizing an isolated temporal pattern. From the practical point of view, the isolated gestures hypothesis needs a cooperation from the user and the later should emphasize the pre and the post strokes. This removes the naturalness of the targeted interface. In this contribution, we focus on having a strong segmentation technique that clusters the body movements into consistent sequences. In this paper, we present a non-parametric stochastic segmentation algorithm that is able to cluster the continuous time series representing body movements into gestures and non-gestures segments. We show as well how this technique allows any novice user creating in a semi-supervised way, his or her, own gestures library. The proposed system is assessed through a real-life example, where a novice user creates an adhoc interface to control an artificial agent in a natural way.

Interacting naturally with artificial agents and environments including virtual avatars and robots rely mainly on gestures. However, the later require heavy setup procedures before any effective use. Specific and personalized training sessions are needed. In addition, performers have to indicate clear separations between gestures, namely, specifying pre- and post-strokes as well as the gesture used to perform the command. Thus, time series segmentation appears as a central problem. Indeed, clustering human motions into meaningful segments or isolating meaningful segments forming a continuous movement flow present the same problem: how to find the post and the pre strokes. For machine learning, this problem is solved by having training sets of carefully labeled data. Good segmentation improves the quality of the gesture recognition-based interface. In our contribution, we focus on developing a non-parametric stochastic segmentation algorithm. Once the segmentation has been validated, we show how any novice user can create in a semi-supervised way, his or her, own gestures library. In addition, we show how the obtained system is efficient in finding meaningful gestures (the once learned earlier) within continuous movements flow, thus removing the constraint of performing manual specification of respectively the beginning of the movement and its end. The proposed technique is assessed through a real-life example, where a novice user creates an ad-hoc interface to control a robot in a natural way.

This paper presents multi-robot simultaneous localization and mapping (SLAM) framework for a team of robots with unknown initial poses. The proposed solution is using feature based Rao-Blackwellised particle filter (RBPF) SLAM for each robot working in an unknown environment equipped only with 2D range sensor and communication module. To represent the environment in compact form, line and corner features (or point features) are used. By sharing and comparing distinct feature based maps of each robot, a global map with known poses is formed without any physical meeting among the robots. This approach can easily applicable to the distributed or centralized robotic systems with ease of data handling and reduced computational cost.

As speech synthesis technology develops more advanced paralinguistic capabilities, open questions emerge regarding how humans perceive the use of such vocal capabilities by robots. Perceptions of spoken interaction are complex and influenced by multiple factors including the linguistic content of a message, social context, perceived intelligence of the agent, and form factor of its embodiment. This paper shares results from a study that controlled for the above factors in order to investigate the effect on human listeners of a male synthetic voice with an expressive range. Participants were randomly assigned to three conditions, counterbalancing for gender and language background, in which how paralinguistic cues were applied was varied. As the voice became more expressive and appropriate for the context, observers were more likely to describe the communication as effective, but were less likely to refer to the unseen agent as a person. Possible effects of the listener gender and cultural-linguistic background are examined. Implications for future methodologies in this field are discussed.

In this paper, we investigated depth perception in real and virtual environments when binocular disparity is the sole distance cue. The observers were asked to estimate the relative depth of spheres verbally in virtual and actual environments. Constant apparent sized stimuli were used to measure the just-noticeable difference in depth perception, thus avoiding providing a size gradient cue. Results of the experiments revealed individual differences in virtual reality in contrast to reality. Specifically a subgroup of observers had difficulty perceiving the depth of virtual spheres in virtual reality, which may indicate that they rely on apparent size for judging depth. Furthermore, the just-noticeable differences were more variable in the virtual environment than with real objects. Our results reveal individual differences when the disparity-driven convergence cue is the only distance cue provided in virtual reality.

Using virtual reality for understanding sports performance allows for systematic investigation of human sensorimotor capabilities and meanwhile promotes the design and comparison of realistic immersive platforms. In this paper, we propose a virtual reality-based experimental design for studying the human ability to intercept spinning balls deflected by the Magnus effect. Compared to the previous approaches, we focused on a tight perception-action coupling. Experienced and novice subjects immersed in a 3D soccer stadium were asked to head realistically simulated balls, free kicked with and without sidespin. Consistent with the former studies, qualitative results show that the interception performance systematically relates to both the ball sidespin direction and arrival position for all the subjects, either experienced or not. However, contrary to those former studies where subjects answered only pseudo-verbally, experienced and novice groups differentiate in quantitative performances, supporting that expertise likely appears when perception is coupled to action. Further analyses will be needed to extract the different information-movement relationships governing the behaviors of experienced subjects and novices.

Industrial implementation of robots is to perform the assigned tasks in the minimum possible time in the cycle comes up to increase productivity and reduce the cost. The cycle time is strongly linked to the robot trajectory cycle to the task. However, the optimization of the robot trajectory cycle the robot visited a set of points which represent the robotics task. Similar to persons in traveling the robot execute the task into shorter time if has a shorter path. However the trajectory cycle of the robot is strongly related to the displacement in coordinate space rather than operational space. In fact, the shorter distance between two task points is the shorter distance between two configurations. Since robot has different configurations in each task point the minimum trajectory should be chosen between each successive configuration. However the order of visiting the task point also affects the trajectory distance. Moreover the relative robot position to the task also has a trivial effect on the task time. In this work we develop a method to optimize the order of visiting the task point taking into consideration the robot configuration and the placement of the robot in the robotized site. Mainly, this method is based on Genetic Algorithms and it takes into consideration the multiplicity solutions of the robot Inverse Kinematics Model (IKM), the task point visit order and the placement of robot at the same time.

This preliminary study looks into how visual information affects motion planning in a gravitational environment. Recently, it has been shown that goal-directed tasks, e.g. simple pointing actions, are performed using a combination of a priori knowledge and closed-loop information, coming from proprioceptive-vestibular and visual feedback. In particular it has been observed that visual information is used by the central nervous system to reorganize motor planning when visuo-motor conflicts occur.The aim of our work is to investigate deeper the contribution of the visual channel in motor planning by means of virtual reality tools. Indeed, virtual reality technology is used here to modify the nature of visual input, leading to test several scenarios from minimalist and impoverished scenes to fully immersive and realistic environments. In fact we want to understand how the richness of the visual information is relevant in influencing our motor planning. We describe here our experimental setup and protocols. Moreover we present some preliminary results and we discuss the ongoing developments.

This paper deals with the usage of virtual reality and scenario languages in the field of teleoperation: how to enable a group of teleoperators to control, in a collaborative way, groups of real robots, in turn collaborating with each other to achieve complex tasks; such tasks include inspecting a dangerous area or exploring a partially unknown environment. The main goal is to obtain efficient, natural and innovative interactions in such a context.We first present the usage of collaborative virtual environments (CVE) to obtain a unified, simplified,virtual abstraction of distributed, complex, real robots. We show how this virtual environment offers a peculiar ability: to free teleoperators from space and time constraints. Then we present our original usage of scenario languages to describe complex and collaborative tasks in a natural and flexible way. Finally, we validate the proposed framework through our teleoperation platform ViRAT.

This study investigated how people achieve pointing tasks using a vibrotactile device. A comparison using visual feedback when doing the same task is presented. Subjects were asked to reach a target as fast and as precise as possible guided by visual information from an LCD screen; and two types of tactile information generated by a specific glove we developed. The first type of tactile information was a constant vibrating stimulation and the second one was a frequency modulation based vibration. For each modality, two experiments were performed. The first one was in 1-D: reaching a target situated on a horizontal left-right line. The second one was in 2-D: reaching a target is in a horizontal plane. The results we obtained showed that tactile based pointing task obey to the classical the Fitts' law. Furthermore, results showed that frequency modulation based vibrating gave better performance in trajectory length, but not in time of execution.

Robotics needs realistic simulators and environments to display not only physical interactions between robots and the environment, but also more and more complex processes. Sensing procedures, communications and other processes like cognitive reasoning may be first run on adapted and competitive simulation platforms to deal with optimal design. Virtual reality (VR) left the ghetto of visual rendering tool to become a more complete mean to handle basic simulation problems and to enable aggregation of complex functions for complex interactions. VR and robotics have met through Teleoperation. Historically, the first teleoperation systems were used as interfaces to control remotely robots by providing input tools and sensory feedback information. With technological improvements, VR shifted to be the main kernel for first, studies on advanced robotics systems studies like bio-inspired robots behavior, multi-robot systems, etc., and second, to provide advanced multi-modal interfaces for real time interactions with a useful level of abstraction. We present in this paper some basic robotics problems and the way VR can solve them. The first part of this paper discuss about the benefits of standardization and VR for complex robots systems. Then, we present some existing solutions and we analyze their drawbacks and advantages. Finally, we introduce an application where we take advantage of those features, in terms of efficiency in the development cycle and of reusability.

This paper deals with the usage of Virtual and Augmented Reality techniques in the field of teleoperation. The global context is a project for collaborative teleoperation of real robots groups. The aim is to enable to a group of teleoprators to control in a collaborative way groups of robots to achieve complex tasks like inspecting an area or to explore unknown parts of an unknown environment. More precisely, we first present the usage of Collaborative Virtual Environments (CVE) system for the representation of the robot's world. This CVE and its interactions capabilities offer an abstraction of the environment and robots for teleoperators. We also present an application of this system throughout the original usage of a head-tracking system combined with a VR helmet. This application enables to control an active vision system on a remote mobile robot. Augmented Reality add-ons are included to ease the control process.

In this paper, we propose a new approach for some augmented reality applications by exploiting minimal geometric knowledge and using Desargues theorem. The idea underlying our approach is to use a generalization of Desargues theorem in uncalibrated images context. This approach allows the realization of three applications that includes: points matching, novel view synthesis and adding a virtual object to real scene. Examples on real and synthetic images are presented.

A virtual control environment for robot teleoperation via the Internet is presented. It comprises a Java3D-based real-time virtual representation of the robot and worksite, and uses a graphic panel, an environment for remote robot programming, and a dataglove with a 6D position tracker as the control interfaces. The use of virtual reality (VR) techniques for Internet teleoperation allow: (1) time delays inherent in IP networks to be suppressed and (2) the operator's work to be simplified and accelerated, compared to methods that use delayed TV images. The system realisation, with its use of open technologies Java, Java3D and 3-tier client/server architecture, provides portability among different computer platforms and types of robots. The efficiency of the VR-based methods developed has been verified for slow communication rates (0.1-0.5 KB/sec), where TV-based control methods are inapplicable. VR systems have been developed for the WWW-based control of the PUMA and CRS industrial robot manipulators. The particulars of these systems, the experiments undertaken, current issues, and directions of future work are presented.