High dynamic capabilities of industrial robots make them dangerous for humans and environment. To reduce this factor and advance collaboration between human and manipulator fast and reliable collision detection algorithm is required. To overcome this problem, we present an approach allowing to detect collision, localize action point and classify collision nature. Internal joint torque and encoder measurements were used to determine potential collisions with the robot links. This work proposes two ways of solving this problem: using classical analytical approach and learning approach implemented with neural network. The suggested algorithms were examined on the industrial robotic arm Kuka iiwa LBR 14 R820, ground truth information on the contact nature and its location were obtained with 3D LIDAR and camera.

One of the main barriers preventing widespread use of formal methods is the elicitation of formal specifications. Formal specifications facilitate the testing and verification process for safety critical robotic systems. However, handling the intricacies of formal languages is difficult and requires a high level of expertise in formal logics that many system developers do not have. In this work, we present a graphical tool designed for the development and visualization of formal specifications by people that do not have training in formal logic. The tool enables users to develop specifications using a graphical formalism which is then automatically translated to Metric Temporal Logic (MTL). In order to evaluate the effectiveness of our tool, we have also designed and conducted a usability study with cohorts from the academic student community and industry. Our results indicate that both groups were able to define formal requirements with high levels of accuracy. Finally, we present applications of our tool for defining specifications for operation of robotic surgery and autonomous quadcopter safe operation.

Spatial interactions between agents (humans, animals, or machines) carry information of high value to human or electronic observers. However, not all the information contained in a pair of continuous trajectories is important and thus the need for qualitative descriptions of interaction trajectories arises. The Qualitative Trajectory Calculus (QTC) (Van de Weghe, 2004) is a promising development towards this goal. Numerous variants of QTC have been proposed in the past and QTC has been applied towards analyzing various interaction domains. However, an inherent limitation of those QTC variations that deal with lateral movements is that they are limited to two-dimensional motion; therefore, complex three-dimensional interactions, such as those occurring between flying planes or birds, cannot be captured. Towards that purpose, in this paper QTC3D is presented: a novel qualitative trajectory calculus that can deal with full three-dimensional interactions. QTC3D is based on transformations of the Frenet–Serret frames accompanying the trajectories of the moving objects. Apart from the theoretical exposition, including definition and properties, as well as computational aspects, we also present an application of QTC3D towards modeling bird flight. Thus, the power of QTC is now extended to the full dimensionality of physical space, enabling succinct yet rich representations of spatial interactions between agents.

Traditional path planning for robots is a well-studied problem. However, the classical setting of the problem is simple to state: plan a path for a robot, starting from an initial point, and ending at a desired target point, given an environmental map, usually in the form of an occupancy grid. In this setting though, no special consideration is given to humans; they are thought of simply, as being obstacles in the environment, equivalent to chairs or walls. However, with more robots entering human spaces, special consideration needs to be given: humans need special treatment as obstacles, and furthermore humans can also serve the goal of goal points, towards starting a social interaction; either individual humans or groups of humans. Also, special mechanisms are required for engaging and disengaging in such interactions, taking into account psychological considerations of proxemics. In this paper, we first introduce our unifying theoretical framework for all the subproblems of social path planning; then, we propose an extended mode for engaging groups of people; and then, by using a special version of the fast-marching square planning method, we present and demonstrate actual algorithmic solutions for the social path planning subproblems. Our results prove the strengths of our approach and its generalizability. Finally, concrete further steps are discussed.

Traditional path planning for robots is a well-studied problem. However, the classical setting of the problem is simple to state: plan a path for a robot, starting from an initial point, and ending at a desired target point, given an environmental map, usually in the form of an occupancy grid. In this setting though, no special consideration is given to humans; they are thought of simply, as being obstacles in the environment, equivalent to chairs or walls. However, with more robots entering human spaces, special consideration needs to be given: humans need special treatment as obstacles, and furthermore humans can also serve the goal of goal points, towards starting a social interaction; either individual humans or groups of humans. Also, special mechanisms are required for engaging and disengaging in such interactions, taking into account psychological considerations of proxemics. In this paper, we first introduce our unifying theoretical framework for all the subproblems of social path planning; then, we propose an extended mode for engaging groups of people; and then, by using a special version of the fast-marching square planning method, we present and demonstrate actual algorithmic solutions for the social path planning subproblems. Our results prove the strengths of our approach and its generalizability. Finally, concrete further steps are discussed.

Although existing robotic systems are interesting to interact with in the short term, it has been shown that after some weeks of quasi-regular encounters, humans gradually lose their interest, and meaningful longer-term human-robot relationships are not established. An underlying hypothesis driving the proposed project is that such relationships can be significantly enhanced if the human and the robot are gradually creating a pool of shared episodic memories that they can co-refer to, and if they are both embedded in a social web of other humans and robots they both know and encounter frequently. Thus, here we propose to use Facebook, a highly successful online networking resource for humans, towards enhancing longer-term human-robot relationships, by helping to address the above two prerequisites. As a starting point, we utilize social information in order to personalize human-robot dialogues, and to include references to past encounters and to encounters with friends within dialogues. A robot equipped with a modular software architecture (with IPC-intercommunicating modules for face recognition, a simple dialog system, a navigation subsystem, and a real-time Facebook connection/local social database) has been deployed, and is encountering humans in the environment of our lab. An early demonstration of a basic form of such encounters is shown in the submitted video. The system is expected to achieve two significant novelties: arguably being one of the first robots to be embedded in a social web, and being the first robot that can purposefully exploit and create social information that is available online. Furthermore, it is expected to provide empirical support for our main driving hypothesis, that the formation of shared episodic memories within a social web can lead to more meaningful long-term human-robot relationships.

Our project aims at supporting the creation of sustainable and meaningful longer-term human-robot relationships through the creation of embodied robots with face recognition and natural language dialogue capabilities, which exploit and publish social information available on the web (Facebook). Our main underlying experimental hypothesis is that such relationships can be significantly enhanced if the human and the robot are gradually creating a pool of shared episodic memories that they can co-refer to (shared memories), and if they are both embedded in a social web of other humans and robots they both know and encounter (shared friends). In this paper, we are presenting such a robot, which as we will see achieves two significant novelties.

Although numerous early attempts towards the use of robotics in conjunction with virtual characters in order to create augmented reality theater and interactive performances have taken place, the state of the art is still in an initial stage on the pathway towards the full exploitation of the technological as well as creative and artistic potentialities that exist. Here we will present the IbnSina center, a novel augmented reality interactive theatre installation, named after the famous polymath of the 10th century, known as Avicenna in the west. The installation consists of a ten-meter stage, multiple stage sensors, a screen, a pseudo-3D holograph transparency, and a seating area for the audience. The stage is populated by a custom-designed humanlike humanoid robot (the ldquoIbnSinardquo robot) and by humans and other entities. The screen and the holograph can display static and moving images, virtual environments as well as online virtual worlds populated by characters, or a windowed/blended mix of the above. The robotic and virtual characters can be autonomous, partially-autonomous, puppetteered, scripted, or real-time-controlled by imitation of human body movements (embodied telepresence). Furthermore, multiple modes of participation of distant humans can be supported: not only through videoconference, but also through control of robots and/or virtual characters. The IbnSina center serves as a platform for multiple purposes: artistic, research as well as educational; and most importantly, the centrality of a progressive character such as IbnSina catalyzes the reconnection of the wider region of the UAE to a past during which scientific inquiry and the arts had flourished; and thus, enables the creation of a future for the region which will emphasize such cultural values.

Although numerous early attempts towards the use of robotics in conjunction with virtual characters in order to create augmented reality theater and interactive performances have taken place, the state of the art is still in an initial stage on the pathway towards the full exploitation of the technological as well as creative and artistic potentialities that exist. Here we will present the IbnSina Center, a novel augmented reality interactive theatre installation, named after the famous polymath of the 10th century, known as Avicenna in the West. The installation consists of a ten-meter stage, multiple stage sensors, a screen, a pseudo-3D holograph transparency, and a seating area for the audience. The stage is populated by a custom-designed humanlike humanoid robot (the ldquoIbnSinardquo robot) and by humans and other entities. The screen and the holograph can display static and moving images, virtual environments as well as online virtual worlds populated by characters, or a windowed / blended mix of the above. The robotic and virtual characters can be autonomous, partially-autonomous, puppetteered, scripted, or real-time-controlled by imitation of human body movements (embodied telepresence). Furthermore, multiple modes of participation of distant humans can be supported: not only through videoconference, but also through control of robots and/or virtual characters. The IbnSina center serves as a platform for multiple purposes: artistic, research as well as educational; and most importantly, the centrality of a progressive character such as IbnSina catalyzes the reconnection of the wider region of the UAE to a past during which scientific inquiry and the arts had flourished; and thus, enables the creation of a future for the region which will emphasize such cultural values.

This video illustrates some of the bahavioral capabilities achieved using an implementation of the proposed GSM architecture on the conversational robot Ripley, as introduced in the video, and as further described in the accompanying paper. The situation model acts as a "theatrical stage" in the robot's mind, filled in with present, past or imagined situations. A special tripley-layered design enables bidirectional translation between the sensory data and linguistic descriptions. In the video, you will see the robot: 1) Answering questions and serving motor commands, and verbalising uncertainty: What color are the objects? 2) "Imagining" objects when informed about their existence through language, talking about them without having seen them. Later, you will see him matching his sensory expectations with existing objects, and integrating sensory-derived information about the objects with language-derived. Imagine a blue object on the left! 3) Remembering past events, resolving temporal referents, and answering questions about the past. How big was the blue object when your head started moving? More videos of the system at author's site.

Our long-term objective is to develop robots that engage in natural language-mediated cooperative tasks with humans. To support this goal, we are developing an amodal representation and associated processes which is called a grounded situation model (GSM). We are also developing a modular architecture in which the GSM resides in a centrally located module, around which there are language, perception, and action-related modules. The GSM acts as a sensor-updated "structured blackboard", that serves as a workspace with contents similar to a "theatrical stage" in the robot's "mind", which might be filled in with present, past or imagined situations. Two main desiderata drive the design of the GSM: first, "parsing" situations into ontological types and relations that reflect human language semantics, and second, allowing bidirectional translation between sensory-derived data/expectations and linguistic descriptions. We present an implemented system that allows of a range of conversational and assistive behavior by a manipulator robot. The robot updates beliefs (held in the GSM) about its physical environment, the human user, and itself, based on a mixture of linguistic, visual and proprioceptive evidence. It can answer basic questions about the present or past and also perform actions through verbal interaction. Most importantly, a novel contribution of our approach is the robot's ability for seamless integration of both language- and sensor-derived information about the situation: for example, the system can acquire parts of situations either by seeing them or by "imagining" them through descriptions given by the user: "There is a red ball at the left". These situations can later be used to create mental imagery and sensory expectations, thus enabling the aforementioned bidirectionality

To build robots that engage in fluid face-to-face spoken conversations with people, robots must have ways to connect what they say to what they see. A critical aspect of how language connects to vision is that language encodes points of view. The meaning of my left and your left differs due to an implied shift of visual perspective. The connection of language to vision also relies on object permanence. We can talk about things that are not in view. For a robot to participate in situated spoken dialog, it must have the capacity to imagine shifts of perspective, and it must maintain object permanence. We present a set of representations and procedures that enable a robotic manipulator to maintain a "mental model" of its physical environment by coupling active vision to physical simulation. Within this model, "imagined" views can be generated from arbitrary perspectives, providing the basis for situated language comprehension and production. An initial application of mental imagery for spatial language understanding for an interactive robot is described.

Human cognition makes extensive use of visualization and imagination. As a first step towards giving a robot similar abilities, we have built a robotic system that uses a perceptually-coupled physical simulator to produce an internal world model of the robot's environment. Real-time perceptual coupling ensures that the model is constantly kept in synchronization with the physical environment as the robot moves and obtains new sense data. This model allows the robot to be aware of objects no longer in its field of view (a form of "object permanence"), as well as to visualize its environment through the eyes of the user by enabling virtual shifts in point of view using synthetic vision operating within the simulator. This architecture provides a basis for our long term goals of developing conversational robots that can ground the meaning of spoken language in terms of sensorimotor representations.

The paper presents several novel 3D image analysis algorithms, applied towards the segmentation and modeling of faces and hands. These are subsequently used to build a face-based authentication system and a system for human-computer interaction based on static and dynamic gestures. The system relies on an active stereo sensor that uses a structured light approach to obtain 3D information. In this paper we demonstrate how the use of 3D information may significantly improve the efficiency of traditional face and gesture recognition techniques that use 2D images only.