The environments in which the collaboration of a robot would be the most helpful to a person are frequently uncontrolled and cluttered with many objects present. Legible robot arm motion is crucial in tasks like these in order to avoid possible collisions, improve the workflow and help ensure the safety of the person. Prior work in this area, however, focuses on solutions that are tested only in uncluttered environments and there are not many results taken from cluttered environments. In this research we present a measure for clutteredness based on an entropic measure of the environment, and a novel motion planner based on potential fields. Both our measure and the planner were tested in a cluttered environment meant to represent a more typical tool-sorting task for which the person would collaborate with a robot. The in-person validation study with Baxter robots shows a significant improvement in legibility of our proposed legible motion planner compared to the current state-of-the-art legible motion planner in cluttered environments. Further, the results show a significant difference in the performance of the planners in cluttered and uncluttered environments, and the need to further explore legible motion in cluttered environments. We argue that the inconsistency of our results in cluttered environments with those obtained from uncluttered environments points out several important issues with the current research performed in the area of legible motion planners.

Facial expressiveness plays a crucial role in a robot’s ability to engage and interact with children. Prior research has shown that expressive robots can enhance child engagement during human-robot interactions. However, many robots used in therapy settings feature non-personalized, static faces designed with traditional facial feature considerations, which can limit the depth of interactions and emotional connections. Digital faces offer opportunities for personalization, yet the current landscape of robot face design lacks a dynamic, user-centered approach. Specifically, there is a significant research gap in designing robot faces based on child preferences. Instead, most robots in child-focused therapy spaces are developed from an adult-centric perspective. We present a novel study investigating the influence of child-drawn digital faces in child-robot interactions. This approach focuses on a design activity with children instructed to draw their own custom robot faces. We compare the perceptions of social intelligence (PSI) of two implementations: a generic digital face and a robot face, personalized using the user’s drawn robot faces. The results of this study show the perceived social intelligence of a child-drawn robot was significantly higher compared to a generic face.

In this paper, we focus on the field of maritime simulation games, also known as serious games or simulators, which serve as a vital tool for maritime education and training. These simulations offer a controlled and risk-free platform to mimic real-world situations, thereby aiding seafarers in learning essential skills such as ship maneuverability, collision prevention, and understanding other naval agents’ intentions. We present an implementation of a Unity-based naval simulator that enables the development of complex, multi-vessel navigation scenarios and provides multiple key capabilities relevant to the naval domain. First, the agent vessels are equipped with mathematical models to assess their capabilities and vulnerabilities. Second, a vulnerability heatmap is developed to illustrate the sensor and defense coverage of an agent or a group of agents. Third, a Closest Point of Approach (CPA) based action heatmap is developed to explore potential threats from the surrounding agents. Furthermore, these heatmaps are fused into a threat heatmap that encodes in real time an agent’s overall coverage and potential threats. In addition, vessel agents are equipped with Hidden Markov Model-based intent recognition models, to analyze the behavior of other agents around them. This paper describes the naval simulator with its capabilities and illustrates its main capabilities in various naval scenarios.

In human-robot collaboration, legible intent of the robot is critical to success as it enables the human to more effectively work with and around the robot. Environments where humans and robots collaborate are widely varied and in the real world are most often cluttered. However, prior work in legible motion utilizes primarily environments which are uncluttered. Success in these environments does not necessarily guarantee success in more cluttered environments. Furthermore, the prior work has been primarily performed based on results from robot-human studies and the problem has not been studied from the prospective of what people do to express intent to each other. Therefore, this work addresses a gap in current research into legible robot arm motion in the following ways: first we perform a human-human study in order to establish the factors which humans use to express their intent through body language, and second we perform the study in a cluttered and varied environment. Through the study we showed that the primary factors which people considered are: timing, kinematic parameters, hand gestures, object proximity, etc. The results also showed that legibility is correlated with perceived safety, perceived social intelligence, the collaborator’s contribution, and trust which further speaks to the importance of legible motion. Future work will utilize the pose data extracted from the study’s video recordings to develop a model for legible motion.

Mobile robots are increasingly populating homes, hospitals, shopping malls, factory floors, and other human environments. Human society has social norms that people mutually accept; obeying these norms is an essential signal that someone is participating socially with respect to the rest of the population. For robots to be socially compatible with humans, it is crucial for robots to obey these social norms. In prior work, we demonstrated a Socially-Aware Navigation (SAN) planner, based on Pareto Concavity Elimination Transformation (PaCcET), in a hallway scenario, optimizing two objectives so the robot does not invade the personal space of people. This article extends our PaCcET-based SAN planner to multiple scenarios with more than two objectives. We modified the Robot Operating System’s (ROS) navigation stack to include PaCcET in the local planning task. We show that our approach can accommodate multiple Human-Robot Interaction (HRI) scenarios. Using the proposed approach, we achieved successful HRI in multiple scenarios such as hallway interactions, an art gallery, waiting in a queue, and interacting with a group. We implemented our method on a simulated PR2 robot in a 2D simulator (Stage) and a pioneer-3DX mobile robot in the real-world to validate all the scenarios. A comprehensive set of experiments shows that our approach can handle multiple interaction scenarios on both holonomic and non-holonomic robots; hence, it can be a viable option for a Unified Socially-Aware Navigation (USAN).

For socially assistive robots (SAR)to be accepted into complex and stochastic human environments, it is important to account for subtle social norms. In this paper, we propose a novel approach to socially-aware navigation (SAN)which garnered an immense interest in the Human-Robot Interaction (HRI)community. We use a multi-objective optimization tool called the Pareto Concavity Elimination Transformation (PaC-cET)to capture the non-linear human navigation behavior, a novel contribution to the community. A candidate point on a trajectory is scored (1)for its progress towards the goal, and (2)based on autonomously-sensed distance-based features that capture the social norms and associated social costs. Rather than use a finely-tuned linear combination of these costs, we use PaCcET to select an optimized future trajectory point, associated with a non-linear combination of the costs. Existing research in this domain concentrates on geometric reasoning, model-based, and learning approaches, which have their own pros and cons. This approach is distinct from prior work in this area. We showed in a simulation that the PaCcET-based trajectory planner not only is able to avoid collisions and reach the intended destination in static and dynamic environments but also considers a human's personal space i.e. rules of proxemics in the trajectory selection process.

Horizon line is a promising visual cue which can be exploited for robot localization or visual geo-localization. Prominent approaches to horizon line detection rely on edge detection as a pre-processing step which is inherently a non-stable approach due to parameter choices and underlying assumptions. We present a novel horizon line detection approach which uses machine learning and Dynamic Programming (DP) to extract the horizon line from a classification map instead of an edge map. The key idea is assigning a classification score to each pixel, which can be interpreted as the likelihood of the pixel belonging to the horizon line, and representing the classification map as a multi-stage graph. Using DP, the horizon line can be extracted by finding the path that maximizes the sum of classification scores. In contrast to edge maps which are typically binary (edge vs no-edge) and contain gaps, classification maps are continuous and contain no gaps, yielding significantly better solutions. Using classification maps instead of edge maps allows for removing certain assumptions such as the horizon is close to the top of the image or that the horizon forms a straight line. The purpose of these assumptions is to bias the DP solution but they fail to produce good results when they are not valid. We demonstrate our approach on three different data sets and provide comparisons with a traditional approach based on edge maps. Although our training set is comprised of a very small number of images from the same location, our results illustrate that our method generalizes well to images acquired under different conditions and geographical locations.

The ability to detect human actions or gestures is key for a wide range of applications that involve interactions between humans and robots. These actions are patterns that have a particular spatio-temporal structure. This paper presents an approach for encoding such patterns using spike-timing networks with axonal conductance delays. The proposed method brings the following contributions: first, it enables the encoding of patterns in an unsupervised manner. Second, it allows us to create models of specific patterns using a very small set of training samples, in contrast with standard pattern recognition approaches that typically require large amounts of training data. Based on these models, the method further enables classification of new patterns using a longest-common subsequence approach for matching between patterns of activated neurons. Third, the approach is invariant to scale and translation and thus it enables generalization across multiple scales and positions. Fourth, the approach also enables early recognition of patterns from only partial information about the pattern. The proposed method is validated on a set of gestures representing the digits from 0 to 9, extracted from video data of a human drawing the corresponding digits. The results are also compared with other state of the art pattern recognition algorithms.

In this paper, we propose a new approach for recognizing intentions of humans by observing their activities with an RGB-D camera. Activities and goals are modeled as a distributed network of inter-connected nodes in an Activation Spreading Network (ASN). Inspired by a formalism in hierarchical task networks, the structure of the network captures the hierarchical relationship between high-level goals and low-level activities that realize these goals. Our approach can detect intentions before they are realized and it can work in real-time. We also extend the formalism of ASNs to incorporate contextual information into intent recognition. A fully functioning system is developed for experimental evaluation. We implemented a robotic system that uses our intent recognition to naturally interact with the user. Our ASN based intent recognizer is tested against two different scenarios involving everyday activities performed by a subject, and our results show that the proposed approach is able to detect low-level activities and recognize high-level intentions effectively in real-time. Further analysis shows that contextual ASN is able to discriminate between otherwise ambiguous goals.

We compare genetic algorithms against hill-climbers for generating competitive unit micro-management for winning real-time strategy game skirmishes. Good group positioning and movement, which are part of unit micro-management can help win skirmishes against equal numbers and types of opponent units or even when outnumbered. In this paper, we use influence maps to generate group positioning and potential fields to guide unit movement. We tested the behaviors obtained from genetic algorithm and two types of hill-climbing search against the default Starcraft AI using the brood war API. Preliminary results show that while our hill-climbers quickly find influence maps and potential fields that generate quality positioning and movement in our simulations, they only find quality solutions fifty to seventy percent of the time. On the other hand, genetic algorithms evolve high quality solutions a hundred percent of the time, but take significantly longer.

This paper investigates the evolution of group tactics and counter tactics for wargaming and real-time strategy games. Inspired by potential field methods in robotics, we compactly represent group behavior as a combination of several potential fields and evolve potential field parameters against hand-coded opponent groups. A novel real-coded evolutionary algorithm encourages tactic diversity by using a new diversity metric to mediate parent selection for recombination. Preliminary results indicate that we can quickly evolve counter tactics that beat hard coded opponent groups.

The ability to learn new concepts is essential for any robot to be successful in real-world applications. This is due to the fact that it is impractical for a robot designer to pre-endow it with all the concepts that it would encounter during its operational lifetime. In this context, it becomes necessary that the robot is able to acquire new concepts, in a real-world context, from cues provided in natural, unconstrained interactions, similar to a human-teaching approach. However, existing approaches on concept learning from visual images and abstract concept learning address this problem in a manner that makes them unsuitable for learning in an embodied, real-world environment. This paper presents a developmental approach to concept learning. The proposed system learns abstract, generic features of objects and associates words from sentences referring to those objects with the features, thus providing a grounding for the meaning of the words. The method thus allows the system to later identify such features in previously unseen images. The paper presents results obtained on data acquired with a Kinect camera and on synthetic images.

This paper addresses the problem of gender recognition by proposing a new feature descriptor to be used in classification. The contribution of this work is an extension to the local binary patterns traditionally used as descriptors. Local binary patterns include information about the relationship between a central pixel value and those of its neighboring pixels in a very compact manner. In the proposed method we incorporate into the descriptor more information from the neighborhood by using four predefined patterns, rather than just one, as in the classic model. We evaluate the performance of our method on the standard FERET database by comparing it to existing methods and show that we can extract more discriminative features and subsequently provide better gender recognition accuracy.

Effective human-robot interaction requires systems that can accurately infer and predict human intentions. In this paper, we introduce a system that uses stacked denoising autoencoders to perform intent recognition. We introduce the intent recognition problem, provide an overview of deep architectures in machine learning, and outline the components of our system. We also provide preliminary results for our system's performance.

As robots become more common across society, there is a pressing need to deal with questions of moral responsibility and legal liability in accidents involving semi-autonomous and autonomous machines. Previous attempts to address these questions have assumed machines with either minimal autonomy or full intelligence, and thus have not adequately considered the current and likely future state of the art in robotics and artificial intelligence. In this paper, we offer general principles to make sense of the foregoing issues, and propose a framework for addressing questions of responsibility and liability in human-robot interaction. This approach is based on the premise that robots can be analogized to animals for the purpose of assigning responsibility and liability when robots are involved in accidents. We provide justification for this approach, consider its implications, and discuss several of its advantages in analyzing human-robot interactions.

Understanding intent is an important aspect of communication among people and is an essential component of the human cognitive system. This capability is particularly relevant for situations that involve collaboration among agents or detection of situations that can pose a threat. In this paper, we propose an approach that allows a robot to detect intentions of others based on experience acquired through its own sensory-motor capabilities, then using this experience while taking the perspective of the agent whose intent should be recognized. Our method uses a novel formulation of Hidden Markov Models designed to model a robot's experience and interaction with the world. The robot's capability to observe and analyze the current scene employs a novel vision-based technique for target detection and tracking, using a non-parametric recursive modeling approach. We validate this architecture with a physically embedded robot, detecting the intent of several people performing various activities.

Background modeling is an essential and important part of many high-level video processing applications. Recently, the Support Vector Data Description (SVDD) has been introduced for novelty detection when only one class of data is available, i.e. background pixels. This paper proposes a method to efficiently train an SVDD and compares the performance of this training algorithm with the traditional SVDD training techniques. We compare the performance of our method with traditional SVDD and other classification algorithms on various data sets including real video sequences.

DNA matching is a crucial step in sequence alignment. Since sequence alignment is an approximate matching process there is a need for good approximate algorithms. The process of matching in sequence alignment is generally finding longest common subsequences. However, finding a longest common subsequence may not be the best solution for either a database match or an assembly. An optimal alignment of subsequences is based on several factors, such as quality of bases, length of overlap, etc. Factors such as quality indicate if the data is an actual read or an experimental error. Fuzzy logic allows tolerance of inexactness or errors in sub sequence matching. We propose fuzzy logic for approximate matching of subsequences. Fuzzy characteristic functions are derived for parameters that influence a match. We develop a prototype for a fuzzy assembler. The assembler is designed to work with low quality data which is generally rejected by most of the existing techniques. We test the assembler on sequences from two genome projects namely, Drosophila melanogaster and Arabidopsis thaliana. The results are compared with other assemblers. The fuzzy assembler successfully assembled sequences and performed similar and in some cases better than existing techniques

Research in context-aware user interfaces aims to improve human-computer interaction by providing more effective, smarter and user-friendlier solutions for computer applications. Currently, software available for performing such research and developing context-aware interfaces is very limited both in scope and possibilities of extension. Sycophant was designed with two objectives in mind: first, to allow easy insertion of new features and capabilities needed for conducting research and, second, to provide a reusable, readily available programming resource for developing new context-aware interactive software applications. Available as open source software, Sycophant's API and the calendaring application we created using it are presented in this paper in terms of functional capabilities, high level architecture, detailed design, and results of use. Procedural steps for developing new context-aware user interfaces using our API are also described in the paper.

In this article we present a computational approach to developing effective training systems for virtual simulation environments. In particular, we focus on a Naval simulation system, used for training of conning officers. The currently existing training solutions require multiple expert personnel to control each vessel in a training scenario, or are cumbersome to use by a single instructor. The inability of current technology to provide an automated mechanism for competitive realistic boat behaviors thus compromises the goal of flexible, anytime, anywhere training. In this article we propose an approach that reduces the time and effort required for training of conning officers, by integrating novel approaches to autonomous control within a simulation environment. Our solution is to developintelligent, autonomous controllersthat drive the behavior of each boat. To increase the system's efficiency we provide a mechanism for creating such controllers, from the demonstration of a navigation expert, using a simple programming interface. In addition, our approach deals with two significant and related challenges: therealism of behaviorexhibited by the automated boats and theirreal‐time response to changesin the environment. In this article, we describe the control architecture we developed that enables the real‐time response of boats and the repertoire of realistic behaviors we designed for this application. We also present our approach for facilitating the automatic authoring of training scenarios and we demonstrate the capabilities of our system with experimental results.

Recognizing human actions from video has been a challenging problem in computer vision. Although human actions can be inferred from a wide range of data, it has been demonstrated that simple human actions can be inferred by tracking the movement of the head in 2D. This is a promising idea as detecting and tracking the head is expected to be simpler and faster because the head has lower shape variability and higher visibility than other body parts (e.g., hands and/or feet). Although tracking the movement of the head alone does not provide sufficient information for distinguishing among complex human actions, it could serve as a complimentary component of a more sophisticated action recognition system. In this article, we extend this idea by developing a more general, viewpoint invariant, action recognition system by detecting and tracking the 3D position of the head using multiple cameras. The proposed approach employs Principal Component Analysis (PCA) to register the 3D trajectories in a common coordinate system and Dynamic Time Warping (DTW) to align them in time for matching. We present experimental results to demonstrate the potential of using 3D head trajectory information to distinguish among simple but common human actions independently of viewpoint.

A critical challenge in robot learning from demonstration is the ability to map the behavior of the trainer onto the robot's existing repertoire of basic/primitive capabilities. Following a behavior-based approach, we aim to express a teacher's demonstration as a linear combination (or fusion) of the robot's primitives. We treat this problem as a state estimation problem over the space of possible linear fusion weights. We consider this fusion state to be a model of the teacher's control policy expressed with respect to the robot's capabilities. Once estimated under various sensory preconditions, fusion state estimates are used as a coordination policy for online robot control to imitate the teacher's decision making. A particle filter is used to infer fusion state from control commands demonstrated by the teacher and predicted by each primitive. The particle filter allows for inference under the ambiguity over a large space of likely fusion combinations and dynamic changes to the teacher's policy over time. We present results of our approach in a simulated and real world environments with a Pioneer 3DX mobile robot

Behavior based architectures have many parameters that must be tuned to produce effective and believable agents. The authors used genetic algorithms to tune simple behavior based controllers for predators and prey. First, the predator tries to maximize area coverage in a large asymmetric arena with a large number of identically tuned peers. Second, the GA tunes the predator against a single prey agent. Then, two predators were tuned against a single prey. The prey evolves against a default predator and an evolved predator. The genetic algorithm finds high-performance controller parameters after a short length of time and outpaces the same controllers hand tuned by human programmers after only a small number of evaluations.

In this paper we address the problem of teaching robots to perform various tasks. We present a behavior-based approach that extends the capabilities of robots, allowing them to learn representations of complex tasks from their own experiences of interacting with a human, and to use the acquired knowledge to teach other robots in turn. A learner robot follows a human or robot teacher and maps its own observations of the environment to its internal behaviors, building at run-time a representation of the experienced task in the form of a behavior network. To enable this, we introduce an architecture that allows the representation and execution of complex and flexible sequences of behaviors and an online algorithm that builds the task representation from observations. We demonstrate our approach in a set of human(teacher)-robot(learner) and robot(teacher)-robot(learner) experiments, in which the robots learn representations for multiple tasks and are able to execute them even in environments with distractor objects that could hinder the learning and the execution process.

We focus on a robotic domain in which a human acts both as a teacher and a collaborator to a mobile robot. First, we present an approach that allows a robot to learn task representations from its own experiences of interacting with a human. While most approaches to learning from demonstration have focused on acquiring policies (i.e., collections of reactive rules), we demonstrate a mechanism that constructs high-level task representations based on the robot's underlying capabilities. Next, we describe a generalization of the framework to allow a robot to interact with humans in order to handle unexpected situations that can occur in its task execution. Without using explicit communication, the robot is able to engage a human to aid it during certain parts of task execution. We demonstrate our concepts with a mobile robot learning various tasks from a human and, when needed, interacting with a human to get help performing them.