This paper presents the first AI/ML system for automating building damage assessment in uncrewed aerial systems (sUAS) imagery to be deployed operationally during federally declared disasters (Hurricanes Debby and Helene). In response to major disasters, sUAS teams are dispatched to collect imagery of the affected areas to assess damage; however, at recent disasters, teams collectively delivered between 47GB and 369GB of imagery per day, representing more imagery than can reasonably be transmitted or interpreted by subject matter experts in the disaster scene, thus delaying response efforts. To alleviate this data avalanche encountered in practice, computer vision and machine learning techniques are necessary. While prior work has been deployed to automatically assess damage in satellite imagery, there is no current state of practice for sUAS-based damage assessment systems, as all known work has been confined to academic settings. This work establishes the state of practice via the development and deployment of models for building damage assessment with sUAS imagery. The model development involved training on the largest known dataset of post-disaster sUAS aerial imagery, containing 21,716 building damage labels, and the operational training of 91 disaster practitioners. The best performing model was deployed during the responses to Hurricanes Debby and Helene, where it assessed a combined 415 buildings in approximately 18 minutes. This work contributes documentation of the actual use of AI/ML for damage assessment during a disaster and lessons learned to the benefit of the AI/ML research and user communities.

This survey article investigates human-initiated touch interaction with a touch surface on a robot, reviewing 37 papers, with seven papers involving unprompted touch interaction and 30 papers involving prompted touch interaction. The 37 papers are examined in depth as to whether the touch interaction was unprompted, which of the nine touch surface characteristics (appearance, haptic feedback, light, material, movement, shape, size, texture, and temperature) were examined, and the methods used to measure the effects of the touch surface characteristics. This survey makes three findings. One is that no unprompted touch interaction provides an understanding of what factors influence its initiation. Two, four studies involving prompted touch suggest two methods for inferring influence on causation from attentiveness or engagement. Three, the touch surface characteristics involved in human-initiated touch interaction consist of nine characteristics, and the limited set of four touch surface characteristics presented within the unprompted touch literature should be expanded to the nine characteristics. The survey sets the foundation for future research into unprompted touch, especially for use in disasters and medical intervention, by (a) expanding the demographics, groups, and scenarios by investigating unprompted touch, (b) investigating the triggers and affordances associated with unprompted touch, and (c) investigating the interaction and affect touch surface characteristics have on one another and the touch interaction when combined.

This work presents the first quantitative study of alignment errors between small uncrewed aerial systems (sUAS) georectified imagery and a priori building polygons and finds that alignment errors are non-uniform and irregular, which negatively impacts field robotics systems and human-robot interfaces that rely on geospatial information. There are no efforts that have considered the alignment of a priori spatial data with georectified sUAS imagery, possibly because straight-forward linear transformations often remedy any misalignment in satellite imagery. However, an attempt to develop machine learning models for an sUAS field robotics system for disaster response from nine wide-area disasters using the CRASAR-U-DROIDs dataset uncovered serious translational alignment errors. The analysis considered 21,608 building polygons in 51 orthomosaic images, covering 16787.2 Acres (26.23 square miles), and 7,880 adjustment annotations, averaging 75.36 pixels and an average intersection over union of 0.65. Further analysis found no uniformity among the angle and distance metrics of the building polygon alignments, presenting an average circular variance of 0.28 and an average distance variance of 0.45 pixels2, making it impossible to use the linear transform used to align satellite imagery. The study’s primary contribution is alerting field robotics and human-robot interaction (HRI) communities to the problem of spatial alignment and that a new method will be needed to automate and communicate the alignment of spatial data in sUAS georectified imagery. This paper also contributes a description of the updated CRASAR-U-DROIDs dataset of sUAS imagery, which contains building polygons and human-curated corrections to spatial misalignment for further research in field robotics and HRI.

This paper details four principal challenges encountered with machine learning (ML) damage assessment using small uncrewed aerial systems (sUAS) at Hurricanes Debby and Helene that prevented, degraded, or delayed the delivery of data products during operations and suggests three research directions for future real-world deployments. The presence of these challenges is not surprising given that a review of the literature considering both datasets and proposed ML models suggests this is the first sUAS-based ML system for disaster damage assessment actually deployed as a part of real-world operations. The sUAS-based ML system was applied by the State of Florida to Hurricanes Helene (2 orthomosaics, 3.0 gigapixels collected over 2 sorties by a Wintra WingtraOne sUAS) and Debby (1 orthomosaic, 0.59 gigapixels collected via 1 sortie by a Wintra WingtraOne sUAS) in Florida. The same model was applied to crewed aerial imagery of inland flood damage resulting from post-tropical remnants of Hurricane Debby in Pennsylvania (436 orthophotos, 136.5 gigapixels), providing further insights into the advantages and limitations of sUAS for disaster response. The four challenges (variation in spatial resolution of input imagery, spatial misalignment between imagery and geospatial data, wireless connectivity, and data product format) lead to three recommendations that specify research needed to improve ML model capabilities to accommodate the wide variation of potential spatial resolutions used in practice, handle spatial misalignment, and minimize the dependency on wireless connectivity. These recommendations are expected to improve the effective operational use of sUAS and sUAS-based ML damage assessment systems for disaster response.

This paper introduces a differentiable representation for the optimization of boustrophedon path plans in convex polygons, explores an additional parameter of these path plans that can be optimized, discusses the properties of this representation that can be leveraged during the optimization process and shows that the previously published attempt at optimization of these path plans was too coarse to be practically useful. Experiments were conducted to show that this differentiable representation can reproduce scores from traditional discrete representations of boustrophedon path plans with high fidelity. Finally, optimization via gradient descent was attempted but found to fail because the search space is far more non-convex than was previously considered in the literature. The wide range of applications for boustrophedon path plans means that this work has the potential to improve path planning efficiency in numerous areas of robotics, including mapping and search tasks using uncrewed aerial systems, environmental sampling tasks using uncrewed marine vehicles, and agricultural tasks using ground vehicles, among numerous others applications.

In a structural collapse, debris piles up in a chaotic and unstable manner, creating pockets and void spaces that are difficult to see or access. Often, these regions have the highest chances of concealing survivors and identifying such regions can increase the success of a search and rescue (SAR) operation while ensuring the safety of both survivors and rescue teams. In this paper, we present an approach for ex post facto void detection in rubble piles by using registered 3D point clouds reconstructed from aerial images captured at multiple times on the scene. We perform a temporal layering of these point clouds to capture the dynamic surface of the rubble pile from multiple days of the SAR operation and analyze this 3D structure to detect candidate regions corresponding to void spaces. The layering is achieved by a parallel 3D point cloud reconstruction of the scene using the COLMAP Structure from Motion pipeline. The void detection is achieved by applying multiple point filtering criteria in thin segments of the 3D point clouds of the rubble. We test our approach on aerial images collected from the Surfside Structural Collapse at Miami in June 2021. Our method achieves an improvement in registration compared to the use of standard point cloud registration methods on individual 3D reconstructions. Through our method, we see translation errors reduce by 82%. Additionally, our method detects 9 out of 10 void spaces that were observed by experts in the rubble.

Data collected at Hurricane Ian (2022) quantifies the demands that small uncrewed aerial systems (UAS), or drones, place on the network communication infrastructure and identifies gaps in the field. Drones have been increasingly used since Hurricane Katrina (2005) for disaster response, however getting the data from the drone to the appropriate decision makers throughout incident command in a timely fashion has been problematic. These delays have persisted even as countries such as the USA have made significant investments in wireless infrastructure, rapidly deployable nodes, and an increase in commercial satellite solutions. Hurricane Ian serves as a case study of the mismatch between communications needs and capabilities. In the first four days of the response, nine drone teams flew 34 missions under the direction of the State of Florida FL-UAS1, generating 636GB of data. The teams had access to six different wireless communications networks but had to resort to physically transferring data to the nearest intact emergency operations center in order to make the data available to the relevant agencies. The analysis of the mismatch contributes a model of the drone data-to-decision workflow in a disaster and quantifies wireless network communication requirements throughout the workflow in five factors. Four of the factors-availability, bandwidth, burstiness, and spatial distribution-were previously identified from analyses of Hurricanes Harvey (2017) and Michael (2018). This work adds upload rate as a fifth attribute. The analysis is expected to improve drone design and edge computing schemes as well as inform wireless communication research and development.

Each natural disaster leaves a trail of destruction and damage that must be effectively managed to reduce its negative impact on human life. Any delay in making proper decisions at the post-disaster managerial level can increase human suffering and waste resources. Proper managerial decisions after any natural disaster rely on an appropriate assessment of damages using data-driven approaches, which are needed to be efficient, fast, and interactive. The goal of this study is to incorporate a deep interactive data-driven framework for proper damage assessment to speed up the response and recovery phases after a natural disaster. Hence, this paper focuses on introducing and implementing the Visual Question Answering (VQA) framework for post-disaster damage assessment based on drone imagery, namely Supervised Attention-Based VQA (SAM-VQA). In visual question answering, query-based answers from images regarding the situation in disaster-affected areas can provide valuable information for decision-making. Unlike other computer vision tasks, visual question answering is more interactive and allows one to get instant and effective scene information by asking questions in natural language from images. In this work, we present a VQA dataset and propose a novel supervised attention-based VQA framework (SAM-VQA) for post-disaster damage assessment on remote sensing images. Our model outperforms state-of-the-art attention-based VQA techniques, including Stacked Attention Networks (SAN) [1] and Multi-modal Factorized Bilinear (MFB) with Co-Attention [2]. Furthermore, our proposed model can derive appropriate visual attention based on questions to predict answers, making our approach trustworthy.

This work creates a model of the value of different external viewpoints of a robot performing tasks. The current state of the practice is to use a teleoperated assistant robot to provide a view of a task being performed by a primary robot; however, the choice of viewpoints is ad hoc and does not always lead to improved performance. This research applies a psychomotor approach to develop a model of the relative quality of external viewpoints using Gibsonian affordances. In this approach, viewpoints for the affordances are rated based on the psychomotor behavior of human operators and clustered into manifolds of viewpoints with the equivalent value. The value of 30 viewpoints is quantified in a study with 31 expert robot operators for four affordances (reachability, passability, manipulability, and traversability) using a computer-based simulator of two robots. The adjacent viewpoints with similar values are clustered into ranked manifolds using agglomerative hierarchical clustering. The results show the validity of the affordance-based approach by confirming that there are manifolds of statistically significantly different viewpoint values, viewpoint values are statistically significantly dependent on the affordances, and viewpoint values are independent of a robot. Furthermore, the best manifold for each affordance provides a statistically significant improvement with a large Cohen's d effect size (1.1-2.3) in the performance (improving time by 14%-59% and reducing errors by 87%-100%) and improvement in the performance variation over the worst manifold. This model will enable autonomous selection of the best possible viewpoint and path planning for the assistant robot.

In this paper, we present a large-scale hurricane Michael dataset for visual perception in disaster scenarios, and analyze state-of-the-art deep neural network models for semantic segmentation. The dataset consists of around 2000 high-resolution aerial images, with annotated ground-truth data for semantic segmentation. We discuss the challenges of the dataset and train the state-of-the-art methods on this dataset to evaluate how well these methods can recognize the disaster situations. Finally, we discuss challenges for future research.

The emergence of small unmanned aerial vehicles (UAV) along with inexpensive sensors presents the opportunity to collect thousands of images after each natural disaster with high flexibility and easy maneuverability for rapid response and recovery. Despite the ease of data collection, data analysis of the big datasets remains a significant barrier for scientists and analysts. Here we propose an integration of densely connected CNN and RNN networks, which is able to accurately segment out semantically meaningful object boundaries with end-to-end learning. The proposed network is applied on UAV aerial images of flooded areas in Houston, TX. We achieved 96% accuracy in detecting flooded areas on a large UAV dataset.

This paper develops and compares two motion planning algorithms for a tethered UAV with and without the possibility of the tether contacting the confined and cluttered environment. Tethered aerial vehicles have been studied due to their advantages such as power duration, stability, and safety. However, the disadvantages brought in by the extra tether have not been well investigated by the robotic locomotion community, especially when the tethered agent is locomoting in a non-free space occupied with obstacles. In this work, we propose two motion planning frameworks that (1) reduce the reachable configuration space by taking into account the tether and (2) deliberately plan (and relax) the contact point(s) of the tether with the environment and enable an equivalent reachable configuration space as the non-tethered counterpart would have. Both methods are tested on a physical robot, Fotokite Pro. With our approaches, tethered aerial vehicles could find their applications in confined and cluttered environments with obstacles as opposed to ideal free space, while still maintaining the advantages from the usage of a tether. The motion planning strategies are particularly suitable for marsupial heterogeneous robotic teams, such as visual servoing/assisting for another mobile, tele-operated primary robot.

This research teams an Unmanned Surface Vehicle (USV) with an Unmanned Aerial Vehicle (UAV) to augment and automate marine mass casualty incident search and rescue in emergency response phase. The demand for real-time responsiveness of those missions requires fast and comprehensive situational awareness and precise operations, which are challenging to achieve because of the large area and the flat nature of the water field. The responders, drowning victims, and rescue vehicle are far apart and all located at the sea level. The long distances mean responders cannot clearly discern the rescue vehicle and victims from the surrounding water. Furthermore, being at the same elevation makes depth perception difficult. Rescue vehicle and victims at different distances from the responder will always appear to be close together. This makes it almost impossible for the responders to accurately drive the USV to the victims in time. This paper proposes the use of a UAV to compensate for the lack of elevation of the responders and to automate search and rescue operations. The benefit of this system is two fold: 1) the UAV provides responders with an overhead view of the field, covers larger area than direct visual, and allows more accurate perception of the situation, and 2) it automates the rescue process so that the responders can focus on task-level needs instead of tediously driving the USV to the victims. Thirty autonomous navigation trials in 4 rescue scenarios prove the first known successful implementation of a small UAV visually navigating a USV.

This article will present a simulation-based approach to testing multiple variables in the behavior of a small Unmanned Aerial Vehicle (sUAV), inspired by insect and animal motions, to understand how these variables impact time of interaction, preference for interaction, and distancing in Human-Robot Interaction (HRI). Previous work has focused on communicating directionality of flight, intentionality of the robot, and perception of motion in sUAVs, while interactions involving direct distancing from these vehicles have been limited to a single study (likely due to safety concerns). This study takes place in a Cave Automatic Virtual Environment (CAVE) to maintain a sense of scale and immersion with the users, while also allowing for safe interaction. Additionally, the two-alternative forced-choice method is employed as a unique methodology to the study of collocated HRI in order to both study the impact of these variables on preference and allow participants to choose whether or not to interact with a specific robot. This article will be of interest to end-users of sUAV technologies to encourage appropriate distancing based on their application, practitioners in HRI to understand the use of this new methodology, and human-aerial vehicle researchers to understand the perception of these vehicles by 64 naive users. Results suggest that low speed (by 0.27m, p < 0.02) and high cyclicity (by 0.28m, p < 0.01) expressions can be used to increase distancing; that low speed (by 4.4s, p < 0.01) and three-dimensional (by 2.6s, p < 0.01) expressions can be used to decrease time of interaction; and low speed (by 10.4%, p < 0.01) expressions are less preferred for passability in human-aerial vehicle interactions.

This work provides a case study of both fixed-wing and rotorcraft small unmanned aerial systems (SUAS) used in a deployment at the SR530 mudslides in Washington state and compares the types of flight paths used by each vehicle type. Previously aerial imagery from SUAS have produced 2D and 3D reconstructions of simple terrain, but have not been used in complex terrain which encompasses both flat areas and drastic changes in the height of ground level, such as a mudslide. In this deployment, both types of SUAS platforms were used to collect imagery over terrain varied nearly 200m in elevation but different paths were used due to the complexity of the terrain, safety, privacy, and platform-specific limitations. The deployment found that paths with fixed-wing platforms can be thought of as stacked horizontal planes while rotorcraft can cover complex terrain with a set of vertical planes. The different paths contribute to autonomous path planning, particularly to accommodate vertical planes, and to general understanding of how different SUAS can be applied to challenging terrains. Future work in path planning should incorporate Geographic Information Systems (GIS) information to facilitate flight paths in vertical planes and to maintain altitude restrictions relative to radically changing elevations of a landscape.

In this exploratory study, participants taking cover from a simulated earthquake interacted with a search-and-rescue robot that assumed one of four different identities, distinguished by varying degrees of autonomy, and whose communications were either clear or noisy. Results showed that identities with low autonomy elicited greater hopefulness from participants than identities with high autonomy. Discussion focuses on design recommendations for search-and-rescue robots, and on the design of immersive HRI experiments.

Sky Writer is a collaborative communication medium that augments the traditional display of a UAV pilot and allows other stakeholders to communicate their needs and intentions to the pilot. UAV pilots engaging in time-critical missions, such as urban disaster responses, often must allocate most of their cognitive capacity towards flight tasks, making communication and collaboration with other stakeholders difficult or dangerous. Sky Writer addresses the needs of stakeholders while requiring minimal cognitive effort from the UAV pilot. The application presents stakeholders with an interface that provides contextual flight information and a live video stream of the flight. Stakeholders are able to sketch directly on the video stream or use a spotlight indicator that is mirrored across all displays in the system, including the pilot's display. The application can be used in any modern web browser and works with traditional and touch devices. Concept experimentation performed at Disaster City with two pilots indicated that the spotlight feature was particularly useful while the UAV was in motion, and the sketching features were most useful while the UAV was stationary. The system will be tested with professional responders soon to determine its efficacy in a simulated response, and to inform the ongoing design process.

This paper introduces and empirically evaluates two scaling functions to alter a robot's physical movements based on proximity to a human. Previous research has focused on individual aspects of proxemics, like the appropriate distance to maintain from a human, but has not explored autonomous methods to adapt robot behavior as proximity changes. This paper proposes that robots in a social role should modify their behavior using a continuous function mapped to proximity. The method developed calculates a gain value from proximity readings, which is used to shape the execution of active behaviors on the robot. In order to identify the effects of different mappings from proximity to gain value, two different scaling functions were implemented on an affective search and rescue robot. The findings from a 72 participant study, in a high-fidelity mock disaster site, are examined with attention given to a new measure to determine proxemic awareness. The results indicated that for attributes of intelligence, likability, proxemic awareness, and submissiveness, a logarithmic-based scaling function is preferred over a linear-based scaling function, and over no scaling function. In areas of participant comfort and participant stress, the results indicated both logarithmic and linear scaling functions were preferred to no scaling.

This study demonstrates that robots can achieve socially acceptable interactions using loosely synchronized head gaze-speech acts. Prior approaches use tightly synchronized head gaze-speech, which requires significant human effort and time to manually annotate synchronization events in advance, restricts interactive dialog, or requires that the operator acts as a puppeteer. This paper describes how autonomous synchronization of head gaze can be achieved by exploiting affordances in the sentence structure and time delays. A 93-participant user study was conducted in a simulated disaster site. The rescue robot “Survivor Buddy” generated head gaze for a victim management scenario using a 911 dialog. The study used pre- and postinteraction questionnaires to compare the social acceptance level of loosely synchronized head gaze-speech against tightly synchronized head gaze-speech (manual annotation) and no head gaze-speech conditions. The results indicated that for attributes of Self-Assessment Manikin, i.e., Arousal, Robot Likeability, Human-Like Behavior, Understanding Robot Behavior, Gaze-Speech Synchronization, Looking at Objects at Appropriate Times, and Natural Movement, the loosely synchronized head gaze-speech is similar to tightly synchronized head gaze-speech and preferred to the no head gaze-speech case. This study contributes to a fundamental understanding of the role of social head gaze in social acceptance for human-machine interaction, how social gaze can be produced, and promotes practical implementation in social robots.

This paper presents the first known human-subject study of comfortable approach distance and height for human interaction with a small unmanned aerial vehicle (sUAV), finding no conclusive difference in comfort with a sUAV approaching a human at above head height or below head height. Understanding the amount, if any, of discomfort introduced by a sUAV flying in close proximity to a human is critical for law enforcement, crowd control, entertainment, or flying personal assistants. Previous work has focused on how humans interact with each other or with unmanned ground vehicles, and the experimental methods typically rely on the human participant to consciously express distress. The approach taken was to duplicate the experimental set up in human proxemics studies, while adding psychophysiological sensing, under the hypothesis that human-robot interaction will mirror human-human interaction. The 16 participant, within-subjects experiment did not confirm this hypothesis. Instead a sUAV above height of a “tall” person in human experiments (2.13 m) did not produce statistically different heart rate variability nor cause the participant to stop the robot further away than for a sUAV at a “short” height (1.52 m). The lack of effect may be due to two possible confounds: i) duplicating prior human proxemics experiments did not capture how a sUAV would likely move or interact and ii) telling the participants that the robot could not hurt them. Despite possible confounding, the results raise the question of whether human-human psychological and physical distancing behavior transfers to human-aerial robot interactions.

This paper surveys the human-machine interaction technologies supporting the Mission Specialist role in unmanned aerial systems (UASs). The Mission Specialist role is one of three formal human team member roles extracted from the UAS-related literature (the others are Flight Director and Pilot), but unlike the Pilot role, the interface needs have not been established. The interfaces used by 17 micro, small, medium altitude long endurance (MALE) and high altitude long endurance (HALE) platforms are examined to determine (1) what type of user interface technologies are present and/or available; (2) how the Mission Specialist currently or could interact with the user interface technology; and (3) what are the perceived positive and negative aspects of this user interface technology in the context of the UAS human-robot team roles. Micro and small UAVs pose significant user interface limitations for the Mission Specialist role and may produce unintentional interaction conflicts between the Mission Specialist role and the Pilot, potentially resulting in suboptimal performance and loss of robustness. The survey is expected to serve as a reference for future design and refinement of user interfaces for UAS and a foundation for better understanding human-robot interaction in UAS.

Humans regulate their social behavior based on proximity to other social actors. Likewise, when a robot fulfills the role of a social actor it too should regulate its interaction based on proximity. This paper describes work in progress to establish methods for autonomous modification of social behavior based on proximity and to quantify human preferences between methods of scaling a robot's social behaviors based on distance from a human. The preliminary results of a 72 participant human study examine the reaction to scaling with linear methods and perception-based methods. Results indicate significantly higher ratings in multiple areas (comfort, natural movement, safety, self-control, intelligence, likability, submissiveness (p<.05) when using a perception-based scaling function, as opposed to a linear or no scaling function. Work in progress is analyzing the biometric measures collected.

This video offers a retrospective of the 26 disasters where land, marine, or aerial robots have been deployed, starting with the first use of rescue robots at the 2001 World Trade Center Collapse and continuing through the 2011 Tohoku Tsunami/Fukushima nuclear event. While robots have not found a living survivor, possibly because they arrive on average 4.5 days after the event, the video clips illustrate the contributions of robots to searching for survivors, reconnaissance and mapping, inspection of buildings, and inspection of bridges and shipping channels. The clips also highlight four open research questions: human-robot interaction, mobile manipulation, reliable wireless networks, and obstacle avoidance for UAVs and UMVs. Use of robots is accelerating and their contributions to disaster prevention, preparedness, response, and recovery will grow and lead to new challenges for research.

This paper presents a literature review on the use of depth for hand tracking and gesture recognition. The survey examines 37 papers describing depth-based gesture recognition systems in terms of (1) the hand localization and gesture classification methods developed and used, (2) the applications where gesture recognition has been tested, and (3) the effects of the low-cost Kinect and OpenNI software libraries on gesture recognition research. The survey is organized around a novel model of the hand gesture recognition process. In the reviewed literature, 13 methods were found for hand localization and 11 were found for gesture classification. 24 of the papers included real-world applications to test a gesture recognition system, but only 8 application categories were found (and three applications accounted for 18 of the papers). The papers that use the Kinect and the OpenNI libraries for hand tracking tend to focus more on applications than on localization and classification methods, and show that the OpenNI hand tracking method is good enough for the applications tested thus far. However, the limitations of the Kinect and other depth sensors for gesture recognition have yet to be tested in challenging applications and environments.

This video describes the Survivor Buddy social medium robot.

We have created a preliminary inference engine for generating gaze acts based on extracting the social context from conversational structure and timing in human-robot dialog.

No abstract available.

Based on a synthesis of eight major studies using six robots involving social gaze in robotics, this research proposes a novel behavioral definition as a mapping G = E(C) from the perception of a social context C to a set of head, eye, and body patterns called gaze acts G that expresses the engagement E. This definition places social gaze within the behavior-based programming framework for robots and agents, providing a guide for principled future implementations. The research also identifies five social contexts, or functions, of social gaze (Establishing agency, Communicating social attention, Regulating the interaction process, Manifesting interaction content and Projecting mental state) along with six discrete gaze acts for social gaze functions (Fixation, Short glance, Aversion, Concurrence, Confusion, and Scan) that have been employed by various robots or in simulation for these contexts. The research contributes to a computational understanding of social gaze that bridges psychological, cognitive, and robotics communities.

This paper describes an open source speech translator toolkit created as part of the "Survivor Buddy" project which allows written or spoken word from multiple independent controllers to be translated into either a single synthetic voice, synthetic voices for each controller, or unchanged natural voice of each controller. The human controllers can work over the internet or be physically co-located with the Survivor Buddy. The toolkit is expected to be of use for exploring voice in general human-robot interaction.

Designing and constructing affective robots on schedule and within costs is especially challenging because of the qualitative, artistic nature of affective expressions. Detailed affective design principles do not exist, forcing an iterative design process. This paper describes a three step design process created for the Survivor Buddy project that engages artists in the design process and allows animation to guide physical implementation. The process combines creative design of believable agents unconstrained by costs with traditional design decision matrices. The paper provides a case study comparing the resulting design of the Survivor Buddy 2.0 robot with the original (Survivor Buddy 1.0). The multi-disciplinary methodology produced a more pleasing and expressive robot that was 50% less expensive, 78% lighter, and up to 700% faster within the same amount of design time. This methodology is expected to contribute to reducing risk in designing cost effective affective robots and robots in general.

A robot is being developed for urban search and rescue missions. USAR Whegs™ implements several new features into Whegs™ robot design. It is the first quadruped Whegs™ robot of this scale. It uses differential steering and the user can rapidly change its running gear to and from tracks and wheel-legs. This is also the first implementation of carbon fiber wheel-legs on a Whegs™ vehicle. The carbon-fiber reduces the mass moment of inertia eight times compared to previous aluminum designs. The running gear can be changed in 30 seconds and the resulting connections are secure. GeoSystems Zippermast allows a camera to be deployed as high as eight feet above the robot. The robot is 47.6 cm long, can travel 1.9 meters per second on its tracks, and can climb 15 cm obstacles using its wheel-legs. A two-speed transmission is being developed to permit it to run more slowly on wheel-legs for better control on irregular terrain.

Summary form only given. TMA was established in 1997 as an R&D software outsourcing company, and has grown from a small group of 6 engineers in 1997, to over 1000 people today, serving many large customers around the world. In 2009, when TMA celebrated the first 12-year cycle, as part of the development strategy for the next cycle, we embarked on a new imitative to set up an R&D center to leverage the human and financial capital we have accumulated from R&D outsourcing, and the infrastructure we have built, to research and develop IT-enabled applications. The emphasis is on applications, based on ideas from ourselves, and from our partners around the world. This presentation will share with you the achievements, the lessons learned, and the call for collaboration.

This half-day tutorial will cover topics related to conducting cognitive task analysis and cognitive work analysis for purposes of informing human-robot interaction design and development. The goal of the tutorial is to provide attendees with an overview and comparison of various cognitive task analysis and cognitive work analysis methods, an understanding of how to conduct these types of analyses, collect the necessary data for analysis, and provide real-world case studies for specific cognitive task analysis and cognitive work analysis. The tutorial will include examples from actual analyses and data collection activities.

The introduction of two types of unmanned aerial vehicles into a production of A Midsummer Night's Dream suggests that social proof informs untrained human groups. We describe the metaphors used in instructing actors, who were otherwise untrained and inexperienced with robots, in order to shape their expectations. Audience response to a robot crash depended on whether the audience had seen how the actors interacted with the robot "baby fairies." If they had not seen the actors treating a robot gently, an audience member would likely throw the robot expecting it to fly or handle it roughly. If they had seen the actors with the robots, the audience appeared to adopt the same gentle style and mechanisms for re-launching the micro-helicopter. The difference in audience behavior suggests that the principle of social proof will govern how untrained humans will react to robots.

On the current situation where robots having functions of communication with humans begin to appear in daily-life fields, it should be considered how symbiosis of humans and robots can be achieved. Many existing studies have focused on how robots can learn from and adapt for humans. This full-day workshop focuses not only on this classical theme but also on how humans can learn in and adapt for environments where robots are acting. In particular, human learning from and adaptation for robots should be covered by interdisciplinary research fields including robotics, computer science, psychology, sociology, and pedagogy.

This paper describes the Survivor Buddy human-robot interaction project and how it was used by four middle-school girls to illustrate the scientific process for an episode of "SciGirls", a Public Broadcast System science reality show. Survivor Buddy is a four degree of freedom robot head, with the face being a MIMO 740 multi-media touch screen monitor. It is being used to explore consistency and trust in the use of robots as social mediums, where robots serve as intermediaries between dependents (e.g., trapped survivors) and the outside world (doctors, rescuers, family members). While the SciGirl experimentation was neither statistically significant nor rigorously controlled, the experience makes three contributions. It introduces the Survivor Buddy project and social medium role, it illustrates that human-robot interaction is an appealing way to make robotics more accessible to the general public, and raises interesting questions about the existence of a minimum set of degrees of freedom for sufficient expressiveness, the relative importance of voice versus non-verbal affect, and the range and intensity of robot motions.

This paper surveys sensing assessment solutions from the literature with a particular focus on techniques which can be used in unknown environments, including the following: sensor fault detection and identification (FDI), sensor or source evaluation, and isolating poorly sensed regions. Each approach is evaluated in terms of its ability to perform sensing assessment tasks in unknown environments and its coverage of the range of potential sensing problems. These tasks include sensing problem detection and characterization, as well as performance evaluation (e.g., estimating accuracy or reliability), for a sensor or group of sensors. This survey shows that over 40 existing approaches are focused on either detection and identification of traditional sensor faults (e.g., drift or physical damage) in known environments or evaluation of the reliability of a source (e.g., sensor or agent). Only eight approaches surveyed have tackled environment-dependent problems (e.g., exteroceptive sensor FDI, miscalibration, or use of an inappropriate sensor) in a useful manner for unknown environments. Even less work (two studies) appears to have been done on isolating poorly sensed regions. The survey concludes with a list of opportunities for future research, including developing methods for detecting and characterizing environment-dependent problems and creating comprehensive sensing assessment systems.

Optical flow computation in vision-based systems demands substantial computational power and storage area. Hence, to enable real-time processing at high resolution, the design of application-specific system for optic flow becomes essential. In this paper, we propose an efficient VLSI architecture for the accurate computation of the Lucas-Kanade (L-K)-based optical flow. The L-K algorithm is first converted to a scaled fixed-point version, with optimal bit widths, for improving the feasibility of high-speed hardware implementation without much loss in accuracy. The algorithm is mapped onto an efficient VLSI architecture and the data flow exploits the principles of pipelining and parallelism. The optical flow estimation involves several tasks such as Gaussian smoothing, gradient computation, least square matrix calculation, and velocity estimation, which are processed in a pipelined fashion. The proposed architecture was simulated and verified by synthesizing onto a Xilinx Field Programmable Gate Array, which utilize less than 40% of system resources while operating at a frequency of 55 MHz. Experimental results on benchmark sequences indicate 42% improvement in accuracy and a speed up of five times, compared to a recent hardware implementation of the L-K algorithm.

This video presents a visual summary of large-scale, complex human study in Human-Robot Interaction (HRI) designed to evaluate whether humans would view interactions with two non-anthropomorphic robots more positively and calming when the robots were operated in an emotive mode versus a standard, non-emotive mode. The video presents actual participants' reactions, the study design, and images from search and rescue operations.

This paper describes preliminary results of a large-scale, complex human study in HRI in which results show that participants were calmer interacting with non-anthropomorphic robots operated in an emotive mode versus a standard, non-emotive mode.

This video captures human-robot interaction which occurred during an evaluation of a novel, snake-like search and rescue robot assisting with victim management. Most of the observations confirmed previous findings- That a 2:1 H-R ratio ratio is appropriate, Team coordination is enhanced by shared visual perception, and Poor interfaces continue to lead to incomplete coverage. However, the victims responded to the robot in two surprising ways: grabbing the robot and being concerned about its appearance.

No abstract available.

A crew organization and four-step operational protocol is recommended based on a cumulative descriptive field study of teleoperated rotary-wing micro air vehicles (MAV) used for structural inspection during the response and recovery phases of Hurricanes Katrina and Wilma. The use of MAVs for real civilian missions in real operating environments provides a unique opportunity to consider human-robot interaction. The analysis of the human-robot interaction during 8 days, 14 missions, and 38 flights finds that a three person crew is currently needed to perform distinct roles: Pilot, Mission Specialist, and Flight Director. The general operations procedure is driven by the need for safety of bystanders, other aircraft, the tactical team, and the MAV itself, which leads to missions being executed as a series of short, line-of-sight flights rather than a single flight. Safety concerns may limit the utility of autonomy in reducing the crew size or enabling beyond line-of-sight-operations but autonomy could lead to an increase in flights per mission and reduced Pilot training demands. This paper is expected to contribute to set a foundation for future research in HRI and MAV autonomy and to help establish regulations and acquisition guidelines for civilian operations. Additional research in autonomy, interfaces, attention, and out-of-the-loop (OOTL) control is warranted.

This video provides a preliminary report of the use of a rescue robot from Aug. 24, to Sept. 2, 2007, at the Crandall Canyon, Utah, mine disaster. The customized Inuktun teleoperated robot was able to traverse over 1,400 feet through an 8 7/8" uncased borehole drilled, enter the mine and travel approximately 7 feet. The robot showed that the walls had deteriorated, indicating that a major collapse had occurred. The large debris combined with dense mud created unfavorable navigational conditions for the robot. The robot was lost on the ascent, approximately 52 feet from the surface, due to eroding borehole conditions. The video identifies open research questions.

This paper presents the results from experiments validating the physics and environmental accuracy of a new robot simulation environment, the search and rescue game environment (SARGE), which is the foundation for series of robot-operator training games. An ATRV-Jr. outfitted with a SICK laser, GPS, and compass was used both in the real-world and in a simulated environment modeled after the real-world testing location in a simulated anomaly detection task. The ARTV-Jr., controlled by the Distributed Field Robotics Architecture, navigated through a series of waypoints in the environment. The simulated ATRV-Jr. matched the actions of the real ATRV-Jr. in both velocity and path similarity within 0.08 m/s and 0.7 m respectively.

Non-facial and non-verbal methods of affective expression are essential for naturalistic social interaction in robots that are designed to be functional and lack expressive faces (appearance-constrained) such as those used in search and rescue, law enforcement, and military applications. This correspondence identifies five main methods of non-facial and non-verbal affective expression (body movement, posture, orientation, color, and sound), and ranks their effectiveness forappearance-constrainedrobots operating within theintimate, personal, andsocialproximity zones of a human corresponding to interagent distances of approximately 3 m or less. This distance is significant because it encompasses the most common human social interaction distances, the exception being thepublicdistance zone used for formal presentations. The correspondence complements prior, broad surveys of affective expression by reviewing the psychology, computer science, and robotics literature specifically relating the impact of social interaction in non-anthropomorphic andappearance-constrainedrobots, and summarizing robotic implementations that utilize non-facial and non-verbal methods of affective expression as their primary means of expression. The literature is distilled into a set of prescriptive recommendations of the appropriate affective expression methods for each of the three proximity zones of interest. These recommendations serve as design guidelines for retroactively adding affective expression through software to a robot without physical modifications or designing a new robot.

This work applies a previously developed set of heuristics for determining when to use non-facial/non-verbal methods of affective expression to the domain of a robot being used for victim assessment in the aftermath of a disaster. Robot-assisted victim assessment places a robot approximately three meters or less from a victim, and the path of the robot traverses three proximity zones (intimate (contact -- 0.46m), personal (0.46 -- 1.22 m), and social (1.22 -- 3.66 m)). Robot- and victim-eye views of an Inuktun robot were collected as it followed a path around the victim. The path was derived from observations of a prior robot-assisted medical reachback study. The victim's-eye views of the robot from seven points of interest on the path illustrate the appropriateness of each of the five primary non-facial/non-verbal methods of affective expression: (body movement, posture, orientation, illuminated color, and sound), offering support for the heuristics as a design aid. In addition to supporting the heuristics, the investigation identified three open research questions on acceptable motions and impact of the surroundings on robot affect.

This study presents mobile robots as a way of augmenting communication in distributed teams through a remote shared visual presence (RSVP) consisting of the robot's view. By giving all team members access to the shared visual display provided by a robot situated in a remote workspace, the robot can serve as a source of common ground for the distributed team. In a field study examining the effects of remote shared visual presence on team performance in collocated and distributed Urban Search & Rescue technical search teams, data were collected from 25 dyadic teams comprised of US&R task force personnel drawn from high-fidelity training exercises held in California (2004) and New Jersey (2005). They performed a 2 x 2 repeated measures search task entailing robot-assisted search in a confined space rubble pile. Multilevel regression analyses were used to predict team performance based upon use of RSVP (RSVP or no-RSVP) and whether or not team members had visual access to other team members. Results indicated that the use of RSVP technology predicted team performance ( ß= -1.24, p<.05). No significant differences emerged in performance between teams with and without visual access to their team members. Findings suggest RSVP may enable distributed teams to perform as effectively as collocated teams. However, differences detected between sites suggest efficiency of RSVP may depend on the user's domain experience and team cohesion.

This report presents a survey of computer based simulators for unmanned vehicles. The simulators examined cover a wide spectrum of vehicles including unmanned aerial vehicles, both full scale and micro size; unmanned surface and subsurface vehicles; and unmanned ground vehicles. The majority of simulators use simple numerical simulation and simplistic visualization using custom OpenGL code. An emerging trend is to used modified commercial game engines for physical simulation and visualization. The game engines that are commercially available today are capable of physical simulations providing basic physical properties and interactions between objects. Newer and/or specialized engines such as the flight simulator X-Plane or Ageia PhysX and Havok physics engines, are capable of simulating more complex physical interactions between objects. Researchers in need of a simulator have a choice of using game engines or available open source and commercially available simulators, allowing resources to be focused on research instead of building a new simulator. We conclude that it is no longer necessary to build a new simulator from scratch.

This paper demonstrates the use of social roles to enable taskability in multi-robot teams based on a study of roles from the social sciences as well as related work in software agents. It first provides a survey of the current state of role- based robotics. Then, it offers specific examples of how roles can enable behavior with a team of heterogeneous robots using the Distributed Field Robot Architecture to integrate with a cognitive agent. The implementation extends the robot persona, previously utilized for allocating resources within a distributed robot team, and constructs a context adapter to allow each robot to assume a role as directed by the cognitive agent.

This paper reviews the literature related to the use of psychophysiology measures in human-robot interaction (HRI) studies in an effort to address the fundamental question of appropriate metrics and methodologies for evaluating HRI research, especially affect. It identifies four main methods of evaluation in HRI studies: (1) self-report measures, (2) behavioral measures, (3) psychophysiology measures, and (4) task performance. However, the paper also shows that using only one of these measures for evaluation is insufficient to provide a complete evaluation and interpretation of the interactions between a robot and the human with which it is interacting. In addition, the paper describes exemplar HRI studies which use psychophysiological measures; these implementations fall into three categories: detection and/or identification of specific emotions of participants from physiological signals, evaluation of participants' responses to a robot through physiological signals, and development and implementation of real-time control and modification of robot behaviors using physiological signals. Two open research questions on psychophysiological metrics were identified as a result of this review.

No abstract available.

On Oct 24, 2005, Hurricane Wilma made landfall at Cape Romano, Florida. Two days later, the Center for Robot-Assisted Search and Rescue (CRASAR) deployed an iSENSYS helicopter and an unmanned surface vehicle to survey damage in parts of Marco Island, 11 miles from landfall. Assistance was provided by the National Science Foundation's industry/university cooperative research center on safety security rescue technologies. The AEOS-1 USV was prototype built for environmental science studies. It was modified to carry a Sound Metric Dual frequency IDentification SONar (DIDSON). The DIDSON was able to show the state of underwater structures, schools of small fish swimming, and find the railings from the collapsed section of a pier. This work validates the concept of using USVs and UAVs together for disaster response, suggests missions, and priorities for autonomy. Besides damage inspection, USV-UAV teams can find safe lanes of sea travel and to detect hazardous materials spills. In addition to providing situation awareness, the UAV can serve as a wireless network relay. Inspection of damage to seawalls, docks, and bridges requires vision above the waterline as well as below poses a new type of Simultaneous Localization and Mapping (SLAM).

This paper shows that the Con metric from Dempster-Shafer theory is a good indicator of sensing quality, where an increase in the conflict metric value correlates with a decrease in map quality (at p ≤ 0.05). Two sets of experiments were conducted. In one, sonar data were gathered from a Nomad 200 robot operating in typical indoor hallways. Another used sonars on an RWI Urban robot in a confined, irregular tunnel built from eight different construction materials. For each set, an occupancy grid map was built and evaluated through a quantitative comparison with the ground truth. It is expected that the results of this study will generalize not only to other sensors and multi-sensor fusion, but any application where consistency can be assumed. It also contributes a design for an inexpensive reconfigurable confined space testbed.

As mobile robot teams become more complex, it is necessary to develop a control architecture to manage the resources present in the team. The Distributed Field Robot Architecture (DFRA) is a distributed, object-oriented implementation of the SFX hybrid robot architecture that allows for dynamic discovery and acquisition of robot resources and the seamless integration of humans and artificial agents in the robot team. This paper introduces the DFRA and details its application to a high-fidelity demining scenario using a heterogeneous team of ground and aerial robots.

This paper presents a detailed look at how unmanned ground vehicles (UGVs) fail in the field using information from 10 studies and 15 different models in Urban Search and Rescue or military field applications. One explores failures encountered in a limited amount of time in a real crisis (World Trade Center rescue response). Another covers regular use of 13 robots over two years. The remaining eight studies are field tests of robots performed by the Test and Evaluation Coordination Office at Fort Leonard Wood. A novel taxonomy of UGV failures is presented which categorizes failures based on the cause (physical or human), its impact, and its repairability. Important statistics are derived and illustrative examples of physical failures are examined using this taxonomy. Reliability in field environments is low, between 6 and 20 hours mean time between failures. For example, during the PANTHER study (F. Cook, 1997) 35 failures occurred in 32 days. The primary cause varies: one study showed 50% of failures caused by effectors; another study showed 54% of failures occurred in the control system. Common causes are: unstable control systems, platforms designed for a narrow range of conditions, limited wireless communication range, and insufficient bandwidth for video-based feedback.

Mobile robot reliability must be guaranteed before they can be employed in hazardous domains like mine clearing or nuclear waste handling, but recent studies of robots used in urban search and rescue and military scenarios have shown a mean time between failures (MTBF) in the field of 6 to 20 hours. This paper extends previous work characterizing robot failures by including recent data and organizing failures according to a novel taxonomy , which includes human failures. Failure type and frequency data were collected from 15 robots representing three manufacturers and seven models over a period of three years, in a variety of environments. Standard manufacturing measures for product reliability were used. The results show that overall MTBF and availability have improved since the previous analysis but are still low. The MTBF across all robot types was 24 hours and availability was 54%. The control system was the most common source of failures (32%), followed by the mechanical platform. Statistical analysis shows that the time between failures, time to repair, and downtime vary widely. For this reason the means reported here are not reliable predictors for future failures, but still provide information on the overall frequency and consequences of mobile robot failures.

The implementations of a distributed, autonomous error handler (EH) and a human-robot interface (HRI) are presented. The interface is combined with the EH to allow a human operator to see that a failure has occurred on a robot and whether or not it has been served by the EH. An experiment was run to test how well the EH and the interface work together, as well as the usefulness of the EH. The results were inconclusive, although the EH and interface worked together successfully.

The purpose of this study is to evaluate the utility of a diagnosis technique, which uses minimum message length (MML) for autonomous mobile robot fault diagnosis. A simulator was developed for a behavior-based robotic system and results were gathered for over 24,000 simulations varying the level of test noise and the components with simulated failures. The results showed that the MML diagnosis technique did not perform well as a turn-key solution. In two different data sets, only 0.59% and 1.19% of the test cases were correctly diagnosed and none of the cases with multiple failures were identified correctly. This paper presents the approach used to evaluate the new technique, the results, and a discussion of why MML diagnosis may not be appropriate for mobile robotics.

This study investigates data collected from operating an Inuktun robot in an urban search and rescue (USAR) confined space training exercise task at Virginia Beach Training Center. Data was collected from coding approximately one hour of video. The video had no sound so all analysis is based on the video feed. Indicators of communication, gestures, physical interactions with the robot, and robot movements were analyzed. The findings indicate that the robot emerges as a virtual presence for the support of the team outside of the confined space. The team members spontaneously responded socially to the robot despite the robot not being engineered to have a social intelligence. This confirms numerous studies in the cognitive science, psychology, and affective computing literature that robots need a social interface regards of domain.

This paper presents a method that integrates MATLAB into a distributed behavioral robotics architecture. The architecture is written in Java and uses the Jini platform for distributed object registration, lookup and remote method invocation. The method described here can be used to integrate MATLAB into any Java-based behavioral architecture. The form of the integration allows a running MATLAB workspace to be accessed as a distributed object within the larger Java/Jini-based architecture. This is beneficial because MATLAB scripts and functions may be called in interpreted form and can make full use of MATLAB tool boxes and have access to the MATLAB workspace environment. This is not possible when MATLAB scripts are compiled into stand-alone C++, Java or p-code. The use of the architecture is demonstrated on an iRobot ATRV-JR robot and remote computer workstation. Experiments have been conducted to quantify GPS and odometry errors in outdoor environments using automated methods supported by the distributed architecture.

As part of a Defense Advanced Research Projects Agency/National Science Foundation study on human-robot interaction (HRI), over sixty representatives from academia, government, and industry participated in an interdisciplinary workshop, which allowed roboticists to interact with psychologists, sociologists, cognitive scientists, communication experts and human-computer interaction specialists to discuss common interests in the field of HRI, and to establish a dialogue across the disciplines for future collaborations. We include initial work that was done in preparation for the workshop, links to keynote and other presentations, and a summary of the findings, outcomes, and recommendations that were generated by the participants. Findings of the study include-the need for more extensive interdisciplinary interaction, identification of basic taxonomies and research issues, social informatics, establishment of a small number of common application domains, and field experience for members of the HRI community. An overall conclusion of the workshop was expressed as the following-HRI is a cross-disciplinary area, which poses barriers to meaningful research, synthesis, and technology transfer. The vocabularies, experiences, methodologies, and metrics of the communities are sufficiently different that cross-disciplinary research is unlikely to happen without sustained funding and an infrastructure to establish a new HRI community.

This paper develops and applies a variant of the Min-Conflict algorithm to the problem of sensor allocation with incomplete information for mobile robots. A categorization of the types of contention over sensing resources is provided, as well as a taxonomy of available information for the sensor scheduling task. The Min-Conflict with Happiness (MCH) heuristic algorithm, which performs sensor scheduling for situations in which no information is known about future assignments, is then described. The primary contribution of this modification to Min-Conflict is that it permits the optimization of sensor certainty over the set of all active behaviors, thereby producing the best sensing state for the robot at any given time. Data are taken from simulation experiments and runs from a pair of Nomad200 robots using the SFX hybrid deliberative/reactive architecture. Results from these experiments demonstrate that MCH is able to satisfy more sensor assignments (up to 142%) and maintain a higher overall utility of sensing than greedy or random assignments (a 7-24% increase), even in the presence of sensor failures. In addition, MCH supports behavioral sensor fusion allocations. The practical advantages of MCH include fast, dynamic repair of broken schedules allowing it to be used on computationally constrained systems, compatibility with the dominant hybrid robot architectural style, and least-disturbance of prior assignments minimizing interruptions to reactive behaviors.

Rescue robotics has been suggested by a recent DARPA/NSF study as an application domain for the research in human-robot integration (HRI). This paper provides a short tutorial on how robots are currently used in urban search and rescue (USAR) and discusses the HRI issues encountered over the past eight years. A domain theory of the search activity is formulated. The domain theory consists of two parts: 1) a workflow model identifying the major tasks, actions, and roles in robot-assisted search (e.g., a workflow model) and 2) a general information flow model of how data from the robot is fused by various team members into information and knowledge. The information flow model also captures the types of situation awareness needed by each agent in the rescue robot system. The article presents a synopsis of the major HRI issues in reducing the number of humans it takes to control a robot, maintaining performance with geographically distributed teams with intermittent communications, and encouraging acceptance within the existing social structure.

Failure data on mobile robots is critical for three reasons: to support the theory of autonomous fault detection, identification, and recovery necessary for success in new domains; provide design and manufacturing feedback to the robotics community; and permit project managers to accurately create development schedules. The failures considered in this paper occurred over a period of 2 years, in a variety of environments. Failure type and frequency data were collected from thirteen robots representing three manufacturers and seven models. The data was analyzed using standard manufacturing measures for the reliability of a product. The mean time between failures represents the average time to the next failure. Availability was used to gauge the impact of a failure on a project. The results show that the reliability for mobile robots is low, with an average MTBF of 8 hours and availability of less than 50%. The platform itself was the source of most failures (42%) for field robots, while the control system was responsible for 29% of the failures.

This paper presents work extending previous research in sensor fault tolerance, classification, and recovery from a single robot to a heterogeneous team of distributed robots. This approach allows teams of robots to share knowledge about the working environment, sensor and task state, to diagnose failures and also communicate to redistribute tasks in the event that a robot becomes inoperable. Our work presents several novel extensions to prior art: distributed fault handling and task management in a dynamic, distributed Java framework. This research was implemented and demonstrated on robots in a lab environment performing a simplified search operation.

The World Trade Center (WTC) rescue response provided an unfortunate opportunity to study the human-robot interactions (HRI) during a real unstaged rescue for the first time. A post-hoc analysis was performed on the data collected during the response, which resulted in 17 findings on the impact of the environment and conditions on the HRI: the skills displayed and needed by robots and humans, the details of the Urban Search and Rescue (USAR) task, the social informatics in the USAR domain, and what information is communicated at what time. The results of this work impact the field of robotics by providing a case study for HRI in USAR drawn from an unstaged USAR effort. Eleven recommendations are made based on the findings that impact the robotics, computer science, engineering, psychology, and rescue fields. These recommendations call for group organization and user confidence studies, more research into perceptual and assistive interfaces, and formal models of the state of the robot, state of the world, and information as to what has been observed.

This paper presents findings from field trials observing human-robot interaction between certified rescue workers and two types of tactical mobile robots at a rescue training site. Data was collected on how members of a fire rescue department directed the use of two types of robots for four tasks (climbing stairs to investigate condition of upper floors, searching dark, cluttered environments with two different sensor suites, and exploring vertical voids). The prototypical workflow, the type and frequency of errors during each task, how the robot workflow compared with existing urban search and rescue (USAR) practices, and any additional information that came out during debriefing is reported for each task. Two major workflow patterns that could be partially or fully automated were identified: stairwell search and topological search. In addition, collaborative teleoperation appeared to be an important multi-robot strategy. Rescue workers rated the robots' performance superior to existing methods for searching and for exploring vertical voids, but not for stairwells.

Previous experiences show that it is possible for agents such as robots cooperating asynchronously on a sequential task to enter deadlock, where one robot does not fulfil its obligations in a timely manner due to hardware or planning failure, unanticipated delays, etc. Our approach uses a formal multilevel hierarchy of emotions where emotions both modify active behaviors at the sensory-motor level and change the set of active behaviors at the schematic level. The resulting implementation of a team of heterogeneous robots using a hybrid deliberative/reactive architecture produced the desired emergent societal behavior. Data collected at two different public venues illustrate how a dependent agent selects new behaviors (e.g., stop serving, move to intercept the refiner) to compensate for delays from a subordinate agent (e.g., blocked by the audience). The subordinate also modifies the intensity of its active behaviors in response to feedback from the dependent agent. The agents communicate asynchronously through knowledge query and manipulation language via wireless Ethernet.

This paper reports on a reactive docking behavior which uses a vision algorithm that grows linearly with the number of image pixels. The docking robot imprints (initializes) on a two-colored docking fiducial upon departing from the dock, then uses region statistics to adapt the color segmentation in changing lighting conditions. The docking behavior was implemented on a marsupial team of robots, where a daughter micro-rover had to reenter the mother robot from an approach zone with a 2 m radius and 140/spl deg/ angular width with a tolerance of /spl plusmn/5 and /spl plusmn/2 cm. Testing during outdoor conditions (noon, dusk) and challenging indoor scenarios (flashing lights) showed that using adaptation and imprinting was more robust than using imprinting atone.

Motion blur arises when motion is fast relative to the shutter time of a camera. Unlike most work on motion blur, which considers the streaks due to motion blur to be noisy artifacts. In this paper we introduce a new method to extract motion information from these streaks. Previous methods with similar goals first extract an optic flow field from local information in the motion streaks and then infer global motion parameters. On the contrary, we adopt a more direct feature-based approach and extract global motion parameters from the motion streaks. We first extract edges in the motion blurred images, which we then group to determine the foci of expansion, the center of rotation, or motion parallel to the image plane. Furthermore, we determine the direction of motion. We present results on real images from a mobile robot in cluttered environments.

For mobile robots employing reactive behaviors, allocation of physical sensors to satisfy sensing needs should be dynamic and fast. It is becoming increasingly apparent that this allocation should also support behavioral sensor fusion, as indicated by experimental data, in order to maximize the use of available sensing hardware and to increase the quality of sensing. These issues are addressed in the context of the min-conflict with happiness algorithm for dynamic sensor allocation, whose execution rates on two real robots ranged from 11 to 17 milliseconds. Experimental results are shown which illustrate the improvements (27.5%-75% of observations) achieved using sensor fusion. The paper also contributes a quantitative representation of sensing quality using t-norms, allowing fused sensors to be compared with single sensors for a behavior.

A key objective for a mobile robot in urban search and rescue (USAR) is to efficiently fined survivors, get near enough to communicate and/or drop off communications and biomedical monitoring gear. This paper discusses a biomimetic search strategy extracted from ethological studies of how insects and animals forage for food, and cognitive studies of how children search for objects. This leads to a biomimetic search organization where a robot partitions the search space based on the semantic understanding of the expected distribution of survivors, then systematically searches each of the volumes in ranked order. While in transit between volumes, the robot conducts a passive opportunistic search. The paper also describes how this search strategy is being implemented and evaluated on a mobile robot for two upcoming USAR competitions.

This paper presents an algorithm for allocating sensing resources for an autonomous mobile robot with logically redundant sensing capabilities. The algorithm creates a partial plan based on the set of requests by behaviors. If two or more behaviors place conflicting requests, a variant of the MIN-CONFLICT algorithm is used to find a replacement logical sensor. Unlike traditional MIN-CONFLICT, our variant maximizes each behavior's preference for a particular sensor ("happiness"). Simulations compared MIN-CONFLICT with Happiness to other methods (random and greedy assignment) for 10 sequences of 20 random requests for 8 sensors from up to 11 concurrent behaviors. Results showed that it is able to generate more schedules (on the order of 71% to 155% more) and that a further variant could maximize happiness better (7% to 30%).

Gibson's ecological approach to perception has recently received significant attention in the robotics literature, The approach relies on the identification and application of affordances for a behaviour. However, determining suitable affordances has been treated as an art rather than a science. This paper presents a methodology for deciding whether an affordance-based approach should be pursued for a specific behaviour and for isolating a reliable affordance. Three case studies on three mobile robots are presented which used this methodology successfully. The case studies illustrate the advantages and limitations of affordances, as well as offer practical insights into applying the ecological approach.

This article offers a case study of how to teach image computation in an upper level elective course on robotics with a significant number of non-Computer Science majors. The MACS 415 course at the Colorado School of Mines is required for the popular interdisciplinary undergraduate minor in Robotics and AI. It is mandated to provide a broad survey of the artificial intelligence tools available to roboticists, including image computation. Teaching image computation in a robotics elective is challenging both because of the limited time that can be spent on computer vision, and because of the attributes of the students. Non-CS majors typically do not have enough programming experience to program DSP algorithms, yet the students' preferred learning style is "hands-on." In order to reconcile this dilemma, we (1) cover a broad set of topics in class, (2) have several laboratory assignments using khoros, and (3) require the students to complete a robot project involving computer vision. The article summarizes the lessons learned to date, which are expected to be applicable to any course with non-majors involving image computation.

This article discusses Dempster-Shafer (DS) theory in terms of its utility for sensor fusion for autonomous mobile robots. It exploits two little used components of DS theory: the weight of conflict metric and the enlargement of the frame of discernment. The weight of conflict is used to measure the amount of consensus between different sensors. A lack of consensus leads the robot to either compensate within certain limits or investigate the problem further, adding robustness to the robot's operation. Enlarging the frame of discernment allows a modular decomposition of evidence. This decomposition offers the advantages of perceptual abstraction, and permits expert knowledge about the domain to be embedded in the frames of discernment, simplifying the construction and maintenance of the knowledge base. Six experiments using this Dempster-Shafer framework are presented. Data from four types of sensor data were collected by a mobile robot and fused with the sensor fusion effects (SFX) architecture.

This paper investigates a range of strategies for determining when to explicitly replan paths. An computationally inexpensive event-driven method is presented which allows the robot to execute an a priori path reactively until a significant deviation in the path occurs, at which point the robot explicitly replans. The event-driven strategy is compared to replanning after every move, and replanning after every n moves on a mobile robot for a wide variety of influences, including different unmodeled obstacle configurations and densities, and quality of the a priori map. The paper concludes that, in general, planning as frequently as resources permits results in smoother actual paths and faster completions. However, if the updated map is inaccurate, the event-driven method is superior.

This article discusses the impact of sensing on each aspect of the design of the CSM autonomous mobile robot architecture, in particular, the overall control scheme, coordination of behaviours, and representation of different types of knowledge. The CSM/ deliberative reactive system uses three novel mechanisms for maintaining robust perception: a perceptual schema for behavioural sensing, a sensing manager to globally allocate sensing resources, and abstract navigation behaviours to coordinate diverse sensing demands in addition to simplifying motor control. Examples of the operation of each are taken from software developed for the two CSM mobile robots operating in indoor and outdoor task domains.

This paper presents a new approach to reactive control utilizing the output of an explicit path planner which considers terrain preferences. To demonstrate the concept, a wavefront propagation, type of planner algorithm, Trulla, is integrated into a reactive framework and used as part of the overall architecture. Terrain preferences are represented as weights or costs which are considered in generating a near optimal path. The algorithm is demonstrated on the CSM Denning-Branch MRV4 mobile robot for three scenarios: navigation out of a box canyon, planning with terrain preferences, and opportunistic path improvement.

While inherent uncertainty in obstacle location for autonomous vehicle path planning has been investigated previously, terrain map uncertainty has largely been ignored. The terrain map is constructed via a priori data that in general, is incorrectly utilized as "perfect information". In this paper a probabilistic approach using empirical loss functions is developed to incorporate terrain map uncertainties into the path planning process. A new graph search method is introduced that allows the inclusion of negative weight edges on a optimally planned path. Representative results from forty simulations are presented which indicate that the incorporation of terrain map uncertainty can affect the planned paths that are generated for an autonomous robotic vehicle.

This paper proposes scripts as a framework for coordinating and controlling a collection of behaviors needed to perform a highly stereotyped task. Scripts facilitate planning by explicitly representing the types of situations the behaviors are suited for, and the default schedule or plan of activities. They support building abstract behaviors from libraries of primitive independent behaviors by providing the meta-knowledge needed to smooth over minor incompatibilities. Scripts enable robust execution, allowing subscripts to be attached for reacting to anomalous conditions. The utility of scripts is demonstrated via a case study of topological navigation. In this case study, scripts provided the foundation for the abstract navigation behaviors used by a mobile robot to travel through indoor office spaces.

This paper presents a novel organization of reactive behaviors which avoids the problems generally associated with arbitration or combination of behaviors. The organization uses tactical behaviors to attempt to safely satisfy the intent of strategic behaviors given the immediate situation (e.g., state of the environment, status of the robot, certainty, etc.) A tactical speed control behavior using fuzzy logic is described in detail. Experiments with a nonholonomic mobile robot navigating a 150 ft course show that a tactical speed control behavior improves navigational performance without requiring either knowledge about the strategic behavior (follow-line) or the complexity of the course. The speed control behavior has also been used in conjunction with shared control and has been transferred to a holonomic robot, demonstrating how the behavioral organization enhances software modularity and portability.

This paper reviews the literature from the biological and cognitive sciences in sensory integration and derives principles for use in constructing intelligent sensor fusion systems. In particular, it presents psychophysical and neurophysical studies on how sensor fusion is accomplished and cognitive models of associated activities, including optimization of sensing configurations, improvement of sensing quality, and filtering of noise. The sensor fusion effects architecture for robot navigation is also presented as one example of how these insights from the biological and computer science can be applied to robotic sensor fusion. Experimental results demonstrates the utility of the biological and cognitive insights, especially that of fusion modes. Other representative architectures for robotic sensor fusion are contrasted with the biological and cognitive principles.

Sensor fusion has an important role in the navigation of autonomous mobile robots. Our research has generated a generic and robust pro- cess model based on the action-oriented percep- tion paradigm. The autonomous execution and ex- ception handling abilities of this model have been implemented as the Sensor Fusion Effects (SFX) architecture. The key aspects of this implemen- tation are the sensing plan, the uncertainty man- agement mechanism, the application of feedback from the sensing process to individual sensors, the detection of exceptions to the sensing plan, and handling of those exceptions. This paper gives an overview of the SFX architecture, concentrating on the sensing plan as the central control struc- ture guiding autonomous execution. This paper also reports on experiments using sensor data col- lected from our mobile robot which demonstrate the use of the sensing plan representation, the ex- ecution sequence, the application of feedback, and how feedback improves the overall sensing capa- bilities of the robot.

Current approaches towards achieving mobility in the workplace are reviewed. The role of automatic guided vehicles (AGVs) and some of the preliminary work of other groups in autonomous vehicles are described. An overview is presented of the autonomous robot architecture (AuRA), a general-purpose system designed for experimentation in the domain of intelligent mobility. The means by which navigation is accomplished within this framework is specifically addressed. A description is given of the changes made to AuRA to adapt it to a flexible manufacturing environment, the types of knowledge that need to be incorporated, and the new motor behaviors required for this domain. Simulations of both navigational planning and reactive/reflexive motor schema-based navigation in a flexible manufacturing systems environment, followed by actual navigational experiments using the mobile vehicle, are presented.>