To help answer questions about the behavior of participants in human-robot systems, we propose the cognitive evaluation of human-robot systems (CEHRS) method based on our work with the personal exploration rover (PER). The CEHRS method consists of six steps: (1) identify all system participants, (2) collect data from all participant groups, including the system's creators, (3) analyze participant data in light of system-wide goals, (4) answer targeted questions about each participant group to determine the flow of knowledge, information, and influence throughout the system, (5) look for inconsistencies in the knowledge and beliefs of different participant groups, and (6) make recommendations for improvement. We offer this comprehensive, human-centered evaluation method as a starting point for future work in understanding cognitive change in human-robot interactions

The Personal Rover Project produces technology, curriculum and evaluation techniques for robotic educational use in formal and informal (after-school, out-of-school) learning environments. Our specific aim for this phase of the project is to create and evaluate human-robot interactions that educate members of the general public in an informal learning environment, specifically museums. Our educational goals are to further an appreciation and understanding of NASA's Mars Exploration Rovers (MERs), to illustrate the role of robotic rovers in scientific exploration, and to provide hands-on learning experiences that demonstrate robot autonomy. We have designed a new robot, the Personal Exploration Rover (PER) and the related interactive components of a museum exhibit to achieve these goals. Here we describe the exhibits developed and the formal evaluation results of the exhibits' educational impact and efficacy. These results suggest techniques by which learning can be measured and used as an indicator of successful human-robot interaction.

During the summer of 2002, Robotic Autonomy was taught to thirty students at Carnegie Mellon West in cooperation with NASA/Ames (Moffett Field, CA). The authors at Carnegie Mellon University's Robotics Institute and at the University of Pittsburgh's Learning Research and Development Center planned a methodology for formally evaluating the educational efficacy of Robotic Autonomy. This article describes the educational analysis methodology and the statistically significant results of our analysis, demonstrating the positive impact of Robotic Autonomy on student learning beyond the boundaries of specific technical concepts in robotics. Educational robotics is gaining traction at all levels of the educational system, however formal analysis of its effectiveness has been lacking. This research project seeks to address this weakness by presenting statistically significant evidence of broad learning.