Transparent and consistent reporting of user studies is essential for advancing scientific knowledge. In human-robot interaction (HRI), studies are often reported incompletely, even in top-tier venues, limiting proper evaluation, replication, and practical application of findings in practice. This study aimed to generate expert consensus on a reporting checklist for HRI user studies and provide a validated tool to improve transparency, reproducibility, and methodological rigor in the field, leading to easier translation of research into practice. A two-round Delphi study was conducted with 34 HRI experts from academia and industry from over 12 countries. An international panel of nine interdisciplinary experts first synthesized a preliminary list of 116 reporting items from the literature. Experts rated the importance of each item and provided qualitative feed- back. Consensus was defined as 70% agreement, and items were iteratively refined through anonymous online surveys. Overall, consensus was achieved on 106 items, encompassing both essential and context-dependent elements in nine domains. The resulting RUSH checklist (Reporting User Studies in Human-Robot Inter- action) provides the first community-endorsed, consensus-based reporting guideline for HRI user studies.

There are a multitude of robotic application domains that touch on the field of human-robot interaction (HRI). From modern manufacturing involving human-robot teams, to personal care robots assisting the elderly, the roles that robots are being tasked with and the nature of interactions with humans are constantly shifting. Even the nature of interaction has changed to incorporate wearable technologies such as exoskeletons to enhance human capabilities, and advanced prosthetics to restore those abilities that have been lost. With this ever-evolving spectrum of HRI, the capacity of measurement science to evaluate, assess, and assure performance and safety struggles to keep up. Building on our previous five-workshop series on Test Methods and Metrics for Effective HRI, NIST presents a new series on evaluative methodologies for accelerating the pipeline from cutting-edge HRI research to state-of-practice. This workshop will address issues regarding 1) data collection and reporting for replicability and system validation, 2) test design and execution for performance verification, and 3) cross-modality artifact design for real-world application-adjacent technology transfer. The goal of this workshop is to accelerate and accommodate accessibility to HRI research results, and address the specific key performance indicators that would establish end-user trust and acceptance of emerging HRI technologies.

In an interdisciplinary and evolving research field like human-robot interaction, clear and precise results reporting is essential for study comparability and replicability. To address the lack of a standard for such reporting and, at the same time, provide guidance for novices in the field, we have developed a web-based reporting form to capture human-robot interaction studies, serving as a model for how conferences could adopt it into the submission pipeline. In this work, we present a formative evaluation of this form regarding its level of detail, format and clarity, and the perceived benefits for authors, reviewers, and the community as a whole. We report the expert review of nine researchers who highlight the substantial value of this tool. In addition, these experts also provide suggestions for improvements to its form and the addition of details surrounding qualitative reporting.

The drive for increasing adoption of HRI technolo-gies is evident through research and development of manufac-turing, social, medical, and service robot solutions. However, novel methods and metrics are required to overcome the barrier between fundamental HRI research and its adoption in real-world environments. Hence, the fourth installment of the annual workshop, 'Test Methods and Metrics for Effective HRI,’ seeks to identify novel and emerging test methods and metrics for the holistic assessment and assurance of HRI performance. Specifically, the focus is on identifying innovative methods for the evaluation of HRI performance and to advance the growth of the HRI community based on the principles of collaboration, data sharing, and repeatability. The goal of this workshop is to break the boundaries between the development and adoption of HRI technologies through the promotion of robust experimental design, test methods, and metrics for assessing interaction and interface designs. This workshop will have participants from var-ious sectors in the HRI research community including academia, industry, and government in order to accomplish its aims.

Standardized metrics and methods are critical to-wards wider adoption of HRI technologies in real-world applications. However, the interdisciplinary nature of HRI creates an inherently decentralized research paradigm that limits the use of standardized metrics for baseline comparisons among studies. This limitation restricts both the real-world adoption and academic replicability of HRI solutions developed by the research community. To identify specific opportunities for reuse of metrics and methods in HRI, this paper presents a comprehensive survey of 1464 papers from the ACM/IEEE International Conference on Human-Robot Interaction (HRI) and the IEEE International Conference on Robot and Human Interactive Communication (Ro-Man) over seven years. By providing a holistic perspective of the metrological tools leveraged in the current state-of-practice of HRI research, we find that a significant portion of HRI studies use custom surveys, thus limiting baseline comparison. Hence, the analysis in this work aims to advance the field of HRI by identifying specific barriers to adoption of HRI technologies in addition to proposing solutions to overcome existing limitations in the context of metrics and methodologies.

This special issue of the Transactions on Human-Robot Interaction highlights, documents, and explores the metrics, test methods, and artifacts used in human-robot interaction (HRI) research. This collection of articles brings to attention the commonalities between the application of measurement science for the assessment and assurance of human-centric robotics in a variety of application domains, including industry, education, and defense. This special issue draws specific attention to the use and impact of metrology toward the advancement of HRI technologies and algorithms, and it promotes the application of measurement science toward the benchmarking and replication of HRI research. Special attention is given to the use cases, data sets, test methodologies, measurement techniques, metrics, and statistical analyses used to evaluate system performance.

We present a comprehensive framework and test methodology for the evaluation of human-machine interfaces (HMI) and human-robot interactions (HRI) in collaborative manufacturing applications. An overview of the challenges that face current- and next-generation collaborative robot systems is presented, specifically focused on the interactions between man and machine, and a series of objectively quantitative and subjectively qualitative metrics are given to guide the development and assessment of interfaces and interactions. A generalized set of guidelines for the design of HMI is also proposed to address these challenges and thereby enable effective and intuitive diagnostics and error corrections when process failures occur. These guidelines are aimed at aiding researchers in developing effective interface and interaction technologies, maximizing operator situation awareness in human-robot collaborative manufacturing teams, promoting effective process and system diagnostics reporting, and enabling faster responses to equipment or application errors.

Verified and validated test methods, being necessary to measure the performance of complex systems, are important tools for driving innovation, benchmarking and improving performance, and establishing trust in collaborative human-robot teams. This full-day workshop aims to explore the metrology necessary for repeatably and independently assessing the collaborative performance of robotic systems in real-world human-robot interaction (HRI) scenarios. This workshop aims to bridge the gaps between the theory and applications of HRI in industry, accelerating the adoption of cutting edge technologies as the industry state-of-practice. The interest in collaborative HRI is evident in the current market as well as standards efforts toward manufacturing, social, medical, and service robot solutions. Though these domains have been considered separate for many years, recent technological and scientific advancements show that, while their applications may differ, the underlying principles of HRI performance impact each identically. As such, this workshop seeks to identify test methods and metrics for the holistic assessment and assurance of collaborative HRI performance. The focus is on identifying the key performance indicators of these seemingly disparate sectors, and additionally to establish a community based on the principles of transparency, repeatability, & establishing trust in the assessment of collaborative HRI. The goal is to aid in the advancement of HRI technologies through the development of experimental scenarios, protocols, test methods, & metrics for the verification and validation of interaction solutions and interface designs.