Brain-computer interface (BCI) systems can translate the human mind into control commands, which makes it feasible to improve the life quality of physically challenged people. However, in real-life situations, it is still difficult for users to utilize robots to provide basic services with BCI systems. We aimed to propose a BCI-based system with a visual servo module to operate a service robot. We recorded single-channel steady-state visual evoked potentials (SSVEP) as input signals for the BCI system of this study. The visual stimuli for inducing SSVEP were modulated at seven different frequencies with the sampled sinusoidal method. Correspondingly, this SSVEP-based BCI system can generate seven control commands for the operation of the service robot, which can provide three fundamental services: mobility, manipulation, and delivery. The visual servo module was established to reduce the burden of users and accelerate service procedures. To evaluate the performance of this system, subjects were recruited to participate in the experiments. All the participants succeed in operating the robot to provide the basic services. According to the experimental results, this SSVEP-based BCI system that incorporates the visual servo module can be effectively used to operate service robots with reduced number of channels and increased ability to perform multiple tasks.

This work proposes a new approach to realize embodied and multimodal HRI between virtual robot and real world human for HRI challenges in RoboCup @Home.

The high cost and complexity to maintain a robot, and the impracticability for long term large scale HRI with real robots, have induced constraints in HRI studies. The purpose of this paper is to demonstrate the feasibility of our proposal to realize embodied and multimodal HRI between virtual robot and real world human in response to the above constraints. Based on our SIGVerse simulator, a 3D virtual environment is established for the virtual robot to interact with real world human via immersive interface. The realizations of embodiment and multimodal interaction are illustrated in several HRI research scenarios, where verbal action instruction and spatial information from the body gesture are simulated.

This work aims to realize multimodal interaction with embodied contextual understanding based on the simple chatterbot system. A system framework is proposed to integrate the dialogue system into a 3D simulation platform, SIGVerse to attain multimodal interaction. The chatterbot AIML implementations are described in the achievement of the conversations with embodied contextual understanding in HRI simulations.

The aim of this work is to propose a cloud computing architecture simulation platform for social human-robot interaction. This paper explains the design and development of this system named SIGVerse in four main components, namely (1) SIGServer as the central server, (2) Agent Controller for user applications, (3) Service Provider, and (4) SIGViewer as the client terminal, and web based development interface, in addressing the two main “human” issues in human-robot interaction with cloud computing architecture, (a) Distributed development platform, and (b) Large scale of human-robot simulation. Three current applications are discussed for the validation of the cloud computing architecture in social human-robot interaction simulations.

A cellular manufacturing system is good at producing diversified products flexibly; however, its assembly efficiency depends mainly on its operators' abilities. As the workforce shrinks in Japan, cellular manufacturing systems are difficult to maintain. In this case, a new assembly system has been developed since 2006 that combines both the dexterities of human operators and the advantages of automatic machinery. Its characteristics consist of three aspects: collaboration between an operator and twin manipulators on a mobile base, assembly information guidance, and safe design for collaboration. To meet the rapid changing tastes of customers, operators have to assemble various products without longtime training. This requires an effective assembly skill transfer system to extract assembly skills from skilled operators, and then transfer them to novice ones. Considering the characteristics of a cellular manufacturing system, an assembly skill transfer system was proposed and used to extract and transfer assembly skills in both cognition and execution aspects. Taking a cable harness task as an example, the proposed skill transfer system was applied in a developed assembly system. The results show that working under the developed assembly system with physical support, informational support, and the assembly skill transfer system, novice operators' assembly performance was greatly improved. This verified the effect of the proposed solution to maintain the cellular manufacturing system in the aging Japanese society.

The purpose of this work is to develop information support for human-robot collaboration in cellular manufacturing based on human-centered approach to improve the human adaptability of the support system. In this paper, six main developments are presented to illustrate the human-centered approach. Before the human-robot collaboration system can be developed, a human-human collaboration experiment is conducted to investigate the interactions within the collaboration. Instead of analyzing the assembly process in conventional mechanical approach, task analysis approach is used to analyze human operations in assembly. This approach explains the assembly process by human operations for better understanding of the assembly by the operator. Collaboration analysis in both quantitative and qualitative analyses is conducted in design phase to understand the collaboration relationship in assembly. The collaboration is analyzed from the given assembly operations and the tasks are reassigned to the appropriate working agent based on the capabilities of the human operator and the robot system. Risk assessment is conducted to enhance safety information in the support system. Based on human cognitive process and experimental studies, a multimodal information support system is developed with horizontal display providing information support by multimedia presentation. Support information design and presentation are developed based on human information processing. A set of design guidelines is developed to prepare the multimedia presentation. The support system is successfully implemented in a prototype production cell for cable harness assembly. Comparison study is conducted to validate the improvement by the support system.

The objective of this work is to enhance man-machine collaboration in cell production by task modeling approach. Task modeling provides a method to define and study the collaboration between man and machine. Six requirements have been derived as basis of development. In this work, the task modeling has extended classical HTA method to represent collaborative operation in a hierarchical structure with the classification of operation task levels and extended operation details as task properties. A modeling tool is developed to provide a development and data management platform. The man-machine collaboration is well defined in a structural format by the ability of task analysis method to address human task in a more flexible way. The collaboration modeling and operation resources information have significantly facilitated operation planning in both assembly and control levels. From the case study, the improvement in safety aspect has proven the ability of this approach in assisting safety design and development of production system. The integration with operation control system has shown the execution potential in real-time operation. The unique development of operation data management has expanded this work into operation information support. Through the multimodal information support system, the human operator is well guided by the operation information corresponded to the task components. The operation information also can work as evaluation functions to measure the human working performance and system performance as a whole.

The challenge of this work is to study the design and development of human-robot collaboration (HRC) in cellular manufacturing. Based on the concept of human collaborative design, four main design factors are being identified and developed in an active HRC prototype production cell for cable harness assembly. Human collaborative design aims to optimize the system design for the advantage of collaboration between human and robots based on human considerations. Task modeling approach is developed to study and analyze the task in order to identify the collaboration tasks to develop the collaboration planning. In the collaboration safety development, five safety designs, cover both hardware and control design, are proposed and developed in the prototype system. Risk assessment is conducted to verify the safety design. Two main experiments were conducted as preliminary study to investigate mental workload in HRC. A multimodal information support system is developed in the study of man-machine interface in this work to provide a comprehensive human-robot interface to facilitate human operator. The system performance evaluation had proven the improvement of prototype production cell with HRC design for cellular manufacturing.

The purpose of this work is to conduct human factors studies in the development of an information support system for human-robot collaborative cellular manufacturing system. Multimedia technologies are being utilized to enhance the support system to function as operational information support and interface between human operator and robot system. In the study, five experiments were carried out in both system hardware and support information developments to investigate the human factors design of the support system. In the information display approach study, comparison of information presentation ability among paper manual, LCD TV and projector was conducted. Similar information was displayed in different visual areas by vertical and horizontal LCD TV configurations to study the effect on human visual working area. The comparison experiment on image, text and voice formats had resulted varying modality effectiveness of information. Different information formats require different human cognitive processes even though the information contents are similar. The investigation on multimedia displays in five combinations of information elements was performed in multimedia display design study. The information support system is implemented in a human-robot collaborative prototype production cell. The cable harness assembly experiment had proven the effectiveness of the information support system in guiding the operators on the assembly operation and also to collaborate well with the robot system.