This paper proposes a new system architecture that includes human action. The system is composed of three parts; sensing, processing, and actuation. Based on the concept, a virtual museum with a spatial immersive display and a locomotion interface was developed. It provides a physical walking experience in the virtual exhibition space. We developed a digital twin of the Artizon Museum as the system’s content. The locomotion interface records the walking trajectory of the participant, which represents their action in the virtual exhibition space. The system was implemented and operated in public for four weeks.

This paper introduces Art Sensorium Project that is founded in Asia AI Institute of Musashino University. A main target of the project is to design and implement a system architecture of unified art collections for virtual art experiences. To provide art experiences, a projection-based VR system, called Data Sensorium, is used to stage art materials in a form of real-sized virtual reality. Furthermore, a system architecture of a multidatabase system for heterogeneous art collection archives is presented, so a set of integrated art data is applied to Data Sensorium for newly generated art experiences.

BigRobot Mk.2, which creates the body motion of a giant, is described. The robot has 10 actuated joints so that it can emulate the body motion of human. A pilot rides on the top of the robot and it moves according with walking action of the pilot. This function provides the pilot to the body sensation that is extended to an 8-m giant. The basic design, implementation, and performance evaluation of BigRobot Mk.2 are presented.

This paper describes about project “Data Sensorium” launched at the Asia AI Institute of Musashino University. Data Sensoriumis a conceptual framework of systems providing physical experience of content stored in database. Spatial immersive display is a key technology of Data Sensorium. This paper introduces prototype implementation of the concept and its application to environmental and architectural dataset.

This paper studies the characteristics of the human perception of a haptic shape-changing interface, capable of altering its size and rigidity simultaneously for presenting characteristics of virtual objects physically. The haptic interface is composed of an array of computer-controlled balloons, with two mechanisms, one for changing size and one for changing rigidity. We manufactured two balloons and conducted psychophysical experiments with twenty subjects to measure perceived sensory thresholds and haptic perception of the change of size and rigidity. The results show that subjects can correctly discriminate different conditions with an acceptable level of accuracy. Our results also suggest that the proposed system can present an ample range of rigidities and variations of the size in a way that is compatible with the human haptic perception of physical materials. Currently, shape-changing interfaces do not hold a defined position in the current VR / AR research. Our results provide basic knowledge for developing novel types of haptic interfaces that can enhance the haptic perception of virtual objects, allowing rich embodied interactions, and synchronize the virtual and the physical world through computationally-controlled materiality.

A furniture-device is a device having the furniture appearance and physical input and output functions. The Escaping Chair is a furniture-device capable of having physical and dynamic interaction with a subject to create self-awareness toward the intent of their actions and personification of the furniture-device. The chair interacts with bystanders by trying to move away from nearby people. By fleeing this, the device tries to make a person unable to sit on it, stimulating their perception toward their sitting action, while also making the person consider the Chair's "personality." We exhibited the chair and collected feedback via conversation and observation. During exhibition the exhibit, the chair succeeded in making subjects perceive their own movements and to feel a semblance of will in the chair during their interactions, as we planned, and even making them sympathize with it. Exhibition also raised further challenges to explore in user interaction.