This paper proposes a method for controlling video images in performance using video signals in the audio domain. It also describes a demonstration performance system using this representation method. This system consists of hardware for converting the component video signal to audio and software for easy control of image to produce the pitch(frequency), intensity(amplitude), rhythm and timbre(wave shape). Also included is data for verification of effectiveness of the proposed system and method of performance operation using a questionnaire survey of subject experiment, and demonstration of audio-visual performance in a public hall. This proposed representation method was created as a new unique relationship between video and audio signals, and also shows a new aspect of computer as a musical instrument.

This video shows the basic functions of HFAMRO-2 equipped with the step-on interface (SOI). In the SOI the projected screen is used as a bilateral interface. It not only presents information from the equipment to the user but also delivers the instructions from the user to the equipment. HFAMRO is intended to represent the concept based on which robots interact with users. It assumes, for example, the ability to play 'tag' - in this case, playing tag with light, similar to 'shadow' tag. The HFAMRO-2 mobile robot, developed to study the SOI's application with mobility, has two sets of the SOI consisting of a projector and a range scanner on a mobile platform. The projector displays a direction screen on a travel surface and the two-dimensional range scanner detects and measures the user's stepping to specify the selected button.

The rehabilitation of the physically challenged is one of the trial applications of the step-on interface (SOI) on a mobile platform as the friendly amusing mobile (FAM) function. This video shows the result of the preliminary trial.