We present ViSTAR, a Virtual Skill Training system in AR that supports self-guided basketball skill practice, with feedback on balance, posture, and timing. From a formative study with basketball players and coaches, the system addresses three challenges: understanding skills, identifying errors, and correcting mistakes. ViSTAR follows the Behavioral Skills Training (BST) framework-instruction, modeling, rehearsal, and feedback. It provides feedback through visual overlays, rhythm and timing cues, and an AI-powered coaching agent using 3D motion reconstruction. We generate verbal feedback by analyzing spatio-temporal joint data and mapping features to natural-language coaching cues via a Large Language Model (LLM). A key novelty is this feedback generation: motion features become concise coaching insights. In two studies (N=16), participants generally preferred our AI-generated feedback to coach feedback and reported that ViSTAR helped them notice posture and balance issues and refine movements beyond self-observation.

Training resources for parasports are limited, reducing opportunities for athletes and coaches to engage with sport-specific movements and tactical coordination. To address this gap, we developed BRIDGE, a system that integrates a reconstruction pipeline, which detects and tracks players from broadcast video to generate 3D play sequences, with an embodiment-aware visualization framework that decomposes head, trunk, and wheelchair base orientations to represent attention, intent, and mobility. We evaluated BRIDGE in two controlled studies with 20 participants (10 national wheelchair basketball team players and 10 amateur players). The results showed that BRIDGE significantly enhanced the perceived naturalness of player postures and made tactical intentions easier to understand. In addition, it supported functional classification by realistically conveying players’ capabilities, which in turn improved participants’ sense of self-efficacy. This work advances inclusive sports learning and accessible coaching practices, contributing to more equitable access to tactical resources in parasports.

There is growing interest in communication artifacts that allow users to connect with others while supporting the emotional well-being that often arises from social interactions. Social presence is central in digital interactions, as it fosters a sense of “being together,” which is essential for affective communication, collaboration, and relationship-building in computer-mediated environments. This study explores the impact of behavioral synchronization through HugBits, a cushion -shaped device designed for remote hug-based interaction, on both perceived social presence and users' prosociality. We conducted two experiments: (1) testing whether hug synchronization increased perceived social presence, and (2) assessing whether synchronization influenced cooperative decision-making in a Prisoner's Dilemma game. In both experiments, the occurrence of behavioral synchronization (i.e., hugging) was controlled by having participants interact with a virtual agent that was designed to probabilistically promote or avoid synchronized hugs. Results show that more synchronized interactions significantly enhanced perceived social presence and promoted cooperative behavior. These findings highlight behavioral synchronization as a promising design strategy for affective communication technologies and provide concrete implications for integrating emotional and behavioral outcomes in computer-mediated communication research.

This paper examines whether the robot’s face-turning behavior toward humans during handover can be a behavior cue that supports the human action plan in accomplishing a task. A series of behavior cues observed in interactions, such as Joint action, are understood as social signals, which can be used as cues to help each other make decisions about their own action plans. We hope that constructing a inference model of empathetic behavior will enable humans and robots to share the intention of their behavioral goals in joint action. We conducted temporal analysis and subjective evaluation of the receiver’s behavior in an object handover experiment. The results showed that the subjective experience did not significantly change when the robot turned its head toward the receiver, but the time to receive the participant significantly increased. This suggests that the robot’s face-turning behavior may have altered the receiver’s human prediction and action plan, causing the receiver to hesitate to respond. This shows the potential for the robot face-turning to a human which could change the human inference and response behavior plan.

Live Performances have been conducted online in recent years due to the influence of the COVID-19 pandemic. This generated a novel problem of losing eye contact with the performers, which deteriorates the viewing experience in the online setting. This study proposes a feedback system based on gaze modeling to augment the perceptual experience of being faced by the performer during an online live performance viewing. This system aims to improve the experience of viewing online live performances by recreating a perceptual experience of interaction between the performer and the audience. The proposed system consists of a gaze recognition unit that judges whether the performer is looking at the camera based on gaze modeling in the video, and a stimulus presentation unit that presents the judgement result to the audience through vibration stimulus. This paper describes the gaze recognition model, experiments to determine the parameters to be used in the model, and the evaluation of live performance viewing experiments using the proposed system.

In this study, we developed a smart mirror device for continuous mental health checkup based on individual response characteristics through simple and short dialogue interaction on a daily basis. Early detection and intervention have a predominant impact on remission in many cases of mental illness, and daily monitoring is essential for early detection. However, existing methods require time-consuming and burdensome measurements and rely on information obtained from patients' experiences, communication, and physical findings for diagnosis. We propose a mental health checkup system that uses supervised learning to model physician's evaluation based on the response characteristics of individuals during short dialogue interaction with the smart mirror. A validation experiment with 10 participants was conducted for two weeks, and the results showed that mental health checkups by physician was successfully reproduced with the proposed algorithm. In conclusion, this study demonstrates the potential of Smart Mirror for mental health checkup in daily life. It provides a potentially more objective and non-invasive method for early detection and monitoring, which could contribute to gradual improvements in mental healthcare.

We propose a method for fabricating custom passive finger prostheses with compliant fingertips. The prosthetic devices detailed in this study are intended to mimic the compliance of biological fingertips: improving the capacity and comfort of keyboard typing, the manipulation of small objects, and instrumental activities of daily living in general. The proposed method utilizes 3D scanning, 3D printing, and computer-aided design to realize the compliant fingertips feature and to create custom-fit prostheses. We present the clinical application of the proposed prostheses on two end users and the evaluation results of typing speed and gross finger dexterity. The results showed successful fitting of the developed prostheses on both participants. Performance tests showed comparable performance in typing speed and accuracy with and without prostheses, and improved comfort with the prostheses. Gross finger dexterity and pinch force did not show improvement with the prostheses. The participants were able to utilize all fingers in keyboard typing and other iADLs instead of only the intact fingers, which indicates the potential for further functional improvement after habituation to the prostheses. In addition to improved comfort, the participants also reported reduced pain in sensitive sites, and aesthetic satisfaction with the prostheses.

The 9TH AAAC Conference on Affective Computing and Intelligent Interaction 2021 was held in a virtual format in the fall of 2021. It was technically co-sponsored by the IEEE Computer Society and featured the recent work on Affective Computing. The six best papers from this conference were selected by the technical program chairs. They were invited to submit their extended version to be considered for this special section at the IEEE Transactions on Affective Computing. Each submission was reviewed by at least three expert reviewers and was evaluated in terms of overall contribution and the adequacy of the additional content to warrant a new article. This special section features five accepted submissions whose major contributions are summarized below.

The amount of force and the timing of force application are necessary training skills in basketball free-throw practice. Coaches cannot readily assess the shooter's motion and provide instructions because the throwing motion is performed in a short time, and because the applied force cannot be directly observed by a second person. In this work we propose a wearable device that measures the wearer's force control during throwing based on electrical impedance and acceleration measurement of the forearm, and provides feedback such that the wearer can adjust their motion appropriately. Temporal features of the electrical impedance and acceleration measured during throwing are determined experimentally to differentiate between experts and beginners. Using these features, we propose a training system for force control using the proposed device and verify its effectiveness in free-throw training.

As the operation of robots is becoming easier, adopting a robot as a companion for families is increasing. When considering that family playtime has a positive influence on all family members, it is important to provide various activities for families to play together. Robots might mediate fun activities to increase family playtime. Therefore, this research was designed to compare play activities between dyadic child-robot interactions and family-robot interactions. We explored the possible activities for family playtime with a NAO robot. The results from three family groups showed that verbal activities, such as giving an instruction to a robot and talking to a robot, increased when family members are playing together with the robot. Also, increased physical activities were involving all family members, such as giving and receiving balls. Therefore, it will be necessary for a robot to recognize the voices of multiple family members and respond to each member appropriately. More importantly, a robot's time allocation for each family member and encouraging the participation of all members will be necessary to maintain or Increase the family playtime.

Autonomous navigation in highly populated areas remains a challenging task for robots because of the difficulty in guaranteeing safe interactions with pedestrians in unstructured situations. In this work, we present a crowd navigation control framework that delivers continuous obstacle avoidance and post-contact control evaluated on an autonomous personal mobility vehicle. We propose evaluation metrics for accounting efficiency, controller response and crowd interactions in natural crowds. We report the results of over 110 trials in different crowd types: sparse, flows, and mixed traffic, with low- (< 0.15 ppsm), mid- (< 0.65 ppsm), and high- (< 1 ppsm) pedestrian densities. We present comparative results between two low-level obstacle avoidance methods and a baseline of shared control. Results show a 10% drop in relative time to goal on the highest density tests, and no other efficiency metric decrease. Moreover, autonomous navigation showed to be comparable to shared-control navigation with a lower relative jerk and significantly higher fluency in commands indicating high compatibility with the crowd. We conclude that the reactive controller fulfils a necessary task of fast and continuous adaptation to crowd navigation, and it should be coupled with high-level planners for environmental and situational awareness.

Distal facial Electromyography (EMG) can be used to detect smiles and frowns with reasonable accuracy. It capitalises on volume conduction to detect relevant muscle activity, even when the electrodes are not placed directly on the source muscle. The main advantage of this method is to prevent occlusion and obstruction of the facial expression production, whilst allowing EMG measurements. However, measuring EMG distally entails that the exact source of the facial movement is unknown. Therefore, we investigated whether we could identify specific Facial Action Units (AUs) from distal facial EMG after an initial calibration phase with Computer Vision (CV). We compared Support Vector Machines (SVM) and Random Forest (RF) with several types of feature engineering and early fusion of the two modalities. The detection performance for AU6 (Orbicularis Oculi) and AU12 (Zygomaticus Major) was estimated by calculating the agreement with Facial Action Coding System (FACS) certified coders. The best results were achieved using Random Forest. Using a fusion of CV and EMG features resulted in F1 scores of 0.83 for AU6; and the fusion of engineered EMG plus CV returned an F1 score of 0.81 for AU12. Both these results are well above the CV baseline that shows F1 scores of 0.56 and 0.62 for AU6 and AU12 respectively. This demonstrates the potential of distal EMG to detect individual facial movements. It also enables researchers to compare the results measured with this wearable device to psychological research on facial expressions using FACS. Using a wearable enables measurements with higher ecological validity. Finally, we observed that EMG activity starts before the onset of visually perceived movement. Because of this, the agreement between EMG-based methods and FACS coders might be underestimating the ground truth.

We propose a method for learning transferable skills among various state-action spaces for reinforcement learning using embedding spaces. Most existing approaches for robot skill transfer assume similar hardware settings between robots. However, in practical applications, identical hardware settings among different robots is not necessarily guaranteed. This study is an attempt to abstract and represent the characteristics of state-action space caused by differences in hardware, such as sensor settings and actuators, using an embedding space. Using this method, the skills become transferable between different hardware settings without requiring heuristic tuning by experts. In this study, we implement the proposed method on a mobile robot in a two-dimensional plane and demonstrate its effectiveness by performing a navigation task in a simulation environment.

The education of children with Autism Spectrum Disorder (ASD) to acquire concepts present in social interaction is essential for facilitating their inclusion in society. This paper proposes a novel portable device to digitalize sessions and provide Visual Support (VS) to aid children during their learning process. The device can record video and estimate the pose of participants during activities, which can be used for further analysis. It uses fisheye lenses for sensing and projecting and a new concept, called Panda Masking, to minimize light illumination on participants’ eyes. Initial tests with the device indicate its feasibility to be used in monitoring therapy sessions and provide Visual Support while preventing face illumination.

In this paper, we propose and evaluate a novel human-machine interface (HMI) for controlling a standing mobility vehicle or person carrier robot, aiming for a hands-free control through upper-body natural postures derived from gaze tracking while walking. We target users with lower-body impairment with remaining upper-body motion capabilities. The developed HMI bases on a sensing array for capturing body postures; an intent recognition algorithm for continuous mapping of body motions to robot control space; and a personalizing system for multiple body sizes and shapes. We performed two user studies: first, an analysis of the required body muscles involved in navigating with the proposed control; and second, an assessment of the HMI compared with a standard joystick through quantitative and qualitative metrics in a narrow circuit task. We concluded that the main user control contribution comes from Rectus Abdominis and Erector Spinae muscle groups at different levels. Finally, the comparative study showed that a joystick still outperforms the proposed HMI in usability perceptions and controllability metrics, however, the smoothness of user control was similar in jerk and fluency. Moreover, users' perceptions showed that hands-free control made it more anthropomorphic, animated, and even safer.

In computer-mediated communication, the amount of non-verbal cues or social signals that machines can support is still limited. By integrating haptic information into computational systems, it might be possible to give a new dimension to the way people convey social signals in mediated communication. This research aims to distinguish different haptic gestures using a physical interface with a cushion-like form designed as a mediator for remote communication scenarios. The proposed interface can sense the user through the cushion's deformation data combined with motion data. The contribution of this paper is the following: 1) Regardless of each participant's particular interpretation of the gesture, the proposed solution can detect eight haptic gestures with more than 80% of accuracy across participants, and 2) The classification of gestures was done without the need of calibration, and independent of the orientation of the cushion. These results represent one step toward the development of affect communication systems that can support haptic gesture classification.

This study aims for the new driving skill improvement support system based on driving skill evaluation. We try to define driving skill in terms of "spatial perception" and "prediction". This research can be roughly divided into two steps. The first step is to establish an objective skill evaluation model. The second step is to verify effects of the support system for driving skill improvement. In this paper, we describe that we evaluated driving skill based on parking performance, operational behavior like steering and cognitive behavior like head motion and gazing points. We conducted driving behavior measurement experiment by using a real vehicle to get operational and cognitive data during driving. In consequence, we confirmed that driving skill could be defined by spatial perception and operational behavior of drivers in approaching to an obstacle.

This paper investigates the effects of real time visual biofeedback for improving sports performance using a large scale immersive mixed reality system in which users are able to play a simulated game of curling. The users slide custom curling stones across the floor onto a projected target whose size is dictated by the user's stress-related physiological measure; heart rate (HR). The higher HR the player has, the smaller the target will be, and vice-versa. In the experiment participants were asked to compete in three different conditions: baseline, with and without the proposed biofeedback. The results show that when providing a visual representation of the player's HR or "choking" in competition, it helped the player understand their condition and improve competition performance (P-value of 0.0391).

We engineered an exoskeleton, which we call HandMorph, that approximates the experience of having a smaller grasping range. It uses mechanical links to transmit motion from the wearer's fingers to a smaller hand with five anatomically correct fingers. The result is that HandMorph miniaturizes a wearer's grasping range while transmitting haptic feedback.

Affective communication technologies are designed to enhance awareness, social connectedness, and affectivity. Design strategies involve alternative methods to convey affection in computer-mediated scenarios, emphasizing on the importance of mediated physical contact. Therefore, we proposed a huggable interface to mediate social touch by sensing the user's hug gestures, transferring them to a paired device, and delivering them as simple cues. We investigated the effect of the huggable interface as a mediator with a physical embodiment and compared it with a similar communication interface represented by an agent with a virtual embodiment on a touch screen. During the experiments, we set up a scenario in which individuals with a close relationship watched movies and communicated with each other. Results showed the effect of both interfaces in terms of perceived social connectedness and stress levels. The discussion pointed out the potential and limitations of the proposed evaluation method, as well as of each type of interface as affective communication technology.

Electro-Dermal Activity (EDA)and head movement have been shown to correlate with felt affect. Given the easiness to measure them, they are suitable as a wearable affective assessment tool. Arguably, autonomic affective responses such as EDA are less affected by volition than the production of other embodied cues of affect such as facial expressions. Moreover, head movement has been shown as a reliable source of information about the intention behind a facial expression. Therefore, we explored the feasibility of using EDA measured from the neck and head movement to make inferences about the nature of facial expressions, in particular, smiles. EDA was measured simultaneously from the hand and the neck of participants displaying spontaneous and posed smiles. Our results show that both measurement locations are highly correlated. Furthermore, EDA signals carry information about the spontaneity of the measured smiles, as shown by a classification accuracy of about 90%. Finally, head movement turned out to be rather revealing, with classification accuracy reaching about 99 %.

In this paper, we propose a new mobile system to support therapists for teaching and tracking socio-communicative behaviors in children with neurodevelopmental disorders during music therapy sessions. The CHIMELIGHT system was designed to deal with the current issues in conventional therapies, such as the difficulty in both evaluating the performance and maintaining engagement of these children during therapeutic activities. The system evaluated movements made by a child with neurodevelopmental disorders playing a musical instrument while delivering contingent visual feedback based on real-time motion analysis. A set of metrics was implemented to evaluate the performance during the therapy activity and quantify specific target behaviors. An evaluation study performed during music therapy group sessions showed that the CHIMELIGHT-delivered visual feedback increased the engagement of children in the activity and decreased targeted negative behaviors. In some participants, we observed potential changes in their positive behaviors. Interviews and questionnaires provided to therapists showed that the developed system was effective for supporting evidence-based music therapy. Accordingly, our research enables new methods for both interactive therapy and mediation of the interaction between therapists and children with neurodevelopmental disorders.

EnhancedTouchX, a bracelet-type interpersonal body area network device, not only detects but also quantifies interpersonal hand-to-hand touch interactions. Without any wired connection, it can identify the direction and gestures of a touch. The developed device can connect to an external device via Bluetooth Low Energy for monitoring and logging where, when, how long, who, and how the touch interactions occurred. These daily augmented touch interactions provided by such contextual information would offer a variety of applications to facilitate social interactions. Our experiment, conducted with several pairs of participants, demonstrates that the devices can identify the direction of a touch (from one initiating the touch (active touch) to the one being touched (passive touch)) with 95% accuracy. In addition, the devices are also capable of identifying four types of touch gestures with 85% accuracy using a simple threshold classifier.

This paper explores how human perceptions, actions, and interactions can be changed through an embodied and active experience of being a smaller person in a real-world environment, which we call an egocentric smaller person experience. We developed a wearable visual translator that provides the perspective of a smaller person by shifting the wearer's eyesight level down to their waist using a head-mounted display and a stereo camera module, while allowing for field of view control through head movements. In this study, we investigated how the developed device can modify the wearer's body representation and experiences based on a field study conducted at a nursing school and museums, and through lab studies. It was observed that the participants changed their perceptions, actions, and interactions because they are considered to have perceived themselves as being smaller. Using this device, designers and teachers can understand the perspectives of other people in an existing environment.

Positive experiences are a success metric in product and service design. Quantifying smiles is a method of assessing them continuously. Smiles are usually a cue of positive affect, but they can also be fabricated voluntarily. Automatic detection is a promising complement to human perception in terms of identifying the differences between smile types. Computer vision (CV) and facial distal electromyography (EMG) have been proven successful in this task. This is the first study to use a wearable EMG that does not obstruct the face to compare the performance of CV and EMG measurements in the task of distinguishing between posed and spontaneous smiles. The results showed that EMG has the advantage of being able to identify covert behavior not available through vision. Moreover, CV appears to be able to identify visible dynamic features that human judges cannot account for. This sheds light on the role of non-observable behavior in distinguishing affect-related smiles from polite positive affect displays.

Wearable robots suffer from issues of usability, cost-performance, and battery life. These drawbacks hinder the development of their market and their implementation in healthcare and industry. Back support exoskeletons in particular rely heavily on mechanical storing of energy but are still mostly using traditional mechanical elements and actuators to achieve the task. The core technology of our work, named MRLink, uses a smart fluid in combination with a compression spring to produce a unique energy store-and-release functionality. With only 5 Watts MRLink can produce over 500N of braking force. The braking force, variable viscosity, can be continuously controlled through the supply current. With appropriate selection of the spring and control method, the required assist energy can be acquired from body weight and movement dynamics, stored momentarily and, released at the appropriate timing to assist the body motion. The MRLink does not require an external power source like pneumatic actuators, does not require gearboxes and large batteries like DC motors, and comes in a compact package. Thus, this technology has the potential to significantly contribute to wearable exoskeletons in general, and back support exoskeletons in particular. In this work we present the development of a back support exoskeleton prototype based on MRLink, we describe its functional details, and a pilot test partially verifying the proposed functions.

In this paper, we propose a novel floor projection drawing tool for AR-aided special needs education, called CANVAS, which can be easily used by teachers or practitioners. We propose a method of using floor projection to support the activities of children with neurodevelopmental disorders (ND) by summarizing the used elements of past activities: Positions (Points), Lines, Areas, and Symbols (PLAS method). This PLAS method fulfills the design requirements of the content that can be created on CANVAS, which is a tablet type drawing system. The content is transmitted in realtime to the gym projectors to realize realtime and interactive activities based on the projection of geometric shapes, at the intended place on the floor. Experimental results demonstrate that simple shapes such as circles and lines are sufficient to support cognition and changes in the physical behaviors of children with ND.

Embodied knowledge of one's experiences are broadly categorized as tacit knowledge, that can be acquired through active physical experiences only and not by conversations or visual media. We hypothesize that shaping a person's bodily sensation, such as physical representation or a physiological characteristic, to that of another person, including a child or a patient, while allowing active, embodied, and social interactions with the surrounding objects and people in a real-world environment would provide an authentic as well as empathic knowledge to school teachers, product designers, and medical staff when designing living environments or communicating with them. We propose and demonstrate a new style to experience one's perspectives, called HYPERSPECTIVE (hyper + perspective), that can be achieved with wearable devices, and discuss its challenges, benefits, and feasibility in real scenarios.

This paper describes the deformation control of a ferrofluid-based robotic sheet for object handling. The ferrofluid sheet consists of ferrofluid wrapped in thin polyethylene sheets and allows us to handle the transport of objects by controlling its deformation due to the characteristics of a ferrofluid. To control sheet deformation, we need to develop the sheet's mechanical model. Although the mechanical model of the ferrofluid has already been researched, the effect of the polyethylene sheet on the ferrofluid has not been considered. In this paper, we first verify the relationship between the fluid shape in the ferrofluid sheet and the thickness of the sheet on the fluid. Next, we verify the transportation method by making the ferrofluid sheet to move two balls simultaneously and a thin circular plate on its surface. We also confirm the deformation control of this robotic sheet in an alternative configuration. The achievement contributes to the technology for soft robotics and advanced control, opening new aspects for fluid-based 3-D manipulations. Note to Practitioners-This paper is motivated by the problem of manipulation of flexible objects. It is difficult for conventional manipulators to handle flexible objects, such as droplets, because their shapes dynamically deform while being handled. We considered that a flexible system should be able to conform to the object shape. Therefore, we investigate the possible use of a ferrofluid as an actuating component because of its fluidity and peculiar characteristics. A ferrofluid has the characteristics of a fluid and magnetic material; therefore, the fluid gathers around the source of a magnetic field and applies force to an object on it. In this paper, we verify the characteristics of the proposed system as well as the object manipulation. Through characteristic verification, the behavior of our system can be modeled, and the method of object transportation can be systematized.

Visual support (VS) is one of the effective ways of facilitating activities of children with neurodevelopmental disorder (ND). This paper reports on an interactive VS provided by a large-scale floor projection system in an augmented gymnasium called FUTUREGYM, designed for children with ND. The study focuses on students' cleaning, and two interactive VS activities-Mop Game, an exergame involving group cleaning, and Mop Guide, a VS for training about vocational cleaning-were designed with the teachers with the aim of motivating students toward cleaning and help them acquire fundamental cleaning skills. The study attempts to design a VS for cleaning that is suitable for the students by conducting an empathic design approach, which helps us understand what are the problems, obtain new perspectives, and gather ideas into demonstrative prototypes by sharing values and thoughts with the teachers and their students. This is a case study of deploying an empathic design approach in a special needs school setting.

This paper proposes a calibration method for a large-scale floor projection system at a school gym. In our system, the projectors are installed on the ceiling of a school gym, and the contents are projected onto the floor. The projection results suffer from both projective distortion and lens distortion. It is hard to use chessboard-based calibration methods or self-calibration methods in our system due to the restrictive deploy environment and practical limitations. Our method is basing on the projector-camera system and "straight lines have to be straight". The experiments show that our method fulfills the on-site requirements of the floor projection system at school gym for learning aid usages.

In this paper, we propose the design of an exoskeleton with support at the knee joint and lower torso for sit-to-stand and stand-to-sit (STS) posture transitions; devised for users with spinal cord injury and other complete lower-body impairments. The STS transitions assistance is achieved through a power transfer mechanism that synchronizes knees and lumbar motion through a cable-driven pulley system. We analyze the human body dynamics in the posture transition with a rigid link model and the interaction interface with the exoskeleton through an impedance model for producing a passive system voluntarily controlled by natural motions of the upper body. Therefore, allowing the potential users to achieve STS transitions with their body residual capabilities without an external power source. Instead, transferring power from their upper-body to lower-body, herewith, controlling a passive energy storage. A prototype was constructed and evaluated with seven healthy subjects observing the proposed motion and muscle activity during the STS transitions. The results show a significant reduction in the muscle activity evaluated, at the erector spinae, gluteus maximus and rectus femoris, with reductions between 30% to 50% at the p <; 0.01 level comparing STS transitions with and without the exoskeleton support. Concluding that the STS transitions support is feasible with the passive exoskeleton envisioned for applications in upright locomotion, STS training, and rehabilitation.

Walking rehabilitation must be performed based on voluntary motion intention, and for this purpose, the development of a control method for an exoskeleton robot based on voluntary intention is investigated. This study proposes a method of exoskeleton robot control to estimate the voluntary lower limb muscle activities lost after a spinal cord injury (SCI). This method is based on the spinal cord mapping of the remaining muscle activities and its matching to the one obtained from healthy participants considering the muscle synergy of the whole body during the walking motion. By implementing the matching procedure at the spinal cord map level and incorporating information of reliable and unreliable spinal cord levels based on a diagnosis, the method has the potential to provide a maximally voluntary locomotion for people with SCI. We report an analysis of the synergy of the whole-body muscle activity during walking and its spinal cord mapping using non-negative matrix factorization and the computation of the transformation matrix to estimate the intended lower limb muscle activity from the remaining spinal cord activity. The implementation of the proposed method using the right leg of the hybrid assistive limb and walking experiments with a healthy participant are also reported.

This paper describes a novel passive exoskeleton for voluntary sitting-standing posture transition for children with lower limb impairment. The design of the exoskeleton is based on the utilization of the center of gravity transition through the user's upper body motion. The passive exoskeleton powered by gas springs allows the user to realize a natural-like posture transition by the user's voluntary upper body motion. We designed the posture transition model such that the user can realize the posture transition in a natural manner based on a minimum jerk criterion. The proposed design aims to permit toilet usage without transferring seating positions between the exoskeleton and toilet seat. Furthermore, the developed mechanism can be integrated with a regular wheelchair, which would allow users to have locomotion capability. We believe that the design can improve children's self-reliant social activities by supporting their voluntary posture transition and toilet use. In this paper, we describe the detailed design process of the exoskeleton and preliminary experiments to investigate its effectiveness through evaluation with a healthy participant.

We propose a calibration method for deploying a large-scale projection-based floor display system. In our system, multiple projectors are installed on the ceiling of a large indoor space like a gymnasium to achieve a large projection area on the floor. The projection results suffer from both perspective distortion and lens distortion. In this paper, we use projector-camera systems, in which a camera is mounted on each projector, with the “straight lines have to be straight” methodology, to calibrate our projection system. Different from conventional approaches, our method does not use any calibration board and makes no requirement on the overlapping among the projections and the cameras' fields of view.

Facial expressions are one of the most salient cues of affect. However, they are also a communication tool that can be expressed voluntarily. Spontaneous and posed facial expressions have different characteristics. In this study, we show the potential of using Electromyography (EMG) in a wearable device to automatically differentiate between posed and spontaneous smiles. This classification is moderately successful when using spatial and magnitude features, and increases significantly if temporal features are included. Hence, the high temporal resolution of EMG-based detection is advantageous in this task.

We present a novel, paired, wearable system for combining the kinesthetic experiences of two persons. These devices allow users to sense and combine muscle contraction and joint rigidity bi-directionally. This is achieved through kinesthetic channels based on electromyogram (EMG) measurement and electrical muscle stimulation (EMS). We developed a pair of wearable kinesthetic input-output (I/O) devices called bioSync that uses specially designed electrodes to perform biosignal measurement and stimulation simultaneously on the same electrodes.

This study proposes a novel facial wearable robot for supporting eyelid movements. The robot is designed for people with facial paralysis, especially on one side of the face. People with facial paralysis are not able to blink, which leads to dry eyes and could cause permanent damage to the cornea. They are also prone to developing synkinesis during recovery, and a biofeedback-enabled rehabilitation is considered to prevent it. Based on these, we developed a robot system to support eyelid movements on the paralyzed side, based on the eye closure on the healthy side. The robot has a novel mechanism for supporting the eyelid control, made from soft material, which is called Eyelid Gating Mechanism (ELGM). ELGM deforms by simple rotational or linear actuation inputs and its deformation is customized to eyelid movements. Therefore, this robot can provide non-invasive and gentle support for eyelid movements. In this paper, we present the development and performance evaluation of the developed facial wearable robot.

It is well-known that walking offers many health benefits for everyone, especially for older people who need to maintain mobility and independence coping with declining of functional capacity. In this paper, we present the design of a humanoid walking trainer that has to monitor and encourage walking in the elderly. This design is based on our target users' preferences. We present as well a preliminary walking experiment that was carried out in order to test the accuracy of the gait data obtained from the laser range sensor, which is positioned on the robot, during motion.

Because visually impaired persons are not able to confirm the appearance of their own face, they are afraid of and uneasy about makeup. We have been developing a system that assists makeup application through verbal feedback according to the appearance of the user's face. The system encourages social communication by helping the user feel confident. In this paper, we introduce a new method of using a dedicated camera device and an image processing algorithm to quantify lip makeup. We designed the camera device to capture face images under different illumination types; white light and green light. In the image processing, we extract the lip area from a segmentation that uses the difference in saturation between two lighting conditions. We also developed a symmetry histogram-based geometrical feature of lip shape to estimate the symmetrical characteristics. The experimental results show that the proposed approach yields results close to a human's subjective evaluation.

In this paper, we propose a computational modeling method to investigate head-eye coordination in face-to-face behavior. The method looks into probability density of individuals' head orientation during looking at others' face. We conducted experiment under two different scenarios in human-human interaction. Under each scenario, individuals' head orientation could be fitted with one Gaussian distribution. Referring to the model, it is possible to calculate several ranges of head orientation, which could be used as surrogate for estimating visual focus of attention. The model also provides theoretical support to the design of automatic systems that measure face-to-face interaction.

This study proposes a novel wearable device, called HARS (Hybrid-based Aspiration and Respiration Sensing), made of a flexible material for respiratory measurement and home monitoring of asthma. This paper mainly mentions about the method of respiratory phase identification using respiratory sounds and chest movements. The main challenge is to implement chest movement acquisition with a small device, which has the advantage of flexibility. The characteristics of the acoustic signals and the movement signals are investigated. On the basis of the results, a method is developed in which the acoustic and movement signals are derived from breathing using sensors that have low sensitivity to positioning. The results of another experiment demonstrate the feasibility of HARS for phase identification during spontaneous breathing.

Facial expressions are usually linked to emotional states of a person, and are among the most salient cues for automatic emotion recognition. They are an indispensable social communication tool, and therefore, they can also be fabricated to face complex situations in social interaction. Despite efforts to either deliberately or unconsciously conceal an emotion, micro-expressions are usually leaked. Because of their revealing nature, potential applications can arise from automatically detecting them, both at a personal and inter-personal levels. Therefore, we explored micro-expression detection using a wearable device that detects distal facial EMG signals unobtrusively, as an alternative to Computer Vision-based detection. EMG recognition is advantageous over commonly used video recognition techniques because of its robustness against occlusion and light changes, good temporal resolution, and independence of movement. We evaluated the performance of the wearable device on micro-smile recognition. The results show the potential of EMG to detect such fast and subtle spontaneous expressions. Finally, we evaluated the device as a tool to provide affective annotations of Advertisement Videos, and social interaction in a face-to-face context while watching these stimuli.

We present EnhancedTouch, a novel bracelet-type wearable device for facilitating human-human physical touch. In particular, we aim to support children with autism spectrum disorder (ASD), who often exhibit particular communication patterns, such as lack of physical touch. EnhancedTouch is a unique device that can measure human-human touch events and provide visual feedback to augment touch interaction. We employ personal area network (PAN) technology for communication with partner devices via modulated electrical current flowing through the users' hands. Our user study show that the visual feedback provided by the developed bracelet motivates children with ASD to touch one another. Moreover, EnhancedTouch offers a function to record the time and duration of a touch event as well as the identity of the touched person. This allows us to identify and evaluate intervention based on physical touch in a quantitative manner.

Although socially assistive robots are popularly studied as a powerful tool in therapeutic activities for children with Autism Spectrum Disorder (ASD), there are several challenges particularly in automation of the robot's behavior due to difficulties of sensing children's social behaviors by the robotic agent. We developed measurement methods to quantitatively measure children's social behaviors, such as smiling and facing behavior, by means of a wearable device and an image processing. Smiling, facing behavior and their synchrony are usually considered as important social behaviors that imply positive affect and engagement of the participant during the therapeutic intervention. Therefore, the proposed approach would contribute not only to develop the robotic agent which is capable of performing in response to social behaviors of children with ASD, but also to support an evidence-based therapeutic intervention. In this article, the measurement performance of the proposed methods were verified by experiments with ten children with ASD, and the relationship between their smile and facing behavior were analyzed.

Engaging and motivational toys used during therapies for children with Autism Spectrum Disorder (ASD) need to be combined with the appropriated evaluation methods. Technology can provide the therapists with tools to facilitate the analysis of the interventions. In this paper we propose a model using paired devices with three different interaction rules made to facilitate turn-taking behaviors. These rules are evaluated in succession using identical spherical devices with simple visual cues designed to mediate turn-taking interaction between children and therapists. The results explored the potential of paired devices with embedded sensors as a method not only to engage children in the activity, but also to measure social cues that describe children performance. The evaluation of the proposed interaction model was used to provide insights for future implementations of paired devices for training turn-taking.

Population aging will have an economic and social impact in the near future. We will witness a declining of labor force and an increase of funding support for health care and pension. An active research field is now aiming at improving mobility of the elderly, since the importance of physical exercise for mental and physical health is an established fact. In order to have a user centered design approach, we conducted a feasibility study bringing the humanoid robot Pepper to a nursing home and testing the reactions of the older people when they have to walk with a robot for the first time. We wondered if the robot in such a situation might be able to monitor and encourage walking. The results of this experiment will lead to improve the design of an acceptable humanoid walking trainer which can help the overworked staff in a nursing home scenario.

This paper presents a synchronous kinesthetic interaction among people through haptic input/output based on biosignal measurement and stimulation. Users are able to bi-directionally transmit kinesthetic experiences such as rigidity of joints or exertion of muscles in addition to physical bodily motion. Such interaction would be very important in the fields of rehabilitation and sports training. In this study, we introduce a set of wearable devices that is capable of both electromyogram (EMG) measurement and electrical muscle stimulation (EMS) simultaneously on the same muscle by using common electrodes. To achieve smooth kinesthetic bi-directional interaction, we propose a new method for discharging the residual potential of the stimulation in order to enable fastest simultaneous operation (40Hz). Through a performance evaluation, participants could recognize the teacher's exertion strength on a 5-point scale without visual information.

We developed a novel wearable device for monitoring electrocardiogram (ECG) of children in a living environment using clothes fitted with dry electrodes on elastic-type sleeve cuffs. The ECG signals are captured at the wrist with a very light pressure applied by the elastic element of the clothes. This reduces the feeling of restraint, and hence, the proposed system is suitable for the ECG measurement of children during their activities of daily life. In this paper, we describe the design details of the proposed system and conduct experiments to determine the effects of the electrode location and the elastic materials on the motion artifacts. We also evaluate the performance of the developed device with a child participant. The accuracy of R-wave detection is more than 96 %, and the error ratio of R-R intervals is less than 1 % in both the resting state and when small movements are made.

There are a few types of step-climbing wheelchairs in the world, but most of them are large and heavy because they are power-assisted. Therefore, they require large space to maneuver, which is not always feasible with existing house architectures. This study proposes a novel step-climbing wheelchair based on lever propulsion control using human upper limbs. The developed step-climbing wheelchair device consists of manual wheels with casters for moving around and a rotary-legs mechanism that is capable of climbing steps. The wheelchair also has a passive mechanism for posture transition to shift the center of gravity of the person between the desired positions for planar locomotion and step-climbing. The proposed design consists of passive parts, and this leads the wheelchair being compact and lightweight. In this paper, we present the design of this step-climbing wheelchair and some preliminary experiments to test its usability.

This study proposes “Gaming Humanoid”, a robot that is able to play video games with human as a partner in the same gaming conditions found typically in a gaming environment, in particular, using body gestures and being able to adapt the skill level. We developed an autonomous video gameplay capability of the humanoid robot NAO. In the autonomous gameplay, we used image processing to recognize the gaming situation and play along with the content. Also we designed the robot's body movement to perform realistic gameplay and to be able to adapt gaming performance. The result with Tennis Game shows that the gaming humanoid was able to play in real gaming environment and also able to control performance of gameplay. The experiment with human showed that the gaming humanoid was not inferior to human level in terms of gameplay. Moreover, the preliminary study with children demonstrated its feasibility of facilitating interactions by implementing emotional/encouraging behaviors. In this paper, we introduce the gaming humanoid as a playmate partner for human, and we describe and evaluate the whole system.

Among treatments for children with ASD, assistive robots are growing popular as they are able to elicit different social behaviors. At the same time, technology is providing methods to automatically collect quantitative data, in order to assist the therapist evaluating the children progress. In this study we introduce a system composed of multiple spherical devices, as well as the design of a turn taking activity using those devices. In previous studies turn taking was found to be an important social skill for development and engaging in activities with others. To evaluate the system performance, a single case experiment with a boy with ASD and the developed device was done. During the activity, the device had two different roles: to engage the child on the turn taking activity and to provide information to the therapist that describe the child's behavior. We could successfully collect quantitative data that represent turn taking as well as we could observe how does the boy manipulate and interact with the device. Based on the results, we are motivated to keep exploring the potential application of devices mediated activities for children with ASD.

This paper presents the design, implementation, and application of a vision-based automatic system that measures facial interaction based on human's cognitive feature. We investigated the feature of people's facial interaction under natural gaze-movement via an experiment, and created criteria of k-degree facial interaction and face-to-face interaction depending only on facial orientation. These criteria were used to develop the automatic system. The system is vision-based. It could be easily embedded into applications. We focused on an application to understand the behavior of children with autism spectrum disorders (ASD), and tested the use of the automatic system in a robot-assisted activity for those children. The results suggested that the system could help to improve the efficiency of behavior analysis during the children's activities with the robot. The facial interaction measured between the children and the robot can be used by therapists to comprehend the children's psychological aspects and state of health.

We propose a novel wearable interface for sharing facial expressions between children with autism spectrum disorders (ASD) and their parents, therapists, and caregivers. The developed interface is capable of recognizing facial expressions based on physiological signal patterns taken from facial bioelectrical signals and displaying the results in real time. The physiological signals are measured from the forehead and both sides of the head. We verified that the proposed classification method is robust against facial movements, blinking, and the head posture. This compact interface can support the perception of facial expressions between children with ASD and others to help improve their communication.

This paper introduces a doll-type interface for real-time teleoperation of a humanoid robot in Robot-Assisted Activity (RAA) for children with Autism Spectrum Disorders (ASD). We developed a prototype of the interface and verified its usability by conducting RAA sessions with children with ASD.

Recent studies on autism spectrum disorders (ASD) have reported that positive emotions can be a good incentive for children with ASD to perform spontaneous positive social behaviors. Based on this findings, we propose an affective robot-assisted activity (ARAA) for fostering social interaction and communication skills among children with ASD by promoting their positive emotional responses through interaction with a robot. As it has been termed spectrum, every child has different social and affective characteristics that should be taken into account. However, due to difficulties in programming the robot's behavior, conventional RAA systems did not allow the therapist to customize the activity according to the characteristics of each individual. To tackle this problem, we developed a comprehensive framework of ARAA that consists of (i) a robot tele-operation method that allows a therapist to improvise a robot's behavior in real-time and (ii) a quantitative measurement method to describe social interaction within both behavioral and affective aspects.

This study proposes a gaming companion robot to facilitate social interaction among children. Games have been one popular social communication tool for people, and we propose a novel approach to gaming communication using a robot. We have implemented a robot that is capable of playing a video game to facilitate social interaction in a game environment. We used autonomous control to allow the robot to play during the videogame, and the Wizard of OZ (WOZ) framework for interaction between human and robot before and after the game. The results show that the robot was able to play the game in a manner similar to how a human plays the game. The preliminary study with children showed that this new approach was useful to facilitate people's interactions. We describe and evaluate the robotic system in this paper.

Social Playware is regarded as cyber-physical systems to support and enhance the experiences on play and social interaction among people by measuring and presenting physical contacts, spatial movement and facial expression. Several wearable or modular devices are used in this study, which enable behavior and affective measurements. It is aimed not only to integrate cyber and physical spaces by using developed wearable devices but also to show the possibility to help people develop their social ability throughout case studies. In this paper, a case-study involving children with developmental disorders such as autism and intellectual disorders is reported. We introduce a wearable device and social robotic device, which enable behavior and affective measurements based on augmented human technology. It is aimed not only to integrate cyber and physical spaces by using developed wearable devices but also to help people develop their social ability.

In this paper we present the design of a wearable device that reads positive facial expressions using physiological signals. We first analyze facial morphology in 3 dimensions and facial electromyographic signals on different facial locations and show that we can detect electromyographic signals with high amplitude on areas of low facial mobility on the side of the face, which are correlated to ones obtained from electrodes on traditional surface electromyographic capturing positions on top of facial muscles on the front of the face. We use a multi-attribute decision-making method to find adequate electrode positions on the side of face to capture these signals. Based on this analysis, we design and implement an ergonomic wearable device with high reliability. Because the signals are recorded distally, the proposed device uses independent component analysis and an artificial neural network to analyze them and achieve a high facial expression recognition rate on the side of the face. The recognized emotional facial expressions through the wearable interface device can be recorded during therapeutic interventions and for long-term facial expression recognition to quantify and infer the user's affective state in order to support medical professionals.

This study proposes a novel personal mobility vehicle for supporting and assisting people with disabled lower limbs. The developed mobile platform is capable of assisting voluntary postural transition between standing and sitting, called the Passively Assistive Limb (PAL) mechanism, in addition to high mobility with upright posture. The device consists of gas-spring-powered passive exoskeleton for postural support and two in-wheel motors for mobility support. The developed robot system allows a user to sit down on chairs and beds, and stand up through easy operations. In addition, a user can move the system in the standing posture without using their hands. In this paper, the development and assessments of the standing mobility vehicle are described.

In this paper, we propose a novel method to support learning of grasping behavior with a myoelectric hand through the use of force-feedback. A force transfer mechanism is implemented in the developed prosthetic hand, which creates a force stimulus that is synchronized with opening and closing of the gripper. For a person with congenital forearm defects, learning how to operate a myoelectric prosthetic hand is quite difficult. The force-feedback is presented at the end of the forearm of the affected side to assist the process of creating a body image. Several experiments were conducted to confirm the ability to generate real-time feedback with the developed hand. An extensive study was carried out on a subject with a congenital forearm defect, and the ability to perceive the stimulus of pressing force applied at the end of the forearm of the affected side was confirmed.

This study proposes a novel wearable device that augments the auditory awareness of hearing impaired people to help them identify the speaker during group conversations. A number of hearing impaired people are able to understand speech by using auditory-oral methods such as lip-reading, however they always need to watch the speaker closely. The proposed device estimates the direction of the sound source and indicates the estimated direction in real time with light-emitting diodes (LEDs), thus aiding hearing impaired people during group conversations. The device has 4 unidirectional microphones and estimates the sound source direction by comparing the signal amplitudes in each microphone. Hearing assistance devices are required to be small and wearable in order to assist anytime in daily life. The proposed device is small, wearable, and can easily be used in various situations.

The accident probability of beginner drivers is significantly higher than that of experienced drivers. It can be assumed that this is due to lack of driving skills which lead to making wrong decisions according to cognition and operating in correct way. In this paper, we propose a novel assisted driving system intended to help drivers to improve their skills for the reverse parking. The system is able to assist the driver by haptic instruction on the steering wheel in order to induce the driver to make the adequate operation. For the validation, we developed a 1/10 scale car simulator as a simulation environment on which we installed the proposed assistance method and conducted reverse parking experiment by using the simulator. According to the experiment, we validated that the parking accuracy and the trajectory similarity of subjects assisted by proposed system significantly increased compare to subjects unassisted. Consequently, the proposed assisted driving system could accelerate the learning of humans' driving skills.

Several methods have been investigated and realized for operation of exoskeleton robots for assistance of human gait. These systems perform motion intention estimation using the bioelectrical signals of muscle activation, body gestures and kinesiological information, or a mixed combination in a hybrid system. For motion intention estimation of the lower limb(s), information of the lower limbs is usually utilized. However, human gait is not only the function of the lower limbs, but also coordination between upper and lower limbs, adding to balance and cognitive functions as well. In this study, we investigate on how to utilize the synergies of upper and lower limbs of human walking in exoskeleton robot control by using the cane (walking aid). We analyse the synergies of human gait with cane in healthy subjects by means of Principal Component Analysis (PCA) in order to investigate the usability of cane for robot-assisted motor rehabilitation. We also implement a semi autonomous control for an exoskeleton robot, single leg version of HAL (Hybrid Assistive Limb) suit, based on the cane and body joint synergies.

This study proposes a novel method to change the apparent stiffness of a mechanical link system according to the given magnetic field. We propose an elastic link mechanism integrated with a magnetorheological (MR) fluid that can be controlled by the current of an electrical coil. The potential applications of the proposed device include sports training, physiotherapy, and motor rehabilitation. Compared to a simple weight lifting system, which requires manual changing of weights, this new device allows users to change the load easily and continuously, with relatively lower power consumption. The advantages of the developed link have been verified through several experiments aimed at evaluating its basic characteristics as well as through application to an upper limb orthosis.

In this paper, we propose a method of spatial and time-series analysis of facial expressions using a real-time depth image, particularly for the measurement of head orientation and for the dynamic analysis of facial expression. The method is based on an evaluation of the 3-dimensional shape of the face and is able to assess facial expressions, independent of the orientation of the head. It aims to extract psychological and physiological features from the physical features of the face's shape. This research focuses on the characteristics of transitions detected during changes of facial expression, particularly using dynamical spatial and time-series analysis. This paper describes the development of the sensing system, the experimental results of head orientation measurements, and the results of facial expression analysis.

This paper presents part of an on-going project to design a wearable supportive device to enhance facial expressiveness, in particular for the facial paralyzed patients. Earlier we introduced the SMA actuator based Robot Mask that can be used to enhance the expressiveness of the face. The basic concept of that design was pulling of the skin through wires attached to the face and we explained the human anatomy based criteria of selecting these pulling points and directions. The major reason to use SMA instead of traditional actuators such as motors was their silent acting nature. However, their dynamic properties been governed by thermal energy and their mechanical properties been affected by the hysteresis due to their metallurgy, SMA based actuators tend not to perform too well under cooling. This paper introduces a novel controlling scenario that use the actuation of only a limited number of SMA wires out of the total connected in series on either sides of a slider to control the direction and amount of movement of the slider. The advantage of this method is by keeping some SMA wires at low temperatures it was possible to achieve a high speed of actuation even when the direction of motion was changed. This paper also investigates on the amount of actuation rates that are required to generate natural looking smiles and later attempts to recreate them using the proposed actuation unit.

In this study, we propose a self-repairing mechanism for modules. This mechanism has been developed by studying the self-repairing processes found in nature. Proposed model is considered with the aid of energy given from the external environment. We investigate a self-repairing process using small modules actuated by vibration. We first describe the proposed self-repairing mechanism and its application to the control of a group of modules. We report the development of the vibrator used in this study and formulate the equation of translational and rotational motions of modules placed on this plate. The proposed mechanism is verified by carrying out numerical simulation and experiments. In addition, we introduce two simple structures based on the proposed mechanism.

This paper presents part of an on-going project to design a wearable supportive device, in particular for the facial paralyzed patients to enhance facial expressiveness. As various complications can result in facial disfigurement and loss of functionality in facial muscles it is required to develop a supporting device for people with such conditions. The previously proposed robot mask, which consists of a head supporter and motor units attempts to recreate facial expressions artificially by pulling the facial skin through cables attached to the skin. Since a facial expression is the result of the full or partial activation of combination of facial muscles, it is necessary to control the amount of displacement of the artificially created skin movement. Furthermore, in order to facilitate interpersonal timing of facial expressions, it is necessary to be able to read the nerve signals and process them in real time. This paper presents a compact and fully controllable actuation unit for the earlier proposed robot mask, and analyzes the relationship between the displacement of the specifically selected areas of the face and the actuation by control unit. It also present a bioelectrical signal based real time signal processing system to determine the requirement for an artificial expression.

The face is such a salient feature of a person that it plays a crucial role in physical, psychological, and emotional makeup. Hence facial paralysis, which is the loss of voluntary muscle movement of one or both sides of the face, apart from making physical disturbances, can also be an alarming and depressing event in onepsilas life. Although therapeutically and surgery based treatment methods are available, they can work on temporary paralysis only. Facial uplifts done on permanently paralyzed patients will only attempt to reduce the facial asymmetry of a neutral faces. Despite facial paralysis been fairly common, over the years only a very limited amount of effort has been put in to the development of supporting systems for facial paralyzed patients and it is also virtually zero in the robotics field. This paper discusses a novel method; an expressive mask based on robotics technology while giving strong references to human anatomy. The developed robot mask uses artificial muscles based on soft actuators as they can closely model natural muscles while producing silent actuations. With 6 artificial muscles, tests were done on one side of the face of a healthy human. We also explain the suitability of proposed artificial muscles through preliminary experiments and conclude by underlining its suitability as a worthy candidate.

In this study, we propose a synthetic methodology that can enable a humanoid robot to understand the dynamics of objects in a psychological framework. The action selection of the robot is carried out based on a selection probability determined by several internal variables that draws upon the motivation mechanism of human beings. This makes the robot classify its action space by a sensory feedback caused by its own action. The system prioritizes actions that are expected to cause distinguishing sensory patterns. The action selection probability allows the robot to explore unknown spaces for understanding the dynamics of objects in a real environment. In our experiment, we demonstrate the performance of object clustering by multi-modal active sensing using the proposed action selection rule. Furthermore, we show that the system allows the robot to build knowledge about common object movements by repeating interactions with several different types of objects, and also to predict the movement of unknown objects.

In recent years, we can easily access an enormous amount of digital video stream in standardized video format such as MPEG etc. However, it is not so easy to find a desired video stream from a video database in a reasonably short time. Some efficient searching methods in terms of computational cost are definitely required for video stream retrieval. In this paper, we propose a novel method for video stream retrieval using a video stream as search-key. The objective is to find a part of the video stream stored in the database that is similar to the given video stream key. This method extracts some characteristics of frame transitions in video streams, and utilizes the characteristics for the similarity matching. Some experimental results are also shown to verify the efficiency of the proposed method.

In this paper, we introduce a novel method of relevance learning by a multi-layer perceptron. The relevance learning is regarded as learning from the relationship among two or more outputs of the network. The learning network architecture is based on a simple multi-layer perceptron with a modified back-propagation learning algorithm. Unlike the conventional multi-layer perceptron that learns from a set of an input feature vector and the target output, the proposed network can obtain a nonlinear mapping between a set of two or more vector inputs and the desired relevance. For instance, the desired relevance represents the dissimilarity among given objects. We show the performance of the proposed network with some experiments with four artificially generated data set. We then discuss the theoretical and mathematical background underlying the network learning with some related works. We evaluate the obtained arrangement of objects in comparison with the result of principle component analysis (PCA) and multidimensional scaling method (MDS). This work also contributes to the measurement of human subjective evaluation for multidimensional perceptual scaling. Some experimental results on the low-dimensional representation of color hue data set and emotional facial images were presented.

A substantial robotic interface is proposed for collaborative work between humans and machines in a multimodal musical environment. The robotic interface is regarded as a "moving instrument" that displays the reactive motion on a stage while producing sound and music by embedded stereo speakers according to the context of the performance. In this paper, we introduce four musical platforms utilizing robotic technology and information technology in different circumstances. These are effective designing environments for artists such as musicians, composers, and choreographers, not only for music creation but also for media coordination including motion and visual effects. The architecture, called the MIDI network, enables them to control the robot movement as well as to compose music. Each of the developed robotic systems works as a sort of reflector to create an acoustic and visual space with multimodality. The proposed approach to equip musical instruments with an autonomous mobile ability promises a new type of computer music performance.

When designing systems that support remote instruction on physical tasks, one must consider four requirements: 1) participants should be able to use non-verbal expressions, 2) they must be able to take an appropriate body arrangement to see and show gestures, 3) the instructor should be able to monitor operators and objects, 4) they must be able to organize the arrangement of bodies and tools and gestural expression sequentially and interactively. GestureMan was developed to satisfy these four requirements by using a mobile robot that embodies a remote instructors actions. The mobile robot mounts a camera and a remote control laser pointer on it. Based on the experiments with the system we discuss the advantage and disadvantage of the current implementation. Also, some implications to improve the system are described.

No abstract available.

A system which supports remote instruction for physical tasks in the real world should support following requirements: 1) flexibility of the body arrangement; 2) awareness of each other's orientations; and 3) communication of gestural information. To satisfy these requirements, we developed a system named "GestureMan". GestureMan is a mobile robot mounted with three cameras and a remotely controlled laser pointer on it. An instructor controls the robot remotely with a joystick and gives instruction to an operator. Two preliminary experiments using a low-speed Satellite link and high-speed ATM network respectively show that GestureMan has an ability to support the remote instruction. Furthermore, gigabit network enabled the natural interaction between the instructor and the operator by alleviating limitations such as time delay and narrow field of view.