Esports have highlighted both their potential for social inclusion and the accessibility challenges faced by individuals with physical disabilities. This study introduces a novel paradigm for inclusive esports by shifting game control from traditional kinematic inputs to kinetic inputs. An EMG-based control interface enables gameplay through force regulation (e.g., muscle activation and inhibition). The aim is to explore how this interface enables common gameplay mechanics among players with and without physical disabilities. User Study 1 involved 20 able-bodied participants performing competitive esports tasks to examine how EMG-based control accuracy is influenced by movement range, such as wrist and elbow motion. User Study 2 extended the investigation to eight participants with physical disabilities to compare control accuracy between disabled and able-bodied users. The findings suggest that the interface enables common gameplay mechanics for individuals who can separately control activation and inhibition of each muscle corresponding to each EMG sensor via calibration adjustment but disability-related involuntary muscle activity and unintended co-contraction remains a major challenge for the interface.

In crowdsourced accessibility mapping, detailed descriptions of accessibility issues in collected images help determine navigability for people with disabilities and assist facility managers in identifying problems.To explore the potential of large language models (LLMs) in generating accessibility descriptions, we conducted two studies.In Study 1, we examined the effect of role assignment on LLM outputs.Although a small number of descriptions included incorrect information, each model produced outputs that reflected the assigned roles of a wheelchair or cane user.In Study 2, we investigated how inter-LLM dialogue influences description quality.The results suggest that dialogue encourages the use of numerical expressions and reduces ambiguity, but does not help eliminate incorrect content.These findings highlight both new opportunities and challenges in leveraging LLMs for automating accessibility assessments.

Praising behavior is an important method of communication. An existing study constructed models to predict praising skill, which indicates the degree to which the praise is done well, by using only unimodal behavior such as speech audio or visual behavior of a praiser who gives praise in dyad interactions. To improve prediction performance, a model should be constructed that uses various additional information. In this study, we propose two approaches to predict praising skill highly accurately. The first uses trimodal (multimodal) behaviors extracted from visual, acoustic, and linguistic modalities. The second uses the behaviors of the receiver of praise since the reaction of the receiver should differ depending on how good the praise is. For this study, we collect trimodal features and the degree of praising skill in each praising scene in a dialogue. We construct multiple models to predict the degree of praising skills using various combinations of the trimodal features from the praiser and receiver. The experimental results show that the model that predicts praising skill most accurately uses multiple features related to both verbal and nonverbal behaviors of the praiser and receiver. Therefore, the two approaches of using trimodal behaviors and using features from both the receiver and praiser are effective for predicting praising skills in dyad interactions.

Esports strengthen connections with society for people with disabilities. However, due to the competitive nature of the gaming industry, many games are not sufficiently playable for individuals with upper limb disabilities. Additionally, unlike non-esports games, esports involves competition through gaming, which means that unconditional assistance for players with disabilities can evoke a sense of unfairness among able-bodied players. To address this problem, we propose an esports support method for players with upper limb disabilities that provides skill assistance based on pre-training. The proposed method is designed so that the assistance is triggered by user actions, and the degree of assistance reflects the user’s prior training. This approach aims to reduce the perceived unfairness of assistance while preserving a sense of agency. Interview survey results suggest that appropriate assistance levels can maintain this sense of agency. However, achieving complete fairness remains challenging.

During dialogues, speakers need to be able to predict whether their partners understand their message. This is important for not only for human-to-human interaction but also human-to-agent interaction. We consider that if the listener's comprehension level can be automatically predicted, interactive agents will be able to communicate appropriately according to the user's comprehension level. However, to the best of our knowledge, there is no case study that reveals how comprehension can be predicted based on multimodal information about the listener. In this study, we attempt to predict comprehension levels on the basis of the listener's multimodal information. First, we construct a dialogue corpus consisting of the listener's comprehension levels and the listener's multimodal information. Next, we construct machine learning models that predict the listener's comprehension levels on the basis of the listener's multimodal information. Our results suggest that our model was able to predict a listener's comprehension level on the basis of a listener's multimodal information. In addition, two movements, the lifting of the cheeks and the pulling up of the corners of the lips, were suggested to be important in assessing the listener's level of comprehension.

The listener's backchannels are an important part of dialogues. With appropriate backchannels, people are able to smoothly promote dialogues. Thus, backchannels are considered to be important in dialogues between not only humans but also humans and agents. Progress has been made in studying dialogue agents that perform natural affable dialogue. However, we have not clarified whether the listener's various backchannel types are predictable using the speaker's multimodal information. In this paper, we attempt to predict a listener's various backchannel types on the basis of the speaker's multimodal information in dialogues. First, we construct a dialogue corpus that consists of multimodal information of a speaker's utterances and a listener's backchannels. Second, we construct machine learning models to predict a listener's various backchannel types on the basis of a speaker's multimodal information. Our results suggest that our model was able to predict a listener's various backchannel types on the basis of a speaker's multimodal information.

Praising behavior is considered to an important method of communication in daily life and social activities. An engineering analysis of praising behavior is therefore valuable. However, a dialogue corpus for this analysis has not yet been developed. Therefore, we develop corpuses for face-to-face and remote two-party dialogues with ratings of praising skills. The corpuses enable us to clarify how to use verbal and nonverbal behaviors for successfully praise. In this paper, we analyze the differences between the face-to-face and remote corpuses, in particular the expressions in adjudged praising scenes in both corpuses, and also evaluated praising skills. We also compare differences in head motion, gaze behavior, facial expression in high-rated praising scenes in both corpuses. The results showed that the distribution of praising scores was similar in face-to-face and remote dialogues, although the ratio of the number of praising scenes to the number of utterances was different. In addition, we confirmed differences in praising behavior in face-to-face and remote dialogues.

In this work, as a first attempt to analyze the relationship between praising skills and human behavior in dialogue, we focus on head and face behavior. We create a new dialogue corpus including face and head behavior information of persons who give praise (praiser) and receive praise (receiver) and the degree of success of praising (praising score). We also create a machine learning model that uses features related to head and face behavior to estimate praising score, clarify which features of the praiser and receiver are important in estimating praising score. The analysis results showed that features of the praiser and receiver are important in estimating praising score and that features related to utterance, head, gaze, and chin were important. The analysis of the features of high importance revealed that the praiser and receiver should face each other without turning their heads to the left or right, and the longer the praiser's utterance, the more successful the praising.

Existing virtual reality (VR) based wheelchair simulators have difficulty providing both visual and motion feedback at low cost. To address this issue, we propose a VR-based wheelchair simulator using a combination of motions attainable by an electric-powered wheelchair and vection-inducing movies displayed on a head-mounted display. This approach enables the user to have a richer simulation experience, because the scenes of the movie change as if the wheelchair performs motions that are not actually performable. We developed a proof of concept using only consumer products and conducted evaluation tasks, confirming that our approach can provide a richer experience for barrier simulations.

We propose a VR-based wheelchair simulator using a combination of vection-inducing movies displayed on a head-mounted display and motions performable by an electric-powered wheelchair. The scenes of the movie change as if the wheelchair performs motions that are not actually performable. We developed a prototype system and conducted an evaluation task, confirming that our simulator can provide a richer experience for barrier simulations.

This paper proposes a new extrinsic calibration of kaleidoscopic imaging system by estimating normals and distances of the mirrors. The problem to be solved in this paper is a simultaneous estimation of all mirror parameters consistent throughout multiple reflections. Unlike conventional methods utilizing a pair of direct and mirrored images of a reference 3D object to estimate the parameters on a per-mirror basis, our method renders the simultaneous estimation problem into solving a linear set of equations. The key contribution of this paper is to introduce a linear estimation of multiple mirror parameters from kaleidoscopic 2D projections of a single 3D point of unknown geometry. Evaluations with synthesized and real images demonstrate the performance of the proposed algorithm in comparison with conventional methods.

Many studies have addressed the affective robot, a robot that can express emotion, in the field of human-robot interaction. Really useful applications, however, can only be designed if the effect of such expressions on the user are completely elucidated. In this paper, we propose a new useful application scenario for the affective robot that shares the user's experience and describe an experiment in which the user's experience is altered by the presence of the affective robot. As the stimulus, we use movie scenes to evoke 4 types of emotion: excitement, fright, depression, and relaxation. Twenty four participants watch different movies under three conditions: no robot present, with robot that offers appropriate emotional expression, and with robot that has random emotional expression. The results show that the participants watching with the appropriate emotion robot experienced stronger emotion with exciting and relaxing movies and weaker emotion with scary movies than is true without the robot. These changes in the viewer's experience did not occur when watching with the random emotion robot. From the results, we extract design points of affective robot behavior for enhancing user experience. This research is novel in terms of examining the impact of robot emotion, seen as appropriate by the viewer, on the viewer's experience.

We introduce a Japanese Kana-character entry method on touch screens for visually impaired people. Our proposal, "Move&Flick," allows the user to move a single finger in any of eight directions twice without lifting the finger on the screen to select a character; lifting the finger decides the character. The method uses radial areas with dead zones to detect each of the eight movement directions. An experiment shows that the dead zones and voice feedback let the user acquire proper finger directions in an easy learning process. The method also has an algorithm to correctly detect the change point from the first movement direction to the second movement direction. We evaluate Move&Flick in an experiment with visually impaired subjects and confirm that Move&Flick works well.

We propose shared emotional experience agents. They enhance the user's emotional experience by emotional contagion. Our experiment has 12 participants watch videos together with a robot that expresses an emotional state by body and voice. The results suggest that the affective robot will make user more excited and relaxed and make user less depressed and afraid than they view it alone.

We present a method of document area identification that utilizes consecutive characters in the non-reading direction as search keys. We use this method to develop a prototype system called Kappan. It enables service providers and users to create hyperlinks in books without markers. Existing techniques generally require markers to be printed on the page if a hyperlink is to be created. We consider that utilizing the concept of the search index makes markers unnecessary. Kappan associates indexed text areas in a large number of books with supporting digital contents. The indexed text areas, freely defined by service providers or users, are identified by subjecting images of small areas of the printed page to OCR (Optical Character Recognition) and extracting from the text so recognized highly specific and efficient search keys. Traditional text indexing methods must extract long character sequences from the partial image in order to identify the area exactly given the sheer number of book pages. However, considering that the average OCR error rate is more than 20 percent if the partial image is captured by a camera-equipped cellular phone, it is highly probable that many characters would be misrecognized and area identification would thus fail. In contrast, our indexing method can extract area-specific clues using fewer characters that can identify the area exactly even when the partial image is small and the extracted text contains misrecognized characters. An experiment proves that our method can identify the exact area from more than one million areas with the high accuracy rates of 99 percent and 96 percent for OCR error rates of 0 percent and 22 percent, respectively.

Currently, there are many occasions to communicate with each other in the collaborative virtual environment. However, it is more difficult to keep high-motivation for participating in communication in such virtual environment than in face-to-face environment. It is considered that the reasons are 1) the difficulty of understanding how their own motivations are and 2) the boring expression due to the fixed view point. To address this issue, we display electroencephalogram information to a virtual space by forward-backward model - zoom in/out of whole virtual space. According to the results of experiment, our system successfully provides an environment in which people can keep high-motivations for participating in communication.