The precision of medical diagnoses based on images is inextricably linked to the quality and clarity of the images. Poor image quality can impede accurate diagnosis and pose challenges for both physicians and machine learning algorithms in interpreting images. By automating the process of fundus image quality assessment during capture, we can ensure that only high-quality images are used, improving diagnostic accuracy. Accordingly, we propose a novel approach for automatically assessing the quality of fundus images using deep learning techniques. Our method incorporates retinal vessel segmentation into RGB images to create four-channel images and then trains a deep learning model on these images to identify the image focus and quality of the images. It can evaluate the quality of fundus images and request image recapture with the adjustment of specific parameters that are classified as poor quality. Our proposed method has the potential to improve the diagnostic accuracy and efficiency of retinal disease diagnosis, particularly in telemedicine settings. By automating the process of fundus image quality assessment, we can ensure that only high-quality images are used for diagnosis, thus improving diagnostic precision. It can serve as an efficient screening tool in the initial stage of acquiring high-quality fundus images.

Eye-writing, the act of drawing letters using eye movements, offers a promising avenue for human-computer interaction, especially when coupled with electrooculogram (EOG) recognition techniques. However, achieving high accuracy in eye-writing recognition using deep learning requires large datasets, which is challenging due to the time-consuming nature of data collection and privacy concerns. In this paper, we introduce a diffusion model capable of producing synthetic EOG signals, demonstrating the feasibility of generating high-quality bio-signal data. This approach not only mitigates the challenges of data collection but also facilitates the improvement of pattern recognition accuracies in small bio-signal datasets.

In digital interfaces, companies often exploit psychological vulnerabilities to increase user engagement, particularly on platforms like YouTube. This study explores these manipulative design tactics, known as dark patterns, and their underexplored effects on attention and psychological well-being. We introduced a feature that issues warnings when certain criteria, like content loads or video views, are met. A user study with ten participants showed that these warnings helped reduce unintended content consumption and time-wasting behaviors, though the impact varied. The research suggests AI-enhanced warnings could offer a more adaptive solution to combat dark patterns.

Wearable electroencephalography (EEG) devices are emerging as crucial tools in human-robot interaction (HRI), enabling intuitive and effective communication between humans and robots. These devices non-invasively measure brain activity, providing real-time insights into a user’s mental state, intentions, and cognitive load. This paper explores the advancements and applications of wearable ear EEG technology in HRI, with a focus on detecting human intent, monitoring emotional and cognitive states, and delivering real-time feedback for adaptive robot behavior. The benefits of wearable EEG over traditional scalp EEG, such as enhanced user comfort, reduced setup time, and improved long-term wearability, are thoroughly examined. Additionally, the paper covers advanced applications, including emotion-aware adaptive interactions, neurofeedback training, and direct robot control via brain-computer interfaces (BCIs). A review of the latest advancements in device miniaturization, integration with other wearable sensors, and applications in virtual reality, gaming, and healthcare is provided. Future directions focus on expanding applications, enhancing human-AI collaboration, and improving accuracy and reliability in HRI. The studies discussed in this paper represent our state-of-the-art research across various fields and applications of wearable EEG systems. This study highlights the potential of wearable Ear-EEG devices to revolutionize human-robot interactions by enhancing customization, responsiveness, and overall efficacy.

Physical inactivity is one of the major health issues in modern society. Music has shown great potential in enhancing exercise performance and motivation, offering an economical and non-invasive way to improve the exercise experience. However, the number of music tracks available to enhance workout performance is limited, and frequent listening to these tracks can lead to fatigue, diminishing their effectiveness. To address this issue, we explored methods for generating personalized workout music. First, we conducted a detailed analysis of the musical and acoustic features of existing workout music to identify which features are most crucial for enhancing exercise performance and motivation. Based on these features, we developed a GANbased music style transfer model that incorporates these highly relevant workout music features, allowing these characteristics to be embedded into music from different genres. Our results show that our new model generates music that has a higher similarity to workout music compared to the baseline model.

Abstract People with color vision deficiency (CVD) have difficulty in distinguishing differences between colors. To compensate for the loss of color contrast experienced by CVD individuals, a lot of image recoloring approaches have been proposed. However, the state-of-the-art methods suffer from the failures of simultaneously enhancing color contrast and preserving naturalness of colors [without reducing the Quality of Vision (QOV)], high computational cost, etc. In this paper, we propose an image recoloring method using deep neural network, whose loss function takes into consideration the naturalness and contrast, and the network is trained in an unsupervised manner. Moreover, Swin transformer layer, which has long-range dependency mechanism, is adopted in the proposed method. At the same time, a dataset, which contains confusing color pairs to CVD individuals, is newly collected in this study. To evaluate the performance of the proposed method, quantitative and subjective experiments have been conducted. The experimental results showed that the proposed method is competitive to the state-of-the-art methods in contrast enhancement and naturalness preservation and has a real-time advantage. The code and model will be made available at https://github.com/Ligeng-c/CVD_swin .

Abstract Color vision deficiency (CVD) is an eye disease caused by genetics that reduces the ability to distinguish colors, affecting approximately 200 million people worldwide. In response, image recoloring approaches have been proposed in existing studies for CVD compensation, and a state-of-the-art recoloring algorithm has even been adapted to offer personalized CVD compensation; however, it is built on a color space that is lacking perceptual uniformity, and its low computation efficiency hinders its usage in daily life by individuals with CVD. In this paper, we propose a fast and personalized degree-adaptive image-recoloring algorithm for CVD compensation that considers naturalness preservation and contrast enhancement. Moreover, we transferred the simulated color gamut of the varying degrees of CVD in RGB color space to CIE L*a*b* color space, which offers perceptual uniformity. To verify the effectiveness of our method, we conducted quantitative and subject evaluation experiments, demonstrating that our method achieved the best scores for contrast enhancement and naturalness preservation.

The positive text reframing (PTR) task which generates a text giving a positive perspective with preserving the sense of the input text, has attracted considerable attention as one of the NLP applications.Due to the significant representation capability of the pre-trained language model (PLM), a beneficial baseline can be easily obtained by just fine-tuning the PLM.However, how to interpret a diversity of contexts to give a positive perspective is still an open problem.Especially, it is more serious when the size of the training data is limited.In this paper, we present a PTR framework, that learns representations where the meaning and style of text are disentangled.The method utilizes pseudopositive reframing datasets which are generated with two augmentation strategies.A simple but effective multi-task learning-based model is applied to fuse the generation capabilities from these datasets.Experimental results on Positive Psychology Frames (PPF) dataset, show that our approach outperforms the baselines, BART by five and T5 by six evaluation metrics.Our source codes and data are available online. DecoderThis is a challenging work.This work is difficult.This work is difficult. Decoder Layer nLayer 2 Layer 1•••

People with metamorphopsia suffer from perceiving things in a distorted way. Various methods for examining metamorphopsia have been suggested in the current literature, with the most advanced techniques demonstrating the ability to yield quantitative measurements. However, these cutting-edge methods necessitate extended examination durations and impose challenging manipulations on patients. In this study, our objective is to enhance the time efficiency of the inspection process and alleviate the burden placed on the user. We propose a novel user-friendly quantitative inspection system which utilizes interactive reinforcement learning. Instead of having users directly operate the system, we ask them to evaluate the stimuli generated by the system. Based on their evaluations, the system gradually refines the deformation map representing the distortion perceived by the user. The reinforcement learning scheme is implemented using relevance feedback approach based on optimum-path forest classifier. To evaluate the effectiveness of the proposed system, subjective evaluation experiments involving simulated and real metamorphopsia participants were conducted in this study. The experimental findings reveal that, when compared to the state-of-the-art method, our proposed system yields comparable inspection out-comes while significantly reducing both the inspection duration and the mental workload.

As society develops, the use of smartwatches is becoming more common, and so too is the demand for more complex functions, including text input. Despite it being known that hand posture affects text input performance, most of the previous research has overlooked the influence of hand posture on smartwatch text input performance. Most of the current input methods have employed language and spatial models to improve text input performance. However, there is still a lack of research on the effect of hand posture on the input method after adding these models. Therefore, this study focused on the effect of the language model and spatial model when different hand postures (index finger support, no finger support, thumb and middle finger support, and thumb finger support) and different movement situations (walking and standing) were used.

Several image recoloring methods have been proposed to compensate for the loss of contrast caused by color vision deficiency (CVD). However, these methods only work for dichromacy (a case in which one of the three types of cone cells loses its function completely), while the majority of CVD is anomalous trichromacy (another case in which one of the three types of cone cells partially loses its function). In this paper, a novel degree-adaptable recoloring algorithm is presented, which recolors images by minimizing an objective function constrained by contrast enhancement and naturalness preservation. To assess the effectiveness of the proposed method, a quantitative evaluation using common metrics and subjective studies involving 14 volunteers with varying degrees of CVD are conducted. The results of the evaluation experiment show that the proposed personalized recoloring method outperforms the state-of-the-art methods, achieving desirable contrast enhancement adapted to different degrees of CVD while preserving naturalness as much as possible.

Metamorphopsia is a symptom caused by abnormalities in the retina, and people with metamorphopsia experience distortions in their field of view. To compensate for this disorder, computer-based distortion assessment and compensation methods have been proposed. In the state-of-the-art method, compensation is carried out by dynamically deforming the image with a manipulation map, which is bound to the symptom of an individual user with metamorphopsia, according to the user's gaze data captured by an eye tracker. However, this method suffers from the instability and inaccuracy of the eye tracker, which leads to a poor compensation effect. In this paper, we propose a novel method for metamorphopsia compensation without using an eye tracker. The proposed method generates a compensation image by simultaneously applying multiple manipulation maps to the input image. To evaluate the effectiveness of the proposed method, a preliminary subjective experiment using a reading task and involving 10 participants with normal vision was conducted. The evaluation results show that the proposed method can compensate for visual distortion during reading tasks.

Feedback latency during the use of interactive displays is an issue currently being considered in the HCI field. Several studies have focused on reducing latency using various approaches. This paper proposes a framework that uses a convolutional neural network to predict user touch points for interactive public displays. The framework predicts user touch events before the finger reaches the display surface to reduce the latency in feedback. As a training dataset, 1,651 tapping actions were collected from 18 participants in front of a display. The training of the convolutional neural network architecture was performed using the collected tapping actions. Validation test results showed that reasonable accuracy could be achieved at 390 ms before touching the display.

Most of the previous studies on smartwatch input methods were conducted in a stable experimental environment (while the user was sitting and using comfortable hand postures), and there have been few studies on the characteristics of input methods using different hand postures and in different body movements. This study evaluated smartwatch performance with different hand postures and in different body movements and compared the result with the ZoomBoard performance. We collected 16 participants' smartwatch input data and analyzed their tapping behavior with four hand postures (index finger support [IS], no finger support, thumb and middle finger support [TMS], and thumb support) and in two different body movements (walking and standing). We studied the input accuracy and speed with different hand postures and the distribution of the touchpoints. The results indicated that IS has the lowest error rate and fastest performance when the user is standing, and TMS when the user is walking. Also, the different hand postures had different touchpoint distributions. Therefore, this paper provides design considerations for future research on human-computer interaction on ultra-small screens, especially smartwatches.

Worldwide, 79.6 million people experience glaucoma, which can cause visual field loss to the individuals. Visual field examination plays an important role in the detection of glaucoma. However, current visual field examination approaches have disadvantages, such as requirements for expensive equipment, long testing time, location restrictions, and more. In the present study, we propose an assessment based on saccadic reaction time (SRT) to overcome the issues in the existing approaches. To confirm the effectiveness of our method, we simulated different stages of glaucoma using a visual field defect simulation system. The result of the visual experiment showed that the discrimination method using SRT can distinguish different symptoms with less testing time.

Homonymous hemianopia (HH) is one kind of visual field defect that the right or left half of the visual fields of both eyes are missing. HH is caused by damage to the neural pathways of the brain and a complete recovery is usually difficult. This study proposes a new information compensation method for HH patients using optical-see-through head mounted display. To avoid causing the occlusion, the proposed technique uses indicators placed at the boundary between the lost and remaining sides of visual field to notify patients about the changes in the lost side. Experiment involving simulated HH participants was conducted to verify the effectiveness of the proposed method and compare with the existing method using overlaid overview window. Objective and subjective evaluation results show that the proposed method can partially alleviate the occlusion problem of overlaid overview window approach and has lower physical and mental demand.

Visual field defects (VFD) can be caused by a variety of conditions. Checking and tracking the progression of VFD is an important part of an eye assessment. Although the use of standard automatic perimetry (SAP) is very popular for VFD diagnosis, it limits the population because of its high requirement for patients. We used a video-watching task as a replacement modality, which precludes the long period of fixation and uses the on-screen gaze to replace the button response. We developed a simulation system to mimic the different types of VFD in people with a normal pattern.We hypothesize that patients with VFD need more eye movement to compensate for the unseen area. We proposed a metric that indicates the gross eye movements toward a specific direction and found a significant difference between the VFD and normal pattern. Furthermore, we found videos that show the unique eye movement pattern in different eye conditions.

Small wearable information devices such as smart-watches are available everywhere. A user can input text in these devices while doing other tasks. However, eyes-free typing, that is, without looking at the input space, on these devices remains difficult to execute. This paper describes our project on developing eyes-free text input systems. Not only an auditory feedback feature was developed for the EdgeWrite text input system for round-face smartwatches, but also a simple template-based gesture recognition engine was implemented. The experimental results show that the developed system maintains reasonable text input speed and error rate for eyes-free under standing and walking conditions.

The number of individuals with Age-related Macular Degeneration (AMD) is rapidly increasing. One of the main symptoms of AMD is "metamorphopsia," or distorted vision, which not only makes it difficult for individuals with AMD to do detailed-oriented tasks but also makes sufferers more vulnerable to certain risks in day-to-day life. Traditional clinical approaches to assess metamorphopsia have lacked mechanisms for quantifying the degree of distortion in space, making it impossible to know exactly how individuals with the condition see things. This paper proposes a new method for quantifying distortion in space and visualizing AMD patients' distorted views via line manipulation. By visualizing the distorted views stemming from metamorphopsia, the method gives doctors and others an intuitive picture of how patients see the world and thereby enables a broad range of options for treatment and support.

Visual field defect (VFD) refers to a symptom in which a patient loses part of his/her field of view (FoV). Medical therapy can halt the progression of VFD, but complete recovery is impossible. In this paper, we propose a computational method for alleviating the restricted FoV with an optical see-through head-mounted display (OST-HMD), where an overview scene captured by the installed camera is overlaid on the persisting FoV. Since the overview window occludes with the real world scene, there is a trade-off between the augmented contextual information and the local unscreened information. We hypothesized that such a trade-off can be resolved by taking into consideration the size of the overview window and its displacement from the center of the unimpaired FoV. We therefore conducted an empirical evaluation through a Whac-A-Mole type of task with ten VFD-imitative subjects, where three sizes of an overview window with a fixed aspect ratio and seven positions in terms of elevation and azimuth were used combinatorially on an OST-HMD to find the best size and position of the overview window. It was statistically proven that for left-sided homonymous VFD-imitative subjects, the performance of the task was better when the medium-sized overview window was placed in the lower right position. The obtained result can legitimate default settings for the proposed VFD alleviation method.

An origami tessellation display is an origami-based input/output device that has a flexible and unique deformable surface. This study achieves shape estimation and user gesture recognition for an origami-based surface without the use of external devices. Six tri-axis accelerometers are employed and embedded inside the surface, and shape estimation and gesture recognition models are constructed by means of neural networks and a random forest. Despite the small number of sensors and simple machine learning approaches, the models demonstrate sufficient estimation and recognition ability. In this paper, a framework in which shape estimation and gesture recognition are performed in real time is also presented.

More than 100 million individuals around the world suffer from color vision deficiency (CVD). Image recoloring algorithms have been proposed to compensate for CVD. This study has proposed a new recoloring algorithm to make up shortages of contrast enhancement and naturalness preservation of the state-of-the-art methods. The recoloring task is formulated as an optimization problem that is solved by using the colors in a simulated CVD color space to maximize contrast and to preserve the original color as much as possible. In addition, the dominant colors are extracted for recoloring. They are then propagated to the whole image so that the optimization problem could be solved at a reasonable cost independent of the image size. In the quantitative evaluation, the results of the proposed method are competitive with those of the best existing method. The evaluation involving subjects with CVD demonstrates that the proposed method outperforms the state-of-the-art method in preserving both the information and the naturalness of images.

Color vision deficiency (CVD) is caused by anomalies in the cone cells of the human retina. It affects approximately 200 million individuals throughout the world. Although previous studies have proposed compensation methods, contrast and naturalness preservation have not been adequately and simultaneously addressed in the state-of-the-art studies. This paper focuses on red–green dichromats’ compensation and proposes a recoloring algorithm that combines contrast enhancement and naturalness preservation in a unified optimization model. In this implementation, representative color extraction and edit propagation methods are introduced to maintain global and local information in the recolored image. The quantitative evaluation results showed that the proposed method is competitive with state-of-the-art methods. A subjective experiment was also conducted and the evaluation results revealed that the proposed method obtained the best scores in preserving both naturalness and information for individuals with severe red–green CVD.

This paper investigates smartwatch touch behaviors of users with different hand postures by conducting a laboratory experiment with 12 participants and analyzing their tap events on a smartwatch with four postures: the no-support (NS) posture, thumb support (TS) posture, index-finger-support (IS) posture, and thumb-and-middle-finger-support (TMS) posture. The participants' movement time and the error rate of the tapping tasks were measured for each posture and for two target positions (edge and center) and three target sizes (large, medium, and small). The results indicate much faster response times for the small, center targets for the TS and TMS postures compared with the NS posture. Based on this result, this paper proposes five design guidelines for arranging touch targets on a smartwatch touchscreen based on four postures.

This paper introduces a smartphone-based system that encourages photography by enhancing a user's awareness of the surrounding environment during a stroll in a city. When a user approaches a pre-registered location for photography, the system notifies the user by vibrating the smartphone. The system only displays the distance to the location and a photograph of the location on the smartphone screen; it does not provide detailed information, such as that given by a map. A user study performed during a stroll in a city confirmed the application of this system. The results demonstrate that ambiguous information provided by the system increased the opportunity for photography and fortuitous discovery.

It is our great pleasure to present this special issue on human–computer interaction (HCI) in the Asia-Pacific Region in the International Journal of Human–Computer Interaction (IJHCI). This is a c...

This paper presents our project of designing Edge Write text entry methods for Japanese language. We are developing two versions of Edge Write text entry method for Japanese language: Roman-Kana Edge Write and Katakana Edge Write. Roman-Kana Edge Write is a straightforward application of the original Edge Write to Japanese phonograms. In Roman-Kana Edge Write, each Japanese phonogram is represented as a combination of a consonant and vowel, each of which is specified by an English character in the original Edge Write. Katakana Edge Write specifies the line stroke directions and writing order of the Japanese Katakana character. The ideal corner sequence pattern of Edge Write for each Katakana character is designed based on its line stroke directions and writing order.

In this paper, we present an approach for sensing object shapes and touching of interactive tables based on diffuse illumination (DI) and frustrated total internal reflection (FTIR) and other infrared-based tracking approaches. Our approach uses tangible objects with series connection of optical fibers as bend and touch sensors. The approach enables detection of object deformations and touched points on the object with no modification of the original tabletop-based tracking system. We present the basic concept and an initial evaluation of the approach.

We introduce Trapco, a website prototyping system developed to demonstrate a transformation-based prototyping method. Trapco enables designers to transform a paper-based prototype of website into a computer-based prototype. It supports typical design activities of creating design ideas in a paper-based format, comparing them, and transforming them into a computer-based format to test their interactivity. The transformation-based design method allows designers to discuss usability issues of paper prototypes in initial design concepts. In this paper, we discuss the basic concept of Trapco and the evaluation result of the Trapco system, which is implemented to demonstrate the basic concept.

We report a case study of designing a mobile phone of the future, involving participatory requirements elicitation using a form of scenario-based design. Participants took photographs and wrote essays that illustrate their personal interests and perspectives on given themes. They then analyzed the photos and essays. They created scenarios and posed questions about the scenarios to envision contexts of future use. The participants produced novel design concepts and provided design insights even though they had no design training.

Eyesight and speech are two channels that humans naturally use to communicate with each other. However both the eye tracking and the speech recognition technique existing are still far from perfect. This work explored how to integrate two (or more) error-prone sources of information on users' selection of objects in a visual interface. The implemented system integrated a commercial speech recognition system with gaze tracking in order to improve recognition results. In addition, we employed a new measure of the rate of mutual disambiguation for the multimodal system and conducted an experimental evaluation.

The recognition ambiguity of a recognition-based user interface is inevitable. Multimodal architecture should be an effective means to reduce the ambiguity, and contribute to error avoidance and recovery, compared with a unimodal one. But does the multimodal architecture always perform better than the unimode at any time? If not, when does it perform better than unimode, and when is it the optimum? Furthermore, how can modalities best be combined to gain the advantage of synergy? Little is known about these issues in the literature available. In this paper we try to give the answer through analyzing integration strategies for gaze and speech modalities, together with an evaluation experiment verifying these analyses. The approach involves studying the mutual correction cases and investigating when the mutual correction phenomena will occur. The goal of this study is to gain insights into integration strategies, and develop an optimum system to make error-prone recognition technologies perform at a more stable and robust level within a multimodal architecture.

This work explores how to use the gaze and the speech command simultaneously to select an object on the screen. Multimodal systems have long been a key mean to reduce the recognition errors of individual components. But the multimodal system generates errors as well. This present study tries to classify the multimodal errors, analyze the reasons causing these errors, and propose the solutions for eliminating them. The goal of this study is to gain insight into multimodal integration errors, and to develop an error self-recoverable multimodal architecture so as to make the error-prone recognition technologies perform at a more stable and robust level within multimodal architecture.

In this paper we propose "Augmented Reality (AR) Clipboard" with "twist-information presentation method" based on the angle difference between user's head and body. The AR Clipboard is a computer-generated information clipboard that allows a user to keep virtual objects on it on a video see-through head mounted display (HMD). The twist-information presentation method provides a user with wide view of a HMD by presenting a virtual object in the peripheral space of a user's view, which is generated by the angle difference between the user's head and body. Using AR Clipboards, a user can carry virtual objects on them around his/her body and can look at one of them as long as the user turns his/her face toward a clipboard. Moreover, a user can grasp a virtual object and post it on one of the clipboards by performing hand postures.

In this paper, the authors propose a decomposition method for a formal specification that divides the specification into two subspecifications composed by a parallel operator. To make these specification behaviors equivalent before and after decomposition, the method automatically synthesizes an additional control specification, which contains the synchronization information of the decomposed subspecifications. The authors prove that a parallel composition of the decomposed subspecifications synchronized with the control specification is strongly equivalent with the original (monolithic) specification. The authors also write formal specifications of the OSI application layer's association-control service and decompose it using their method as an example of decomposition of a practical specification. Their decomposition method can be applied to top-down system development based on stepwise refinement.

The problem of modifying an existing protocol specification of a communicating system in Labelled Transition System (LTS) based on changes of it's service specification in LTS is discussed. Two types of changes of a service specification are treated: addition of a behaviour expression to the service specification and deletion of some parts of the behaviour expression from the service specification. For each change, a modification method is proposed and a property of the method is shown. From the point of view of verification and validation of a specification, it is desirable to separate a changed part of the specification from the existing specification. Thus, especially for the former type of changes that is addition of a behaviour expression to the service specification, the method to express changes of the service specification by using a concurrent LTS is proposed. The method proposed in this paper is efficient and desirable when the size of the changed part of the service specification is small in terms of behaviour expressions.

An algorithm for transforming LOTOS specifications from monolithic style to constraint-oriented style is proposed. The objective of this transformation is to decompose monolithic LOTOS behavior into two processes composed by the parallel operator. In order to make the behaviors observed congruent before and after the decomposition, the algorithm creates an additional process from the given specification and adds to the decomposed processes using a parallel operator. The additional process consists of n interleaved subprocesses (n /spl ges/ 1). There are no internal interaction gates between decomposed processes and the added process. The interactions among them occur only at the visible gates. This transformation has the following applications: (1) modularization or decomposition of a specification, (2) restructuring of complex specifications, and (3) derivation of protocol specifications from service specifications.>