The importance of cyber-physical-social systems (CPSS) lies in their key role in bridging the gap between the physical, cyber, and social worlds. Integrating topological spaces into CPSS can provide relational structures derived from observations and measurements in the real environment, thereby reducing the risk of accidents and failures when robots operate in social environments. The topological maps of the topological space can be gradually learned by Growing Neural Gas (GNG) from real data, making it suitable for online and real-time applications. However, the training process of GNG encounters difficulties in handling high-density data, leading to significant computational overhead. To address this limitation, this paper proposes a new method to represent the topological space as a multi-layer topological map. This method enables the extraction of information related to different tasks from different topological maps. To enable GNG to effectively learn multiple topologies with different densities, a two-phase approach is proposed: a top-down phase for building a tree structure to accelerate nearest neighbor search, and a bottom-up phase for refining the topology of the upper layer. The experimental results show that it is possible to learn a topology that aligns with the data, overcoming the challenges of conventional methods, and achieving improvements in efficiency and reductions in computational cost, effectively integrating topological space into CPSS.

Extended Reality, artificial intelligence, and big data technologies offer new opportunities for advancing rehabilitation diagnosis and training. This study presents a method for extracting behavioral features from a visual search task conducted in an immersive virtual reality environment. To identify motion patterns specific to individual patients, we employ a topological mapping approach based on Growing Neural Gas (GNG), which adapts its structure dynamically using node activation and error-based edge management. While GNG effectively captures spatial characteristics, it lacks the ability to model temporal relationships and is sensitive to hyperparameter settings. To address these limitations, we introduce a spatiotemporal topological clustering method, along with a hierarchical framework that enables segmentation at multiple levels of granularity. Furthermore, to evaluate feasibility, we conducted a visual search task with three patients, including one with USN, and performed a comparative analysis of their extracted motion features.

Event cameras offer superior performance in terms of temporal resolution, dynamic range, latency, and power efficiency compared to conventional cameras. However, due to their unique operational principles, which involve measuring changes in luminance values, tasks such as human pose estimation become more challenging. In this study, we propose a real-time feature extraction method from event data for human pose estimation using event cameras. Specifically, after adjusting resolutions with Multi-resolution Maps, we employ Growing Neural Gas for topological clustering. Subsequently, spectral clustering is performed based on the resulting graph structure, enabling the segmentation of human body parts with a focus on the graph structure.

|Unilateral Spatial Neglect (USN) is characterized by an attention deficit to one side of space, where individuals struggle to perceive stimuli on that side without a lack of sensation. Traditional paper-pencil tasks like line cancellation and copying tests are commonly used to assess USN, but they have limitations in evaluating neglect areas confined to a two-dimensional plane. Immersive VR systems and multimodal sensing systems offer a more sensitive approach for diagnosis and training. In related works, AR/VR systems and eye-tracking devices are utilized for measuring, evaluating, and creating assessment tasks for USN. However, these approaches can only analyze relationships between perception and movements in specific environments. In this study, we propose a method for cognitive modeling based on perceiving-acting cycle in USN, utilizing computational intelligence techniques to establish a structured coupling framework, aiming to contribute to a novel and effective approach for understanding and addressing USN.

This paper presents a method for human motion segmentation aimed at motion analysis in healthcare and rehabilitation. Motion segmentation involves extracting small movements, known as motion primitives, from a sequence of behavioral patterns. While previous works have utilized unsupervised clustering methods as effective approaches for motion segmentation, many of these methods require prior knowledge to enhance performance. To overcome these challenges, we propose a hierarchical topological clustering method capable of representing spatiotemporal features using GNG and the Pulse Neuron Model. Additionally, we present experiments and discussions to validate the applicability of the proposed method for motion analysis in exercise.

In recent years, exercise recognition has become increasingly popular for exercise monitoring and rehabilitation for older adults. However, to identify non-learned exercises, another dataset needs to be collected for differentiation purposes. This study aims to provide an add-on technique to the encoder model to distinguish between learned and non-learned exercises without training with non-learned exercise data. First, we form a list of activation nodes based on the output of the encoder by using the k-means algorithm. Afterward, these nodes are used to compute activation scores from encoded features. These activation scores are used to differentiate non-learned exercises by a threshold value. After differentiation, the activation scores are then passed to a multi-layer perceptron (MLP) for exercise classification. Meanwhile, we proposed a unique method to compute a distinguishing score to find the optimal$k-\mathbf{nodes}$. We demonstrate the proposed method using the MM-Fit dataset, showing that it can identify fitness exercises and distinguish non-learned exercises without much performance loss. Lastly, we collected a dataset about Chair-Fitness activity to validate the proposed method's effectiveness further and welcome other researchers to utilize the dataset.

To realize a super-smart society (Society 5.0) where humans and robots coexist, there is a need for a perceptual system that can recognize unknown objects in various unknown environments quickly and flexibly. In unknown environments, the characteristics of objects cannot be known in advance, so prior learning-based recognition methods such as deep reinforcement learning cannot fully cover the problem. There have been many studies on environment recognition (clustering, etc.) using a combination of RGB images and distance images, but the recognition performance is unstable because it strongly depends on the lighting conditions of the environment. Therefore, in this study, we construct a 3D topological map of the environment in real-time using Growing Neural Gas (GNG), which can learn 3D topological structures even for unlearned objects, using only 3D point cloud data as input. In the real world, due to the characteristics of RGB-D cameras, sample density decreases for more far-away objects and only sparse depth information can be obtained, so conventional GNG cannot generate high-density topological structures of unknown objects. Therefore, if the object category labels of the winner nodes (nearest nodes) for the input vector (3D point cloud) match the unknown object and are within a predefined tolerance area, then it is judged to be useful input information for learning the topological structure of the unknown object, and the topological structure of the unknown object is learned. We propose Add-if-Silent rule-based GNG (AiS-GNG) which can generate high-density topological structures for far-away objects by directly adding input data as a reference vector. We verify the effectiveness of the proposed method through experiments using a 3D dynamics simulator.

We aim to develop a simple motion capture system for home environments. As users need to install their own cameras, a calibration-free system is required. Therefore, we propose a 3D pose estimation method based on 3D joint angles of humans using multiple smart devices with a hybrid optimization method that combines Particle Swarm Optimization and steepest descent method. We also estimate the relative angles between humans and cameras to facilitate camera calibration. In this paper, we discuss the impact of the combination of global and local search capabilities of the optimization method on the system's performance. Specifically, we propose an optimization method that gradually changes the number of iterations of Particle Swarm Optimization and Steepest Descent Method and compare it with a simple sequential combination.

Theaim of this study is to model the cognitive processes based on a perceiving-acting cycle in patients with unilateral spatial neglect (USN). USN is the inability to perceive features of the environment, body, or objects on one side. To extract the cognitive characteristics of USN patients in a multifaceted manner, we constructed a multimodal sensing system using immersive VR. In this paper, we present a system that predicts movement of a subject while performing a search task using the measurement results and an LSTM neural network.

In order to realize a super-smart society (Society 5.0) where humans and robots coexist, there is a need for a perceptual system that can recognize the environment quickly and flexibly in an environment that changes from moment to moment. In an unknown environment, the characteristics of objects cannot be known in advance, and thus prior learning-based recognition methods such as deep reinforcement learning may not be able to cope with this situation. In this study, we construct a 3D topological map of the environment in real-time using Growing Neural Gas (GNG), which can learn 3D topological structures even for unlearned objects. However conventional GNG have the problem that they cannot generate nodes with high-density for distant objects and cannot identify whether an unknown object is static or dynamic. Therefore, by directly adding useful input data as a new node (reference vector) based on the object category labels of the winner nodes (nearest nodes) to the input vector (3D point cloud), it is possible to generate high-density topological structures even for distant objects. We proposed the Add-if-Silent rule-based GNG with Amount of Movement (AiS-GNG-AM), which can identify between static and dynamic objects based on the past amount of movement of a node. The effectiveness of the proposed method is verified through experiments using a 3D dynamics simulator.

Gesture learning and recognition are essential challenges for developing human friendly robots. Classification methods and machine learning technics have been used to classify gestures and produce motions for robots. However, human behaviors generally differ according to cultures, characteristics of the region, and personality traits of individuals. Thus, the capability of learning in unsupervised manner is required for social robots to adaptively acquire the skills. In this study, we use growing neural gas (GNG) algorithm for the clustering of primitive motion patterns on gesture trajectory. Moreover, we propose a hierarchical learning architecture for decomposing imitative motions and reconstructing gesture movements. Furthermore, we show an experimental example to discuss the effectiveness and applicability of the proposed method.

In this study, we aim to develop a system of remote-controlled avatar robot for elderly and disabled patients. Most of teleoperation systems have interfaces to visually present the state of the robots including feedback information and receive the control commands manually sent from the operator. However, in elderly and patients with disabilities, they might have difficulty in the manual control of the robot. This paper therefore presents a multimodal interface for remotely controlling a robotic avatar. We furthermore propose a cognitive platform for remotely controlling a robot based on a concept of perceiving-acting cycle. The platform consists of a perceptual system for incremental environment modeling and an action system for extracting patterns of operator behavior. In each system, a self-organized neural network based on unsupervised learning is used. Moreover, we use a spiking neural network for spatial-temporal modeling of interaction between an operator and environment.

In this study, we aim to develop a measurement system for evaluating walking ability in daily life. Health promotion is one of the most important tasks to improve quality of life and quality of community for elderly people. Disabilities related to loss of independence in performing activities of daily living can lead to their social isolation and loneliness that can induce immobility and depression, producing the vicious cycle. Various methods have been proposed to measure lower limb joint angles and positions by using wearable systems and motion capture systems. However, such systems are too expensive and big for elderly's daily self-monitoring. This paper presents a method of lower limb joint angle estimation using a Kinect sensor. The sensor has a built-in processor to detect joint positions. However, inverse kinematics problem is required to be addressed in order to derive the joint angles. We therefore propose a hybrid approach for lower limb joint angle estimation using genetic algorithm and feed forward neural network.

In this study, we aim to develop an educational environment and platform based on color science, merging information technology, robotics and artificial intelligence. In the educational environment, the robot requires functional capability for human-like communication. However, the robot may have difficulty in grasping the small cues to represent feelings and emotional changes of human by itself. This paper presents a measurement system with pneumatic pressure sensor for heart rate variability analysis. We proposed a fuzzy spiking neural network to extract heartbeat signal from the measured data. We furthermore conducted experiments to investigate the effects on student's psychological states while solving some conundrums by using the measurement system.

This paper presents a multi-modal sensing system for USN assessment with eye tracker, 3D image sensor and tablet PC. First, we introduce the multi-modal sensing system based on the concept of computational system rehabilitation. Next, we propose a computational approach to extract behavioral and perceptional features of USN patients from the heterogeneous data. Furthermore, we show an experimental example to discuss the effectiveness and applicability to the feature extraction.

Aging society in Japan can be a big serious problem. However, the number of caregivers is currently not enough, and it is not expected to sufficiently increase in future. Elderly care has been shifting from hospital care to community-based care and home care, but this can lead to raise the burden on their family members. Therefore, elderly people should take care of their heath in daily life in order to prevent mental and physical depression. This paper presents a daily exercise support system with a robot partner utilized as an exercise instructor. We discuss the robot human-robot interaction in terms of framing effect. The framing effect is an example of cognitive bias that influences someone's choice depending on whether it is presented as a positive thing or as a negative thing. Here, we implemented verbal communication contents base on positive frame or negative frame. Furthermore, we conducted a demonstration experiment to examine the effect of each expression on elderlies' motivation.

Recently, the number of elderly people living alone is increasing and has become a serious problem in Japan. On the other hand, the stability of both social rhythm and biological rhythm is very important for extension of healthy life expectancy. It is difficult for elderly people to understand the current stability of social rhythm and biological rhythm in daily life. Therefore, we propose several visualization and management systems based on daily life monitoring. First, we explain a measurement method of human daily life logs. Next, we show a social rhythm management support system based on three different types of classification methods, such as (1) ICF (International Classification of Functioning, Disability and Health), (2) NHK (Nippon Hoso Kyokai) Activity classification, and (3) Social Rhythm Metric (SRM). Finally, we discuss the effectiveness of the proposed methods and future works.

Human-like-motion performed by robots can have a contribution to exert a strong influence on human-robot interaction, because bodily expressions convey important and effective information. If the robots could adapt the features of human behavior to their motions and skills, the communication would become more smooth and natural. In this paper, we develop a posture measurement system for a robot imitation using a 3D image sensor. This paper proposes a method of robot posture generation based on a steady-state genetic algorithm (SSGA). SSGA is one of evolutionary optimization methods using selection, mutation, and crossover operators. Since SSGA is a simplified model, it is easy to implement into a real-time processing. Furthermore, we apply a continuous model of generation for an adaptive search in dynamical environment.

In this study, we aim to develop a system for improving daily lives of elderly people to ensure health. In order to realize an enriched life style among elderly people, daily health care is important. Therefore, we have proposed a system where robot partners will assist in exercising activity among elderly people. This paper proposes a method of relational trust modeling based on reinforcement learning. We apply a concept of relational trust defined as the expectation that a person is disposed to act in a trustworthy manner toward "me," no matter what the person does to others. In the experiment, we discuss the effectiveness of relational trust for robot communication.

In order to keep healthy health of elderly people, daily exercise is an important factor. Therefore, we have developed an exercise support system with robot partner to provide the daily exercise program. Furthermore, Human-like-motion can have a contribution to exert a strong influence on the person through the human-robot interaction, because bodily expressions convey important and effective information. If robots could adapt the features of human behavior to their motions and skills, the communication would become more smooth and natural. In this paper, we propose a learning structure for imitation learning.

In this research, we propose a methodology for getting joint angles by Kinect sensor for rehabilitation evaluation support. We measure the motion of the arm of a patient with hemiplegia before and after the rehabilitation, and estimate the range of the motion by using genetic algorithm and neural network. The range after the rehabilitation is bigger than before the rehabilitation. Based on this result, our methodology is able to evaluate the change of the motion before and after the rehabilitation for patients with hemiplegia.

This paper deals with the information extraction of daily life log measured by smart phone sensors. Two types of neural computing are applied for estimating the human activities based on the time series of the measured data. Acceleration, angular velocity, and movement distance are measured by the smart phone sensors and stored as the entries of the daily life log together with the activity information and timestamp. First, growing neural gas performs clustering on the data. Then, spiking neural network is applied to estimate the activity. Experiments are performed for verifying the effectiveness of the proposed method.

This paper proposes a human localization method in informationally structured space based on sensor network First, we explain informationally structured space, robot partners, and sensor networks developed in this study. Next, we apply a fuzzy spiking neural network to extract a person from the measured data by the sensor network. Furthermore, we propose a learning method of fuzzy spiking neural network based on the time series of measured data. Finally, we discuss the effectiveness of the proposed methods through experimental results in a living room.

This paper proposes a measurement method of human position based on sensor network First, we explain informationally structured space, robot partners, and sensor networks developed in this study. Next, we discuss the applicability of the sensor network and robot partners for human interaction. Next, we apply a steady-state genetic algorithm using template matching to extract a person in 3D distance image based on differential extraction. Finally, we discuss the effectiveness of the proposed methods through several experimental results.

This paper proposes an intelligent monitoring system based on emotional model in sensor networks. Emotional models are very useful to understand human behaviors. First, we explain the recent works on emotional models, and discuss the applicability of emotional models to real world problems. Next, we propose an emotional model composed of emotion, feeling, and mood. These three components are coupling, but their time scales are different. Next, we explain perceptual system based on image processing and the monitoring system based on the proposed emotional model. Finally, we discuss the effectiveness of the proposed method through several experimental results.