As the space of causal sequence modeling architectures continues to grow, the need to develop a general framework for their analysis becomes increasingly important. With this aim, we draw insights from classical signal processing and control theory, to develop a quantitative measure of memory utilization: the internal mechanisms through which a model stores past information to produce future outputs. This metric, which we call effective state-size (ESS), is tailored to the fundamental class of systems with input-invariant and input-varying linear operators, encompassing a variety of computational units such as variants of attention, convolutions, and recurrences. Unlike prior work on memory utilization, which either relies on raw operator visualizations (e.g. attention maps), or simply the total memory capacity (i.e. cache size) of a model, our metrics provide highly interpretable and actionable measurements. In particular, we show how ESS can be leveraged to improve initialization strategies, inform novel regularizers and advance the performance-efficiency frontier through model distillation. Furthermore, we demonstrate that the effect of context delimiters (such as end-of-speech tokens) on ESS highlights cross-architectural differences in how large language models utilize their available memory to recall information. Overall, we find that ESS provides valuable insights into the dynamics that dictate memory utilization, enabling the design of more efficient and effective sequence models.

Sit-to-stand and stand-to-sit motions are important in daily activities. However, elderly individuals often find these motions difficult to perform with declining lower limb strength, which causes a considerable reduction to their quality of life. In this study, a sensing method for controlling robotic assistive devices was proposed. This method utilizes electromyographic measurements and a deep neural network to predict motion initiation, and it estimates the timing of triggering assistive devices. Experimental results indicate that four muscle synergy patterns are required to represent the sit-to-stand and stand-to-sit motions together, with two of them being shared between both movements. Subsequently, a long short-term memory network was designed to forecast these two motions, and the result indicates that the prediction accuracy reached 92.95% ± 0.83% with forecasting time of 300 ms.

This paper proposes a method for generating maps in indoor environments that include transparent objects by using a stereo polarization camera and projector. Conventional sensors like LiDAR and stereo cameras struggle with glass, as they rely on diffuse reflection, while glass allows light to pass through. In contrast, polarization cameras can measure light polarization and estimate surface normals, enabling depth estimation by combining polarization and RGB information. However, when measuring transparent objects, reflected and transmitted light cancel each other out, reducing polarization contrast, and the RGB information causes the depth estimation to output the depth of objects behind the glass. To address this issue, this paper proposes a novel method that (1) improves the S/N ration in polarization measument via diffuse reflection on non-glass regions and (2) masks out the RGB color from polarimetric depth estimation to not compute depth map of objects behind the glass to obtain depth images that include glass surfaces. Additionally, (3) in the mapping part, depth estimation is repeated at multiple locations, and the results are integrated using self-localization to generate a complete environmental map. Experiments in an indoor environment confirmed the effectiveness of the proposed method, enabling glass-inclusive depth estimation and successful map generation on a mobile robot.

Self-efficacy is crucial for the effective application of assistive technology and rehabilitation. This study proposes a novel approach to assess the impact of motor interventions on motor self-efficacy, relevant for human-robot interaction in rehabilitation, by focusing on the perceived reachable space. Twelve healthy adults underwent an arm movement restriction intervention using a robotic arm (KINARM), and changes in the perceived reachable space and muscle activity were measured before and after the intervention. The results indicated a reduction in the perceived reachable space and an adaptive decrease in muscle activity for unreachable targets following motor restriction. This suggests that the perceived reachable space can serve as an objective proxy for task-specific motor self-efficacy, which is valuable for evaluating user adaptation to robotic interfaces. Furthermore, these findings imply that in rehabilitation using interactive robots, a patient’s effort levels may be influenced by their perception of task achievability.

Many elderly individuals experience a decline in motor function. To provide appropriate rehabilitation programs, a sufficient and convenient evaluation method is necessary. In this study, we focused on the sit-to-stand (STS) motion, a crucial activity in daily life, and used handrail to obtain force data safely and easily. Previous studies have proposed methods for estimating scores such as the Timed Up and Go test from forces applied to the hand, hip, and foot, classifying elderly individuals into several motor function categories. However, these indicators are insufficient for evaluating the function of specific muscles or joints in the lower extremities individually. The primary objective of this study was joint torque, which more directly represents the function of specific muscles and joints. We measured the time-series data of forces acting on the body during STS motions and developed a model using Long Short-Term Memory to estimate lower limb joint torques. As a result, knee and hip joint torques were accurately estimated from force applied to hand, hip and foot. Furthermore, this method demonstrated the potential for early detection of joint disorders. This approach allows for a detailed assessment of the state of the knee and hip joints simply by standing up while holding a handrail.

We approach designing a state-space model for deep learning applications through its dual representation, the transfer function, and uncover a highly efficient sequence parallel inference algorithm that is state-free: unlike other proposed algorithms, state-free inference does not incur any significant memory or computational cost with an increase in state size. We achieve this using properties of the proposed frequency domain transfer function parametrization, which enables direct computation of its corresponding convolutional kernel’s spectrum via a single Fast Fourier Transform. Our experimental results across multiple sequence lengths and state sizes illustrates, on average, a 35% training speed improvement over S4 layers – parametrized in time-domain – on the Long Range Arena benchmark, while delivering state-of-the-art downstream performances over other attention-free approaches. Moreover, we report improved perplexity in language modeling over a long convolutional Hyena baseline, by simply introducing our transfer function parametrization. Our code is available at https://github.com/ruke1ire/RTF.

Mobile robot navigation in a human-populated environment has been of great interest to the research community in recent years, referred to as crowd navigation. Currently, offline reinforcement learning (RL)-based method has been introduced to this domain, for its ability to alleviate the sim2real gap brought by online RL which relies on simulators to execute training, and its scalability to use the same dataset to train for differently customized rewards. However, the performance of the navigation policy suffered from the distributional shift between the training data and the input during deployment, since when it gets an input out of the training data distribution, the learned policy has the risk of choosing an erroneous action that leads to catastrophic failure such as colliding with a human. To realize risk sensitivity and improve the safety of the offline RL agent during deployment, this work proposes a multipolicy control framework that combines offline RL navigation policy with a risk detector and a force-based risk-avoiding policy. In particular, a Lyapunov density model is learned using the latent feature of the offline RL policy and works as a risk detector to switch the control to the risk-avoiding policy when the robot has a tendency to go out of the area supported by the training data. Experimental results showed that the proposed method was able to learn navigation in a crowded scene from the offline trajectory dataset and the risk detector substantially reduces the collision rate of the vanilla offline RL agent while maintaining the navigation efficiency outperforming the state-of-the-art methods.

Assisting patients with dementia is a significant social issue. Currently, to assist patients with dementia, a multimodal care technique called Humanitude is gaining popularity. In Humanitude, the patients are assisted through various techniques to stand up independently by utilizing their motor functions as much as possible. Humanitude care techniques encourage caregivers to increase the area of contact with patients during the sit-to-stand motion. However, Humanitude care techniques are not accurately performed by novice caregivers. Therefore, in this study, a smock-type wearable sensor was developed to measure the proximity between caregivers and care recipients during sit-to-stand motion assistance. A measurement experiment was conducted to evaluate the proximity differences between Humanitude care and simulated novice care. In addition, the effects of different care techniques on the center of mass (CoM) trajectory and muscle activity of the care recipients were investigated. The results showed that the caregivers tend to bring their top and middle trunk closer in Humanitude care compared with novice simulated care. Furthermore, it was observed that the CoM trajectory and muscle activity under Humanitude care were similar to those observed when the care recipient stands up independently. These results validate the effectiveness of Humanitude care and provide useful information for teaching techniques in Humanitude.

This study develops a training system for a multimodal comprehensive care methodology for dementia patients called Humanitude. Humanitude has attracted much attention as a gentle and effective care technique. It consists of four main techniques, namely, eye contact, verbal communication, touch, and standing up, and more than 150 care elements. Learning Humanitude thus requires much time. To provide an effective training system for Humanitude, we develop a training system that realizes sensing and interaction simultaneously by combining a real entity and augmented reality technology. To imitate the interaction between a patient and a caregiver, we superimpose a three-dimensional CG model of a patient's face onto the head of a soft doll using augmented reality technology. Touch information such as position and force is sensed using the whole body wearable tactile sensor developed to quantify touch skills. This training system enables the evaluation of eye contact and touch skills simultaneously. We build a prototype of the proposed training system and evaluate the usefulness of the system in public lectures.

It is important to understand the mechanism of human standing-up motion to improve the declined physical ability of the elderly people. This study employs the concept of muscle synergies (modular structure of coordinative muscle activation) to understand how humans coordinate their muscles to achieve the standing-up motion. Neuro-musculoskeletal model was developed to represent human body to generate standing-up motion. Using the developed model, forward dynamic simulation was used to analyze how humans utilized the muscle synergies to realize the motion. Results showed that the developed model could generate the standing-up motion with four muscle synergies rather than controlling individual muscles. Moreover, further analysis showed that three different strategies of the standing-up motion could be generated only by changing the start time of the particular muscle synergy.

The standing-up motion is one of the most important activities of daily livings. In order to understand the strategy to achieve the standing-up motion, muscle synergy analysis is applied to the measured data during human standing-up motion. In addition, musculoskeletal model which consists of three body segments and nine muscles in lower limb is developed to ensure that the standing-up motion can be generated by muscle synergies. As a result, three muscle synergies have been extracted from the human standing-up motion, and each synergy strongly corresponded to characteristic kinematic events: momentum flexion, momentum transfer, and posture stabilization. Results of forward dynamic simulation show that the standing-up motion can be achieved by controlling time-varying weighting coefficient of three muscle synergies instead of controlling individual nine muscles.

Although standing-up motion is an important activity of daily living, it remains unclear how people perform the motion in different situations. As described in this paper, muscle synergy analysis is applied to standing-up motions performed at different circumstances, such as two different heights and at three different speeds. Results elucidated three invariant groups of synchronized muscle activations: The first synergy pulls the ankle and raises the hip. The second synergy extends the upper body. The third synergy stabilizes posture. Results also show that people controlled the activation coefficient of each synergy differently during all motions. The slower the standing-up motion is, the longer each synergy activates to adapt to the slower motion speed. Results of this study show that people use the same group of synchronized muscle activation and only control the activation coefficient to achieve adaptive standing-up motion.

As the population continues to age, the number of elderly people requiring healthcare is increasing. In order to improve their physical function, they need to get physical training. There are activities which require integrated arm movements and lower body movements. However there are no quantitative testing methods of the degree of recovery for the coordination between arm movements and lower body movements. In this study, we focus on dart throwing motion as arm movements in lower body movements and suggest the quantitative evaluation of the coordination between arm movements and lower body movements in dart throwing motion. Normalized correlation coefficient (NCC) between arm and lower body was computed at different throwing distances. In addition the standard deviation of the NCC was computed in order to investigate the stability of the joint correlation evaluation. This analysis shows that the correlation between elbow and ankle, or between elbow and knee, are increased at throwing long distance. We suggest that the NCC between elbow angle and right knee angle may be used for the evaluation of the joint correlation between arm movements and lower body movements in dart throwing motion.

Recently healthcare of the elderly people has become a serious issue in medical and rehabilitation areas. In order to know their functional mobility and provide sufficient medical treatment, it is important to measure their body state precisely and objectively. Therefore we developed a wearable and wireless sensor of gyroscope and accelerometer (PocketIMU2) as an easy and precise measurement of human motions. In the sensor, we employed a small and high accurate LiNbO3 crystal to achieve joint angle computation with simple integration of angular velocity. In the current paper, we evaluate the accuracy of the sensor in two important basic motion, such as a walking and sit-to-stand motions. Computed joint angles of ankle, knee, and hip are compared to the reference data measured from a optical motion capture system in term of coefficients of correlation and root mean square error. As a result, coefficient of correlation showed very high value for all joint angles, and root mean square error was adequately small. This strongly supports the usage of our developed gyroscope and accelerometer sensor for monitoring human body movement for medical usage.