Robots and intelligent systems that understand and support human behavior are increasingly used in various settings, including public facilities, homes, offices, and manufacturing sites. To effectively understand human behavior, such systems must accurately segment and classify behaviors. Unsupervised learning, which does not require labeled data, is crucial for this task, as manually labeling every behavior in advance is challenging. Conventional unsupervised segmentation methods typically assume that instances of the same behavior share uniform characteristics. However, this assumption can lead to inaccurate segmentation when individual differences are present. To address this limitation, we propose the individuality-conditioned Gaussian process-hidden semi-Markov model (IC-GP-HSMM), an extension of the GP-HSMM. The original GP-HSMM uses unsupervised learning to model behavioral patterns as Gaussian processes based on observed sequences. Our extension assumes that information regarding individuals performing the behaviors is observable and incorporates this information as an individuality vector. This enables the model to learn behavior-specific Gaussian processes that consider individual variation, resulting in more accurate segmentation. Experiments conducted on synthetic and motion capture datasets demonstrate that IC-GP-HSMM outperforms conventional methods in segmenting behaviors with individual differences. The proposed IC-GPHSMM enables intelligent systems to more accurately recognize and adapt to individual variations in human behavior, enhancing their reliability and effectiveness in real-world applications.

In this paper, we propose a semi-autonomous teleoperation method to reduce operator workload and enhance task execution efficiency. The proposed method uses action chunking with transformers (ACT), an imitation learning method, to predict long-duration motion trajectories from a robot’s egocentric images and pose information during teleoperation. Because the images and poses input into ACT are continuously updated during teleoperation, the predicted trajectory dynamically reflects the control history of the operator. These continuously updated trajectories are displayed in real time in a simulation environment, serving as visual feedback for the operator. When the operator determines that the visualized trajectory aligns with their intended trajectory, they can switch from manual to autonomous control. Subsequently, the robot autonomously executes motion based on the predicted trajectory, thereby reducing operator workload and improving task efficiency. To verify the effectiveness of the proposed method, an object-grasping task was conducted using a human support robot, comparing manual control, fully autonomous control, and the proposed semi-autonomous method. The results demonstrated that the proposed method effectively reduces task completion time and improves success rates, even with unknown objects and environments.

In this paper, we propose RFF-GP-HSMM, a fast unsupervised time-series segmentation method that incorporates random Fourier features (RFF) to address the high computational cost of the Gaussian process hidden semi-Markov model (GP-HSMM). GP-HSMM models time-series data using Gaussian processes, requiring inversion of an N × N kernel matrix during training, where N is the number of data points. As the scale of the data increases, matrix inversion incurs a significant computational cost. To address this, the proposed method approximates the Gaussian process with linear regression using RFF, preserving expressive power while eliminating the need for inversion of the kernel matrix. Experiments on the Carnegie Mellon University (CMU) motion-capture dataset demonstrate that the proposed method achieves segmentation performance comparable to that of conventional methods, with approximately 278 times faster segmentation on time-series data comprising 39,200 frames.

Humans perceive continuous high-dimensional information by dividing it into significant segments such as words and units of motion. We believe that such unsupervised segmentation is also important for robots to learn topics such as language and motion. To this end, we previously proposed a hierarchical Dirichlet process-Gaussian process-hidden semi-Markov model (HDP-GP-HSMM). However, an important drawback to this model is that it cannot divide high-dimensional time-series data. Further, low-dimensional features must be extracted in advance. Segmentation largely depends on the design of features, and it is difficult to design effective features, especially in the case of high-dimensional data. To overcome this problem, this paper proposes a hierarchical Dirichlet process-variational autoencoder-Gaussian process-hidden semi-Markov model (HVGH). The parameters of the proposed HVGH are estimated through a mutual learning loop of the variational autoencoder and our previously proposed HDP-GP-HSMM. Hence, HVGH can extract features from high-dimensional time-series data, while simultaneously dividing it into segments in an unsupervised manner. In an experiment, we used various motion-capture data to show that our proposed model estimates the correct number of classes and more accurate segments than baseline methods. Moreover, we show that the proposed method can learn latent space suitable for segmentation.

Humans recognize perceived continuous information by dividing it into significant segments such as words and unit motions. We believe that such unsupervised segmentation is also an important ability that robots need to learn topics such as language and motions. Hence, in this paper, we propose a method for dividing continuous time-series data into segments in an unsupervised manner. To this end, we proposed a method based on a hidden semi-Markov model with Gaussian process (GP-HSMM). If Gaussian processes, which are nonparametric models, are used, unit motion patterns can be extracted from complicated continuous motion. However, this approach requires the number of classes of segments in the time-series data in advance. To overcome this problem, in this paper, we extend GP-HSMM to a nonparametric Bayesian model by introducing a hierarchical Dirichlet process (HDP) and propose the hierarchical Dirichlet processes-Gaussian process-hidden semi-Markov model (HDP-GP-HSMM). In the nonparametric Bayesian model, an infinite number of classes is assumed and it becomes difficult to estimate the parameters naively. Instead, the parameters of the proposed HDP-GP-HSMM are estimated by applying slice sampling. In the experiments, we use various synthetic and motion-capture data to show that our proposed model can estimate a more correct number of classes and achieve more accurate segmentation than baseline methods.