Synthesizing virtual Inertial Measurement Unit (IMU) data from 3D human motion sequences has emerged as a promising strategy to mitigate the scarcity of labeled datasets in IMU-based Human Activity Recognition (HAR). However, existing virtual IMU data driven methods typically necessitate an extensive datasets of diverse 3D motion sequences to ensure model performance, which imposes significant challenges in terms of motion resource acquisition. To address this, we propose LimbAug, a framework designed to alleviate the high demand for 3D motion resources through intelligent limb movement augmentation. By employing a conditional Variational Autoencoder (cVAE), LimbAug learns and generates limb movement differences to augment existing 3D motion sequences, effectively mimicking real-world intra-class diversity. This approach enables the synthesis of large-scale, diverse virtual IMU data from a limited number of 3D motion samples. Our experiments demonstrate that LimbAug not only reduces the reliance on vast 3D motion libraries but also significantly enhances the generalization and recognition accuracy of HAR models on real-world data.

Human activity recognition (HAR) based on skeletal motion data is an important research direction in the field of human-computer interaction, as it is more robust to environmental variations compared to traditional IMU and RGB video data. However, the collection of datasets often requires a significant investment of human and material resources. Therefore, this paper proposed a skeletal motion data synthesis method based on 3D human motion generation, using the acquired dataset to construct a human behavior recognition model. The method first generates diversified human motion sequences using a text-driven motion generation platform. Then it extracts large-scale virtual skeletal motion data from the generated motion sequences to construct a training dataset. Using virtual data to replace real data for model pre-training effectively reduces the cost of dataset collection. In addition, this paper proposes a masked self-encoder architecture for virtual skeletal training data, which utilizes a small amount of real data to fine-tune the pre-trained model, thereby completing the construction of the HAR model. The final model can then be directly used to recognize target motions. Finally, this paper evaluates the proposed method through a series of experiments. Under the recognition task of six motions, each person only needs to use 90 seconds of data for fine-tuning each motion to achieve 86% recognition accuracy, which verifies the effectiveness of the proposed method.

Human Activity Recognition (HAR) with Inertial Measurement Units (IMUs) sensor is often constrained by the high cost of labeled data and the domain gap between virtual and real signals. To address this, we propose a virtual-to-real domain adaptation framework that integrates a Conditional Variational Autoencoder (CVAE) with contrastive learning. The CVAE generates label-conditioned virtual IMU samples, while contrastive pretraining learns transferable representations, compatible with various self-supervised frameworks. We evaluate on a custom anaerobic exercise dataset with six activities, using three-axis accelerometer signals. Results show that our method improves accuracy from 56.8% to 81.7% with only 40 seconds of labeled data per class, demonstrating its effectiveness in few-shot cross-domain HAR.

Touch sensing is a prominent pillar technique in various human-computer interactive scenarios, especially when touchscreen-based capacitive touch sensing has become a representative in user-end electronics. An intelligent touch-sensing system captures the capacitive touch-sensing images to recognize the objects via machine learning techniques. However, collecting the training dataset is usually laborious and time-consuming, requiring specific coding skills and knowledge. In this article, we introduced vCapTouch, a data generation method to synthesize the touch sensing data, which can be directly employed to train a machine learning model and recognize the real touching behavior, significantly lowering the need for real dataset collection. The presented method is primarily based on the idea of the digital twin. We implemented the method with Unity3D, a game engine that enables high interactivity, is easy to use, and has a low cost. We evaluated the proposed method on eight users with different touch screen devices and proved the feasibility of synthesizing the touch sensing data.

The rise of the Internet of Things (IoT) has given birth to transformative and massively deployed computing applications that raise the significant issue of energy sources. It is impractical and irresponsible to rely on wires and batteries to power trillion-level devices. One promising prediction is that energy harvesting technologies will serve as alternative power sources for IoT devices. However, we might be losing this prophecy for lack of understanding of how novice developers comprehend energy in developing IoT. In response, we conducted a mentored physical prototyping study with a two-day workshop involving eight novice developers. The study consisted of qualitative and quantitative analyses, the artifacts, interviews with both novice developers and an expert, and implications of designs for future tools. The findings reveal informational gaps that demand educational efforts and assistive features to facilitate novice developers. We present major findings from the study and implications for the design of future tools.

Guiding users with limb exercise can assist in muscle training or physical recovery. However, traditional vision-based methods often require multiple camera angles to help users understand the motions and require them to be within the range of the screen. Therefore, we propose a non-visual system that can guide users with multiple-directional limb motions utilizing spatial audio, AudioMove , with commercial-off-the-shelf (COTS) devices (i.e., smartphones and earphones). The proposed system addresses the challenge of conveying directional information encompassing multiple planes in real-time. We conduct a mixed-method user study to evaluate the effectiveness of the system with three methods combining motion data with spatial audio perception. Additionally, a user interface is built to collect users' comments. The results conclude that spatial audio guidance could create a natural, pervasive, and non-visual exercise training solution in daily life.

Human activities can introduce variations in various environmental cues, such as light and sound, which can serve as inputs for interfaces. However, one often overlooked aspect is the airflow variation caused by these activities, which presents challenges in detection and utilization due to its intangible nature. In this paper, we have unveiled an approach using mist to capture invisible airflow variations, rendering them detectable by Time-of-Flight (ToF) sensors. We investigate the capability of this sensing technique under different types of mist or smoke, as well as the impact of airflow speed. To illustrate the feasibility of this concept, we created a prototype using a humidifier and demonstrated its capability to recognize motions. On this basis, we introduce potential applications, discuss inherent limitations, and provide design lessons grounded in mist-based airflow sensing.

While boxing games allow players to learn techniques at home, they seldom check players’ movements. We propose tracking the players’ performance by smartphone sensors and classifying punches with convolutional neural network models. As a result, we achieved accuracies of 79.2% and 84.6% for 10 participants with two experiments, implying the possibilities and facilitation of smartphone sensors in boxing classifications of people with varying experiences.