Human pose estimation (HPE) is a critical technology for multimedia applications such as virtual reality (VR) and other interactive systems, where efficient, accurate and cost-effective pose estimation is essential. However, high-frequency and precise pose estimation often requires advanced equipment like high-speed cameras or time-of-flight sensors and prohibitive amount of computational resources, which are expensive and hinder widespread adoption. Previous fusion based methods always rely on synchronized signal input which is throttle by low frequency sensors and vulnerable to signal drops. To address this challenge, we propose integrating low-cost electromyography (EMG) and visual-inertial odometry (VIO) data for lower-body HPE using a novel multi-modal neural network. Instead of relying on synchronized sensor inputs, we reformulate the fusion problem as an outdated-signal-guided HPE prediction task, achieving latency as low as 1.6 ms per prediction. We validate our approach on a dataset of 1,000 lower-body pose clips from 10 subjects, specifically curated for this task. Experimental results demonstrate that our method achieves accurate, high-frequency pose estimation. The implementation is publicly available at https://github.com/k9tming/ALIVE.

Federated Learning (FL) is an emerging distributed machine learning technology. However, in practical applications, it frequently encounters the challenge of client resource heterogeneity. This can result in long wait times or even model training failures during the communication process of FL. To address this problem, we propose a dynamic hierarchical federated learning mechanism for financial models (DHFM-FLM). The local client adopts a dynamic model training design that leverages the property of resource heterogeneity to enhance the performance of the local model. To avoid prolonged wait times for failing clients, a dynamic communication detection design is proposed at three critical junctures. In each round, the model hierarchical reservation communication design is employed to collect models in segments, thus reducing communication congestion and preventing malicious attacks on the communication process. Experiments with heterogeneous computation and communication resources demonstrate that utilizing the DHFM-FLM boosts model performance by approximately 5-8% and reduces communication time by about 15%. Additionally, DHFM-FLM increases the success rate of the FL task by approximately 6%.

The fast-growing techniques of measuring and fusing multi-modal biomedical signals enable advanced motor intent decoding schemes of lower-limb exoskeletons, meeting the increasing demand for rehabilitative or assistive applications of take-home healthcare. Challenges of exoskeletons’ motor intent decoding schemes remain in making a continuous prediction to compensate for the hysteretic response caused by mechanical transmission. In this paper, we solve this problem by proposing an ahead-of-time continuous prediction of lower-limb kinematics, with the prediction of knee angles during level walking as a case study. Firstly, an end-to-end kinematics prediction network(KinPreNet),1consisting of a feature extractor and an angle predictor, is proposed and experimentally compared with features and methods traditionally used in ahead-of-time prediction of gait phases. Secondly, inspired by the electromechanical delay(EMD), we further explore our algorithm’s capability of compensating response delay of mechanical transmission by validating the performance of the different sections of prediction time. And we experimentally reveal the time boundary of compensating the hysteretic response. Thirdly, a comparison of employing EMG signals or not is performed to reveal the EMG and kinematic signals’ collaborated contributions to the continuous prediction. During the experiments, EMG signals of nine muscles and knee angles calculated from inertial measurement unit (IMU) signals are recorded from ten healthy subjects. Our algorithm can predict knee angles with the averaged RMSE of 3.98 deg which is better than the 15.95-deg averaged RMSE of utilizing the traditional methods of ahead-of-time prediction. The best prediction time is in the interval of 27ms and 108ms. To the best of our knowledge, this is the first study of continuously predicting lower-limb kinematics in an ahead-of-time manner based on the electromechanical delay (EMD).

The cybertwin-driven 6G that can obtain static and dynamic data stream of users provide an exciting potential for a novel muscular human cybertwin beyond traditonally used artificial neural networks (ANNs) and musculoskeletal models (MSMs). In this article, we propose the conceptual design of the muscular human cybertwin and construct a baseline model with an improved generalization ability over ANN and an easier adaptation to new data distributions over MSMs. In particular, we for the first time propose to combine ANN and MSM, which benefits from the combination of learning-based approaches and analytical approaches. We then experimentally compare different manners of the combination and demonstrate the better combining manner on our testing case. Finally, we evaluate our method on an open-sourced dataset and on data from wearable sensors from the aspects of joint moment prediction accuracy, data efficiency, generalization ability, and time efficiency of personalization. Our proposed method achieves accuracy similar with ANN and over 30$\%$better than MSM with sufficient training data. Compared with ANN, the improved data efficiency is presented by the better accuracies with a small amount of training data, and the generalization ability to unseen walking conditions and new subjects are demonstrated by the over 70$\%$accuracy improvements. Moreover, when fine-tuning the model, our algorithm is demonstrated by the time 75$\%$shorter than calibrated MSM and the accuracy improvements.

The surface electromyography (sEMG) signal-based human-machine interface (HMI) has been widely used for various scenarios of physical human-robot interaction. However, current HMIs based on bipolar myoelectric sensors are hindered by the limitations of global sEMG features, which are prone to variability and delay. In this letter, we define a HMI that takes advantage of the underlying neural information of spinal module activations from bipolar sEMG signals, inspired by recent findings of neural codes. Firstly, the spinal module activations are identified by the spiking trains of the muscle synergies extracted from bipolar sEMG signals. Secondly, we extract the information encoded in both firing rates and spike timings of the spinal module activation in a population coding manner, which follows the information encoding principle of neurons. Thirdly, we map the series of spinal module activations into gait phases, locomotion modes, joint moment and human identity in order to experimentally reveal the physiological information contained in the spinal module activations. The contained information and the benefit of our design are demonstrated and experimentally explained by the presented results and comparisons with the traditionally used global sEMG features. The proposed bipolar myoelectric sensor-enabled human-machine interface could contribute to various scenarios of physical human-robot interaction.

In this paper we present a multisensor-multitarget tracking testbed for large-scale distributed scenarios. The objective is to develop a testbed capable of handling multiple, heterogeneous sensors in a hierarchical architecture for maritime surveillance. The testbed consists of a scenario generator that can generate simulated data from multiple sensors including radar, sonar, IR and ESM as well as a tracker framework into which different tracking algorithms can be integrated. In the current stage of the project, the IMM/Assignment tracker, and the Particle Filter (PF) tracker are implemented in a distributed architecture and some preliminary results are obtained. Other trackers like the Multiple Hypothesis Tracker (MHT) are also planned for the future.

In this article, we consider the bias issue in a passive tracking system which utilizes a mobile passive sensor network, where bearing-only sensors such as Inferred or ESM are used. Biases due to nonlinear transformations have already been recognized, but have not been studied for this particular case of converted pseudo measurements in a mobile passive sensor network. Based on the Taylor series, the bias equations for a network of two passive sensors are derived. Monte Carlo simulation is used for analysis. There are two other non-linear transformations which are related to this study: 1. range/azimuth to X/Y; 2. range/azimuth to latitude/longitude. Insightful studies with explicit expressions are available for the first nonlinear transformation, but not for the second and the new nonlinear transformations. This article will provide an approximate solution and simulation study for the new nonlinear transformation.