Spatio-Temporal Video Grounding (STVG) aims to localize a spatio-temporal tube that corresponds to a given language query in an untrimmed video. This is a challenging task since it involves complex vision-language understanding and spatiotemporal reasoning. Recent works have explored weakly-supervised setting in STVG to eliminate reliance on fine-grained annotations like bounding boxes or temporal stamps. However, they typically follow a simple late-fusion manner, which generates tubes independent of the text description, often resulting in failed target identification and inconsistent target tracking. To address this limitation, we propose a Tube-conditioned Reconstruction with Mutual Constraints (TubeRMC) framework that generates text-conditioned candidate tubes with pre-trained visual grounding models and further refine them via tube-conditioned reconstruction with spatio-temporal constraints. Specifically, we design three reconstruction strategies from temporal, spatial, and spatio-temporal perspectives to comprehensively capture rich tube-text correspondences. Each strategy is equipped with a Tube-conditioned Reconstructor, utilizing spatio-temporal tubes as condition to reconstruct the key clues in the query. We further introduce mutual constraints between spatial and temporal proposals to enhance their quality for reconstruction. TubeRMC outperforms existing methods on two public benchmarks VidSTG and HCSTVG. Further visualization shows that TubeRMC effectively mitigates both target identification errors and inconsistent tracking.

Edge labels are typically at various granularity levels owing to the varying preferences of annotators, thus handling the subjectivity of per-pixel labels has been a focal point for edge detection. Previous methods often employ a simple voting strategy to diminish such label uncertainty or impose a strong assumption of labels with a pre-defined distribution, e.g., Gaussian. In this work, we unveil that the segment anything model (SAM) provides strong prior knowledge to model the uncertainty in edge labels. Our key insight is that the intermediate SAM features inherently correspond to object edges at various granularities, which reflects different edge options due to uncertainty. Therefore, we attempt to align uncertainty with granularity by regressing intermediate SAM features from different layers to object edges at multi-granularity levels. In doing so, the model can fully and explicitly explore diverse ``uncertainties'' in a data-driven fashion. Specifically, we inject a lightweight module (~ 1.5% additional parameters) into the frozen SAM to progressively fuse and adapt its intermediate features to estimate edges from coarse to fine. It is crucial to normalize the granularity level of human edge labels to match their innate uncertainty. For this, we simply perform linear blending to the real edge labels at hand to create pseudo labels with varying granularities. Consequently, our uncertainty-aligned edge detector can flexibly produce edges at any desired granularity (including an optimal one). Thanks to SAM, our model uniquely demonstrates strong generalizability for cross-dataset edge detection. Extensive experimental results on BSDS500, Muticue and NYUDv2 validate our model's superiority.

Wireless inertial motion capture holds promise for real-time human-machine interfaces and home-based rehabilitation applications. However, wireless data drop can cause significant estimation errors deteriorating performance or even making the system unusable. It is currently unclear how to estimate non-periodic kinematics with wearable inertial measurement units (IMUs) in the presence of wireless data drop (packet loss). We thus propose a novel inference encoder-decoder network model for real-time kinematics during dynamic movement. Twenty-four healthy subjects performed yoga, golf, swimming, dance, and badminton movement activities while wearing IMUs and 10-90% of each IMU's data were randomly removed to determine the effects of data drop on estimation accuracy with and without the proposed model. Results demonstrated a reduction in RMSE of 45.2% to 51.5% in the upper limb kinematic estimation of the proposed model compared to the No Prediction strategy, and a reduction of 19.1% to 31.3% of the proposed model compared with an baseline LSTM model. In addition, the proposed model has significantly less error (p<0.05) than the No Prediction strategy and the baseline LSTM model for 10%, 20%, 30%, 40%, 50%, 60%, 70%, and 80% data drop. These results could enable wearable, wireless IMU dynamic motion analysis and assessment with reduced kinematic estimation error in the presence of varying amounts of wireless data drop and thus could further facilitate human-machine interaction and home-based medical assessment and treatment.

The development of artificial intelligence (AI) provides opportunities for the promotion of deep neural network (DNN)-based applications. However, the large amount of parameters and computational complexity of DNN makes it difficult to deploy it on edge devices which are resource-constrained. An efficient method to address this challenge is model partition/splitting, in which DNN is divided into two parts which are deployed on device and server respectively for co-training or co-inference. In this paper, we consider a split federated learning (SFL) framework that combines the parallel model training mechanism of federated learning (FL) and the model splitting structure of split learning (SL). We consider a practical scenario of heterogeneous devices with individual split points of DNN. We formulate a joint problem of split point selection and bandwidth allocation to minimize the system latency. By using alternating optimization, we decompose the problem into two sub-problems and solve them optimally. Experiment results demonstrate the superiority of our work in latency reduction and accuracy improvement.

Walking, one of the most common daily activities, causes unwanted movement artifacts which can significantly deteriorate hand gesture recognition accuracy. However, traditional hand gesture recognition algorithms are typically developed and validated with wrist-worn devices only during static human poses, neglecting the critical importance of dynamic effects on gesture accuracy. Thus, we developed and validated a signal decomposition approach via empirical mode decomposition to accurately segment target gestures from coupled raw signals during dynamic walking and a transfer learning method based on distribution adaptation to enable gesture recognition through domain transfer between dynamic walking and static standing scenarios. Ten healthy subjects performed seven hand gestures during both walking and standing experiments while wearing an IMU wrist-worn device. Experimental results showed that the signal decomposition approach reduced the gesture detection error by 83.8%, and the transfer learning approach (20% transfer rate) improved hand gesture recognition accuracy by 15.1%. This ground-breaking work demonstrates the feasibility of hand gesture recognition while walking via wrist-worn sensing. These findings serve to inform real-life and ubiquitous adoption of wrist-worn hand gesture recognition for intuitive human-machine interaction in dynamic walking situations.

Wearable activity recognition can collate the type, intensity, and duration of each child's physical activity profile, which is important for exploring underlying adolescent health mechanisms. Traditional machine-learning-based approaches require large labeled data sets; however, child activity data sets are typically small and insufficient. Thus, we proposed a transfer learning approach that adapts adult-domain data to train a high-fidelity, subject-independent model for child activity recognition. Twenty children and twenty adults wore an accelerometer wristband while performing walking, running, sitting, and rope skipping activities. Activity classification accuracy was determined via the traditional machine learning approach without transfer learning and with the proposed subject-independent transfer learning approach. Results showed that transfer learning increased classification accuracy to 91.4% as compared to 80.6% without transfer learning. These results suggest that subject-independent transfer learning can improve accuracy and potentially reduce the size of the required child data sets to enable physical activity monitoring systems to be adopted more widely, quickly, and economically for children and provide deeper insights into injury prevention and health promotion strategies.