The dynamics of multivariate time series (MTS) data are jointly characterized by its nonlinear temporal dependencies and complex variable dependencies, making unsupervised time series anomaly detection a challenging task. Existing methods primarily rely on prediction or reconstruction errors, neglecting the valuable information within the variable dependencies. In this paper, we propose a variable dependency discrepancy-based Transformer (VDDFormer) for unsupervised MTS anomaly detection. VDDFormer comprises a variable correlation encoder, a temporal dependency encoder, and a reconstruction decoder. The variable correlation encoder capitalizes on a variable dependency attention mechanism, which employs self-attention to learn the global variable dependencies; meanwhile, the local variable dependencies are captured by the adaptive correlation matrix. The global and local variable dependencies are then used to compute the variable dependency discrepancy as a new intrinsic property to distinguish between normal and abnormal patterns. By integrating this new discrepancy with the reconstruction error, the model effectively enhances its anomaly differentiation capability. Extensive experiments on five real-world anomaly detection datasets demonstrate that VDDFormer effectively and robustly detects group anomaly patterns by leveraging the variable dependency discrepancy and achieves state-of-the-art performance on four out of the five datasets.

Accomplishing a plug-and-play steady-state visual evoked potential (SSVEP)-based brain-computer interface (BCI) remains a critical challenge, due to the unsatisfying performance of calibration-free decoding algorithms.A current method called online adaptive canonical correlation analysis (OACCA) has proved efficient in enhancing calibration-free performance by self-adaptation merely with online data.However, OACCA only concerns the adaptation of spatial filters and excludes other useful adaptive procedures like individual template estimation, hindering fully exploitable model decoding and adaptation. This study proposes a new online adaptation framework termed online adaptive extended correlation analysis (OAECA) to augment the calibration-free online adaptation loop. OAECA first recalls and cleans the online trials for reliable data learning, then tunes individual templates and spatial filters for complete model updating, and finally adopts extended feature matching to improve target recognition. The simulation results on two public SSVEP datasets revealed that OAECA significantly outperformed OACCA for almost all 105 subjects, and both offline and online experiments further confirmed the effectiveness of OAECA. Particularly, OAECA achieved the highest average information transfer rate (ITR) of 202.17 bits/min in the online experiment, significantly exceeding the state-of-the-art OACCA of 177.02 bits/min. This study enhanced the calibration-free performance through comprehensive online adaptation, hopefully advancing SSVEP-based BCIs toward practical plug-and-play real-world applications.

With the aim of reducing the transmission latency, this article proposes a scheme of user-assisted base station (BS) caching and cooperative prefetching for high-speed railway (HSR) communications. In the model under consideration, neighboring BSs periodically exchange information, including the coverage area and communication rate, to facilitate content caching and prefetching. As an additional means, users can cache contents that are different from BSs. Specifically, we construct an optimization problem for content caching and prefetching that minimizes the overall transmission latency. Moreover, this problem is decomposed into two subproblems, namely, the one aiming to maximize the available throughput of BSs and the other attempting to maximize the user cache utilization subject to the constraint of the BS cache. We demonstrate that the objective function of the first subproblem is equivalent to a monotone submodular function subject to matroid constraints. Therefore, it can be efficiently solved with a greedy algorithm. Meanwhile, the user cache issue can be viewed as a weighted sum maximization problem. Simulation results are presented to verify that the proposed scheme can not only reduce the average transmission latency but also improve the hit probability and system throughput.

Tracking body and hand motions in 3D space is essential for social and self-presence in augmented and virtual environments. Unlike the popular 3D pose estimation setting, the problem is often formulated as egocentric tracking based on embodied perception (e.g., egocentric cameras, handheld sensors). In this article, we propose a new data-driven framework for egocentric body tracking, targeting challenges of omnipresent occlusions in optimization-based methods (e.g., inverse kinematics solvers). We first collect a large-scale motion capture dataset with both body and finger motions using optical markers and inertial sensors. This dataset focuses on social scenarios and captures ground truth poses under self-occlusions and body-hand interactions. We then simulate the occlusion patterns in head-mounted camera views on the captured ground truth using a ray casting algorithm and learn a deep neural network to infer the occluded body parts. Our experiments show that our method is able to generate high-fidelity embodied poses by applying the proposed method to the task of real-time egocentric body tracking, finger motion synthesis, and 3-point inverse kinematics.

Fully immersive experiences in AR/VR depend on re-constructing the full body pose of the user without restricting their motion. In this paper we study the use of body-worn electromagnetic (EM) field-based sensing for the task of 3D human pose reconstruction. To this end, we present a method to estimate SMPL parameters from 6-12 EM sensors. We leverage a customized wearable system consisting of wireless EM sensors measuring time-synchronized 6D poses at 120 Hz. To provide accurate poses even with little user instrumentation, we adopt a recently proposed hybrid framework, learned gradient descent (LGD), to iteratively estimate SMPL pose and shape from our input measurements. This allows us to harness powerful pose priors to cope with the idiosyncrasies of the input data and achieve accurate pose estimates. The proposed method uses AMASS to synthesize virtual EM-sensor data and we show that it generalizes well to a newly captured real dataset consisting of a total of 36 minutes of motion from 5 subjects. We achieve reconstruction errors as low as 31.8 mm and 13.3 degrees, outperforming both pure learning- and pure optimization-based methods. Code and data is available under https://ait.ethz.ch/projects/2021/em-pose.

We present a system for real-time hand-tracking to drive virtual and augmented reality (VR/AR) experiences. Using four fisheye monochrome cameras, our system generates accurate and low-jitter 3D hand motion across a large working volume for a diverse set of users. We achieve this by proposing neural network architectures for detecting hands and estimating hand keypoint locations. Our hand detection network robustly handles a variety of real world environments. The keypoint estimation network leverages tracking history to produce spatially and temporally consistent poses. We design scalable, semi-automated mechanisms to collect a large and diverse set of ground truth data using a combination of manual annotation and automated tracking. Additionally, we introduce a detection-by-tracking method that increases smoothness while reducing the computational cost; the optimized system runs at 60Hz on PC and 30Hz on a mobile processor. Together, these contributions yield a practical system for capturing a user's hands and is the default feature on the Oculus Quest VR headset powering input and social presence.

We propose a new approach to action classification in video, which uses deep appearance and motion features extracted from spatio-temporal volumes defined along body part trajectories to learn mid-level classifiers called deep moving poselets. A deep moving poselet is a classifier that captures a characteristic body part configuration, with a specific appearance and undergoing a specific movement. By having this mid-level representation of a body part be shared across action classes and by learning it jointly with action classifiers, we obtain a representation that is interpretable, shared and discriminative. In addition, by using sparsity-inducing norms to regularize action classifiers, we can reduce the number of deep moving poselets used by each class without hurting performance. Experiments show that the proposed method achieves state-of-the-art performance on the popular and challenging sub-JHMDB and MSR Daily Activity datasets.