Zero-shot action recognition (ZSAR) requires collaborative multi-modal spatiotemporal understanding. However, finetuning CLIP directly for ZSAR yields suboptimal performance, given its inherent constraints in capturing essential temporal dynamics from both vision and text perspectives, especially when encountering novel actions with fine-grained spatiotemporal discrepancies. In this work, we propose Spatiotemporal Dynamic Duo (STDD), a novel CLIP-based framework to comprehend multi-modal spatiotemporal dynamics synergistically. For the vision side, we propose an efficient Space-time Cross Attention, which captures spatiotemporal dynamics flexibly with simple yet effective operations applied before and after spatial attention, without adding additional parameters or increasing computational complexity. For the semantic side, we conduct spatiotemporal text augmentation by comprehensively constructing an Action Semantic Knowledge Graph (ASKG) to derive nuanced text prompts. The ASKG elaborates on static and dynamic concepts and their interrelations, based on the idea of decomposing actions into spatial appearances and temporal motions. During the training phase, the frame-level video representations are meticulously aligned with prompt-level nuanced text representations, which are concurrently regulated by the video representations from the frozen CLIP to enhance generalizability. Extensive experiments validate the effectiveness of our approach, which consistently surpasses state-of-the-art approaches on popular video benchmarks (i.e., Kinetics-600, UCF101, and HMDB51) under challenging ZSAR settings.

Video anomaly detection (VAD) is an important computer vision problem. Thanks to the mode coverage capabilities of generative models, the likelihood-based paradigm is catching growing interest, as it can model normal distribution and detect out-of-distribution anomalies. However, these likelihood-based methods are blind to the anomalies located in local modes near the learned distribution. To handle these ``unseen" anomalies, we dive into three gaps uniquely existing in VAD regarding scene, motion and appearance. Specifically, we first build a noise-conditioned score transformer for denoising score matching. Then, we introduce a scene-dependent and motion-aware score function by embedding the scene condition of input sequences into our model and assigning motion weights based on the difference between key frames of input sequences. Next, to solve the problem of blindness in principle, we integrate unaffected visual information via a novel autoregressive denoising score matching mechanism for inference. Through autoregressively injecting intensifying Gaussian noise into the denoised data and estimating the corresponding score function, we compare the denoised data with the original data to get a difference and aggregate it with the score function for an enhanced appearance perception and accumulate the abnormal context. With all three gaps considered, we can compute a more comprehensive anomaly indicator. Experiments on three popular VAD benchmarks demonstrate the state-of-the-art performance of our method.

—Weakly supervised video anomaly detection (WS-VAD) is challenging because it relies on video-level binary annotations to make frame-level predictions. Existing methods often convert WSVAD into a multiple instance learning (MIL) task, focusing on isolated segments that contribute most to the classification while neglecting the temporal context and detailed feature distinctions. In this paper, we propose a contrastive clustering strategy that enhances the representation of normal and abnormal features. Specifically, we treat the clustering center features and their corresponding categories as positive sample pairs, while features from different categories are treated as negative samples. This approach enables the network to better explore the distinction between normal and abnormal features. Furthermore, we address the bias in pre-trained models, where I3D pre-training features tend to overfit to normal videos and CLIP features exhibit a bias towards abnormal videos. To mitigate this, we introduce a simple early fusion method that combines pre-trained features to eliminate bias and obtain more comprehensive spatio-temporal representations. Extensive experiments on the UCF-Crime and XD-Violence datasets demonstrate the effectiveness of our approach, achieving state-of-the-art performance.

Autonomous aerial vehicle (AAV)-enabled Internet of Things (IoT) exhibits great application potential with its wide coverage, flexible network topology, and diversified services. However, ensuring communication security and efficient spectrum resource utilization in multiuser access scenarios is challenging, given the open nature of AAV channels and the proliferation of communication devices in IoT. To address the above challenges, this article proposes a novel reconfigurable intelligent surface (RIS)-aided AAV collaborative communication framework, where RIS-equipped AAV flexibly serves multiple users. In this work, a rate splitting multiple access (RSMA)-based secure transmission scheme is proposed, where the split public information serves both as useful signals and noise to disrupt eavesdropping. For the proposed scheme, a sum secrecy rate maximization problem is formulated and solved by optimally deploying the AAV’s location, designing the RIS’s phase shift, and power allocation. For this nonconvex problem with a couple of variables, we decompose it and form three separate subissues. Specifically, leveraging the successive convex approximation (SCA) and semidefinite relaxation (SDR) techniques, we first exploit an iterative algorithm for optimizing beamforming vectors and phase-shift matrix of RIS, and the optimal position of the AAV is obtained according to the deep deterministic policy gradient (DDPG). Then, we design an alternating optimization (AO) framework for joint solving. Finally, simulation results validate the efficacy of the proposed scheme in enhancing security, e.g., relative to the nonorthogonal multiple access (NOMA) scheme and benchmark scheme, the secrecy rate of the proposed scheme increased by 29.7% and 71.9%, respectively.

Artificial intelligence (AI) techniques are increasingly being used for the inverse design of microwave devices. However, several challenges, including intensive computation costs for training samples, high-dimensional data, nonuniformity, and low-quality samples in the design space, can negatively impact the final modeling performance. To alleviate these issues, a high-quality sampling inverse design scheme incorporating wavelet transform deep learning (HQS-WTDL) is proposed to achieve customized, automated microwave filter design. In the forward simulation-based sampling, particle swarm optimization (PSO) is used to tentatively select rule-defined samples. Multilabel synthetic minority over-sampling technique (MLSMOTE) is then applied to enlarge the training samples and improve their uniformity in the design space. An inverse modeling approach using neural networks to map the nonlinear relationship between a given set of S-parameters and required filter structural parameters is presented. Given that the dimension of the S-parameters is much higher than that of the structure parameters, the corresponding neural network used in this approach is deep and complex, with multiple layers and neurons. To reduce the number of input variables, the S-parameters are subjected to wavelet transformation, allowing for more efficient representation by the neural network. The proposed method is validated using a band-pass microstrip hairpin filter as an example. Results demonstrate that the proposed approach achieves better modeling effectiveness and inverse design efficiency than conventional methods. In addition, the proposed method allows for fast customization of device parameters, such as center frequencies and bandwidths with good prediction accuracy.

Egocentric action anticipation is a challenging task that aims to make advanced predictions of future actions from current and historical observations in the first-person view. Most existing methods focus on improving the model architecture and loss function based on the visual input and recurrent neural network to boost the anticipation performance. However, these methods, which merely consider visual information and rely on a single network architecture, gradually reach a performance plateau. In order to fully understand what has been observed and capture the dependencies between current observations and future actions well enough, we propose a novel visual-semantic fusion enhanced and Transformer-GRU-based action anticipation framework in this paper. Firstly, high-level semantic information is introduced to improve the performance of action anticipation for the first time. We propose to use the semantic features generated based on the class labels or directly from the visual observations to augment the original visual features. Secondly, to take advantage of both the parallel and autoregressive models, we design a Transformer-based encoder for long-term sequential modeling and a GRU-based decoder for flexible iteration decoding. This hybrid architecture allows for better performance with fewer parameters and computations. Thirdly, an effective visual-semantic fusion module is proposed to make up for the semantic gap and fully utilize the complementarity of different modalities. Extensive experiments on two large-scale first-person view datasets and two third-person datasets validate the effectiveness of our proposed method, which achieves new state-of-the-art performance, outperforming previous approaches by a large margin. The code will be released after acceptance athttps://github.com/sunze992/VS-TransGRU.

It is a challenging task for a continuous-wave Doppler radar sensor (DRS) to linearly separate independent motions without any prior information. In this paper, a low-IF single-input multiple-output (SIMO) DRS system is designed and implemented for motion separation. The SIMO DRS system detects the signals scattered off multiple objects, and a two-step blind motion separation approach is proposed to separate motions from the DRS output. Experiments show that the SIMO DRS and the proposed motion separation approach can successfully separate combinations of triangular, sinusoidal, and random motions, as long as the velocities or initial phases of such motions are different. With the two-step approach sequentially tackling the signal separation and the nonlinear demodulation problems, both small- and large-scale independent motions can be linearly separated. These results imply the potential in practical applications of short-range tracking and localization of moving objects.