Point-level weakly-supervised temporal action localization (P- TAL) aims to localize action instances in untrimmed videos through the use of single-point annotations in each instance. Existing methods predict the class activation se-quences without any boundary information, and the unreli-able sequences result in a significant misalignment between the quality of proposals and their corresponding confidence. In this paper, we surprisingly observe the most salientframe tend to appear in the central region of the each instance and is easily annotated by humans. Guided by the temporal saliency information, we present a novel proposal-level plug-in framework to relearn the aligned confidence of proposals generated by the base locators. The proposed approach consists of Center Score Learning (CSL) and Alignment-based Boundary Adaptation (ABA). In CSL, we design a novel center label generated by the point annotations for predicting aligned center scores. During inference, we first fuse the center scores with the predicted action probabilities to obtain the aligned confidence. ABA utilizes the both aligned confidence and IoU information to enhance localization completeness. Extensive experiments demon-strate the generalization and effectiveness of the proposed framework, showcasing state-of-the-art or competitive per-formances across three benchmarks. Our code is available at https://github.com/zyxial0091CVPR2024-TspNet.

Semantic segmentation of high-resolution remote sensing (HRRS) images is an important task in the field of remote sensing image analysis. However, the presence of a large number of complex ground objects in HRRS images poses challenges for its semantic segmentation. In this paper, we propose a high-order transformer semantic segmentation network (HOT-Net) for HRRS images. The network uses ResNet-50 as backbone to capture local features at the encoder stage. In order to expand the receptive field of the network and enhance its ability to perceive global contextual information, several proposed high-order transformer blocks are uesd to establish global dependencies in local features. Global enhancement attention modules (GEAM) are used to enhance the representation of global features during the decoder stage. We conducte ablation and comparative experiments on the LoveDA dataset, and the experimental results show that our method has excellent performance compared to other popular methods.

Online action detection aims to identify ongoing actions within untrimmed video streams, with extensive applications in real-life scenarios. However, in practical applications, video frames are received sequentially over time and new action categories continually emerge, giving rise to the challenge of catastrophic forgetting - a problem that remains inadequately explored. Generally, in the field of video understanding, researchers address catastrophic forgetting through class-incremental learning. Nevertheless, online action detection is based solely on historical observations, thus demanding higher temporal modeling capabilities for class-incremental learning methods. In this paper, we conceptualize this task as Class-Incremental Online Action Detection (CIOAD) and propose a novel framework, TS-ILM, to address it. Specifically, TS-ILM consists of two components: task-level temporal pattern extractor and temporal-sensitive exemplar selector. The former extracts the temporal patterns of actions in different tasks and saves them, allowing the data to be comprehensively observed on a temporal level before it is input into the backbone. The latter selects a set of frames with the highest causal relevance and minimum information redundancy for subsequent replay, enabling the model to learn the temporal information of previous tasks more effectively. We benchmark our approach against SoTA class-incremental learning methods applied in the image and video domains on THUMOS'14 and TVSeries datasets. Our method outperforms the previous approaches.

In the domain of remote sensing (RS) image analysis, capturing global context is the key for precise semantic segmentation. Current vision transformer (ViT) advance this field by addressing convolutional neural network’s (CNN) local receptive field limitations. However, ViT predominantly rely on the first-order information in image to establish global relationships, often overlooking the potential of second-order information, which is crucial for enhancing the discrimination of ground objects that exhibit high similarity and constant changes. To address this issue, we propose a global adaptive second-order transformer network (GASOT-Net). Specifically, the proposed global adaptive second-order transformer (GASOT) enhances the existing ViT structure by mining second-order information and adaptively fusing it with the first-order information during the process of establishing global dependency relationships. This approach enables the extraction of more discriminative features, thereby enriching the representation of global features. In addition, the local feature aggregation module (LFAM) is proposed to effectively aggregate features from different stages of CNN as input to the GASOT blocks. Moreover, to refine boundaries of complex ground objects, the global feature enhancement module (GFEM) is used in the decoder stage. In particular, GFEM includes two sub modules—feature shift module (FSM) and hierarchical feature fusion module (HFFM). FSM is used to enhance the local feature representation at first, and then, HFFM hierarchically aggregates local and global features from different stages. We conduct extensive experiments on four benchmark RS datasets, and the results show that our GASOT-Net outperforms other state-of-the-art methods. The code will be available at:https://github.com/j136812832/GASOT-Net.

Online action detection plays a vital role in video action understanding and can be widely used in various video analysis applications. This task aims to detect actions at the current moment within long untrimmed video streams. However, accurately identifying action-background transitions that are ambiguous in terms of time during detection can be challenging due to the similarity between the action and background clips, adding to the difficulty in finding a suitable division between them. To address this issue, we propose a hard video clip mining method based on deep metric learning for online action detection named HCM. The HCM method first selects video clips that are hard to distinguish to determine the optimization objects. Then, a hard clip mining loss is adopted to push the features toward the centers of the categories to which they belong and away from others. Furthermore, we introduce an intra-class feature compaction loss to constrain the divergence of action features, ensuring the stability of their distribution. We evaluated the proposed method on two challenging online action detection datasets, THUMOS14 and TVSeries. The results show that HCM is effective and efficient in online action detection and action anticipation tasks.

Although the convolutional neural network (CNN)-based and vision transformer (ViT)-based methods have achieved effective remote sensing scene classification results in the past few years, the CNN's inductive bias and the ViT's single spatial scale limit the further improvement of accuracy. To address these problems, in this paper, we proposed a novel multi-scale vision transformer (MS-ViT) for remote sensing scene image classification, consists of two different spatial scale information streams that extract spatial multi-scale information, a hybrid attention module that fuses the information and a classifier to predict the scene category. We evaluated the effectiveness of our proposed method on two different remote sensing datasets, namely NWPU-RESISC45 and AID. The experimental results also show that our method outperforms CNN-based methods and the original ViT-based method in performance.