Vision-language pre-training (VLP) methods are blossoming recently, and its crucial goal is to jointly learn visual and textual features via a transformer-based architecture, demonstrating promising improvements on a variety of vision-language tasks. Prior arts usually focus on how to align visual and textual features, but strategies for improving the robustness of model and speeding up model convergence are left insufficiently explored.In this paper, we propose a novel method ViLTA, comprising of two components to further facilitate the model to learn fine-grained representations among image-text pairs. For Masked Language Modeling (MLM), we propose a cross-distillation method to generate soft labels to enhance the robustness of model, which alleviates the problem of treating synonyms of masked words as negative samples in one-hot labels. For Image-Text Matching (ITM), we leverage the current language encoder to synthesize hard negatives based on the context of language input, encouraging the model to learn high-quality representations by increasing the difficulty of the ITM task. By leveraging the above techniques, our ViLTA can achieve better performance on various vision-language tasks. Extensive experiments on benchmark datasets demonstrate that the effectiveness of ViLTA and its promising potential for vision-language pretraining.

Abstract Identifying anomalies, especially weak anomalies in constantly changing targets, is more difficult than in stable targets. In this article, we borrow the dynamics metrics and propose the concept of dynamics signature (DS) in multi‐dimensional feature space to efficiently distinguish the abnormal event from the normal behaviors of a variable star. The corresponding dynamics criterion is proposed to check whether a star's current state is an anomaly. Based on the proposed concept of DS, we develop a highly optimized DS algorithm that can automatically detect anomalies from millions of stars' high cadence sky survey data in real‐time. Microlensing, which is a typical anomaly in astronomical observation, is used to evaluate the proposed DS algorithm. Two datasets, parameterized sinusoidal dataset containing 262,440 light curves and real variable stars based dataset containing 462,996 light curves are used to evaluate the practical performance of the proposed DS algorithm. Experimental results show that our DS algorithm is highly accurate, sensitive to detecting weak microlensing events at very early stages, and fast enough to process 176,000 stars in less than 1 s on a commodity computer.

Abstract Commonly used convolutional neural networks (CNNs) usually compress high‐resolution input images. Although it reduces the computation requirements into a reasonable range, the downsampling operation causes information loss, which affects the accuracy of image classification. How to adopt high‐resolution image inputs to improve the quality of input information and thus improve the classification accuracy without changing the overall structure of the pre‐defined CNN model or increasing the model parameters is an important issue. Here, a CNN model with wavelet domain inputs is proposed to provide a solving scheme. Specifically, the proposed method applies wavelet packet transform or dual‐tree complex wavelet transform to extract information from input images with higher resolutions in the image pre‐processing stage. Some subband image channels are selected as the inputs of conventional CNNs where the first several convolutional layers are removed, so that the networks directly learn in the wavelet domain. Experiment results on the Caltech‐256 dataset and the Describable Textures Dataset with the ResNet‐50 show that the classification accuracy of our method can have a maximum improvement of 2.15% and 10.26%, respectively. These validate the effectiveness of our proposed scheme. This code is publicly available at https://github.com/BeBeBerr/wavelet‐cnn .

In the field of automated home video editing, exploring the dependence relations between who (character) and where (scene) makes great sense to end-users for content selection. However, such techniques have not been well developed in real applications due to their computational intensity. The emerging multi-core architectures provide an opportunity to speed up those compute expensive algorithms if shift from serial thinking to parallelism. This demonstration presents a scalable parallel system for home video editing. In a realtime processing speed, the system analyzes how many characters and scenes are captured and provides end-users with flexible preference customization. Through kernel module optimization and data-level parallelization, evaluations on a real 8-core machine indicates a near linear speed up could be achieved along with the increasing number of cores.