Vision-Language Pretraining (VLP) has significantly improved the performance of various vision-language tasks with the matching of images and texts. In this paper, we propose VL-Match, a Vision-Language framework with Enhanced Token-level and Instance-level Matching. At the token level, a Vision-Language Replaced Token Detection task is designed to boost the substantial interaction between text tokens and images, where the text encoder of VLP works as a generator to generate a corrupted text, and the multimodal encoder of VLP works as a discriminator to predict whether each text token in the corrupted text matches the image. At the instance level, in the Image-Text Matching task that judges whether an image-text pair is matched, we propose a novel bootstrapping method to generate hard negative text samples that are different from the positive ones only at the token level. In this way, we can force the network to detect fine-grained differences between images and texts. Notably, with a smaller amount of parameters, VL-Match significantly outperforms previous SOTA on all image-text retrieval tasks.

Single Image Super-Resolution (SISR) based on deep learning methods has been widely studied for applications on remote sensing images. With limited remote sensing images, most of the existing SISR methods simply adopt the regular data augmentation approaches (such as flip) in natural images to improve model performance. Considering the fact that remote sensing images are all taken from a bird’s-eye view and objects appear in multiple directions, we first introduce rotation augmentation method in remote sensing images to promote diversity of samples dramatically, as rotation does not cause semantic problems like people standing upside down in natural images. However, image rotation at various angles implemented by interpolation will cause the inconsistent pixel distribution problem for the pixel level task. Thus, we propose Transformation Consistency Loss Function (TCLF) to narrow the gap between the augmented and original distribution, while expanding the feature space with rotation augmentation method. Extensive experiments are performed on UC-Merced Land-use dataset of 21 remote sensing scenes, and the results as well as ablation studies demonstrate our proposed method outperforms mainstream methods.

This paper proposes a deep learning-based remote sensing image segmentation method for estimating the proportion of urban public space, which is an important urban planning problem. Remote sensing images contain diverse landforms and different scales of objects, making image segmentation a challenging task. Most current image segmentation methods use convolutional neural networks, which are deep neural networks that can automatically learn image features and perform classification or regression. However, existing convolutional neural networks are usually pre-trained on natural image datasets such as ImageNet, which are very different from remote sensing images, resulting in pre-trained models that cannot fully exploit the characteristics of remote sensing images. To address this issue, this paper proposes a Mixlabel Autoencoder (MLAE) to further pre-train remote sensing images by image reconstruction. Unlike natural images, remote sensing images are complex and difficult to reconstruct; therefore, we use partial labels to guide the reconstruction process. Our method involves replacing random patches of the input image with corresponding labels and reconstructing the patches using an encoder-decoder architecture. Experimental results show that our method achieves higher segmentation accuracy and better visual effects in downstream tasks. Our method provides valuable guidance for urban planning and construction by identifying the proportion of pixels within each type of area in an image.

Although single image super-resolution(SISR) method with deep neural network has already been explored in depth in natural images, further research on SISR in remote sensing image is still desired as aerial imagery has distinctive characteristics such as varied scenes across wide areas. In this paper, considering the scarcity of remote sensing images that may restrict the performance of data-driven neural network, we first propose a new data augmentation method, RotBlur, to promote sample diversity dramatically with a rotated cropped block at random angle. And we also design a Dynamic Multi-Scale Network(DMSN) to enhance details of low-resolution(LR) remote sensing images adaptively according to current scene of various images. Experiments performed on the UC Merced Land-Use dataset demonstrate that our DMSN outperforms several state-of-the-art methods in terms of PSNR of reconstructed images.

High-quality remote sensing images are difficult to obtain due to limited conditions and high cost for data acquisition. With the development of machine vision and deep learning, some image generation methods (e.g., GANs) are introduced into this field, but it’s still hard to generate images with good texture details and structural dependencies. We establish Skip Attention Mechanism to deal with this problem, which learns dependencies between local points on low-resolution feature maps, and then upsample the attention map and combine it with high-resolution feature maps. With this method, long-range dependencies learned from low-resolution are used for generating remote sensing images with more structural details. We name this method as Skip Attention GAN, which is the first method applying cross-scale attention mechanism for unsupervised remote sensing image generation. Experiments show that our method outperforms previous methods under several metrics. Visual and ablation results of attention layers show that Skip Attention has learned long-distance structural dependencies between similar targets.

Recently, multi-resolution networks (such as Hourglass, CPN, HRNet, etc.) have achieved significant performance on pose estimation by combining feature maps of various resolutions. In this paper, we propose a Resolution-wise Attention Module (RAM) and Gradual Pyramid Refinement (GPR), to learn enhanced resolution-wise feature maps for precise pose estimation. Specifically, RAM learns a group of weights to represent the different importance of feature maps across resolutions, and the GPR gradually merges every two feature maps from low to high resolutions to regress final human keypoint heatmaps. With the enhanced resolution-wise features learnt by CNN, we obtain more accurate human keypoint locations. The efficacies of our proposed methods are demonstrated on MS-COCO dataset, achieving state-of-the-art performance with average precision of 77.7 on COCO va12017 set and 77.0 on test-dev2017 set without using extra human keypoint training dataset.

In recent years Transfer Learning has become an important research issue and it is employed to solve the lack of data problem in remote sensing field. As one kind of the most popular generative models, Generative Adversarial Networks (GANs) can generate hierarchy representations and realistic textures of objects. Now GANs have become the most successful and usual way to synthesize images. In this paper we train different GANs and generate images on the UC Merced Land Use Dataset and the aircraft carrier dataset compiled by ourselves. Our images are better than the ones generated by MARTA GAN in remote sensing field. We also creatively adopt the Image Similarity Deep Ranking model (ISDR) to evaluate our GANs and reuse high-quality generated images in an objective detection task. We train the object detection model SSD on our dataset. With the augmentation of generated images, the SSD model achieves a better detection accuracy, which means GANs are beneficial for addressing the lack of training data problem in remote sensing field.

MOTIVATION: There have been several studies on the micro-inversions between human and chimpanzee, but there are large discrepancies among their results. Furthermore, all of them rely on alignment procedures or existing alignment results to identify inversions. However, the core alignment procedures do not take very small inversions into consideration. Therefore, their analyses cannot find inversions that are too small to be detected by a classic aligner. We call such inversions pico-inversions. RESULTS: We re-analyzed human-chimpanzee alignment from the UCSC Genome Browser for micro-inplace-inversions and screened for pico-inplace-inversions using a likelihood ratio test. We report that the quantity of inplace-inversions between human and chimpanzee is substantially greater than what had previously been discovered. We also present the software tool PicoInversionMiner to detect pico-inplace-inversions between closely related species. AVAILABILITY: Software tools, scripts and result data are available at http://faculty.cs.niu.edu/~hou/PicoInversion.html. CONTACT: [email protected].

With the digital technology developments, Digital Cinema outperforms traditional films and attracts the movie industry. The first edition Digital Cinema System Specification was released by Digital Cinema Initiatives, LLC (DCI) in 2005, which advances the Digital Cinema development. We have developed a Digital Cinema Playback system compliant with the DCI Specification. It can extract and play the 2 K-resolution component from the 2 K/4 K DCP (Digital Cinema Package) files at 24 fps, and support an up to 350 Mbps video streaming service. Connected to Barco projector and Dolby digital device, this system can display the standard fully or partially encrypted test materials fluently and correctly. The synchronization of audio and video meets the DCI Specification.

The Cell Broadband Engine is a typical heterogeneous chip multiprocessor which provides potential high performance for computing-intensive applications. Our researches focus on how to use Cell to speed up block cryptography applications. In this paper, we propose a mapping framework for block cryptography working in ECB mode and corresponding optimizing strategy. We take four algorithms(RC5, 3DES, AES, and Twofish) as benchmark and implement these four algorithms using Cell programming language. In order to enhance the performance, we present an optimizing strategy and evaluate the effects of the optimizing methods including compiler optimization, dual buffering, vectorization, and loop unrolling. The experiments indicate that all these four algorithms can obtain 5-20 times speedup compared with traditional processors, which shows that our mapping framework and optimizing strategy are effective for the block cryptography algorithms.