Low-precision computation is an effective method to improve energy efficiency when processing AI models at the edge. The design of numeric format is important to maintain good accuracy while reducing energy consumption. However, current fixed-point based formats or floating-point based formats have either limitations in representation range or precision, and thus efficiency or accuracy is compromised. Posit formats can achieve both large dynamic range and high precision, however, the computation overhead is too high. Inspired by the recent microscaling format, in this paper, a novel microscaling posit format and its corresponding dot-product based processing element are proposed. By designing a specific format, the dotproduct computation overhead of the original posit format is significantly reduced. Implementation results show that the proposed processing element can achieve up to $79 \%$ area reduction and $74 \%$ power reduction when compared with other designs available in the literature, which makes the proposed designs especially suitable for edge AI computation.

As the fundamental unit of life, cells coordinate biological activities through the interaction between microscopic molecular mechanisms and macroscopic tissue organization. Traditional research studies, experiments, and biochemical analyses, give rise to important insights, although they are restricted in spatiotemporal resolution and processing power, thereby precluding the understanding of dynamic cross-scale biological events . Breakthroughs in artificial intelligence (AI) have given birth to the AI virtual cell (AIVC) as a new way to do research. By integrating multi-omics data and mixing methods from multidisciplinary models, AIVC establishes a digital twin system to simulate cell functions and behaviors. AIVC still faces a number of pressing challenges that need to be addressed in its current development stage. In this review, we are proposing a unified definition and technical framework for AIVC and analyze in detail the cross-scale coupling mechanisms of the "gene-protein-pathway-cell" hierarchy. Furthermore, we decompose the technical construction framework of AIVC from cross-scale representation engineering, functional submodule design, and multi-component dynamic regulation mechanisms. Additionally, we summarize the existing models and datasets in the field to provide reference resources for researchers. Finally, we deeply discuss the challenges faced by AIVC, such as data heterogeneity and model interpretability, and aim to accelerate the research progress in the AIVC field while driving the life sciences to shift from observational analysis to a paradigm that integrates predictability and innovation. Despite being in the early stage, AIVC is a trending topic that has garnered widespread interest. This review aims to integrate existing models, datasets, and technical ideas to provide a unified framework for field development.

The self-supervised contrastive learning strategy has attracted considerable attention due to its exceptional ability in representation learning. However, current contrastive learning tends to learn global coarse-grained representations of the image that benefit generic object recognition, whereas such coarse-grained features are insufficient for fine-grained visual recognition. In this paper, we incorporate subtle local fine-grained feature learning into global self-supervised contrastive learning through a pure self-supervised global-local fine-grained contrastive learning framework. Specifically, a novel pretext task called local discrimination (LoDisc) is proposed to explicitly supervise the self-supervised model’s focus toward local pivotal regions, which are captured by a simple but effective location-wise mask sampling strategy. We show that the LoDisc pretext task can effectively enhance fine-grained clues in important local regions and that the global-local framework further refines the fine-grained feature representations of images. Extensive experimental results on different fine-grained object recognition tasks demonstrate that the proposed method can lead to a decent improvement in different evaluation settings. The proposed method is also effective for general object recognition tasks.

Accurate prediction of protein-ligand affinity (PLA) is critical for drug discovery. Recent deep learning approaches have adopted data-driven models for PLA prediction by learning intrinsic patterns from one-dimensional (1D) sequential or two-dimensional (2D) graph representations of proteins and ligands. However, these low-dimensional methods overlook the three-dimensional (3D) geometric features, which are hypothesized to be critical in binding interaction. To address the above problem, we present a Geometric deep learning approach with Hybrid message passing strategies--HybridGeo, for protein-ligand affinity prediction. We adopt dual-view graph learning to model the intra- and inter-molecular atomic interactions and propose to aggregate the spatial information with hybrid strategies. In addition, to fully model the inter-residue dependency upon message aggregation, we adopt a geometric graph transformer on the residue-scale graph of protein pockets. Extensive experiments on the PDBbind dataset show that HybridGeo achieves state-of-the-art performance with a Root Mean Square Error (RMSE) of 1.172. HybridGeo also achieves the best among all baseline models on three external test sets, showcasing good generalizability and robustness. Through systematic ablation experiments, we validated the effectiveness of the proposed modules, and further demonstrated the superior performance of HybridGeo in predicting the binding affinity of macrocyclic compound complexes through case studies. Visualization analysis further indicates the biological interpretability of the model predictions.

Recent advances have shown great promise in mining multimodal protein knowledge for better protein-protein interaction (PPI) prediction by enriching the representation of proteins. However, existing solutions lack a comprehensive consideration of both local patterns and global dependencies in proteins, hindering the full exploitation of modal information. Additionally, the inherent disparities between modalities are often disregarded, which may lead to inferior modality complementarity effects. To address these issues, we propose a cross-modality enhanced PPI prediction method from the perspectives of protein sequence and structure modalities, namely SSPPI. In this framework, our main contribution is that we integrate both sequence and structural modalities of proteins and employ an alignment and fusion method between modalities to further generate more comprehensive protein representations for PPI prediction. Specifically, we design two modal representation modules (Convformer and Graphormer) tailored for protein sequence and structure modalities, respectively, to enhance the quality of modal representation. Subsequently, we introduce a Cross-modality enhancer module to achieve alignment and fusion between modalities, thereby generating more informative modal joint representations. Finally, we devise a cross-protein fusion (CPF) module to model residue interaction processes between proteins, thereby enriching the joint representation of protein pairs. Extensive experimentation on four benchmark datasets demonstrates that our proposed model surpasses all current state-of-the-art (SOTA) methods. The source codes are publicly available at the following link https://github.com/bixiangpeng/SSPPI/.

In this study, we propose a pre-trained graph neural network-based model, called SuperCardioTox, for predicting hERG cardiotoxicity. The model employs a high-quality dataset derived from the CHEMBL database to quantify the inhibitory capacity of compounds through pIC50 values, and uses the GraphMAE graph autoencoder to learn the molecular structure characterisation of drugs. Its functionality is based on the masking of node features and their reconstruction, thereby enabling the learning of the intrinsic laws governing graph data. Furthermore, a drug toxicity prediction model that integrates graph neural network (GNN) and multilayer perceptron (MLP) has been developed to achieve accurate drug toxicity prediction by extracting deep features of graph data. The objective of this study is to enhance the precision of drug design and toxicity prediction through the utilisation of extensive unlabelled molecular data, thereby providing robust support for novel drug development and ensuring the safety of patients' medication. The experimental results demonstrate that the model demonstrates robust performance in drug toxicity prediction, thereby validating its utility and generalisability.

Understanding the functions of proteins is of great importance for deciphering the mechanisms of life activities. To date, there have been over 200 million known proteins, but only 0.2% of them have well-annotated functional terms. By measuring the contacts among residues, proteins can be described as graphs so that the graph leaning approaches can be applied to learn protein representations. However, existing graph-based methods put efforts in enriching the residue node information and did not fully exploit the edge information, which leads to suboptimal representations considering the strong association of residue contacts to protein structures and to the functions. In this article, we propose SuperEdgeGO, which introduces the supervision of edges in protein graphs to learn a better graph representation for protein function prediction. Different from common graph convolution methods that uses edge information in a plain or unsupervised way, we introduce a supervised attention to encode the residue contacts explicitly into the protein representation. Comprehensive experiments demonstrate that SuperEdgeGO achieves state-of-the-art performance on all three categories of protein functions. Additional ablation analysis further proves the effectiveness of the devised edge supervision strategy. The implementation of edge supervision in SuperEdgeGO resulted in enhanced graph representations for protein function prediction, as demonstrated by its superior performance across all the evaluated categories. This superior performance was confirmed through ablation analysis, which validated the effectiveness of the edge supervision strategy. This strategy has a broad application prospect in the study of protein function and related fields.

Frequent occurrences of marine extreme climate and weather events pose significant threats to human life and property, underscoring the practical significance of meteorological data forecasting methods. Notably, significant advancements in meteorological forecasting fields have been achieved by data-driven deep learning techniques, which leverage observed meteorological datasets and employ deep networks to capture complex patterns. However, challenges remain in accurately extracting local details and capturing spatial-temporal correlations when dealing with multiple meteorological forecasting tasks that exhibits diverse temporal and spatial scales. Hence, in this paper, we propose a Multi-Scale Spatial Temporal Transformer (MS-STT) framework to achieve efficient and accurate meteorological data forecasting. Specifically, to achieve more detailed and multi-scale representation of meteorological data, we design the regionally coherent encoding strategy and multi-scale feature aggregation for visual representation. To enhance the multi-scale ability in terms of learning spatial-temporal correlations, we propose a multi-scale spatial-temporal transformer network, which integrates a multi-scale spatial transformer to learn the spatial association between local patches and multi-scale regions and a temporal transformer to learn the temporal dynamic evolution properties. Extensive quantitative and qualitative experiments on three popular spatial temporal forecasting tasks validate the effectiveness of the proposed method. In particular, compared to the representative data-driven deep learning ENSO forecasting method Earthformer, our approach achieves a 3.7% performance improvement with only one-third of the parameters.

Phytoplankton, a crucial component of aquatic ecosystems, requires efficient monitoring to understand marine ecological processes and environmental conditions. Traditional phytoplankton monitoring methods, relying on non-in situ observations, are time-consuming and resource-intensive, limiting timely analysis. To address these limitations, we introduce PhyTracker, an intelligent in situ tracking framework designed for automatic tracking of phytoplankton. PhyTracker overcomes significant challenges unique to phytoplankton monitoring, such as constrained mobility within water flow, inconspicuous appearance, and the presence of impurities. Our method incorporates three innovative modules: a Texture-enhanced Feature Extraction (TFE) module, an Attention-enhanced Temporal Association (ATA) module, and a Flow-agnostic Movement Refinement (FMR) module. These modules enhance feature capture, differentiate between phytoplankton and impurities, and refine movement characteristics, respectively. Extensive experiments on the PMOT dataset validate the superiority of PhyTracker in phytoplankton tracking, and additional tests on the MOT dataset demonstrate its general applicability, outperforming conventional tracking methods. This work highlights key differences between phytoplankton and traditional objects, offering an effective solution for phytoplankton monitoring.

Tropical cyclones (TCs) are one of the most destructive climatic phenomena, making accurate estimation of their intensity crucial for assessing disaster risks. However, due to the complex and indistinguishable structure of their satellite images, existing deep learning methods struggle to differentiate between images of varying intensity, which leads to unsatisfactory accuracy in intensity estimation. In this article, we propose a novel knowledge-guided prompt learning (KGPL) method, KGPL, for TC intensity estimation. KGPL utilizes a pretrained VLM to encode both historical intensity and current satellite image for estimating TC intensity. To minimize domain disparities and facilitate the transfer of a general large model to the TC domain, we introduce a cross-modal interactive prompt learning strategy. Specifically, we embed shared prompts in the text and vision encoders, aiming to learn domain knowledge and promote collaboration and interaction between these two modalities. Furthermore, we design a subregion contrastive learning strategy, which sets constraints on the intensity differences of convective activity in different subregions and guides the model to focus on learning strong convective areas such as the eye and eyewall. Extensive experiments show that our KGPL achieves a significant 41.1% reduction in root mean square error (RMSE) with only one-ninth of the trainable parameters compared to the state-of-the-art method, which validates the effectiveness of our method for TC intensity estimation. The code and data are available athttps://github.com/LiYue-TC/KGPL.

Accurate prediction of Sea Surface Temperature (SST) plays a crucial role in climate research, resource development, marine disaster prevention, and environmental protection. Traditional numerical models, while demonstrating excellent predictive accuracy, heavily rely on precise initial and boundary conditions. In contrast, data-driven approaches compensate for these shortcomings with their flexibility and lower computational costs; however, the lack of physical constraints and interpretability limits their effectiveness. Therefore, enhancing model efficiency and interpretability while ensuring accuracy has become a key challenge in current research. This paper proposes a coupled dual-stream SST prediction model that integrates numerical simulation and data-driven methods to improve predictive accuracy and physical consistency. The model includes a bidirectional information compensation module, a physical equation constraint module, and an anomaly compensation module. First, the bidirectional information compensation module constructs physical information flow and complex process flow using Conv-LSTM networks and designs an interaction mechanism between the two streams to enhance the model’s ability to capture complex processes that cannot be explained by partial differential equations. Next, the physical equation constraint module employs data assimilation techniques based on partial differential equations to simulate the dynamics of energy transfer in fluids, ensuring that the prediction results adhere to physical laws. Finally, the anomaly compensation module integrates the prediction results of physical processes and complex processes to output the final SST prediction. Experimental results demonstrate that, compared to existing methods, this model achieves superior predictive accuracy across multiple datasets.

In recent years, global climate change has intensified, leading to frequent occurrences of short-term extreme rain-fall events. How to accurately predict short-term precipitation within the next few hours has become a concern for people, especially for meteorologists. Traditional Numerical Weather Prediction (NWP) models rely on solving complex physical equations, resulting in significant computational costs and low temporal resolution, which limit their suitability for nowcasting tasks. Weather radar echo extrapolation, which forecasts future echoes from historical observations, has thus become the predominant approach for precipitation nowcasting. In the realm of deep learning-based methods, convolutional or recurrent neural network-based extrapolation pipelines inherently struggle in processing sequential data and capturing the multi-scale spatial features inherent in radar echo images. Additionally, current models often focus on enhancing dense prediction capabilities while neglecting to mine or utilize the evolutionary patterns of historical echoes, leading to diminished accuracy in long-term sequence forecasting. Moreover, high precipitation areas are critical for public safety, but current models often fail to forecast them accurately due to insufficient emphasis during learning. In this paper, we propose a memory-aware multi-scale predictive transformer (MMPFormer) for precipitation nowcasting. Specifically, we designed the Multi-Scale Spatial-Temporal Transformer (MS-STT), which incorporates multi-scale convolution techniques alongside self-attention mechanisms to effectively extract and model the spatiotemporal correlations within historical radar echoes. Additionally, we have designed a ConvGRU-Instructed Memory Mechanism (CIMM) to alleviate the degradation of radar echo details during extended extrapolation periods, enabling accurate forecasts over the next three hours. Furthermore, the Key Area Attention Module (KAAM), a plug-and-play module, has been introduced. It emphasizes high precipitation areas through attention mechanisms, mitigating the negative impacts of missing high precipitation by previous learning-based methods. Quantitative and qualitative experimental results on radar echo datasets demonstrate the superior performance of our method in modeling spatiotemporal dynamics and long-term extrapolation for precipitation forecasting.

Ocean data has strong spatiotemporal correlations and is also characterized by multi-source, decentralized distribution, presenting significant challenges for distributed collaboration. To address these issues, in this paper, we propose DGLN, a distributed GCN-based prediction architecture that employs a decentralized training paradigm comprising local and distributed systems. The local system is designed to capture scale-specific features of marine data across different processors, while the distributed system learns and coordinates global cross-timescale features. The graph learning module in the distributed system integrates graph features from other processors without transmitting raw data, thereby ensuring local data security. To accelerate training and enhance model performance, a joint learning module is composed by cross-timescale computation with a new global weighted consensus algorithm, which is able to adaptively adjust the weights of different scales. Experimental results show that DGLN is able to improve the local prediction accuracy and capture cross-timescale features effectively while improving the training efficiency.

Recently, remote sensing text-image retrieval has gained increasing attention among researchers due to its capability to provide abundant, inclusive, and multi-perspective information. To extract the salient representation of the two modalities and realize information alignment, existing methods usually apply convolution calculation, matrix multiplication, and attention mechanisms to model features in the spatial domain. However, unlike natural images, remote sensing data contain a considerable amount of noise, and the spatial domain calculation methods will lead to information smoothing and noise exchange, ultimately reducing the robustness of feature representation. In this article, a Frequency and Spatial-domain Saliency Network (FSSN) is proposed by extracting saliency features in both frequency and spatial domains to further enhance the efficiency of retrieval networks. Specifically, the FSSN first designs a Frequency-domain Intra-modal Low-pass Filter (FILF) by using the Fourier Transform to convert the spatial domain representation into the frequency domain representation, and leverage low-pass filtering to filter out the noise information of the image. Afterwards, the Space-domain Inter-modal Boundary-based Saliency module (SIBS) is devised, which fully utilizes the positive and negative boundaries and draws an innovative assessment mechanism to automatically recognize the effective regions and words without noise shifting in the spatial domain. Finally, the Frequency and Spatial-domain Saliency Fusion (FSSF) is designed to realize the effective integration of features obtained from the frequency and spatial domain according to the consistency of the identity masks. Quantitative and qualitative experiments are implemented on four remote sensing benchmarks to showcase the notable effectiveness achieved by the joint modeling network of multiple domains. Specifically, the proposed FSSN outperforms the best model RemoteClip by 7.58% of the mR evaluation matrix on the UCM dataset.

In recent years, cross-modal text-image retrieval in remote sensing has gained prominence as a research focus due to its potential to provide abundant, inclusive, and multi-perspective information. However, existing methods usually focus on salient features, but these salient features cannot describe the image or text completely, resulting in the loss of some important details and discriminable information. In this article, a Whole Semantic Sparse Coding Network (WSSCN) is proposed for remote sensing image-text retrieval to build a complete and reliable features description for further improving the performance of the retrieval model. Specifically, the WSSCN first designs a Whole Semantic Sparse Representation Coding (WSSRC) module by constructing a robust semantic library to transform the dense features matrix of the image and text into a whole semantic sparse matrix that enables multiple semantic decoupling and leads to a more precise and detailed features expression. Afterwards, the Intra- and Inter-Modal Consistency (IIMC) module is devised to improve the intra-modal and inter-modal consistency of the whole semantic sparse representation from different models. Finally, the Salient and Whole Semantic Adaptive Learning (SWSAL) loss is proposed to focus on salient information or whole semantic information by calculating an adjusted parameter. Quantitative and qualitative experiments are performed on four extensive datasets for cross-modal retrieval in remote sensing to showcase the notable effectiveness achieved by implementing a whole semantic sparse coding network.

Accurately identifying drug-target affinity (DTA) plays a significant role in promoting drug discovery and has attracted increasing attention in recent years. Exploring appropriate protein representation methods and increasing the abundance of protein information is critical in enhancing the accuracy of DTA prediction. Recently, numerous deep learning-based models have been proposed to utilize the sequential or structural features of target proteins. However, these models capture only the low-order semantics that exist in a single protein, while the high-order semantics abundant in biological networks are largely ignored. In this article, we propose HiSIF-DTA-a hierarchical semantic information fusion framework for DTA prediction. In this framework, a hierarchical protein graph is constructed that includes not only contact maps as low-order structural semantics but also protein-protein interaction (PPI) networks as high-order functional semantics. Particularly, two distinct hierarchical fusion strategies (i.e., Top-down and Bottom-Up) are designed to integrate the different protein semantics, therefore contributing to a richer protein representation. Comprehensive experimental results demonstrate that HiSIF-DTA outperforms current state -of-the-art methods for prediction on the benchmark datasets of the DTA task. Further validation on binary tasks and visualization analysis demonstrates the generalization and interpretation abilities of the proposed method.

Drug-target affinity prediction is a key challenge in the drug discovery process. Recent advances have demonstrated the great potential of deep learning in predicting affinities; however, existing approaches learn the representation of drug-target complex insufficiently, leading to suboptimal performance. Here, we propose a Multiscale Hybrid Attention Network for the Drug-Target Affinity prediction, named MHAN-DTA, which aims to address the problem of insufficient feature mining thereby improving the prediction performance. To empower the model with global perception ability, a pocket-oriented feature aggregation and extraction module is developed based on self-attention mechanisms, together with a hierarchical strategy applied to the target proteins. We further introduce a cross-modal fusion module and a cross-entity interaction module for mining the multiscale intra-molecular and inter-molecular features within the binding sites. Comprehensive evaluations on four benchmark test sets, including an internal and three external benchmark datasets, demonstrate that the proposed approach achieves superior and robust performance.

The detection of image tampering, specifically copy detection, is an important problem in many domains such as military, media, and public opinion outlets. Effective means to detect such tampering is crucial in controlling the dissemination of false information. However, a major challenge in achieving high detection accuracy lies in the variability of the scale of the copied targets. To tackle this problem, we introduce an all-encompassing methodology called Cross-Scale Modeling and Alternating Refinement (CANet) to detect the genuine source and tampered region at the pixel level. CANet consists of three modules: the Cross-Scale Similar Region Detection (CS) module, the Edge-Supervised Tamper Region Detection (ET) module, and the Alternating Refinement (AR) module. The CS module extracts coarse similar region features by cross-scale correlation modeling, which can alleviate the scale gap between the source and tampered region. The obtained coarse similar region feature is refined by the AR module, in which we introduce the source and the tampered region as the auxiliary information and employ a two-stage process that sequentially models their global feature representations. The tampered region used in the AR module is obtained from the ET module using edge supervision with a salient edge selection scheme, and the source region is generated by the implicit modeling. We conducted experiments on the USC-ISI, CASIA v2.0, CoMoFoD, and MICC-F220 datasets separately. Results show that our method outperforms the state-of-the-art.

Visual question answering (VQA) is a challenging task that requires models to understand both visual and linguistic inputs and produce accurate answers. However, VQA models often exploit biases in datasets to make predictions rather than reasoning based on the inputs. Prior approaches to debiasing have suggested the implementation of a supplementary model, deliberately designed to exhibit bias, which subsequently informs the training of a resilient target model. Nevertheless, such techniques merely quantify the model’s divergence based on the statistical distribution of labels within the training dataset or in relation to unimodal branches. In this work, we propose a novel method of generating bias from the target model itself, called LEGO, which aims to combat the language guidance bias. Specifically, LEGO framework employs a generative network that assimilates the biases inherent in the target model by integrating adversarial goals with the principles of knowledge distillation. Then, we use a debiased contrastive learning strategy to model the language guidance bias of caption and question. In the process of modeling, in order to obtain robust semantic coreference, the multimodal representations of two semantic granularity are modeled by mutual information fusion and contrast learning difference modeling. We evaluate our method on various VQA-biased datasets, including VQA-CP2, GQA-OOD, and RSICD, and show that it outperforms similar methods.

The U-net and Transformer have garnered significant attention in precipitation nowcasting due to their impressive capabilities in modeling sequential information. However, the performance is still constrained by the computational complexity of attention mechanism and the persistence of redundant information transmission between encoding and decoding stages. To address the above problems, we propose a novel hierarchical transformer with lightweight attention (HTLA) for precipitation nowcasting, which can integrate the Transformer and U-Net architectures to comprehensively explore the intrinsic characteristics of rainfall data with less complexity. Specifically, HTLA incorporates cross-channel self-attention with lightweight and dual feedforward module as fundamental components for encoding and decoding, efficiently fusing the advantages of Transformer and U-Net. A Gaussian pooling skip-connection strategy is proposed to adaptively weight information, effectively suppressing the redundant interference from the encoder to the decoder. The experimental results demonstrate the effectiveness and robustness of our HTLA, achieving improvements of 5.6% and 5.1% in terms of critical success index (CSI) and Heidke skill score (HSS) with only 3.6% parameters compared to the state-of-the-art method. The code is available athttps://github.com/precipitation-zy/HTLA.

Remote sensing text-image retrieval aims to retrieve valuable information from diverse and complex remote sensing data, attracting significant attention. However, the performance is limited due to the complexity of scenes and their substantial content differences from natural domain images. To address these issues, we propose a simple but effective Text-guided Knowledge Transfer (TGKT) method for remote sensing image-text retrieval. TGKT utilizes CLIP to encode remote sensing data and transfer its rich semantic knowledge from natural to remote sensing domain. The textual information without significant domain differences is employed to bridge the semantic gap between these two domains, thereby enhancing image features. The extensive experimental results on both RSICD and RSITMD datasets demonstrate the effectiveness of our method.

Retrosynthesis prediction is a fundamental problem in organic chemistry and drug synthesis. We proposed an end-to-end deep learning model called CTsynther (Contrastive Transformer for single-step retrosynthesis prediction model) that could provide single-step retrosynthesis prediction without external reaction templates or specialized knowledge. The model introduced the concept of contrastive learning in Transformer architecture and employed a contrastive learning language representation model at the SMILES sentence level to enhance model inference by learning similarities and differences between various samples. Mixed global and local attention mechanisms allow the model to capture features and dependencies between different atoms to improve generalization. We further investigated the embedding representations of SMILES learned automatically from the model. Visualization results show that the model could effectively acquire information about identical molecules and improve prediction performance. Experiments showed that the accuracy of retrosynthesis reached 53.5% and 64.4% for with and without reaction types, respectively. The validity of the predicted reactants is improved, showing competitiveness compared with semi-template methods.

Unsupervised deep hashing has demonstrated significant advancements with the development of contrastive learning. However, most of previous methods have been hindered by insufficient similarity mining using global-only image representations. This has led to interference from background or non-interest objects during similarity reconstruction and contrastive learning. To address this limitation, we propose a novel unsupervised deep hashing framework named Fine-grained Similarity-preserving Contrastive learning Hashing (FSCH), which explores fine-grained semantic similarity among different images and their augmented views more comprehensively. It mainly comprises two modules: the global-local fine-grained similarity consistency preservation module and the local fine-grained similarity contrast preservation module. Specifically, we reconstruct local pairwise similarity structures by matching fine-grained patches, in conjunction with global similarity structures based on global hash codes cosine similarity, to generate hash codes with the ability to preserve global-local similarity consistency. Moreover, the preservation of local fine-grained similarity among augmented views is accomplished through the common regional features mutual representation between patches, then we enhance the discriminability of hash codes by mitigating the potential features difference during contrastive learning. Experimental results on four benchmark datasets demonstrate that our FSCH achieves an excellent retrieval performance compared to state-of-the-art unsupervised hashing methods.

Typhoon poses a significant threat to casualties and economic damage in coastal areas. Accurate forecasting of typhoon intensity is crucial for effective disaster management and mitigation strategies. Despite the certain progress has been made in integrating this task with deep learning techniques, it still faces limitations due to insufficient consideration of the intra-modal characteristics and the inter-modal discrepancy. In this paper, we propose a novel multi-modal adversarial fusion approach for typhoon intensity forecasting, which contains feature encoding and multi-modal fusion modules. Within the feature encoding module, we adopt a novel block attention mechanism to efficiently reveal the relationships of observational factors in multiple latent spaces, thereby obtaining more comprehensive representations feature of the observational data. Moreover, we introduce a complex convolutional operation to encode the components of wind speed within the reanalysis data, which can maintain their characteristics through the real and imaginary parts of complex numbers during the encoding procedure. Finally, we design the adversarial fusion scheme to effectively narrow the modality gap and facilitate a more comprehensive fusion for typhoon intensity forecasting. Comprehensive experiments are conducted on CMA-BST/ERA-Interim data and extensive results verify the effectiveness of our method for typhoon intensity forecasting.

Exploring protein-protein interaction (PPI) is of paramount importance for elucidating the intrinsic mechanism of various biological processes. Nevertheless, experimental determination of PPI can be both time-consuming and expensive, motivating the exploration of data-driven deep learning technologies as a viable, efficient, and accurate alternative. Nonetheless, most current deep learning-based methods regarded a pair of proteins to be predicted for possible interaction as two separate entities when extracting PPI features, thus neglecting the knowledge sharing among the collaborative protein and the target protein. Aiming at the above issue, a collaborative learning framework CollaPPI was proposed in this study, where two kinds of collaboration, i.e., protein-level collaboration and task-level collaboration, were incorporated to achieve not only the knowledge-sharing between a pair of proteins, but also the complementation of such shared knowledge between biological domains closely related to PPI (i.e., protein function, and subcellular location). Evaluation results demonstrated that CollaPPI obtained superior performance compared to state-of-the-art methods on two PPI benchmarks. Besides, evaluation results of CollaPPI on the additional PPI type prediction task further proved its excellent generalization ability.

This study delves into multi-variable meteorological spatial-temporal prediction, focusing on the simultaneous forecasting of key meteorological parameters such as temperature, wind speed, and atmospheric pressure. The core challenge of this task lies in identifying commonalities across different variables while capturing their unique features and the interactions among them. To address this, we propose a novel multi-task learning framework tailored for multi-variable meteorological forecasting. Our framework integrates a convolutional variable-specific visual representation module and a variable-interactive spatial-temporal inference module. The former extracts distinct variable information independently for each variable, while the latter employs a tri-level attention mechanism across space, time, and variables to uncover both commonalities and interactions among the variables. An adaptive multi-loss optimization strategy and a local information aggregation module are introduced to balance task optimization complexities and enhance representation stability. Comprehensive experiments across various meteorological prediction tasks confirm the effectiveness of our methods, showcasing superior performance over existing approaches.

Domain adaptation has been extensively explored as a means of transferring knowledge from the labeled source domain to the unlabeled target domain with disparate data distributions. However, the absence of target annotations and significant domain discrepancies pose a great challenge to transfer knowledge directly from source domain to target domain. To address this challenge, we propose a Progressive Fourier Adversarial Domain Adaptation (PFADA) framework, an effective and versatile framework which can generalize across multiple domain adaptation tasks. Firstly, we propose a Fourier-based style transfer strategy to generate a Fourier intermediate domain that incorporates source images with target domain-specific styles, while preserving the domain-invariant representations of the source data. Secondly, we introduce a progressive adversarial domain adaptation approach that utilizes the Fourier intermediate domain to facilitate the learning of domain-invariant representations. Finally, we present cross-domain semantic alignment and discriminative enhancement approach, which effectively guides the learning of discriminative cross-domain representations utilizing labeled source and intermediate domain data. Extensive experimental evaluations consistently validate the superior performance of the proposed method across diverse visual tasks, encompassing multiple domain adaptive image classification and retrieval scenarios.

Unsupervised domain adaptation (UDA) involves the transfer of knowledge from a labelled source domain to an unlabelled target domain. Recent studies have introduced the concept of intermediate domains to handle significant domain discrepancies between source and target domains. Constructing an appropriate intermediate domain is a crucial step in handling scenarios with substantial domain differences. In this paper, we propose a novel progressive UDA method called adaptive multi-scale intermediate domain via progressive training (AMPT), which has achieved remarkable effectiveness in alleviating large discrepancies between domains. We design a multi-scale similarity metrics module to solve the issue of different scales across different domains by simultaneously computing the pairwise distance between source domain images and the target domain images on multiple scales. Furthermore, we explore a progressive training strategy that facilitates smooth adaptation of the source domain to the target domain by utilizing the intermediate domain. During the progressive training process, considering a positive feedback mechanism, we iteratively leverage task losses and distillation loss through a dynamic threshold. This ensures that the trained intermediate domain branch progressively constrains the target domain branch, and the intermediate domain can be generated dynamically, leading to a smoother gradual adaptation from the source domain to the target domain. Extensive experimental results demonstrate the superiority of our proposed method AMPT on well-known UDA datasets, including Office-Home, Office-31 and DomainNet.

In real classification scenarios, the number distribution of modeling samples is usually out of proportion. Most of the existing classification methods still face challenges in comprehensive model performance for imbalanced data. In this article, a novel theoretical framework is proposed that establishes a proportion coefficient independent of the number distribution of modeling samples and a general merge loss calculation method independent of class distribution. The loss calculation method of the imbalanced problem focuses on both the global and batch sample levels. Specifically, the loss function calculation introduces the true-positive rate (TPR) and the false-positive rate (FPR) to ensure the independence and balance of loss calculation for each class. Based on this, global and local loss weight coefficients are generated from the entire dataset and batch dataset for the multiclass classification problem, and a merge weight loss function is calculated after unifying the weight coefficient scale. Furthermore, the designed loss function is applied to different neural network models and datasets. The method shows better performance on imbalanced datasets than state-of-the-art methods.

Image-text retrieval in remote sensing aims to provide flexible information for data analysis and application. In recent years, state-of-the-art methods are dedicated to “scale decoupling” and “semantic decoupling” strategies to further enhance the capability of representation. However, these previous approaches focus on either the disentangling scale or semantics but ignore merging these two ideas in a union model, which extremely limits the performance of cross-modal retrieval models. To address these issues, we propose a novel Scale-Semantic Joint Decoupling Network (SSJDN) for remote sensing image-text retrieval. Specifically, we design the Bidirectional Scale Decoupling (BSD) module, which exploits Salience Extraction Map (SEM) and Salience Suppression Map (SSM) units to adaptively extract potential features and suppress cumbersome features at other scales in a bidirectional pattern to yield different scale clues. Besides, we design the Label-supervised Semantic Decoupling (LSD) module by leveraging the category semantic labels as prior knowledge to supervise images and texts probing significant semantic-related information. Finally, we design a Semantic-guided Triple Loss (STL), which adaptively generates a constant to adjust the loss function to improve the probability of matching the same semantic image and text and shorten the convergence time of the retrieval model. Our proposed SSJDN outperforms state-of-the-art approaches in numerical experiments conducted on four benchmark remote sensing datasets.

We present MobileSky, the first automatic method for real-time high-quality sky replacement for mobile AR applications. The primary challenge of this task is how to extract sky regions in camera feed both quickly and accurately. While the problem of sky replacement is not new, previous methods mainly concern extraction quality rather than efficiency, limiting their application to our task. We aim to provide higher quality, both spatially and temporally consistent sky mask maps for all camera frames in real time. To this end, we develop a novel framework that combines a new deep semantic network called FSNet with novel post-processing refinement steps. By leveraging IMU data, we also propose new sky-aware constraints such as temporal consistency, position consistency, and color consistency to help refine the weakly classified part of the segmentation output. Experiments show that our method achieves an average of around 30 FPS on off-the-shelf smartphones and outperforms the state-of-the-art sky replacement methods in terms of execution speed and quality. In the meantime, our mask maps appear to be visually more stable across frames. Our fast sky replacement method enables several applications, such as AR advertising, art making, generating fantasy celestial objects, visually learning about weather phenomena, and advanced video-based visual effects. To facilitate future research, we also create a new video dataset containing annotated sky regions with IMU data.

Data partitioning is the most fundamental procedure before parallelizing complex analysis on very big graphs. As a classical NP-complete problem, graph partitioning usually employs offline or online/streaming heuristics to find approximately optimal solutions. However, they are either heavyweight in space and time overheads or suboptimal in quality measured by workload balance and the number of cutting edges across partitions, both of which cannot scale well with the ever-growing demands of quickly analyzing big graphs. This paper thereby proposes a new vertex partitioner for better scalability. It preserves the lightweight advantage of existing streaming heuristics, and more importantly, fully utilizes the knowledge embedded in the local view when streaming a vertex, which significantly improves the quality. We present a sliding window technique to compensate for the additional memory costs caused by knowledge utilization. Also, a parallel technique with dependency detection optimization is designed to further enhance efficiency. Experiments on a spread of real-world datasets validate that our proposals can achieve overall success in terms of partitioning quality, memory consumption, and runtime efficiency.

Local differential privacy (LDP) has been established as a strong privacy standard for collecting sensitive information from users. Although it has attracted much research attention in recent years, the majority of existing works focus on applying LDP to frequency distribution estimation for each individual value in a discrete domain. This paper concerns the important range queries involving multiple discrete values. Till now, only a few works target this problem. They all rely on the B-ary tree to construct a uniform and hierarchical decomposition, so as to decrease the error when answering large range queries. However, the uniform splitting manner ignores the properties of decomposed sub-domains and processes them equally without preferences, which leads to significant performance penalty.In this paper, we tackle the problem head on: our proposal, privNUD, is a novel domain hierarchical decomposition mechanism. It dynamically decomposes each domain with a tailored granularity into some sub-domains, which sensitively considers the potential chances to answer one range query. The issue of granularity is carefully analyzed for better performance. It also can smartly prune the sub-domains with small frequencies. Besides, an adaptive user allocation technique is designed to dynamically decide the scale of users that are involved in each sub-domain’s frequency estimation. Extensive experiments using real and synthetic datasets demonstrate that privNUD achieves significantly higher result accuracy compared to the up-to-date solutions.

Remote sensing cross-modal image-text retrieval has attracted increasing attention due to its important roles in multiple domains. Existing methods perform salient modeling for the feature that has high relevance between different modalities. However, most works consider the relevance between different modalities but ignore the irrelevance between different modalities, resulting in incomplete modeling of the relevance and irrelevance between different modalities. In this paper, we propose a Relevance and Irrelevance Considered Subspace Mapping Neural Networks (RIR-SMNNs) to simultaneously consider the relevance and irrelevance between different modalities. Specifically, we first utilize Multiscale Image Feature Extraction (MIFE) and Multiscale Text Feature Extraction (MTFE) to extract the multiscale feature of image and text. Then, we perform Local Space Building Module (LSB), which constructs local space that realizes scale alignment. Finally, we perform the Relevance and Irrelevance Local Space Mapping (RIRLSM) to consider the relevance and irrelevance of different modalities in multiple spaces. Experimental results on several remote sensing datasets demonstrate our model outperforms the state-of-the-art approaches.

With the emergence of multidrug-resistant bacteria, antimicrobial peptides (AMPs) offer promising options for replacing traditional antibiotics to treat bacterial infections, but discovering and designing AMPs using traditional methods is a time-consuming and costly process. Deep learning has been applied to the de novo design of AMPs and address AMP classification with high efficiency. In this study, several natural language processing models were combined to design and identify AMPs, i.e. sequence generative adversarial nets, bidirectional encoder representations from transformers and multilayer perceptron. Then, six candidate AMPs were screened by AlphaFold2 structure prediction and molecular dynamic simulations. These peptides show low homology with known AMPs and belong to a novel class of AMPs. After initial bioactivity testing, one of the peptides, A-222, showed inhibition against gram-positive and gram-negative bacteria. The structural analysis of this novel peptide A-222 obtained by nuclear magnetic resonance confirmed the presence of an alpha-helix, which was consistent with the results predicted by AlphaFold2. We then performed a structure-activity relationship study to design a new series of peptide analogs and found that the activities of these analogs could be increased by 4-8-fold against Stenotrophomonas maltophilia WH 006 and Pseudomonas aeruginosa PAO1. Overall, deep learning shows great potential in accelerating the discovery of novel AMPs and holds promise as an important tool for developing novel AMPs.

MOTIVATION: In the field of pharmacochemistry, it is a time-consuming and expensive process for the new drug development. The existing drug design methods face a significant challenge in terms of generation efficiency and quality. RESULTS: In this paper, we proposed a novel molecular generation strategy and optimization based on A2C reinforcement learning. In molecular generation strategy, we adopted transformer-DNN to retain the scaffolds advantages, while accounting for the generated molecules' similarity and internal diversity by dynamic parameter adjustment, further improving the overall quality of molecule generation. In molecular optimization, we introduced heterogeneous parallel supercomputing for large-scale molecular docking based on message passing interface communication technology to rapidly obtain bioactive information, thereby enhancing the efficiency of drug design. Experiments show that our model can generate high-quality molecules with multi-objective properties at a high generation efficiency, with effectiveness and novelty close to 100%. Moreover, we used our method to assist shandong university school of pharmacy to find several candidate drugs molecules of anti-PEDV. AVAILABILITY AND IMPLEMENTATION: The datasets involved in this method and the source code are freely available to academic users at https://github.com/wq-sunshine/MomdTDSRL.git.

With the preferable efficiency in storage and computation, hashing has shown potential application in large-scale multimedia retrieval. Compared with traditional hashing algorithms using hand-crafted characteristics, deep hashing inherits the representational capacity of deep neural networks to jointly learn semantic features and hash functions, encoding raw data into compact binary codes with significant discrimination. Generally, most of the current multi-wise hashing methods view the similarity margins between image pairs as constant values in training process. When the distance between sample pairs exceeds the fixed margin, the hashing network would not learn anything. Besides, available hashing methods commonly introduce the random sampling strategy to build training batches and ignore the sample distribution, which is harmful to parameter optimization. In this paper, we propose a novel Deep Adaptive Quadruplet Hashing with probability sampling (DAQH) for discriminative binary code learning. Specifically, with exploring the distribution relationship of raw samples, a non-uniform probability sampling strategy is proposed to build more informative and representative training batches, while maintaining the diversity of training samples. By introducing the prior similarity of sample pairs to calculate corresponding margins, an adaptive margin quadruplet loss is designed to dynamically preserve the underlying semantic relationships with its neighbors. To tune the attributes of binary codes, by combining quadruple regularization and orthogonality optimization, binary code constraint is developed to make the learned embedding with significant discrimination. Extensive experimental results on various benchmark datasets demonstrate our proposed DAQH framework achieves state-of-the-art visual similarity search performance.

In recent years, semantic segmentation technology plays an important role in land resource management tasks. However, many classic semantic segmentation methods often fail to obtain satisfactory results for remote sensing images with a large amount of interference information. In order to improve this situation, we propose Semantic Category Balance-Aware Involved Anti-Interference Network(SCBANet). SCBANet has an encoder-decoder structure similar to DeeplabV3+ [1]. On this basis, we propose Clustering Guided Semantic Decoupling Module(CGSDM), Consistency Based Anti-Interference Feature Extraction Module(CAFEM), Relevance Based Anti-Interference Feature Extraction Module(RAFEM) and Optional Decoder Module Based on Semantic Category Balance(ODMSCB) to improve the accuracy of semantic segmentation. CGSDM aims to obtain the information of different semantic categories through K-Means clustering algorithm. CAFEM performs an average operation on the feature vectors in each semantic category to obtain semantic consistency information. RAFEM deeply excavates the information contained in each semantic category through the modeling method with self attention mechanism as the core, making the relationship between pixels within each semantic category to be better understood by the model. ODMSCB classifies the feature map according to the balance of different semantic categories, so that different decoders can be applied to feature maps with different semantic category balance. These four parts complement each other, greatly improving the model’s anti-interference ability while also enhancing the ability to handle category imbalance issue. We compared our method with several of the most advanced deep learning methods on the Vaihingen and Potsdam data sets. The final results demonstrate the superiority of our method.

Changes in oceanic variables, such as sea surface temperature (SST) and chlorophyll-a (Chl-a), have important implications for marine ecosystems and global climate change. The deep learning methods relying on convolutional neural networks can be employed to extract the spatial correlation for the prediction of oceanic variables. However, these methods are inflexible in the cases where some regions, e.g., land and islands, are invalid for the prediction of oceanic variables. By contrast, the graph convolutional network (GCN) is capable of capturing the large-scale spatial dependency existing in the irregular data. Owing to this, in this paper, we propose a GCN-based method for the prediction of oceanic variables, including SST and Chl-a, with high accuracy, which is referred to as OVPGCN. The proposed OVPGCN consists of three modules aiming to fully extract the spatial correlation and temporal dependency via modeling the multi-characteristics of the spatio-temporal dynamic evolution. In particular, three modules are implemented to extract the stationary and non-stationary variations in the recent spatio-temporal sequences, the spatial differences between different sites, and the periodic features in historical data, respectively. The well-designed OVPGCN is applied to the monthly SST and Chl-a prediction in the Bohai Sea and the Northern South China Sea (NSCS). The performance demonstrates that the proposed OVPGCN is highly effective and enables to achieve much higher prediction accuracy than the state-of-the-art methods.

Low-order features based on convolution kernel are easy to be distorted when encountering dramatic view angle transformation and atmospheric scattering in remote sensing (RS) images. To address this concern, this article first proposes to operate semantic segmentation of RS images based on the high-order information, which can represent the relative relationship of low-order features and is robust and stable when suffering feature distortion. Besides, semantic decouples have recently been well researched and have achieved significant improvement in image understanding. Thus, in this article, a high-order semantic decoupling network (HSDN) is proposed to disentangle features by semantics based on high-order features. Specifically, HSDN first represents each pixel by calculating the pixel-level affinity as a high-order feature and then clusters these pixels into different semantics. Afterward, an attention-like mask generation module is designed for both intra-semantic and inter-semantic groups, leading to three kinds of masks, including the semantic decoupling mask (SDM), which utilizes each high-order cluster centroid as a mask to compact features intracluster and expand different interclusters, so as to improve semantic disentangle performance to a better extent; semantic enhancement mask (SEM), which records pixel-level relative correlation within a class to sufficiently exploit high-order features and could enhance feature robustness; and boundary supplementary mask (BSM), which aims to process borderline pixels to reduce cluster errors. Finally, by applying masks on pixels both within classes and on borderlines, semantic decoupled features are generated and concatenated to realize segmentation. The quantitative and qualitative experiments are conducted on two large-scale fine-resolution RS image datasets to demonstrate the significant performance of adopting high-order representation. Besides, we also implement numerous experiments to validate the effectiveness of the proposed semantic decouple framework in dealing with complicated and distortion-prone RS image segmentation tasks.

With the rapid growth of virtual drug data- bases, the need for efficient molecular docking tools for large-scale screening is also growing. We have developed Vina@QNLM 2.0, a novel molecular docking system that leverages the logical processing units and computational processing arrays of heterogeneous multicore architecture processors. Compared to Vina@QNLM, the new version optimizes the docking speed without sacrificing accuracy. This greatly improves the scoring capability for large molecules (molecular weight > 500). Simultaneously, the new system provides enhanced support for applications such as reverse target finding through an improved parallel strategy. Vina@QNLM 2.0 achieves a speedup 20 times higher than that, using logical processing units only during a single docking process. Additionally, we successfully scaled the reverse target finding a task to 122,401 kernel groups with a robust scalability of 80.01%. In practice, we completed a reverse target-seeking for nine glycan molecules with 10,094 proteins within 1 hour.

Predicting drug-target affinity (DTA) is a crucial step in the process of drug discovery. Efficient and accurate prediction of DTA would greatly reduce the time and economic cost of new drug development, which has encouraged the emergence of a large number of deep learning-based DTA prediction methods. In terms of the representation of target proteins, current methods can be classified into 1D sequence- and 2D-protein graph-based methods. However, both two approaches focused only on the inherent properties of the target protein, but neglected the broad prior knowledge regarding protein interactions that have been clearly elucidated in past decades. Aiming at the above issue, this work presents an end-to-end DTA prediction method named MSF-DTA (Multi-Source Feature Fusion-based Drug-Target Affinity). The contributions can be summarized as follows. First, MSF-DTA adopts a novel "neighboring feature"-based protein representation. Instead of utilizing only the inherent features of a target protein, MSF-DTA gathers additional information for the target protein from its biologically related "neighboring" proteins in PPI (i.e., protein-protein interaction) and SSN (i.e., sequence similarity) networks to get prior knowledge. Second, the representation was learned using an advanced graph pre-training framework, VGAE, which could not only gather node features but also learn topological connections, therefore contributing to a richer protein representation and benefiting the downstream DTA prediction task. This study provides new perspective for the DTA prediction task, and evaluation results demonstrated that MSF-DTA obtained superior performances compared to current state-of-the-art methods.

With the development of computer vision, the semantic segmentation of remote sensing images, which has become an important topic, has been utilized in various applications for image content analysis and understanding, such as urban planning, natural disaster monitoring, and land resource management. Many approaches have been proposed to address these problems. However, due to obvious differences in resolution, spatial structure, and semantics between remote sensing images and ordinary images, the semantic segmentation of remote sensing images is still challenging. In this paper, we propose a novel multiscale image generation network (MIGN) that can efficiently generate high-resolution segmentation results by considering both details and boundary information. In particular, a multi-attention mechanism method for semantic segmentation of remote sensing images is designed. The attention weight is calculated by capturing the interaction of cross dimensions in a two-branch structure, which can learn the underlying feature information and guarantee the performance of each pixel feature for final classification. We also propose an edge supervised module to ensure that the segmentation boundary has a more accurate performance. A multiscale image fusion algorithm based on the Bayes model is proposed to improve the accuracy of the segmentation module. The performance of our model is evaluated on the ISPRS Vaihingen and Potsdam datasets. The results show that our method is superior to the most advanced image segmentation methods in terms of MIoU and pixel accuracy.

Recently, recommendation systems have been widely usedin online business scenarios, which can improve the online experience by learning the user or item characteristics to predict the user’s future behavior and to realize precision marketing. However, data sparsity and cold-start problems limit the performance of recommendation systems in some emerging fields. Thus, cross-domain recommendation has been proposed to handle the abovementioned problems. Nonetheless, many cross-domain recommendations only consider modeling a single user’s representation and ignore user-group information (this group has similar behavior and interests). Additionally, most studies are based on matrix factorization for generating embeddings, which results in a weak generalization ability of user latent features. In this paper, we propose a novel cross-domain recommendation model viaUser-Clustering andMultidimensional informationFusion (UCMF) that attempts to enhance user representation learning in a data sparsity scenario for accurate recommendation. In addition, we consider a user’s individual information and cross-domain feature information. A novel multidimensional information fusion is proposed to guarantee the robustness of the user features. In particular, we apply a graph neural network to learn the user-group features, which can effectively save the correlation among users’ information and guarantee feature performance. In other words, the Wasserstein autoencoder is utilized to learn the cross-domain user features, which can guarantee the consistency of user features from different domains. Experiments conducted on real-world datasets empirically demonstrate that our proposed method outperforms the state-of-the-art methods in cross-domain recommendation.

Myriad of machine learning (ML) algorithms refine model parameters iteratively. Existing synchronous data-parallel frameworks can accelerate training with convergence guarantees. However, the pre-assigned workload-based synchronous design still poses great challenges, since fast workers must wait for slow, straggling ones, especially in a heterogeneous computing cluster. Asynchronous alternatives can bypass this performance bottleneck, but at expense of potentially losing convergence guarantees. This article proposes a new time-based flexible synchronous parallel framework (FSP). It provides strict convergence analysis by consistently updating parameters, as well as significant cost reduction by completely unleashing the power of fast workers. It identifies the optimal synchronization frequency, by online balancing costs of parameters’ update and benefits brought by their freshness. Besides the basic goal of keeping all workers fully CPU-utilized, FSP also aims to keep data spread over the cluster fully utilized, so that they can contribute to convergence with equal opportunities. These proposals are all implemented in a prototype system Flegel, with additional engineering optimizations for further performance enhancement and programming facilitation. Experiments demonstrate that Flegel significantly outperforms recent studies.

With the development of deep neural networks, especially generation networks, gray image coloring technology has made great progress. As one of the fields, remote sensing image colorization needs to be solved urgently. This is because remote sensing images cannot obtain clear color images due to the limitations of shooting equipment and transmission equipment. Compared with ordinary images, remote sensing images are characterized by the uneven spatial distribution of objects, therefore, it is a great challenge to ensure the spatial consistency of coloring. To embrace this challenge, we propose a new joint stream DCGAN including a micro stream and a macro stream, in which the latter is set as a prior to constrain the former for colorization. In addition, the Low-level Correlation Feature Extraction (LCFE) module is proposed to obtain the salient shallow detail feature with global correlation, which is used to enhance the global constraints as well as supplement the low-level information to the micro stream. What's more, we propose the Gated Selection (GSM) module by selecting useful information using a gated scheme to fuse features from two streams appropriately. Comprehensive comparison and ablation experiments are implemented and verify the proposed method performs surpasses other methods in both qualitative and quantitative metrics.

Deep learning shortens the cycle of the drug discovery for its success in extracting features of molecules and proteins. Generating new molecules with deep learning methods could enlarge the molecule space and obtain molecules with specific properties. However, it is also a challenging task considering that the connections between atoms are constrained by chemical rules. Aiming at generating and optimizing new valid molecules, this article proposed Molecular Substructure Tree Generative Model, in which the molecule is generated by adding substructure gradually. The proposed model is based on the Variational Auto-Encoder architecture, which uses the encoder to map molecules to the latent vector space, and then builds an autoregressive generative model as a decoder to generate new molecules from Gaussian distribution. At the same time, for the molecular optimization task, a molecular optimization model based on CycleGAN was constructed. Experiments showed that the model could generate valid and novel molecules, and the optimized model effectively improves the molecular properties.

In recent years, convolutional regression trackers have shown increasing attention for visual object tracking due to their favorable performance and easy implementation. However, most of them are restricted to features from a certain layer and hardly benefit from temporal spatial information, which limits the potential to significant appearance changes. In this work, we go beyond the traditional deep regression trackers and build a novel twofold tracking network, which exploits rich hierarchical features and incorporates both temporal and spatial information to boost the tracking performance. The proposed network is composed of two streams, i.e., an appearance stream and a semantic stream, each stream is independently learned from different convolutional layers. Specially, we propose temporal and spatial mechanism for robust target representation by considering historical information in previous frames as well as spatial information. By design, the proposed twofold convolutional regression tracking network with spatial and temporal mechanism can better tolerate the target appearance changes and improve the tracking accuracy. Extensive experimental results on the benchmarks OTB-2015, Temple-Color, UAV123, and VOT-2018 demonstrate the effectiveness of our method, as compared with a number of state-of-the-art trackers.

As we all know, remote sensing (RS) images contain multi-scale and numerous RS objects, along with massive and complex spatial topological relationships, such as the adjacency, proximity relations of same-scale objects, and inclusion relations of cross-scale objects. However, the existing semantic segmentation methods have never explored the cross-scale relations, which are especially important when comes to the situation that the RS objects cannot be accurately identified, they could be supplemented by the surrounding contents. To address the above concern, we propose a scale-relation joint decoupling network (SRJDN) for the semantic segmentation of RS images by simultaneously considering decoupling scales and decoupling relations to excavate more complete relationships of multi-scale RS objects. The SRJDN is performed by following three steps, namely scale decoupling (SD), relation decoupling (RD), and fine-granularity guided fusion (FGF). The SD module uses dilated convolution with different rates to decouple RS objects into different scale feature groups, from small to large scales. Afterward, the RD considers all the spatial topological relationships and decouples these relationships according to the scale, which is divided into two parts, including same-scale relation extraction (SSRE) and cross-scale relation extraction (CSRE). The SSRE establishes the graph structures at each scale independently to mine the relationships of same-scale RS objects and the CSRE constructs the graph in a unified pattern between cross-scales to explore cross-scale target relationships. Third, the FGF module regards small-scale features as fine-granularity representation and applies its attention map to guide the learning of other scale features, which could mine more reliable and comprehensive saliency information and improve the feature consistency. Numerical experiments conducted on two large-scale fine-resolution RS image datasets empirically demonstrate the robustness of the proposed joint decoupling strategy and the effectiveness of the fine-granularity guided fusion in RS image semantic segmentation tasks.

Virtual marine scene authoring plays an important role in generating large-scale 3D scenes and it has a wide range of applications in computer animation and simulation. Existing marine scene authoring methods either produce periodic patterns or generate unnatural group distributions when tiling marine entities such as schools of fish and groups of reefs. To this end, we propose a new large-scale marine scene authoring method based on real examples in order to create more natural and realistic results. Our method first extracts the distribution of multiple marine entities from real images to create Octahedral Blocks, and then we use a modified Wang Cubes algorithm to quickly tile the 3D marine scene. As a result, our method is able to generate aperiodic tiling results with diverse distributions of density and orientation of entities. We validate the effectiveness of our method through intensive comparative experiments. User study results show that our method can generate satisfactory results which are in accord with human preferences.

With the advance of Convolutional Neural Network, deep hashing methods have shown the great promising performance in large-scale image retrieval. Without depending on extensive human-annotated data, unsupervised hashing is more applicable to image retrieval tasks compared to supervised methods. However, due to the lack of fine-grained supervised signals and multi-similarity constraints, most state-of-the-art unsupervised deep hashing algorithms cannot ensure the correct fine-grained similarity ranking for image pairs. In this paper, we propose a novel unsupervised deep multi-similarity hashing framework to learn compact binary codes by jointly exploiting global-aware and spatial-aware representations, called Unsupervised Deep Multi-Similarity Hashing with Semantic Structure (UDMSH). Specifically, to obtain distinguishing characteristics, we develop a sub-network by jointly learning global semantic structures from Convolutional Neural Network (CNN) and inherent spatial structures from Fully Convolutional Network (FCN). By computing the cosine distance for deep features from image pairs, we construct a similarity matrix with semantic structure, then utilize this matrix to guide hash code learning process. Based on it, we carefully design a multi-level pairwise loss to preserve the correct fine-grained similarity ranking. Furthermore, we introduce Hamming-isometric mapping into unsupervised hashing framework to decrease the quantization errors. Extensive experiments on three widely used benchmarks prove that our proposed UDMSH outperforms several state-of-the-art unsupervised hashing with respect to different evaluation metrics.

In the big data era, distributed computation is becoming a preferred solution for iterative graph analysis. However, graphs are rapidly growing in size and more importantly, there exist a lot of messages across iterations. For better scalability, many distributed systems keep graph data and message data on disk. Now these systems solely employ either pushing or pulling mode to manage data, but neither can always work well during the entire computation. This is mainly because I/O access patterns are dynamic and complex. This article proposes a hybrid solution. It achieves the optimal performance in different scenarios by dynamically and adaptively switching modes between pushing and pulling. Specifically, we first devise a new block-centric pulling technique. It pulls messages much more I/O-efficiently than the existing vertex-centric pulling mode. We then combine pushing and pulling. For general-purpose, we categorize graph algorithms and accordingly present two seamless switching frameworks. We also design performance prediction components specialized to the two frameworks, to decide how and when we can switch modes. Some optimization strategies are also given to further enhance performance, such as priority scheduling and lightweight fault-tolerance. Extensive experiments against state-of-the-art solutions confirm the effectiveness of our proposals.

Three-dimensional (3D) shape recognition is a popular topic and has potential application value in the field of computer vision. With the recent proliferation of deep learning, various deep learning models have achieved state-of-the-art performance. Among them, multiview-based 3D shape representation has received increased attention in recent years, and related approaches have shown significant improvement in 3D shape recognition. However, these methods focus on feature learning based on the design of the network and ignore the correlation among views. In this article, we propose a novel progressive feature guide learning network (PGNet) that focuses on the correlation among multiple views and integrates multiple modalities for 3D shape recognition. In particular, we propose two information fusion schemes from visual and feature aspects. The visual fusion scheme focuses on the view level and employs the soft-attention model to define the weights of views for visual information fusion. The feature fusion scheme focuses on the feature dimension information and employs the quantified feature as the mask to further optimize the feature. These two schemes jointly construct a PGNet for 3D shape representation. The classic ModelNet40 and ShapeNetCore55 datasets are applied to demonstrate the performance of our approach. The corresponding experiment also demonstrates the superiority of our approach.

Air pollution has been long recognized as a hazardous factor for the human cardiovascular system. Sulfur dioxide (SO2) is a common ambient air pollutant that is able to cause detrimental effects on hearts. Though the cardiotoxicity effects by sulfur dioxide were well documented in epidemiological reports, however, the underlying mechanisms remain unclear owing to the technical limitations that exist in traditional experimental measures. In this article, we developed a multi-scale virtual ventricular tissue, which incorporated electrophysiological activities from subcellular to tissue levels and could provide comprehensive records and insightful mechanisms of the SO2induced ventricular arrhythmias. Based on the available cellular and molecular experimental data, our findings provide a rationale at tissue level in support of epidemiologic studies pointing to the deleterious effects of SO2pollution on cardiac function.

Due to the advantages of retrieval speed and storage space, deep hashing methods have become a research hotspot in the field of large-scale image retrieval. Most of existing deep hashing methods pay close attention to similarity between images without images at the top of the ranking list similar to query targets. In the paper, a novel deep hashing model is proposed to preserve top images similar to the query images and optimize the quality of hash codes for image retrieval. Specifically, the optimized AlexNet is utilized to extract discriminative image representations and learn hashing functions simultaneously. The loss function based on acceleration strategy is designed to ensure similarity between returned images at the top of the ranking list and query images. In addition, we implement the model training in a batch-process fashion to low the image storage. Moreover, our extensive experiments on standard benchmarks demonstrate that our method outperforms several state-of-the-art deep hashing methods.

BACKGROUND: Binding sites are the pockets of proteins that can bind drugs; the discovery of these pockets is a critical step in drug design. With the help of computers, protein pockets prediction can save manpower and financial resources. RESULTS: In this paper, a novel protein descriptor for the prediction of binding sites is proposed. Information on non-bonded interactions in the three-dimensional structure of a protein is captured by a combination of geometry-based and energy-based methods. Moreover, due to the rapid development of deep learning, all binding features are extracted to generate three-dimensional grids that are fed into a convolution neural network. Two datasets were introduced into the experiment. The sc-PDB dataset was used for descriptor extraction and binding site prediction, and the PDBbind dataset was used only for testing and verification of the generalization of the method. The comparison with previous methods shows that the proposed descriptor is effective in predicting the binding sites. CONCLUSIONS: A new protein descriptor is proposed for the prediction of the drug binding sites of proteins. This method combines the three-dimensional structure of a protein and non-bonded interactions with small molecules to involve important factors influencing the formation of binding site. Analysis of the experiments indicates that the descriptor is robust for site prediction.

Tracking live fish in an open underwater environment to investigate their behavior is of great value for many applications, e.g. biological and robotic research. However, tracking fish in real world environment is a challenging task due to complex non-rigid deformation and abrupt movement of fish. In this paper, we explore and incorporate motion property of fish and propose a real-time fish tracking method based on novel adaptive multi-appearance models and tracking strategy, which can be adapted to various changes of the fish appearance caused by non-rigid deformation. Experimental results show the promising performance of the proposed method can outperform the previous method by 13.4% in accuracy on average and is robust to real-time underwater fish tracking.

In this work, we studied time management behavior, performance of students, and their association in online learning. We propose three novel time-related features, i.e., video-watching frequency, video-watching interval and learning efficiency. Leveraging these features, we train classifiers for identifying study habits of students and assessing the impact of behavior on performance. Experimental results on dataset from our online learning platform demonstrate that the proposed features are effective for learning behavior description. Comprehensive experiments show that our method can get promising performance.

Exercise is considered as an effective mean against overweight and obesity-related diseases. In this paper, a real-time activity recognition and counting approach is proposed to evaluate amount of exercise only using a wearable smart watch. First, accelerometer and gyroscope data are collected to extract efficient features. Then Support Vector Machine classifiers are trained to recognize nine common exercise activities in real time. In order to measure the frequency of repetitive activity, a general activity counting algorithm based on gyroscope is proposed which is applicable for different types of activity. Various activities can be counted uninterruptedly using the proposed general method without frequently changing algorithms. Through experiments, it is demonstrated that the extracted features are efficient for real time exercise activity recognition. Moreover, our comparative experiments have shown that our counting approach is more accurate than other products on the market.

Everyday, people choose their preferred clothing to wear before leaving home for various activities. It has been recognized that each person may consciously or unconsciously express individual personality through their wearing styles. It will be interesting to ask: Are you what you look like? This paper presents a novel scheme developed to infer personality type from their wearing. The proposed research is justifiably rooted in the psychological findings that reveal intrinsic correlations between clothing style and wearer's inner self in self-image, mood, and social aspirations. First, we build a relatively large dataset with more than 300 persons and over 10,000 portraits, each is labeled with personality type. Then, personality-related clothing features are explored through statistical analysis based on psychological theories. To extract the clothing features from the images, a suite of algorithms, including body detection, GrabCut algorithm and saliency detection have been developed. Binary logistic regression is then applied to verify the Significance Level of the extracted features for predicting personality types. Experimental results demonstrate that the proposed scheme is able to predict several types of personality combination or type pairs with relatively high precisions.

Whenever looking at a stranger's portrait, besides observable appearance, we always build a personality impression implicitly in our subconscious. It is quite interesting to ask how a portrait impresses people. This paper presents a novel method to infer personality impression from portrait. Firstly, a questionnaire is applied to demonstrate the consistence of people's impression. And then personality-related features are explored through the statistical analysis method. Finally, features are trained using Support Vector Machine. Experimental results demonstrate our method could achieve a precision of 52.14% and a recall of 52.78% on inferring 4 personalities from 2,463 randomly selected portraits of people downloaded from "Google images". Improvements of 44.04% and 37.91% are reported compared to a baseline method. And features contribution analysis deeply unveils the correspondence between portrait contents and personality impressions. Demonstrations with respect to visual patterns in portrait collages of different personalities further prove the effectiveness of the proposed method. Furthermore, we apply our method to analyze portraits of Hillary Clinton and obtain an interesting multifaceted figure of this famous politics, which is another proof of both our concept and method.

ABSTRACT Reputation models in mobile ad hoc networks are confronted with the problem of strategically reporting dishonest recommendations given by a selfish group whose purpose is to maximize its benefit. To solve this problem, a truthful and group strategyproof reputation mechanism based on cooperative game theory is proposed in this paper. We first study two supermodular total compensatory payment functions. On the basis of the selection game, we construct a corresponding indirect revelation mechanism and a direct revelation mechanism called claiming compensation mechanism, which is proved to be group strategyproof. Then we choose the extended Shapley value as the concrete compensatory payment scheme and propose an algorithm to generate possible outcome groups for the claiming compensation mechanism. Simulation results show that when we use the total compensatory payments function, which does not relate to cost values, the proposed mechanism is group strategyproof. Finally, all possible situations for a liar are demonstrated and discussed. Copyright © 2014 John Wiley & Sons, Ltd.

The conventional unified parallel computation model becomes more and more complicated which has weak pertinence and little guidance for each parallel computing phase. Therefore, a general layered and heterogeneous idea for parallel computation model research was proposed in this paper. The general layered heterogeneous parallel computation model was composed of parallel algorithm design model, parallel programming model, parallel execution model, and each model correspond to the three computing phases respectively. The properties of each model were described and research spots were also given. In parallel algorithm design model, an advanced language was designed for algorithm designers, and the corresponding interpretation system which based on text scanning was proposed to map the advanced language to machine language that runs on the heterogeneous software and hardware architectures. The parallel method library and parameter library were also provided to achieve the comprehensive utilization of the different computing resources and assign parallel tasks reasonably. Theoretical analysis results show that the general layered heterogeneous parallel computation model is clear and single goaled for each parallel computing phase.

A new color image watermarking algorithm with resistance to JPEG lossy compression based on quantization index modulation (QIM) is proposed in this paper. As it is known, QIM method can achieve a good balance between the embedding bit rate, robustness and distortion between the original image and the composited image by modulating the source signal into different clusters. The corresponding DCT coefficients margins of any two color channels selected from the three as the source signal is substantiated could achieve a high level embedding robustness and a low level distortion in this paper. However, JPEG lossy compression could bring a destructive influence to the watermark since it discards pretty much image information. In this paper, a compensation function is designed to correct the errors caused by JPEG compression. Experiments show the algorithm proposed has a good performance to resist the high compression ratio JPEG lossy compression and other attacks.