Transformer-based models excel in various tasks but their generalization capabilities, especially in arithmetic reasoning, remain incompletely understood.Arithmetic tasks provide a controlled framework to explore these capabilities, yet performance anomalies persist, such as inconsistent effectiveness in multiplication and erratic generalization in modular addition (e.g., modulo 100 vs. 101).This paper develops a unified theoretical framework for understanding the generalization behaviors of transformers in arithmetic tasks, focusing on length generalization.Through detailed analysis of addition, multiplication, and modular operations, we reveal that translation invariance in addition aligns with relative positional encoding for robust generalization, while base mismatch in modular operations disrupts this alignment.Experiments across GPT-family models validate our framework, confirming its ability to predict generalization behaviors.Our work highlights the importance of task structure and training data distribution for achieving data-efficient and structure-aware training, providing a systematic approach to understanding of length generalization in transformers.

Federated learning(FL) is a popular multi-party collaborative privacy computing method. Due to the distributed characteristics of its client, it is more vulnerable to backdoor attacks. In this attack, the attacker will inject the manipulated model update into the aggregation process of the federated model, so that the final generated model can provide targeted wrong prediction for the specific input selected by the attacker. Current defense methods against backdoor attacks mainly focus on detecting and filtering malicious client model updates. These defense methods adopt HDBSCAN clustering, K-means clustering, etc, which generally fail to defense against backdoor attacks in decoy models that contain indicators and there is no obvious clipping threshold for benign groups further clipping after clustering defense. To address these shortcomings, we propose the FLUDP method, which uses UMAP Magnify the angular difference between updates, DBSCAN clustering technology, and a clear threshold clipping technique. We have evaluated FLUDP on MNIST, Tiny-Imagenet, and CIFAR10 datasets. FLUDP achieves lower ABA and EBA rates and maintains a good MA rate compared to previous defense methods.

Advanced persistent threats (APTs) have become a major network threat due to their persistence, complexity, and multi-stage nature. Existing APT detection methods based on provenance graphs have been proven to be effective. However, most studies based on provenance graphs are limited to single event detection, and rely on rule design and prior knowledge, making it difficult to effectively extract contextual information hidden in the graph. To address these issues, this paper innovatively proposes an APT detection model based on self-supervised contrastive learning, APT-SSC. First, the model uses an improved GraphSage(Graph Sample and Aggregate) encoder combined with a multi-head attention mechanism to learn the embedding representation of nodes; then, through context-based (local-global) contrastive learning, the robustness and context-awareness of node embeddings are effectively enhanced; finally, the optimized node embeddings are used as the input of the classification detection model to achieve node-level anomaly detection and tactics detection. Experiments are carried out on two public datasets, CICAPT-IIoT2024 (containing a variety of complex tactics) dataset and DAPRA TC-E3 dataset. The experimental results show that APT-SSC improves F1-Scores by 5% and 7% in anomaly detection and tactic detection experiments, respectively. APT-SSC has been proven to be significantly superior to existing methods in APT detection and can also effectively identify APT tactics.

As a key driving factor of land-atmosphere system, land surface temperature (LST) is widely applied in geoscience studies across various fields. Among numerous LST retrieval methods, the Split-Window (SW) algorithm has been widely used because of its advantage of free of atmospheric profile data. However, some satellites provide only one single available thermal infrared (TIR) channel, which limits the direct application of the SW algorithm. To overcome this shortcoming, this study takes Landsat-8 as an example, whose TIR channel-11 is affected by degraded calibration accuracy caused by stray light, and develops a method to construct a virtual channel using MODIS TIR data, enabling the application of the SW algorithm to Landsat-8 data for LST retrieval. During the construction, the angular normalization is adopted to the MODIS TIR data in advance. The validation results derived from the simulated dataset shows that the RMSE of LST retrieval based the virtual channel using the SW method is less than 1.2 K. Further validation with ground-based measurements from the FPK station results in an RMSE of 2.44 K, demonstrating better accuracy than the result from single channel algorithm. Moreover, the angular normalization applied to MODIS data leads to an improvement of 0.36 K in LST retrieval accuracy. The results demonstrate the advantages of LST retrieval from Landsat-8 data with virtual channel and extend the applicability of the SW algorithm.

This research proposes a coordination method for multi-connected and automated vehicles (CAVs) and traffic signal. It aims at reducing stop-and-go maneuvers of CAVs and enhancing traffic efficiency. The proposed method has the following highlights: i) Adaptive to actual CAV and humandriven vehicle (HV) mixed traffic; ii) Jointly optimization of both vehicle trajectory and signal timing via formulating in the spatial domain; iii) Parallel distributed computing. Simulation test results demonstrate that the proposed coordinated control significantly outperforms the benchmark method. The proposed method reduces the average travel delay by 29.56%, enhances fuel efficiency by 18.87%, and reduces stop count by 87.10%. The proposed parallel distributed computing algorithm ensures a computation time basically within 10 milliseconds. It indicates that the proposed method is ready for real-time large-scale implementation.

Abstract Lyapunov equation is extensively applied in engineering areas, and zeroing neural networks (ZNN) are very effective in solving this kind of equation. In this paper, two predefined-time stability theorems are used to devise new activation functions. Then, we obtain two new ZNN models, which are applied in solving the Lyapunov equation. This type of model is called the predefined-time stability-based zeroing neural network model. Compared with the ZNN models which have existed, the proposed model retains the noise-tolerant virtue and gains a new advantage: predefined-time convergence. Lastly, we verify that the model developed in this paper is superior to the known models in solving the time-variant Lyapunov equation via numerical simulations.

In the face of rapid global climate change and increasing occurrence of extreme weather events, acquiring seamless land surface temperature (LST) with high spatial and temporal resolution on a global scale has become increasingly crucial. However, the limited ability of Thermal Infrared (TIR) Remote Sensing to penetrate cloud cover has hindered the widespread application of TIR LST datasets. To address this limitation, we propose a novel reconstruction approach for cloud-covered pixels, which is established based on the annual surface temperature cycle. It shifted previous reconstruction from directly modelling LST to indirectly modelling the residual term derived from the LST observations and the annual surface temperature cycle (ATC) model fitted values. A random forest regression was used to build this estimation model and the model was applied to cloud-covered pixels to derive their LSTs. Taking the Iberia Peninsula as the study area, the proposed method was applied to generate the all-weather LST product of whole year 2021. The visual assessment demonstrates its robust performance across different seasons and weather conditions. Additionally, through the validation with the masked clear-sky LST observations, it reveals that the proposed method achieves a stable estimation accuracy, with the average value of the coefficient of determination (R2) and Root Mean Squared Error (RMSE) of above 0.8 and 1.08 K under different climatic conditions. In comparison, the validation with the ERA-5 land reanalysis data also indicates a relatively good consistency between the performance of the reconstructed LST and the clear-sky LST, although with a slight decline in R2and RMSE. Additionally, the indirect validation with near surface air temperature (NSAT) also shows the comparable ability of the reconstructed LST in NSAT estimation as the clear-sky LST, with an increase of RMSE no more than 0.95 K. In general, the proposed method shows good potentials in reconstructing cloud-covered LSTs with relatively stable performance under different cloud cover conditions and it can be applied for generating all-weather LST product.

Although the current coronavirus disease 2019 (COVID-19) vaccines have been used worldwide to halt spread of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the emergence of new SARS-CoV-2 variants with E484K mutation shows significant resistance to the neutralization of vaccine sera. To better understand the resistant mechanism, we calculated the binding affinities of 26 antibodies to wild-type (WT) spike protein and to the protein harboring E484K mutation, respectively. The results showed that most antibodies (~85%) have weaker binding affinities to the E484K mutated spike protein than to the WT, indicating the high risk of immune evasion of the mutated virus from most of current antibodies. Binding free energy decomposition revealed that the residue E484 forms attraction with most antibodies, while the K484 has repulsion from most antibodies, which should be the main reason of the weaker binding affinities of E484K mutant to most antibodies. Impressively, a monoclonal antibody (mAb) combination was found to have much stronger binding affinity with E484K mutant than WT, which may work well against the mutated virus. Based on binding free energy decomposition, we predicted that the mutation of four more residues on receptor-binding domain (RBD) of spike protein, viz., F490, V483, G485 and S494, may have high risk of immune evasion, which we should pay close attention on during the development of new mAb therapeutics.

Target prediction and virtual screening are two powerful tools of computer-aided drug design. Target identification is of great significance for hit discovery, lead optimization, drug repurposing and elucidation of the mechanism. Virtual screening can improve the hit rate of drug screening to shorten the cycle of drug discovery and development. Therefore, target prediction and virtual screening are of great importance for developing highly effective drugs against COVID-19. Here we present D3AI-CoV, a platform for target prediction and virtual screening for the discovery of anti-COVID-19 drugs. The platform is composed of three newly developed deep learning-based models i.e., MultiDTI, MPNNs-CNN and MPNNs-CNN-R models. To compare the predictive performance of D3AI-CoV with other methods, an external test set, named Test-78, was prepared, which consists of 39 newly published independent active compounds and 39 inactive compounds from DrugBank. For target prediction, the areas under the receiver operating characteristic curves (AUCs) of MultiDTI and MPNNs-CNN models are 0.93 and 0.91, respectively, whereas the AUCs of the other reported approaches range from 0.51 to 0.74. For virtual screening, the hit rate of D3AI-CoV is also better than other methods. D3AI-CoV is available for free as a web application at http://www.d3pharma.com/D3Targets-2019-nCoV/D3AI-CoV/index.php, which can serve as a rapid online tool for predicting potential targets for active compounds and for identifying active molecules against a specific target protein for COVID-19 treatment.

BACKGROUND: Knowledge of protein motions is significant to understand its functions. While currently available databases for protein motions are mostly focused on overall domain motions, little attention is paid on local residue motions. Albeit with relatively small scale, the local residue motions, especially those residues in binding pockets, may play crucial roles in protein functioning and ligands binding. RESULTS: A comprehensive protein motion database, namely D3PM, was constructed in this study to facilitate the analysis of protein motions. The protein motions in the D3PM range from overall structural changes of macromolecule to local flip motions of binding pocket residues. Currently, the D3PM has collected 7679 proteins with overall motions and 3513 proteins with pocket residue motions. The motion patterns are classified into 4 types of overall structural changes and 5 types of pocket residue motions. Impressively, we found that less than 15% of protein pairs have obvious overall conformational adaptations induced by ligand binding, while more than 50% of protein pairs have significant structural changes in ligand binding sites, indicating that ligand-induced conformational changes are drastic and mainly confined around ligand binding sites. Based on the residue preference in binding pocket, we classified amino acids into "pocketphilic" and "pocketphobic" residues, which should be helpful for pocket prediction and drug design. CONCLUSION: D3PM is a comprehensive database about protein motions ranging from residue to domain, which should be useful for exploring diverse protein motions and for understanding protein function and drug design. The D3PM is available on www.d3pharma.com/D3PM/index.php .

Discovering efficient drugs and identifying target proteins are still an unmet but urgent need for curing coronavirus disease 2019 (COVID-19). Protein structure-based docking is a widely applied approach for discovering active compounds against drug targets and for predicting potential targets of active compounds. However, this approach has its inherent deficiency caused by e.g. various different conformations with largely varied binding pockets adopted by proteins, or the lack of true target proteins in the database. This deficiency may result in false negative results. As a complementary approach to the protein structure-based platform for COVID-19, termed as D3Docking in our previous work, we developed in this study a ligand-based method, named D3Similarity, which is based on the molecular similarity evaluation between the submitted molecule(s) and those in an active compound database. The database is constituted by all the reported bioactive molecules against the coronaviruses, viz., severe acute respiratory syndrome coronavirus (SARS), Middle East respiratory syndrome coronavirus (MERS), severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), human betacoronavirus 2c EMC/2012 (HCoV-EMC), human CoV 229E (HCoV-229E) and feline infectious peritonitis virus (FIPV), some of which have target or mechanism information but some do not. Based on the two-dimensional (2D) and three-dimensional (3D) similarity evaluation of molecular structures, virtual screening and target prediction could be performed according to similarity ranking results. With two examples, we demonstrated the reliability and efficiency of D3Similarity by using 2D × 3D value as score for drug discovery and target prediction against COVID-19. The database, which will be updated regularly, is available free of charge at https://www.d3pharma.com/D3Targets-2019-nCoV/D3Similarity/index.php.