Understanding protein functions in biological processes is pivotal for disease elucidation and drug discovery. Despite notable progress, existing approaches primarily focus on function transfer under in-distribution (ID) settings, where training and test proteins exhibit high sequence similarity. As a result, their performance often degrades when applied to novel, diverse, and low-homology protein sequences, posing a major challenge for out-of-distribution (OOD) generalization encountered in practice. Towards this end, we develop ProteinScore, a graph transformer approach tailored to improve protein function prediction in OOD settings. ProteinScore integrates a label-invariant variational subgraph generator with self-supervised contrastive learning, thereby identifying meaning substructures within proteins. By highlighting informative features while filtering out redundant ones, ProteinScore improves generalization to diverse and low-homology sequences. Experiments on datasets with both experimentally resolved and AlphaFold2-predicted structures demonstrate that ProteinScore consistently outperforms strong baselines and provides biologically meaningful interpretability through accurately identifying binding sites. In addition, ProteinScore generalizes effectively to two additional downstream tasks, drug-target interaction classification and subcellular localization prediction, achieving superior predictive performance and reliable interpretability.

MOTIVATION: In recent years, high-throughput sequencing technologies have made large-scale protein sequences accessible. However, their functional annotations usually rely on low-throughput and pricey experimental studies. Computational prediction models offer a promising alternative to accelerate this process. Graph neural networks have shown significant progress in protein research, but capturing long-distance structural correlations and identifying key residues in protein graphs remains challenging. RESULTS: In the present study, we propose a novel deep learning model named Hierarchical graph transformEr with contrAstive Learning (HEAL) for protein function prediction. The core feature of HEAL is its ability to capture structural semantics using a hierarchical graph Transformer, which introduces a range of super-nodes mimicking functional motifs to interact with nodes in the protein graph. These semantic-aware super-node embeddings are then aggregated with varying emphasis to produce a graph representation. To optimize the network, we utilized graph contrastive learning as a regularization technique to maximize the similarity between different views of the graph representation. Evaluation of the PDBch test set shows that HEAL-PDB, trained on fewer data, achieves comparable performance to the recent state-of-the-art methods, such as DeepFRI. Moreover, HEAL, with the added benefit of unresolved protein structures predicted by AlphaFold2, outperforms DeepFRI by a significant margin on Fmax, AUPR, and Smin metrics on PDBch test set. Additionally, when there are no experimentally resolved structures available for the proteins of interest, HEAL can still achieve better performance on AFch test set than DeepFRI and DeepGOPlus by taking advantage of AlphaFold2 predicted structures. Finally, HEAL is capable of finding functional sites through class activation mapping. AVAILABILITY AND IMPLEMENTATION: Implementations of our HEAL can be found at https://github.com/ZhonghuiGu/HEAL.