Understanding complex three-dimensional (3D) structures of graphs is essential for accurately modeling various properties, yet many existing approaches struggle with fully capturing the intricate spatial relationships and symmetries inherent in such systems, especially in large-scale, dynamic molecular datasets. These methods often must balance trade-offs between expressiveness and computational efficiency, limiting their scalability. To address this gap, we propose a novel Geometric Tensor Network (GotenNet) that effectively models the geometric intricacies of 3D graphs while ensuring strict equivariance under the Euclidean group E(3). Our approach directly tackles the expressiveness-efficiency trade-off by leveraging effective geometric tensor representations without relying on irreducible representations or Clebsch-Gordan transforms, thereby reducing computational overhead. We introduce a unified structural embedding, incorporating geometry-aware tensor attention and hierarchical tensor refinement that iteratively updates edge representations through inner product operations on high-degree steerable features, allowing for flexible and efficient representations for various tasks. We evaluated models on QM9, rMD17, MD22, and Molecule3D datasets, where the proposed model consistently outperforms state-of-the-art methods in both scalar and high-degree property predictions, demonstrating exceptional robustness across diverse datasets, and establishes GotenNet as a versatile and scalable framework for 3D equivariant Graph Neural Networks.

Representation learning on spatial networks is emerging as a distinct area in machine learning and is attracting much attention in diverse domains. Some applications include molecular graphs for drug discovery and screening, brain networks for neuroscience, social networks for recommender systems, etc. However, most proposed approaches either use only spatial or network data that cannot distinguish certain types of graphs and limit their network expressivity or are tied to specific input data domains and network architectures. To address this gap, we introduce an equivariant message-passing network architecture that simultaneously leverages the spatial and network properties. In addition, we propose to take advantage of the geometric representations and extend the classical scalar features with 3D vectors. To exploit the spatial and network features, we present a Spatial Vector Neuron, which can be easily incorporated into the existing graph neural network architectures and allow the model to scale up by stacking more layers for larger receptive fields. A comprehensive set of experiments on both synthetic and real-world datasets demonstrate the strength of our proposed method and the potential of geometric representation learning for Spatial Networks.

Three-dimensional (3D) graph representations have gained significant importance in numerous scientific domains, such as molecular dynamics and astrophysics. In these applications, a precise and effective representation of 3D graphs is crucial. Although various neural network architectures have been proposed to encapsulate the intricate relationships inherent to these graphs, many remain sensitive to challenges such as reflection variances observed in molecular structures. In this paper, we present an SE(3) equivariant neural network architecture tailored for 3D graphs. Our approach uniquely integrates equivariance to both rotations and translations, ensuring robustness against these spatial variances. Distinctively, our model leverages an innovative message passing mechanism, endowing it with an intrinsic understanding of reflection variances. This mechanism not only facilitates superior 3D graph representation but also promotes computational efficiency. Unlike many existing methods, our architecture sidesteps the need for computationally demanding higher-order representations in intermediary layers while still achieving competitive or superior performance metrics. Empirical evaluations conducted on synthetic and real-world datasets underscore the superior performance of our approach in comparison to prevalent models in terms of both accuracy and efficiency.

With the rapid development of the airline industry, maximizing the market share with a constrained budget is an urgent econometric problem for an airline. We investigate the problem by adjusting flight frequencies on different flight routes. Owing to the large search space of solutions and the difficulty of predicting the market, this problem is in general daunting to solve. This paper proposes a novel two-stage optimization method to address the challenges. On the higher level, we use a signal to guide the optimization process toward a constrained satisfying solution. On the lower level, we consider the consecutive itineraries in real scenarios and model the unseen correlations between routes in itineraries for market share prediction. In theory, we prove the convergence of our optimization approach. In the experiment, we empirically verify the superiority of both our prediction model and optimization approach over existing works with large-scale real-world data. Our code has been released at: https://github.com/codingAndBS/AirlineMarket.

Geometric representation learning of molecules is challenging yet essential for applications in multiple domains. Despite the impressive breakthroughs made by geometric deep learning in various molecular representation learning tasks, effectively capturing complicated geometric features across spatial dimensions is still underexplored due to the significant difficulties in modeling efficient geometric representations and learning the inherent correlation in 3D structural modeling. These include computational inefficiency, underutilization of vectorial embeddings, and limited generalizability to integrate various geometric properties. To address the raised concerns, we introduce an efficient and effective framework, Scalable Vector Network (SaVeNet), designed to accommodate a range of geometric requirements without depending on costly embeddings. In addition, the proposed framework scales effectively with introduced direction noise. Theoretically, we analyze the desired properties (i.e., invariance and equivariant) and framework efficiency of the SaVeNet. Empirically, we conduct a comprehensive series of experiments to evaluate the efficiency and expressiveness of the proposed model. Our efficiency-focused experiments underscore the model's empirical superiority over existing methods. Experimental results on synthetic and real-world datasets demonstrate the expressiveness of our model, which achieves state-of-the-art performance across various tasks within molecular representation learning.

Exploring drug-target interaction remains one of the essential tasks in drug discovery, and it is critical to gain a thorough understanding of the biological process and disease mechanisms. Despite recent successes in the application of machine learning approaches, drug-target interaction studies are still largely under-explored due to significant challenges in modeling different types of representations and capturing the inherent correlation between targets and drugs from low-level representations. What is more, the length of the target protein sequences and the complexity of the drug-target binding complex make the problem hard to handle. In this work, we focus on increasing the generalizability and interpretability of the drug-target prediction models and propose an Extrinsic-Intrinsic Representation learning model (EIR) intended to discover the inner correlation between target proteins and drugs on both the extrinsic and intrinsic levels. Our experimental results show that EIR makes more accurate predictions than the state-of-the-art method in both drug-target affinity prediction and drug-target interface prediction tasks and demonstrate the potential of the structural-free method for drug discovery.

The study of protein-ligand interaction is critical for gaining a thorough understanding of biological processes and uncovering disease mechanisms. This exciting topic has attracted much interest. Accurate and reliable prediction of the protein-ligand interactions can be challenging because the model requires successful transformation and creation of the computer interpretable representations of both protein and ligand information. Despite the recent successes in the application of the convolution neural network and graph neural network-based approaches, they remain largely under-explored due to the significant challenges in modeling the complex representations and capturing the inherent correlation between protein and ligand from low-level representations. Several challenges include: 1) Structural-free protein representation learning model is needed to successfully discover the inner correlation between protein and ligand. 2) Limited open literature to date has focused on the generalizability and interpretability for the protein-ligand interface prediction. 3) The length of the protein sequences and the complexity of the protein-ligand binding complex make the problem hard to handle. To address these problems, we propose an end-to-end model framework APIP that could learn effective representations across compound atoms and protein residues for protein-ligand interface prediction. Our experimental results show that APIP makes more accurate predictions than the state-of-the-art method in this task and demonstrate the potential of the attention-based method for drug discovery.

Matching equivalent entities across Knowledge graphs is a pivotal step for knowledge fusion.Previous approaches usually study the problem in Euclidean space.However, recent works have shown that hyperbolic space has a higher capacity than Euclidean space and hyperbolic embedding can represent the hierarchical structure in a knowledge graph.In this paper, we propose a localized geometric method to find equivalent entities in hyperbolic space.Specifically, we use a hyperbolic neural network to encode the lingual information of entities and the structure of both knowledge graphs into a low-dimensional hyperbolic space.To address the asymmetry of structure on different KGs and the localized nature of relations, we learn an instance-specific geometric mapping function based on rotation to match entity pairs.A contrastive loss function is used to train the model.The experiment verifies the power of low-dimensional hyperbolic space for entity matching and shows that our method outperforms the state of the art by a large margin.

Representation learning of protein 3D structures is challenging and essential for applications, e.g., computational protein design or protein engineering. Recently, geometric deep learning has achieved great success in non-Euclidean domains. Although protein can be represented as a graph naturally, it remains under-explored mainly due to the significant challenges in modeling the complex representations and capturing the inherent correlation in the 3D structure modeling. Several challenges include: 1) It is challenging to extract and preserve multi-level rotation and translation equivariant information during learning. 2) Difficulty in developing appropriate tools to effectively leverage the input spatial representations to capture complex geometries across the spatial dimension. 3) Difficulty in incorporating various geometric features and preserving the inherent structural relations. In this work, we introduce geometric bottleneck perceptron, and a general SO(3)-equivariant message passing neural network built on top of it for protein structure representation learning. The proposed geometric bottleneck perceptron can be incorporated into diverse network architecture backbones to process geometric data in different domains. This research shed new light on geometric deep learning in 3D structure studies. Empirically, we demonstrate the strength of our proposed approach on three core downstream tasks, where our model achieves significant improvements and outperforms existing benchmarks. The implementation is available at https://github.com/sarpaykent/GBPNet.

Determining a protein's 3D from its sequences is one of the most challenging problems in biology. Recently, geometric deep learning has achieved great success on non-Euclidean domains including social networks, chemistry, and computer graphics. Although it is natural to present protein structures as 3D graphs, existing research has rarely studied protein structures as graphs directly. The present research explores the geometry deep learning of three-dimensional graphs on protein structures and proposes a graph neural network architecture to address these challenges. The proposed Protein Geometric Graph Neural Network (PG-GNN) models both distance geometric graph representation and dihedral geometric graph representation by geometric graph convolutions. This research shed new light on protein 3D structure studies. We investigated the effectiveness of graph neural networks over five real datasets. Our results demonstrate the potential of GNNs for 3D structure prediction.