Various neural solvers have been devised for combinatorial optimization (CO), which are often tailored for specific problem types, e.g., TSP, CVRP and SAT, etc. Yet, it remains an open question how to achieve universality regarding problem representing and learning with a general framework. This paper first proposes **UniCO**, to unify a set of CO problems by reducing them into the *general* TSP form featured by distance matrices. The applicability of this strategy depends on the efficiency of the problem reduction and solution transition procedures, which we show that at least ATSP, HCP, and SAT are readily feasible. The hope is to allow for the effective and even simultaneous use of as many types of CO instances as possible to train a neural TSP solver, and optionally finetune it for specific problem types. In particular, unlike the prevalent TSP benchmarks based on Euclidean instances with 2-D coordinates, our studied domain of TSP could involve non-metric, asymmetric or discrete distances without explicit node coordinates, which is much less explored in TSP literature while poses new intellectual challenges. Along this direction, we devise two neural TSP solvers with and without supervision to conquer such matrix-formulated input, respectively: 1) **MatPOENet** and 2) **MatDIFFNet**. The former is a reinforcement learning-based sequential model with pseudo one-hot embedding (POE) scheme; and the latter is a Diffusion-based generative model with the mix-noised reference mapping scheme. Experiments on ATSP, 2DTSP, HCP- and SAT-distributed general TSPs show the strong ability towards arbitrary matrix-encoded TSP with structure and size variation.

Graph-structured data is ubiquitous, and graph learning models have recently been extended to address complex problems like mixed-integer linear programming (MILP). However, studies have shown that the vanilla message-passing based graph neural networks (GNNs) suffer inherent limitations in learning MILP instance representation, i.e., GNNs may map two different MILP instance graphs to the same representation. In this paper, we introduce an expressive quantum graph learning approach, leveraging quantum circuits to recognize patterns that are difficult for classical methods to learn. Specifically, the proposed General Quantum Graph Learning Architecture (GQGLA) is composed of a node feature layer, a graph message interaction layer, and an optional auxiliary layer. Its generality is reflected in effectively encoding features of nodes and edges while ensuring node permutation equivariance and flexibly creating different circuit structures for various expressive requirements and downstream tasks. GQGLA is well suited for learning complex graph tasks like MILP representation. Experimental results highlight the effectiveness of GQGLA in capturing and learning representations for MILPs. In comparison to traditional GNNs, GQGLA exhibits superior discriminative capabilities and demonstrates enhanced generalization across various problem instances, making it a promising solution for complex graph tasks.

Tensor network has been a promising numerical tool for computational problems across science and AI. For their emerging and fast development especially in the intersection between AI and science, this paper tries to present a compact review, regarding both their applications and its own recent technical development including open-source tools. Specifically, we make the observations that tensor network plays a functional role in matrix compression and representation, information fusion, as well as quantum-inspired algorithms, which can be generally regarded as Science4AI in our survey. On the other hand, there is an emerging line of research in tensor network in AI4Science especially like learning quantum many-body physics by using e.g. neural network quantum state. Importantly, we unify tensorization methodologies across classical and modern architectures, and particularly show how tensorization bridges low-order parameter spaces to high-dimensional representations without exponential parameter growth, and further point out their potential use in scientific computing. We conclude the paper with outlook for future trends.

We rethink the basic operations, i.e., inner product and matrix multiplication used in neural networks. A quantum-inspired alternative is proposed, utilizing the power of high-dimensional Hilbert space by devising a high-order form of tensor product. We re-parameterize the original (low-order) vectors/matrices into an expressive high-order form, without incurring extra model parameters, and the extra computational overhead is negligible (e.g., about 2%). As an in-place transparent atomic operation, we show its use in the key components in Transformers: token embeddings, attentions (query, key, value) and the MLP. Due to its inherent compatibility to vanilla multiplicative operations, we propose C2Q-SFT, i.e., classic-to-quantum (C2Q) protocol for supervised fine-tuning (SFT): it continues to train a given model by transparently replacing the standard operations with ours. As shown by our experiments, it shows advantages for both training from scratch and fine-tuning on downstream tasks across scales of LLMs. C2Q-SFT consistently outperforms standard SFT, with relative improvements on MMLU (+0.56%) and GSM8k (+0.61%). It sheds light on the innovation of operations in networks, orthogonal to the efforts on new architecture, position encoding, and training algorithms, etc. See project page at: https://github.com/Thinklab-SJTU/LLM/QI-LLM.

For the essential operation, namely inner product (IP) as widely adopted in classic computing e.g. matrix multi-plication, its quantum counterpart: quantum inner product (QIP), has also been recently theoretically explored with a verifiable lower complexity on quantum computers. How-ever, it remains unclear for the embodiment of the quantum circuits (QC) for QIP, let alone a (thorough) evaluation of the QIP circuits, especially in a practical context in the NISQ era by applying QIP to ML via hybrid quantum-classic pipelines. In this paper, we carefully design the QIP circuits from scratch, whose complexity is in accordance with the theoretical complexity. To make the simulation tractable on classic computers, especially when it is integrated in the gradient-based hybrid ML pipelines, we further devise a highly-efficient simulation scheme by directly simulates the output state. Experiments show that the scheme acceler-ates the simulation for more than 68k times compared with the previous circuit simulator. This allows our empirical evaluation on typical machine learning tasks, ranging from supervised and self-supervised learning via neural nets, to K-Means clustering. The results show that the calculation error brought by typical quantum mechanisms would incur in general little influence on the final numerical results given sufficient qubits. However, certain tasks e.g. ranking in K-Means could be more sensitive to quantum noise.

Network embedding (NE) is a prominent technique for network analysis where the nodes are represented as vectorized embeddings in a continuous space. Existing works tend to resort to the low-dimensional embedding space for efficiency and less risk of over-fitting. In this paper, we explore a new NE paradigm whose embedding dimension goes exponentially high w.r.t. the number of parameters, yet being very efficient and effective. Specifically, the node embeddings are represented as product states that lie in a super high-dimensional (e.g. $2^{32}$-dim) quantum Hilbert space, with a carefully designed optimization approach to guarantee the robustness to work in different scenarios. In the experiments, we show diverse virtues of our methods, including but not limited to: the overwhelming performance on downstream tasks against conventional low-dimensional NE baselines with the similar amount of computing resources, the super high efficiency for a fixed low embedding dimension (e.g. 512) with less than 1/200 memory usage, the robustness when equipped with different objectives and sampling strategies as a fundamental tool for future NE research. As a relatively unexplored topic in literature, the high-dimensional NE paradigm is demonstrated effective both experimentally and theoretically.

Estimating the properties of quantum systems such as quantum phase has been critical in addressing the essential quantum many-body problems in physics and chemistry. Deep learning models have been recently introduced to property estimation, surpassing conventional statistical approaches. However, these methods are tailored to the specific task and quantum data at hand. It remains an open and attractive question for devising a more universal task-agnostic pretraining model for quantum property estimation. In this paper, we propose LLM4QPE, a large language model style quantum task-agnostic pretraining and finetuning paradigm that 1) performs unsupervised pretraining on diverse quantum systems with different physical conditions; 2) uses the pretrained model for supervised finetuning and delivers high performance with limited training data, on downstream tasks. It mitigates the cost for quantum data collection and speeds up convergence. Extensive experiments show the promising efficacy of LLM4QPE in various tasks including classifying quantum phases of matter on Rydberg atom model and predicting two-body correlation function on anisotropic Heisenberg model.

Skip-gram models are popular in large-scale network embedding for their cost-effectiveness. The objectives of many skip-gram based methods relate to the word2vec model which closely relates to Noise Contrastive Estimation (NCE). Among existing embedding methods, the differences mostly lie in how the node neighborhood is modeled e.g. by different ways of random walk, which leads to different learning strategies. Orthogonal to these efforts, we take a unified view that the NCE based methods commonly involve two basic NCE components in the learning objective. This perspective allows a natural generalization of the objectives by taking different forms of scoring function in the NCE components. We theoretically analyze how the vanilla NCE-based objectives suffer from the slow convergence speed and challenge in first-/second-order proximity preservation. We also prove the fundamental difficulty for NCE methods to capture non-linearity of complex networks. To mitigate such issues, we devise a general distance-based term added to the used NCE term, inspired by its physical meaning. The distance functions include Wasserstein-k distance and Laplacian/Gaussian kernel functions, with relatively little additional time overhead. The effectiveness of our approach is verified both by prototype examples as well as real-world datasets, for the task of node classification and network reconstruction.

Multi-order proximity is useful for effective network embedding. In contrast to many previous works that only consider order-level weights, this paper proposes to explore a more expressive node-level weighting mechanism to encode the diverse local structure, with a scalable and theoretically justified sampling strategy for its learning. Specifically, we start with a formal definition of multi-order proximity matrix which leads to our new multi-order objective based on Laplacian Eigenmaps and Skip-Gram. Then we instantiate the node-specific multi-order weights in the objective with the help of neighborhood size estimation, which indicates node-specific multi-order information. For objective learning, it is implicitly fulfilled with our proposed branching tree-like random walk strategy termed by BTWalk, which differs from the dominant chain-like walk in existing sampling techniques. BTWalk is designed by a synergetic combination of BFS (breadth-first search) and DFS (depth-first search), which is modulated according to the weights of the considered proximity orders. We theoretically analyze its cost-efficiency, and further propose the so-called Vec4Cross framework that incorporates joint node embedding and network alignment for two partially overlapped networks based on the seed matchings, whereby BTWalk is also adopted for embedding. Promising experimental results are obtained on real-world datasets across popular tasks.

Despite its success in learning network node representations, network embedding is still relatively new for multiplex networks (MNs) with multiple types of edges. In such networks, the inter-layer anchor links are usually missing, which represent the alignment relations between nodes on different layers and are a crucial prerequisite for many cross-network applications like network alignment. For mining such anchor links between layers for MNs, multiplex network embedding (MNE) has become one of the most promising techniques. In this paper, we consider two problems for MNs: 1) edges can be missing to different extent, and data augmentation may mitigate this issue; 2) the known alignment anchor links between layers can be misleading since the behaviors of nodes on different layers are not always consistent, so the most informative ones should be emphasized compared with those misleading ones. However, most existing works neglect the two problems and simply 1) adopt one structural view for all the layers (e.g. random walk with the same window size) and 2) equally extract information from all the anchor links. We propose an end-to-end contrastive framework called cM2NE for MNE, utilizing multiple structural views for each layer and learning with several plug-in components for different scenarios. Through end-to-end optimization on three levels, the intra-view, inter-view, and inter-layer level, our framework achieves to select the fitted views for different layers and maximize the inter-layer mutual information by emphasizing those most informative anchor links. Extensive experimental results on real-world datasets for node classification and multi-network alignment show that our approach consistently outperforms peer methods.