Despite the outstanding capabilities of large language models (LLMs), knowledge-intensive reasoning still remains a challenging task due to LLMs' limitations in compositional reasoning and the hallucination problem. A prevalent solution is to employ chain-of-thought (CoT) with retrieval-augmented generation (RAG), which first formulates a reasoning plan by decomposing complex questions into simpler sub-questions, and then applies iterative RAG at each sub-question. However, prior works exhibit two crucial problems: inadequate reasoning planning and poor incorporation of heterogeneous knowledge. In this paper, we introduce AtomR, a framework for LLMs to conduct accurate heterogeneous knowledge reasoning at the atomic level. Inspired by how knowledge graph query languages model compositional reasoning through combining predefined operations, we propose three atomic knowledge operators, a unified set of operators for LLMs to retrieve and manipulate knowledge from heterogeneous sources. First, in the reasoning planning stage, AtomR decomposes a complex question into a reasoning tree where each leaf node corresponds to an atomic knowledge operator, achieving question decomposition that is highly fine-grained and orthogonal. Subsequently, in the reasoning execution stage, AtomR executes each atomic knowledge operator, which flexibly selects, retrieves, and operates atomic level knowledge from heterogeneous sources. We also introduce BlendQA, a challenging benchmark specially tailored for heterogeneous knowledge reasoning. Experiments on three single-source and two multi-source datasets show that AtomR outperforms state-of-the-art baselines by a large margin, with absolute F1 score improvements of 9.4% on 2WikiMultihop and 9.5% on BlendQA. We release our code and data https://github.com/THU-KEG/AtomR.git.

Recently, deep semi-supervised graph embedding learning has shown much promise on text and image recognition tasks when the number of labeled data is limited. By introducing an auxiliary unsupervised task of predicting the neighborhood context in the graph, these approaches effectively mine the structure information provided by abundant unlabeled data. However, existing methods usually adapt to datasets whose graph is explicitly given or predefined, which cannot handle large-scale datasets with unknown graphs. Besides, the edge connections and weights are fixed during the training process in these methods, which fails to use the current feature information extracted by the model. In this paper, we propose a novel deep semi-supervised dynamic anchor graph embedding learning algorithm. More specifically, we build a two-branch architecture to learn the single-sample local features and the global features in the graph simultaneously. The first branch constrains its outputs to be consistent with different perturbations of the same single sample. And the output features are dedicated to constructing the dynamic anchor graph. The second branch utilizes the graph and the model prediction to sample the context. Then the global graph embed dings are learned based on the context. Finally, the outputs of the two branches are aggregated to jointly predict the class labels. Extensive experimental results on several image and text datasets have shown that the proposed method is able to improve the performance of existing graph embedding learning methods and outperform many state-of-the-art methods on image classification.