Neural networks remain black-box systems, unsure about their outputs, and their performance may drop unpredictably in real applications. An open question is how to qualitatively extend neural networks, so that they are sure about their reasoning results, or reasoning-for-sure. Here, we introduce set-theoretic relations explicitly and seamlessly into neural networks by extending vector embedding into sphere embedding, so that part-whole relations can explicitly encode set-theoretic relations through sphere boundaries in the vector space. A reasoning-for-sure neural network successfully constructs, within a constant number M of epochs, a sphere configuration as its semantic model for any consistent set-theoretic relation. We implement Hyperbolic Sphere Neural Network (HSphNN), the first reasoning-for-sure neural network for all types of Aristotelian syllogistic reasoning. Its construction process is realised as a sequence of neighbourhood transitions from the current towards the target configuration. We prove M=1 for HSphNN. In experiments, HSphNN achieves the symbolic level rigour of syllogistic reasoning and successfully checks both decisions and explanations of ChatGPT (gpt-3.5-turbo and gpt-4o) without errors. Through prompts, HSphNN improves the performance of gpt-3.5-turbo from 46.875% to 58.98%, and of gpt-4o from 82.42% to 84.76%. We show ways to extend HSphNN for various kinds of logical and Bayesian reasoning, and to integrate it with traditional neural networks seamlessly.

Zijun Yao, Chengjiang Li, Tiansi Dong, Xin Lv, Jifan Yu, Lei Hou, Juanzi Li, Yichi Zhang, Zelin Dai. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

A Triple in knowledge-graph takes a form that consists of head, relation, tail. Triple Classification is used to determine the truth value of an unknown Triple. This is a hard task for 1-to-N relations using the vector-based embedding approach. We propose a new region-based embedding approach using fine-grained type chains. A novel geometric process is presented to extend the vectors of pre-trained entities into n-balls (n-dimensional balls) under the condition that head balls shall contain their tail balls. Our algorithm achieves zero energy cost, therefore, serves as a case study of perfectly imposing tree structures into vector space. An unknown Triple (h,r,x) will be predicted as true, when x’s n-ball is located in the r-subspace of h’s n-ball, following the same construction of known tails of h. The experiments are based on large datasets derived from the benchmark datasets WN11, FB13, and WN18. Our results show that the performance of the new method is related to the length of the type chain and the quality of pre-trained entityembeddings, and that performances of long chains with welltrained entity-embeddings outperform other methods in the literature. Source codes and datasets are located at https: //github.com/GnodIsNait/mushroom.

Hailong Jin, Lei Hou, Juanzi Li, Tiansi Dong. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

Fine-grained entity typing aims at identifying the semantic type of an entity in KB. Type information is very important in knowledge bases, but are unfortunately incomplete even in some large knowledge bases. Limitations of existing methods are either ignoring the structure and type information in KB or requiring large scale annotated corpus. To address these issues, we propose an attributed and predictive entity embedding method, which can fully utilize various kinds of information comprehensively. Extensive experiments on two real DBpedia datasets show that our proposed method significantly outperforms 8 state-of-the-art methods, with 4.0% and 5.2% improvement in Mi-F1 and Ma-F1, respectively.

Assertions are intensively used to facilitate correctness reasoning and error detection in daily programming. However, composing assertions for mobile robotic programs can be painfully inconvenient, because classic Hoare logic lacks the expressing power on spatial knowledge, which is crucial when robots interact with their physical environments. The problem is especially evident for Behavior-Based Robotics (BBR) where the world is not explicitly represented with program variables. In this paper, we propose to incorporate spatial reasoning capability in the assertion framework of Hoare logic. The proposed framework features a two-dimensional region calculus and additional axioms for robot movements. The calculus makes the world representation and the specification of mobile robotic program natural and intuitive, and the axioms enable the reasoning about program correctness. We illustrate the use of the framework with a typical behavior-based robotic program. In addition, we present a runtime error detection and recovery mechanism for BBR programs based on the assertion framework. Preliminary experiments with NAO robots demonstrate the effectiveness.

Joint representation learning of words and entities benefits many NLP tasks, but has not been well explored in cross-lingual settings.In this paper, we propose a novel method for joint representation learning of cross-lingual words and entities.It captures mutually complementary knowledge, and enables cross-lingual inferences among knowledge bases and texts.Our method does not require parallel corpora, and automatically generates comparable data via distant supervision using multi-lingual knowledge bases.We utilize two types of regularizers to align cross-lingual words and entities, and design knowledge attention and crosslingual attention to further reduce noises.We conducted a series of experiments on three tasks: word translation, entity relatedness, and cross-lingual entity linking.The results, both qualitatively and quantitatively, demonstrate the significance of our method.1 https://en.wikipedia.org/wiki/Help: Interlanguage_links• We did qualitative analysis to have an intuitive impression of our embeddings, and quantitative analysis in three tasks: word translation, entity relatedness, and crosslingual entity linking.Experiment results show that our method demonstrates significant improvements in all three tasks.

Chang [S. K. Chang, A general framework for slow intelligence systems, International Journal of Software Engineering and Knowledge Engineering, 20(1) (2010) 1–18.] argues that slow intelligence is a property shared by a large number of intelligent systems, and proposes a general framework, which has quick/slow decision cycles. The aim of this paper is to show that the general framework is applicable for modeling human intelligence. We show that the quick/slow decision cycles model the procedural/declarative memory system, and collect supports from adults' second language acquisition and aphasic bilinguals. As spatial cognition is fundamental in human intelligence, we show in detail how object tracing can be simulated within slow intelligence framework. We introduce a psychological experiment on object tracing, and present two rules in psychology which govern object tracing results under certain conditions, namely, the rule of minimal spatial transformation, and the rule of categorical continuity. We review related works in spatial knowledge representation and reduce object tracing problems into the problem of object mapping. We show that the following capabilities can be simulated in the slow intelligence framework: the capability of acquiring knowledge about one scene, and the capability of mapping object between scenes. An experimental result is presented.