Chart question answering (ChartQA) is challenged by the heterogeneous composition of chart elements and the subtle data patterns they encode.This work introduces a novel joint multimodal scene graph framework that explicitly models the relationships among chart components and their underlying structures.The framework integrates both visual and textual graphs to capture structural and semantic characteristics, while a graph contrastive learning strategy aligns node representations across modalities-enabling their seamless incorporation into a transformer decoder as soft prompts.Moreover, a set of tailored Chain-of-Thought (CoT) prompts is proposed to enhance multimodal large language models (MLLMs) in zero-shot scenarios by mitigating hallucinations.Extensive evaluations on benchmarks including ChartQA, OpenCQA, and ChartX demonstrate significant performance improvements and validate the efficacy of the proposed approach.

Projecting knowledge graph queries into an embedding space using geometric models (points, boxes and spheres) can help to answer queries for large incomplete knowledge graphs. In this work, we propose a symbolic learning-free approach using fuzzy logic to address the shape-closure problem that restricted geometric-based embedding models to only a few shapes (e.g. ConE) for answering complex logical queries. The use of symbolic approach facilitates non-closure geometric models (e.g. point, box) to handle logical operators (including negation). This enabled our newly proposed spherical embeddings (SpherE) in this work to use a polar coordinate system to effectively represent hierarchical relation. Results show that the SpherE model can answer existential positive first-order logic and negation queries. We show that SpherE significantly outperforms the point and box embeddings approaches while generating semantically meaningful hierarchy-aware embeddings.

Automatic Chart Question Answering (ChartQA) is challenging due to the complex distribution of chart elements with patterns of the underlying data not explicitly displayed in charts. To address this challenge, we design a joint multimodal scene graph for charts to explicitly represent the relationships between chart elements and their patterns. Our proposed multimodal scene graph includes a visual graph and a textual graph to jointly capture the structural and semantical knowledge from the chart. This graph module can be easily integrated with different vision transformers as inductive bias. Our experiments demonstrate that incorporating the proposed graph module enhances the understanding of charts' elements' structure and semantics, thereby improving performance on publicly available benchmarks, ChartQA and OpenCQA.

We present an overview of our triple extraction system for the ICDM 2019 Knowledge Graph Contest. Our system uses a pipeline-based approach to extract a set of triples from a given document. It offers a simple and effective solution to the challenge of knowledge graph construction from domain-specific text. It also provides the facility to visualise useful information about each triple such as the degree, betweenness, structured relation type(s), and named entity types.

In many machine learning applications there exists prior knowledge that the response variable should be non-decreasing in one or more of the features. For example, the chance of a tumour being malignant should not decrease with increasing diameter (all else being equal). While a number of classification algorithms make use of monotone knowledge, many are limited to full monotonicity (in all features). Taking inspiration from instance based classifiers, we present a framework for monotone additive rule ensembles that is the first to cater for partial monotonicity (in some features). We demonstrate it by developing a partially monotone instance based classifier based on L1 cones. Experiments show that the algorithm produces reasonable results on real data sets while ensuring perfect partial monotonicity.

The meaning of a multi-word phrase not only depends on the meaning of its constituent words, but also the rules of composing them to give the so-called compositional semantic. However, many deep learning models for learning compositional semantics target specific NLP tasks such as sentiment classification. Consequently, the word embeddings encode the lexical semantics, the weights of the networks are optimised for the classification task. Such models have no mechanisms to explicitly encode the compositional rules, and hence they are insufficient in capturing the semantics of phrases. We present a novel recurrent computational mechanism that specifically learns the compositionality by encoding the compositional rule of each word into a matrix. The network uses a recurrent architecture to capture the order of words for phrases with various lengths without requiring extra preprocessing such as part-of-speech tagging. The model is thoroughly evaluated on both supervised and unsupervised NLP tasks including phrase similarity, noun-modifier questions, sentiment distribution prediction, and domain specific term identification tasks. We demonstrate that our model consistently outperforms the LSTM and CNN deep learning models, simple algebraic compositions, and other popular baselines on different datasets.