Tanishq Gupta, Mohd Zaki, Devanshi Khatsuriya, Kausik Hira, N M Anoop Krishnan, Mausam -. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.

Augmenting pre-trained language models with knowledge graphs (KGs) has achieved success on various commonsense reasoning tasks. However, for a given task instance, the KG, or certain parts of the KG, may not be useful. Although KG-augmented models often use attention to focus on specific KG components, the KG is still always used, and the attention mechanism is never explicitly taught which KG components should be used. Meanwhile, saliency methods can measure how much a KG feature (e.g., graph, node, path) influences the model to make the correct prediction, thus explaining which KG features are useful. This paper explores how saliency explanations can be used to improve KG-augmented models' performance. First, we propose to create coarse (Is the KG useful?) and fine (Which nodes/paths in the KG are useful?) saliency explanations. Second, to motivate saliency-based supervision, we analyze oracle KG-augmented models which directly use saliency explanations as extra inputs for guiding their attention. Third, we propose SalKG, a framework for KG-augmented models to learn from coarse and/or fine saliency explanations. Given saliency explanations created from a task's training set, SalKG jointly trains the model to predict the explanations, then solve the task by attending to KG features highlighted by the predicted explanations. On three popular commonsense QA benchmarks (CSQA, OBQA, CODAH) and a range of KG-augmented models, we show that SalKG can yield considerable performance gains --- up to 2.76% absolute improvement on CSQA.

With the advancement in the technology, objects can be represented effectively in their 3D digital models which accurately represents their physical counterparts. Navigation services and mapping based on geographical data have become very popular in supporting our everyday lives. Much of these services are currently available mostly for outdoor purposes, however applications for indoor purposes are being explored where most of the human activities takes place. This can help transform cities into “Smart Cities”. The aim of this study is to develop an indoor mapping system for data collection in a building environment by exploring new, efficient and cost effective scanning devices. The conventional devices currently in use are expensive which makes them difficult to implement for large scale applications. The data will be collected using a 3D scanning camera technology which develops depth maps of various locations. Xbox's Kinect Sensor and Stereolab's ZED camera are being used and compared in this study. Comparisons based on resolution, lighting, accuracy, speed and memory are being made in this study. Their pros and cons over conventional scanning devices are also discussed. The study shows the possibility of using this technology in a large scale building environment in an autonomous method for the future. This technology can then be potentially used for commercial purposes especially to track progress at construction sites, security purposes, facility management, retail and augmented reality applications.