Multi-hop Question Answering (MHQA) adds layers of complexity to question answering, making it more challenging.When Language Models (LMs) are prompted with multiple search results, they are tasked not only with retrieving relevant information but also employing multi-hop reasoning across the information sources.Although LMs perform well on traditional question-answering tasks, the causal mask can hinder their capacity to reason across complex contexts.In this paper, we explore how LMs respond to multi-hop questions by permuting search results (retrieved documents) under various configurations.Our study reveals interesting findings as follows: 1) Encoderdecoder models, such as the ones in the Flan-T5 family, generally outperform causal decoderonly LMs in MHQA tasks, despite being significantly smaller in size; 2) altering the order of gold documents reveals distinct trends in both Flan T5 models and fine-tuned decoderonly models, with optimal performance observed when the document order aligns with the reasoning chain order; 3) enhancing causal decoder-only models with bi-directional attention by modifying the causal mask can effectively boost their end performance.In addition to the above, we conduct a thorough investigation of the distribution of LM attention weights in the context of MHQA.Our experiments reveal that attention weights tend to peak at higher values when the resulting answer is correct.We leverage this finding to heuristically improve LMs' performance on this task.Our code is publicly available at https://github. com/hwy9855/MultiHopQA-Reasoning.

Offer alignment is a key step in a product knowledge graph construction pipeline. It aims to align retailer offers of the same product for better coverage of product details. With the rapid development of online shopping services, the offer alignment task is applied in ever larger datasets. This work aims to build an offer alignment system that can efficiently be used in large-scale offer data. The key components of this system include: 1) common offer encoders for encoding text offer data into representations; 2) trainable LSH partitioning module to divide similar offers into small blocks; 3) lightweight sophisticated late-interactions for efficient filtering and scoring of offer alignment candidate pairs. We evaluate the system on public WDC offer alignment dataset, as well as DBLP-Scholar and DBLP-ACM.

Wenyu Huang, Mirella Lapata, Pavlos Vougiouklis, Nikos Papasarantopoulos, Jeff Pan. Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.