Abhirut Gupta, Ananya B. Sai, Richard Sproat, Yuri Vasilevski, James Ren, Ambarish Jash, Sukhdeep Sodhi, Aravindan Raghuveer. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.

Back-translation (BT) of target monolingual corpora is a widely used data augmentation strategy for neural machine translation (NMT), especially for low-resource language pairs.To improve effectiveness of the available BT data, we introduce HintedBT-a family of techniques which provides hints (through tags) to the encoder and decoder.First, we propose a novel method of using both high and low quality BT data by providing hints (as source tags on the encoder) to the model about the quality of each source-target pair.We don't filter out low quality data but instead show that these hints enable the model to learn effectively from noisy data.Second, we address the problem of predicting whether a source token needs to be translated or transliterated to the target language, which is common in crossscript translation tasks (i.e., where source and target do not share the written script).For such cases, we propose training the model with additional hints (as target tags on the decoder) that provide information about the operation required on the source (translation or both translation and transliteration).We conduct experiments and detailed analyses on standard WMT benchmarks for three crossscript low/medium-resource language pairs: {Hindi,Gujarati,Tamil}→English.Our methods compare favorably with five strong and well established baselines.We show that using these hints, both separately and together, significantly improves translation quality and leads to state-of-the-art performance in all three language pairs in corresponding bilingual settings.

Machine translation of user-generated codemixed inputs to English is of crucial importance in applications like web search and targeted advertising.We address the scarcity of parallel training data for training such models by designing a strategy of converting existing non-code-mixed parallel data sources to codemixed parallel data.We present an mBERT based procedure whose core learnable component is a ternary sequence labeling model, that can be trained with a limited code-mixed corpus alone.We show a 5.8 point increase in BLEU on heavily code-mixed sentences by training a translation model using our data augmentation strategy on an Hindi-English codemixed translation task.

Technical support problems are often long and complex.They typically contain user descriptions of the problem, the setup, and steps for attempted resolution.Often they also contain various non-natural language text elements like outputs of commands, snippets of code, error messages or stack traces.These elements contain potentially crucial information for problem resolution.However, they cannot be correctly parsed by tools designed for natural language.In this paper, we address the problem of segmentation for technical support questions.We formulate the problem as a sequence labelling task, and study the performance of state of the art approaches.We compare this against an intuitive contextual sentence-level classification baseline, and a state of the art supervised text-segmentation approach.We also introduce a novel component of combining contextual embeddings from multiple language models pre-trained on different data sources, which achieves a marked improvement over using embeddings from a single pre-trained language model.Finally, we also demonstrate the usefulness of such segmentation with improvements on the downstream task of answer retrieval.

Technical support problems are very complex. In contrast to regular web queries (that contain few keywords) or factoid questions (which are a few sentences), these problems usually include attributes like a detailed description of what is failing (symptom), steps taken in an effort to remediate the failure (activity), and sometimes a specific request or ask (intent). Automating support is the task of automatically providing answers to these problems given a corpus of solution documents. Traditional approaches to this task rely on information retrieval and are keyword based; looking for keyword overlap between the question and solution documents and ignoring these attributes. We present an approach for semantic parsing of technical questions that uses grammatical structure to extract these attributes as a baseline, and a CRF based model that can improve performance considerably in the presence of annotated data for training. We also demonstrate that combined with reasoning, these attributes help outperform retrieval baselines.