Automatic radiology report generation (RRG) has emerged as a promising approach to reduce clinicians' workload, yet existing systems are vulnerable to biases induced by spurious correlations across data, models, and evaluation pipelines.Such biases raise serious fairness concerns and may adversely affect patient care, making their mitigation critical in clinical settings.Leveraging causal inference to identify true cause-effect relationships can mitigate many biases and yield fair, reliable systems with clinically meaningful outputs.Existing surveys on RRG primarily emphasize deep learning approaches while overlooking the critical role of causality.This survey addresses this gap by analyzing bias across the RRG pipeline, formalizing RRG as a causal modeling problem, and reviewing representative causal techniques from the literature.Based on the level of intervention, we organize existing mitigation strategies into a three-tier taxonomy.We further examine commonly used public medical imaging datasets and evaluation metrics through a causal lens, revealing their biases and limitations in capturing causal alignment and clinical fidelity.To address these limitations, we advocate broader demographic coverage and causal-aware evaluation metrics to improve fairness and reliability, and identify important directions for future work.

Automatic Post-Editing (APE) is a widely studied approach for enhancing the output quality of Neural Machine Translation (NMT) systems. While most prior work has focused on general-purpose APE, the potential of domain-specific APE, such as for personalized or specialized content, remains underexplored due to the scarcity of domain-labeled training data. In this work, we investigate domain adaptation for APE using adapter-based methods. Our proposed multitask learning-based domain adaptation framework includes the use of a domain classifier to get a weighted combination of parallel domain-specific adapters at inference time, without requiring prior domain knowledge. This design allows the model to leverage cross-domain similarities, making it especially robust in low-resource domain scenarios. Our experimental results on English–German, English–Marathi, and English–Tamil pairs across different domains for each pair show substantial improvements over their respective general-purpose APE baselines. To facilitate further research, we will release human-annotated domain labels for triplets in WMT22 English–Marathi, and WMT24 English–Tamil APE datasets and the code.

Kishan Maharaj, Vitobha Munigala, Srikanth G. Tamilselvam, Prince Kumar, Sayandeep Sen, Palani Kodeswaran, Abhijit Mishra, Pushpak Bhattacharyya. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Automated Grammatical Error Correction (GEC) has been extensively researched in Natural Language Processing (NLP), primarily focusing on English and other resource-rich languages. This paper shifts the focus to GEC for a scarcely explored low-resource language, specifically Hindi, which presents unique challenges due to its intricate morphology and complex syntax. To address data resource limitations, this work explores various GEC data generation techniques. Our research introduces a carefully extracted and filtered, high-quality dataset, HiWikiEdits, which includes human-edited 8,137 instances sourced from Wikipedia, encompassing 17 diverse grammatical error types, with annotations performed using the ERRANT toolkit. Furthermore, we investigate Round Trip Translation (RTT) using diverse languages for synthetic Hindi GEC data generation, revealing that leveraging high-resource linguistically distant language for error generation outperforms mid-resource linguistically closer languages. Specifically, using English as a pivot language resulted in a 6.25% improvement in GLEU score compared to using Assamese or Marathi. Finally, we also investigate the neural model-based synthetic error-generation technique and show that it achieves comparable performance to other synthetic data generation methods, even in low-resource settings.

Reduplication and repetition, though similar in form, serve distinct linguistic purposes. Reduplication is a deliberate morphological process used to express grammatical, semantic, or pragmatic nuances, while repetition is often unintentional and indicative of disfluency. This paper presents the first large-scale study of reduplication and repetition in speech using computational linguistics. We introduce IndicRedRep, a new publicly available dataset containing Hindi, Telugu, and Marathi text annotated with reduplication and repetition at the word level. We evaluate transformer-based models for multi-class reduplication and repetition token classification, utilizing the Reparandum-Interregnum-Repair structure to distinguish between the two phenomena. Our models achieve macro F1 scores of up to 85.62% in Hindi, 83.95% in Telugu, and 84.82% in Marathi for reduplication-repetition classification.

A prevalent approach to synthetic APE data generation uses source (src) sentences in a parallel corpus to obtain translations (mt) through an MT system and treats corresponding reference (ref) sentences as post-edits (pe). While effective, due to independence between ‘mt’ and ‘pe,’ these translations do not adequately reflect errors to be corrected by a human post-editor. Thus, we introduce a novel and simple yet effective reference-focused synthetic APE data generation technique that uses ‘ref’ instead of src’ sentences to obtain corrupted translations (mt_new). The experimental results across English-German, English-Russian, English-Marathi, English-Hindi, and English-Tamil language pairs demonstrate the superior performance of APE systems trained using the newly generated synthetic data compared to those trained using existing synthetic data. Further, APE models trained using a balanced mix of existing and newly generated synthetic data achieve improvements of 0.37, 0.19, 1.01, 2.42, and 2.60 TER points, respectively. We will release the generated synthetic APE data.

Code-switching is a widely prevalent linguistic phenomenon in multilingual societies like India. Building speech-to-text models for code-switched speech is challenging due to limited availability of datasets. In this work, we focus on the problem of spoken translation (ST) of code-switched speech in Indian languages to English text. We present a new end-to-end model architecture CoSTA that scaffolds on pretrained automatic speech recognition (ASR) and machine translation (MT) modules (that are more widely available for many languages). Speech and ASR text representations are fused using an aligned interleaving scheme and are fed further as input to a pretrained MT module; the whole pipeline is then trained end-to-end for spoken translation using synthetically created ST data. We also release a new evaluation benchmark for code-switched Bengali- English, Hindi-English, Marathi-English and Telugu-English speech to English text. CoSTA significantly outperforms many competitive cascaded and end-to-end multimodal baselines by up to 3.5 BLEU points.

Masked diffusion language models (MDMs) have recently gained traction as a viable generative framework for natural language. This can be attributed to its scalability and ease of training compared to other diffusion model paradigms for discrete data, establishing itself as the state-of-the-art non-autoregressive generator for discrete data. Diffusion models, in general, have shown excellent ability to improve the generation quality by leveraging inference-time scaling either by increasing the number of denoising steps or by using external verifiers on top of the outputs of each step to guide the generation. In this work, we propose a verifier-based inference-time scaling method that aids in finding a better candidate generation during the denoising process of the MDM. Our experiments demonstrate the application of MDMs for standard text-style transfer tasks and establish MDMs as a better alternative to autoregressive language models. Additionally, we show that a simple soft-value-based verifier setup for MDMs using off-the-shelf pre-trained embedding models leads to significant gains in generation quality even when used on top of typical classifier-free guidance setups in the existing literature.

Aakash Kumar Agarwal, Saprativa Bhattacharjee, Mauli Rastogi, Jemima S. Jacob, Biplab Banerjee, Rashmi Gupta, Pushpak Bhattacharyya. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

Grammatical Error Correction (GEC) for low-resource Indic languages faces significant challenges due to the scarcity of annotated data. In this work, we introduce the Mask-Translate&Fill (MTF) framework, a novel approach for generating high-quality synthetic data for GEC using only monolingual corpora. MTF leverages a machine translation system and a pretrained masked language model to introduce synthetic errors and tries to mimic errors made by second-language learners. Our experimental results on English, Hindi, Bengali, Marathi, and Tamil demonstrate that MTF consistently outperforms other monolingual synthetic data generation methods and achieves performance comparable to the Translation Language Modeling (TLM)-based approach, which uses a bilingual corpus, in both independent and multilingual settings. Under multilingual training, MTF yields significant improvements across Indic languages, with particularly notable gains in Bengali and Tamil, achieving +1.6 and +3.14 GLEU over the TLM-based method, respectively. To support further research, we also introduce the IndiGEC Corpus, a high-quality, human-written, manually validated GEC dataset for these four Indic languages, comprising over 8,000 sentence pairs with separate development and test splits.

Sameer Pimparkhede, Pushpak Bhattacharyya. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

ABSTRACT This paper studies neural machine translation (NMT) of code‐mixed (CM) text. Specifically, we generate synthetic CM data and how it can be used to improve the translation performance of NMT through the data augmentation strategy. We conduct experiments on three data augmentation approaches viz. CM‐Augmentation, CM‐Concatenation, and Multi‐Encoder approaches, and the latter two approaches are inspired by document‐level NMT, where we use synthetic CM data as context to improve the performance of the NMT models. We conduct experiments on three language pairs, viz. Hindi–English, Telugu–English and Czech–English. Experimental results demonstrate that the proposed approaches significantly improve performance over the baseline model trained without data augmentation and over the existing data augmentation strategies. The CM‐Concatenation model attains the best performance.

Abstract Evaluating social biases in language models (LMs) is crucial for ensuring fairness and minimizing the reinforcement of harmful stereotypes in AI systems. Existing benchmarks, such as the Bias Benchmark for Question Answering (BBQ), primarily focus on Western contexts, limiting their applicability to the Indian context. To address this gap, we introduce BharatBBQ,1 a culturally adapted benchmark designed to assess biases in Hindi, English, Marathi, Bengali, Tamil, Telugu, Odia, and Assamese. BharatBBQ covers 13 social categories, including 3 intersectional groups, reflecting prevalent biases in the Indian sociocultural landscape. Our dataset contains 49,108 examples in one language that are expanded using translation and verification to 392,864 examples in eight different languages. We evaluate five multilingual LM families across zero- and few-shot settings, analyzing their bias and stereotypical bias scores. Our findings highlight persistent biases across languages and social categories and often amplified biases in Indian languages compared to English, demonstrating the necessity of linguistically and culturally grounded benchmarks for bias evaluation.

This article proposes a meta-learning-based, few-shot image captioning framework based on an ensemble of object-relational transformer models and a self-distillation strategy. Unlike traditional approaches, the proposed framework can train models using non-paired images and captions. In each iteration, multiple base models are trained on distinct data samples, forming an ensemble that generates pseudo-captions with confidence-based weighting. An efficient pseudo-feature generation method based on gradient descent enables learning from non-paired captions. These pseudo-features and pseudo-captions are then used to train the base models in subsequent iterations. The obtained results show that using just 1% of the paired image-caption training data significantly improves performance and generates meaningful captions. The proposed framework demonstrates the potential to reduce reliance on large, paired datasets while maintaining high-quality image captioning performance. 1

Continual learning, alternatively referred to as incremental learning or lifelong learning, represents a learning paradigm enabling an agent to acquire new information without compromising its retention of previously learned knowledge. Continual learning research has resulted in a number of ways to prevent catastrophic forgetting in deep neural networks. Apparently, continuous learning of recurrent models has never been used to solve the image captioning problems in the Hindi language. This research presents a novel approach to employ continual learning in the context of generating image captions in Hindi. Given its widespread usage in South Asia and its official status in India, Hindi ranks as the third most spoken language worldwide. We look at the continual learning of Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) based models for image captioning in a systematic way. In continuous image captioning problems, we incorporate an attention-based method that explicitly addresses the transient aspect of vocabulary. Using MS-COCO datasets, we employ our methods to solve the incremental image captioning problem. Our findings show that the proposed method can learn five captioning tasks in consecutive order without forgetting the one it has already learned.

Hate speech (HS) is a growing concern in many parts of the world, including India, where it has led to numerous instances of violence and discrimination. The development of effective counter-narratives (CNs) is a critical step in combating hate speech, but there is a lack of research in this area, especially in non-English languages. In this paper, we introduce a new dataset, IndicCONAN, of counter-narratives against hate speech in Hindi and Indian English. We propose a scalable human-in-the-loop approach for generating counter-narratives by an auto-regressive language model through machine generation - human correction cycle, where the model uses augmented data from previous cycles to generate new training samples. These newly generated samples are then reviewed and edited by annotators, leading to further model refnement. The dataset consists of over 2,500 exam- ˜ ples of counter-narratives each in both English and Hindi corresponding to various hate speeches in the Indian context. We also present a framework for generating CNs conditioned on specifc CN type with a mean perplexity of 3.85 for English and 3.70 for Hindi, a mean toxicity score of 0.04 for English and 0.06 for Hindi, and a mean diversity of 0.08 for English and 0.14 for Hindi. Our dataset and framework provide valuable resources for researchers and practitioners working to combat hate speech in the Indian context.

Sarcasm is a widespread linguistic phenomenon that poses a considerable challenge to explain due to its subjective nature, absence of contextual cues, and rooted personal perspectives. Even though the identification of sarcasm has been extensively studied in dialogue analysis, merely detecting sarcasm falls short of enabling conversational systems to genuinely comprehend the underlying meaning of a conversation and generate fitting responses. It is imperative to not only detect sarcasm but also pinpoint its origination and the rationale behind the sarcastic expressions to capture its authentic essence. In this paper, we delve into the discourse structure of conversations infused with sarcasm and introduce a novel task - Sarcasm Initiation and Reasoning in Conversations (SIRC). Embedded in a multimodal environment and involving a combination of both English and code-mixed interactions, the objective of the task is to discern the trigger or starting point of sarcasm. Additionally, the task involves producing a natural language explanation that rationalizes the satirical dialogues. To this end, we introduce Sarcasm Initiation and Reasoning Dataset (SIRD) to facilitate our task and provide sarcasm initiation annotations and reasoning. We develop a comprehensive model named Sarcasm Initiation and Reasoning Generation (SIRG), which is designed to encompass textual, audio, and visual representations. To achieve this, we introduce a unique shared fusion method that employs cross-attention mechanisms to seamlessly integrate these diverse modalities. Our experimental outcomes, conducted on the SIRC dataset, demonstrate that our proposed framework establishes a new benchmark for both sarcasm initiation and its reasoning generation in the context of multimodal conversations. The code and dataset can be accessed from https://www.iitp.ac.in/∼ai-nlp-ml resources.html#sarcasm-explain and https://github.com/GussailRaat/SIRG-Sarcasm-Initiation-and-Reasoning-Generation.

Prince Jha, Raghav Jain, Konika Mandal, Aman Chadha, Sriparna Saha, Pushpak Bhattacharyya. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

We introduce a simple and effective prompt ing technique called incontext mixing (ICM) for effective incontext learning (ICL) with multilingual large language models (MLLMs).With ICM, we modify the fewshot examples within ICL prompts to be intrasententially codemixed by randomly swapping content words in the target languages with their English translations.We observe that ICM prompts yield superior performance in NLP tasks such as disfluency correction, grammar error cor rection and text simplification that demand a close correspondence between the input and out put sequences.Significant improvements are observed mainly for lowresource languages that are underrepresented during the pretrain ing and finetuning of MLLMs.We present an extensive set of experiments to analyze when ICM is effective and what design choices con tribute towards its effectiveness.ICM works consistently and significantly better than other prompting techniques across models of varying capacity such as mT0XXL, BloomZ and GPT 4. Code, prompts and datasets are available here.

Tejpalsingh Siledar, Swaroop Nath, Sankara Muddu, Rupasai Rangaraju, Swaprava Nath, Pushpak Bhattacharyya, Suman Banerjee, Amey Patil, Sudhanshu Singh, Muthusamy Chelliah, Nikesh Garera. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

Over the last few years, artificial intelligence-based clinical assistance has gained immense popularity and demand in telemedicine, including automatic disease diagnosis. Patients often describe their signs and symptoms to doctors using visual aids, which provide vital evidence for identifying a medical condition. In addition to learning from our experiences, we learn from well-established theories/ knowledge. With the motivation of leveraging visual cues and medical knowledge, we propose a transformer-based, knowledge-infused multi-modal medical dialogue generation (KI-MMDG) framework. In addition, we present a discourse-aware image identifier (DII) that recognizes signs and their severity by leveraging the current conversation context in addition to the image of the signs. We first curate an empathy and severity-aware multi-modal medical dialogue (ES-MMD) corpus in English, which is annotated with intent, symptoms, and visual signs with severity information. Experimental results show the superior performance of the proposed KI-MMDG model over uni-modal and non-knowledge infused generative models, demonstrating the importance of visual signs and knowledge infusion in symptom investigation and diagnosis. We also observed that the DII model surpasses the existing state-of-the-art model by 7.84%, indicating the crucial significance of dialogue context for identifying a sign image surfaced during conversations. The code and dataset are available at https://github.com/NLP-RL/KI-MMDG.

Prince Jha, Krishanu Maity, Raghav Jain, Apoorv Verma, Sriparna Saha, Pushpak Bhattacharyya. Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

In document-level neural machine translation (DocNMT), multi-encoder approaches are common in encoding context and source sentences. Recent studies (CITATION) have shown that the context encoder generates noise and makes the model robust to the choice of context. This paper further investigates this observation by explicitly modelling context encoding through multi-task learning (MTL) to make the model sensitive to the choice of context. We conduct experiments on cascade MTL architecture, which consists of one encoder and two decoders. Generation of the source from the context is considered an auxiliary task, and generation of the target from the source is the main task. We experimented with German–English language pairs on News, TED, and Europarl corpora. Evaluation results show that the proposed MTL approach performs better than concatenation-based and multi-encoder DocNMT models in low-resource settings and is sensitive to the choice of context. However, we observe that the MTL models are failing to generate the source from the context. These observations align with the previous studies, and this might suggest that the available document-level parallel corpora are not context-aware, and a robust sentence-level model can outperform the context-aware models.

In this paper, we explore the utility of Translationese as synthetic data created using machine translation for pre-training language models (LMs) for low-resource languages (LRLs).Our simple methodology consists of translating large amounts of web-crawled monolingual documents (clean) into the LRLs, followed by filtering the translated documents using tiny LMs trained on small but clean LRL data.Taking the case of Indian languages, we pre-train LMs from scratch with 28M and 85M parameters, and then fine-tune them for 5 downstream natural language understanding (NLU) and 4 generative (NLG) tasks.We observe that pretraining on filtered synthetic data leads to relative performance drops of only 0.87% for NLU and 2.35% for NLG, compared to pre-training on clean data, and this gap further diminishes upon the inclusion of a small amount of clean data.We also study the impact of synthetic data filtering and the choice of source language for synthetic data generation.Furthermore, evaluating continually pre-trained larger models like Gemma-2B and Llama-3-8B in few-shot settings, we observe that using synthetic data is competitive with using clean data.Our findings suggest that synthetic data shows promise for bridging the pre-training gap between English and LRLs.

Contemporary deep learning models effectively handle languages with diverse morphology despite not being directly integrated into them.Morphology and word order are closely linked, with the latter incorporated into transformerbased models through positional encodings.This prompts a fundamental inquiry: Is there a correlation between the morphological complexity of a language and the utilization of positional encoding in pre-trained language models?In pursuit of an answer, we present the first study addressing this question, encompassing 22 languages and 5 downstream tasks.Our findings reveal that the importance of positional encoding diminishes with increasing morphological complexity in languages.Our study motivates the need for a deeper understanding of positional encoding, augmenting them to better reflect the different languages under consideration.

Sameer Pimparkhede, Mehant Kammakomati, Srikanth G. Tamilselvam, Prince Kumar, Ashok Pon Kumar, Pushpak Bhattacharyya. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024.

Nihar Sahoo, Pranamya Kulkarni, Arif Ahmad, Tanu Goyal, Narjis Asad, Aparna Garimella, Pushpak Bhattacharyya. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.

Text-to-Image (T2I) models are being increasingly adopted in diverse global communities where they create visual representations of their unique cultures. Current T2I benchmarks primarily focus on faithfulness, aesthetics, and realism of generated images, overlooking the critical dimension of cultural competence. In this work, we introduce a framework to evaluate cultural competence of T2I models along two crucial dimensions: cultural awareness and cultural diversity, and present a scalable approach using a combination of structured knowledge bases and large language models to build a large dataset of cultural artifacts to enable this evaluation. In particular, we apply this approach to build CUBE (CUltural BEnchmark for Text-to-Image models), a first-of-its-kind benchmark to evaluate cultural competence of T2I models. CUBE covers cultural artifacts associated with 8 countries across different geo-cultural regions and along 3 concepts: cuisine, landmarks, and art. CUBE consists of 1) CUBE-1K, a set of high-quality prompts that enable the evaluation of cultural awareness, and 2) CUBE-CSpace, a larger dataset of cultural artifacts that serves as grounding to evaluate cultural diversity. We also introduce cultural diversity as a novel T2I evaluation component, leveraging quality-weighted Vendi score. Our evaluations reveal significant gaps in the cultural awareness of existing models across countries and provide valuable insights into the cultural diversity of T2I outputs for underspecified prompts. Our methodology is extendable to other cultural regions and concepts and can facilitate the development of T2I models that better cater to the global population.

Cyberbullying is a malady of social media, and its automatic detection is critically important considering its virulence, velocity of spreading, and the scale of the havoc it can wreak. However, the problem is challenging due to its disguised behavior, noise in the content, and, in recent times, introduction of code-mixing. In this work, we propose MTBullyGNN a novel graph neural network (GNN)-based multitask (MT) framework that solves sentiment-aided cyberbullying detection (CD) from code-mixed language. The GNN helps detect unlabelled or noisy label nodes (sentences) accurately by aggregating information from similarly labeled nodes. To connect nodes, we apply cosine similarity between sentences and create a single text graph for a benchmark code-mixed cyberbullying corpus, BullySent. Experimental results illustrate that MTBullyGNN outperforms the state-of-the-art (SOTA) methods for both the single (CD) and MT (CD and sentiment) settings by up to 4.46% and 4.92% in classification accuracy, respectively. Furthermore, another benchmark Hindi–English code-mixed single-task dataset has also been considered to illustrate the robustness of our proposed model. The code will be made publicly available in the camera-ready version.

Conversational systems have shown immense growth in their ability to communicate like humans. With the emergence of large pre-trained language models (PLMs) the ability to provide informative responses have improved significantly. Despite the success of PLMs, the ability to identify and generate engaging and empathetic responses is largely dependent on labelled-data. In this work, we design a prompting approach that identifies the emotion of a given utterance and uses the emotion information for generating the appropriate responses for conversational systems. We propose a two-step prompting method that first recognises the emotion in the dialogue utterance and in the second-step uses the predicted emotion to prompt the PLM to generate the corresponding em- pathetic response in a few-shot setting. Experimental results on three publicly available datasets show that our proposed approach outperforms the state-of-the-art approaches for both automatic and manual evaluation.

With the advancement of telemedicine, both researchers and medical practitioners are working hand-in-hand to develop various techniques to automate various medical operations, such as diagnosis report generation. In this paper, we first present a multi-modal clinical conversation summary generation task that takes a clinician-patient interaction (both textual and visual information) and generates a succinct synopsis of the conversation. We propose a knowledge-infused, multi-modal, multi-tasking medical domain identification and clinical conversation summary generation (MM-CliConSummation) framework. It leverages an adapter to infuse knowledge and visual features and unify the fused feature vector using a gated mechanism. Furthermore, we developed a multi-modal, multi-intent clinical conversation summarization corpus annotated with intent, symptom, and summary. The extensive set of experiments, both quantitatively and qualitatively, led to the following findings: (a) critical significance of visuals, (b) more precise and medical entity preserving summary with additional knowledge infusion, and (c) a correlation between medical department identification and clinical synopsis generation. Furthermore, the dataset and source code are available at https://github.com/NLP-RL/MM-CliConSummation

Although there has been a plethora of work on open-domain conversational systems, most of these lack the mechanism of controlling the concept transitions in a dialogue.For activities like switching from casual chit-chat to taskoriented conversation, an agent with the ability to manage the flow of concepts in a conversation might be helpful.The user would find the dialogue more fascinating and engaging and be more receptive to such transitions if these concept transitions were made while taking into account the user's persona.Focusing on personaaware concept transitions, we propose a Reinforced Persona-aware Topic-guiding Conversational System (RPTCS).Due to the lack of a persona-aware topic transition dataset, we propose a novel conversation dataset creation mechanism in which the conversational agent leads the discourse to drift to a set of target concepts depending on the persona of the speaker and the context of the conversation.To avoid scarcely available expensive human resources, the entire data-creation process is mostly automatic with human-in-loop only for quality checks.This created conversational dataset named PTCD is used to develop the RPTCS in two steps.First, a maximum likelihood estimation loss-based dialogue model is trained on PTCD.The trained model is then fine-tuned in a Reinforcement Learning (RL) framework by employing novel reward functions to assure persona, topic, and context consistency with non-repetitiveness in generated responses.Our experimental results demonstrate the strength of the proposed system with respect to strong baselines 1 .

Timely generation of radiology reports and diagnoses is a challenge worldwide due to the enormous number of cases and shortage of radiology specialists.In this paper, we propose a Knowledge Graph Augmented Vision Language BART (KGVL-BART) model that takes as input two chest X-ray images-one frontal and the other lateral-along with tags which are diagnostic keywords, and outputs a report with the patient-specific findings.Our system development effort is divided into 3 stages: i) construction of the Chest X-ray KG (referred to as chestX-KG), ii) image feature extraction, and iii) training a KGVL-BART model using the visual, text, and KG data.The dataset we use is the well-known Indiana University Chest X-ray reports with the train, validation, and test split of 3025 instances, 300 instances, and 500 instances respectively.We construct a Chest X-Ray knowledge graph from these reports by extracting entity1-relation-entity2 triples; the triples get extracted by a rule-based tool of our own.Constructed KG is verified by two experienced radiologists (with experience of 30 years and 8 years, respectively).We demonstrate that our model-KGVL-BART-outperforms State-ofthe-Art transformer-based models on standard NLG scoring metrics.We also include a qualitative evaluation of our system by experienced radiologist (with experience of 30 years) on the test data, which showed that 73% of the reports generated were fully correct, only 5.5% are completely wrong and 21.5% have important missing details though overall correct.To the best of our knowledge, ours is the first system to make use of multi-modality and domain knowledge to generate X-ray reports automatically.

Medical Question Answering (MedQA) is one of the most popular and significant tasks in developing healthcare assistants. When humans extract an answer to a question from a document, they first (a) understand the question itself in detail and (b) utilize relevant knowledge/experiences to determine the answer segments. In multi-span question answering, it becomes increasingly important to comprehend the query accurately and possess relevant knowledge, as the interrelationship among different answer segments is essential for achieving completeness. Motivated by this, we first propose a transformer-based query semantic and knowledge (QueSemKnow) guided multi-span question-answering model. The proposed QueSemKnow works in a two-phased manner; in the first stage, a multi-task model is proposed to extract query semantics: (i) intent identification and (ii) question type prediction. In the second stage, QueSemKnow selects a relevant subset of the knowledge graph as the underlying context/document and extracts answers depending on the semantic information extracted from the first stage and context. We build a multi-task query semantic extraction model for query intent and query type identification to investigate the co-relation among these tasks. Furthermore, we created a semantically aware medical question-answering corpus named QueSeMSpan MedQA wherein each question is annotated with its corresponding semantic information. The proposed model outperforms several baselines and existing state-of-the-art models by a large margin on multiple datasets, which firmly demonstrates the effectiveness of the human-inspired multi-span question-answering methodology.

With the rise of social media and online communication, the issue of cyberbullying has gained significant prominence.While extensive research is being conducted to develop more effective models for detecting cyberbullying in monolingual languages, a significant gap exists in understanding code-mixed languages and the need for explainability in this context.To address this gap, we have introduced a novel benchmark dataset named Bully-Explain for explainable cyberbullying detection in code-mixed language.In this dataset, each post is meticulously annotated with four labels: bully, sentiment, target, and rationales, indicating the specific phrases responsible for identifying the post as a bully.Our current research presents an innovative unified generative framework, GenEx, which reimagines the multitask problem as a text-to-text generation task.Our proposed approach demonstrates its superiority across various evaluation metrics when applied to the BullyExplain dataset, surpassing other baseline models and current state-of-the-art approaches. 1

Query-focused Summarization (QfS) deals with systems that generate summaries from document(s) based on a query.Motivated by the insight that Reinforcement Learning (RL) provides a generalization to Supervised Learning (SL) for Natural Language Generation, and thereby performs better (empirically) than SL, we use an RL-based approach for this task of QfS.Additionally, we also resolve the conflict of employing RL in Transformers with Teacher Forcing.We develop multiple Policy Gradient networks, trained on various reward signals: ROUGE, BLEU, and Semantic Similarity, which lead to a 10 -point improvement over the State-of-the-Art approach on the ROUGE-L metric for a benchmark dataset (ELI5).We also show performance of our approach in zero-shot setting for another benchmark dataset (Debate-Pedia) -our approach leads to results comparable to baselines, which were specifically trained on DebatePedia.To aid the RL training, we propose a better semantic similarity reward, enabled by a novel Passage Embedding scheme developed using Cluster Hypothesis.Lastly, we contribute a gold-standard test dataset to further research in QfS and Long-form Question Answering (LfQA).

We present a novel retrofitting method to induce emotion aspects into pre-trained language models (PLMs) such as BERT and RoBERTa.Our method updates pre-trained network weights using contrastive learning so that the text fragments exhibiting similar emotions are encoded nearby in the representation space, and the fragments with different emotion content are pushed apart.While doing so, it also ensures that the linguistic knowledge already present in PLMs is not inadvertently perturbed.The language models retrofitted by our method, i.e., BERTEmo and RoBERTaEmo, produce emotion-aware text representations, as evaluated through different clustering and retrieval metrics.For the downstream tasks on sentiment analysis and sarcasm detection, they perform better than their pre-trained counterparts (about 1% improvement in F1-score) and other existing approaches.Additionally, a more significant boost in performance is observed for the retrofitted models over pretrained ones in few-shot learning setting.

Sarcasm is a complex linguistic construct with incongruity at its very core.Detecting sarcasm depends on the actual content spoken and tonality, facial expressions, the context of an utterance, and personal traits like language proficiency and cognitive capabilities.In this paper, we propose the utilization of synthetic gaze data to improve the task performance for multimodal sarcasm detection in a conversational setting.We enrich an existing multimodal conversational dataset, i.e., MUStARD++ with gaze features.With the help of human participants, we collect gaze features for < 20% of data instances, and we investigate various methods for gaze feature prediction for the rest of the dataset.We perform extrinsic and intrinsic evaluations to assess the quality of the predicted gaze features.We observe a performance gain of up to 6.6% points by adding a new modality, i.e., collected gaze features.When both collected and predicted data are used, we observe a performance gain of 2.3% points on the complete dataset.Interestingly, with only predicted gaze features, too, we observe a gain in performance (1.9% points).We retain and use the feature prediction model, which maximally correlates with collected gaze features.Our model trained on combining collected and synthetic gaze data achieves SoTA performance on the MUStARD++ dataset.To the best of our knowledge, ours is the first predict-and-use model for sarcasm detection.We publicly release 1 the code, gaze data, and our best models for further research.

In this paper, we compare two approaches to train a multilingual language model: (i) simple multilingual learning using data-mixing, and (ii) meta-learning. We examine the performance of these models by extending them to unseen language pairs and further finetune them for the task of unsupervised NMT. We perform several experiments with varying amounts of data and give a comparative analysis of the approaches. We observe that both approaches give a comparable performance, and meta-learning gives slightly better results in a few cases of low amounts of data. For Oriya-Punjabi language pair, meta-learning performs better than multilingual learning when using 2M, and 3M sentences.

Abstract Moderators often face a double challenge regarding reducing offensive and harmful content in social media. Despite the need to prevent the free circulation of such content, strict censorship on social media cannot be implemented due to a tricky dilemma – preserving free speech on the Internet while limiting them and how not to overreact. Existing systems do not essentially exploit the correlatedness of hate-offensive content and aggressive posts; instead, they attend to the tasks individually. As a result, the need for cost-effective, sophisticated multi-task systems to effectively detect aggressive and offensive content on social media is highly critical in recent times. This work presents a novel multifaceted transformer-based framework to identify aggressive and hate posts on social media. Through an end-to-end transformer-based multi-task network, our proposed approach addresses the following array of tasks: (a) aggression identification, (b) misogynistic aggression identification, (c) identifying hate-offensive and non-hate-offensive content, (d) identifying hate, profane, and offensive posts, (e) type of offense. We further investigate the role of emotion in improving the system’s overall performance by learning the task of emotion detection jointly with the other tasks. We evaluate our approach on two popular benchmark datasets of aggression and hate speech, covering four languages, and compare the system performance with various state-of-the-art methods. Results indicate that our multi-task system performs significantly well for all the tasks across multiple languages, outperforming several benchmark methods. Moreover, the secondary task of emotion detection substantially improves the system performance for all the tasks, indicating strong correlatedness among the tasks of aggression, hate, and emotion, thus opening avenues for future research.

Social chatbots have gained immense popularity, and their appeal lies in their capacity to respond to diverse requests, but also in their ability to develop an emotional connection with users. To develop and promote social chatbots, we need to concentrate on increasing user interaction and consider both the intellectual and emotional quotient in conversational agents. In this work, we propose the task of empathetic, personalized dialogue generation giving the machine the capability to respond emotionally and in accordance with the persona of the user. We design a generative adversarial framework, named EP-GAN (Empathy and Persona aware Generative Adversarial Network) to generate responses that are sensitive to the emotion of the user and corresponds to the persona. The persona information is encoded as memory vectors that, along with the dialogue history, are fed to the decoder for generation. An interactive adversarial learning framework is implemented to verify whether the generated responses elicit the emotion and persona in dialogues. Experimental results show that the EP-GAN framework significantly outperforms the existing baselines for both automatic and manual evaluation. We achieve an improvement of 1 point BLEU-4 and 2 points in the emotion accuracy metric compared to the best performing baseline.

Image captioning frameworks usually employ an encoder-decoder paradigm, with the encoder receiving abstract image feature vectors as input and the decoder for language modeling. Nowadays, most prominent architectures employ features from region proposals derived from object detection modules. In this work, we propose a novel architecture for image captioning. We employ the object detection module integrated with transformer architecture as an encoder and GPT-2 (Generative Pre-trained Transformer) as a decoder. The encoder utilizes the information of the spatial relationships among detected objects. We introduce a unique methodology for image caption generation in Hindi, which is widely spoken in South Asia and India and is the world’s third most spoken language as well as India’s official language. In terms of BLEU scores, the proposed approach’s performance is comparable to those of other baselines, and the results illustrate that the proposed approach outperforms the other baselines. The efficacy of the proposed approach in generating correct captions is further determined by human assessment in terms of adequacy and fluency.

Dense image captioning is a task that requires generating localized captions in natural language for multiple regions of an image. This task leverages its functionalities from both computer vision for recognizing regions in an image and natural language processing for generating captions. Numerous works have been carried out on dense image captioning for resource-rich languages like English; however, resource-poor languages like Hindi are not explored. Hindi is one of India’s official languages and is the third most spoken language in the world. This article proposes a dense image captioning model to describe different segments of an image by generating more than one caption in the Hindi language. For localized image recognition and language modeling, we employ Faster R-CNN and Long Short-Term Memory (LSTM), respectively. Apart from this, we conduct various experiments using gated recurrent units (GRUs) and attention mechanism. By manually translating the well-known Visual Genome dataset from English to Hindi, a dataset has been created for dense image captioning in Hindi. The experiments conducted on the newly constructed Hindi dense image captioning dataset illustrate the efficacy of the proposed method over the state-of-the-art methods.

In sequence-to-sequence modeling tasks, such as image captioning, machine translation, and visual question answering, encoder-decoder architectures are state of the art. An encoder, convolutional neural network (CNN) encodes input images into fixed dimensional vector representation in the image captioning task, whereas a decoder, a recurrent neural network, performs language modeling and generates the target descriptions. Recent CNNs use the same operation over every pixel; however, all the image pixels are not equally important. To address this, the proposed method uses a dynamic convolution-based encoder for image encoding or feature extraction, Long-Short-Term-Memory as a decoder for language modeling, and X-Linear attention to make the system robust. Encoders, attentions, and decoders are important aspects of the image captioning task; therefore, we experiment with various encoders, decoders, and attention mechanisms. Most of the works for image captioning have been carried out for the English language in the existing literature. We propose a novel approach for caption generation from images in Hindi. Hindi, widely spoken in South Asia and India, is the fourth most-spoken language globally; it is India’s official language. The proposed method utilizes dynamic convolution operation on the encoder side to obtain a better image encoding quality. The Hindi image captioning dataset is manually created by translating the popular MSCOCO dataset from English to Hindi. In terms of BLEU scores, the performance of the proposed method is compared with other baselines, and the results obtained show that the proposed method outperforms different baselines. Manual human assessment in terms of adequacy and fluency of the captions generated further determines the efficacy of the proposed method in generating good-quality captions.

In the natural language processing community, open-domain conversational agents, also known as chatbots, are gaining popularity. One of the difficulties is getting them to communicate in an emotionally intelligent manner. To generate dialogues, current neural response generation methods depend solely on end-to-end learning from large scale conversation data. Therefore, we introduce a large-scale multi Emotion and Intent guided Multimodal Dialogue (EmoInt-MD) dataset labelled with 32 emotions and 15 empathetic intents having 32 k dialogues taken from different movie genres. We propose a novel multi-task multimodal contextual Transformer framework for simultaneously identifying the emotions and intents in a given utterance utilizing audio and visual features in addition to the textual information. Experimental analysis proves that the proposed framework outperforms several unimodal and multimodal baselines on theEmoInt-MDdataset. This dataset along with our baseline and proposed framework implementations will be made publicly available for research purposes.

Politeness enhances interactions by improving relations between the participants. If there is a display of rudeness, even the finest conversation can fall through. In addition, if lathered with kindness, even the most angst-prone circumstance can be expressed with far less suffering. Previously, researchers have focused upon including politeness in conversations. But the existing research does not focus on variations in politeness according to the user profile. Therefore, in this article, we propose a novel task of generating polite personalized dialog responses in accordance with the user profile and consistent with the conversational history. We design a novel Polite Personalized Dialog Generation (PoPe-DG) framework that employs a reinforced deliberation network. We create human-annotated politeness templates according to user profiles to induce politeness variation in the generated responses for the proposed task. Precisely, the personality profile is transformed and normalized into a vector using the fusion attention combined with dialogue utterances to build context. Furthermore, the context modules and the annotated templates are appended to initialize the deliberation decoder. Experimental analysis validates that our proposed approach inculcates politeness in responses in accordance with the user profile and the conversational history.

Thanks to the digital age, online speech and information may now be disseminated anonymously without regard for repercussions. Regulators face a unique problem with social media platforms because of the speed and volume of material and the lack of editorial supervision. The existing datasets on hate speech or offensive language identification lack diversity in the dataset’s content. In this article, we create a multi-domain hate speech corpus (MHC) of English tweets that includes hate speech against religion, nationality, ethnicity, and gender in general and cover diverse domains, such as current affairs, politics, terrorism, technology, natural disasters, and human/drugs trafficking. Each instance in our dataset is manually annotated as hate or non-hate. We use the existing state-of-the-art models and present a stacked-ensemble-based hate speech classifier (SEHC) to identify hate speech from Twitter data. Our results indicate that the proposed method may serve as a strong baseline for future studies using this dataset. (The dataset is available athttps://www.iitp.ac.in/~ai-nlp-ml/resources.html#MHC.)

The advent of the Internet is a boon to society. However, many of its banes cannot be undermined, cyberbullying being one of them. The emotional state and sentiment of a person have a significant influence on the intended content. The current work is the first attempt in investigating the role of sentiment and emotion information for identifying cyberbullying in the Indian scenario. From Twitter, a benchmark Hind–English code-mixed corpus called BullySentEmo has been developed as there is no dataset available labeled with bully, sentiment, and emotion. Moreover, emoji information available with tweet texts can provide better understanding of user intention. The developed dataset consists of both modalities, tweet text, and emoji. In India, the majority of communication on different social media platforms is based on Hindi and English and language switching is a common practice in digital communication. An attention-based multimodal, adversarial multitasking framework is proposed for cyberbully detection (CBD) considering two auxiliary tasks: sentiment analysis (SA) and emotion recognition (ER). Experimental results indicate that compared to unimodal and single-task variants, the proposed framework improves the performance of the main task, i.e., CBD, by 3.59% and 2.56% in terms of accuracy and F1-Score, respectively. Furthermore, two different benchmark datasets (Twitter dataset and Aggression dataset) have been considered to show the robustness of our proposed model.

Mental health disorder continues to be a grievous concern plaguing humans worldwide. The scarcity of mental health professionals (MHPs) has driven novel efforts lately to combat mental illness by developing automated systems capable of assisting MHPs. However, lack of high-quality conversational data due to privacy concerns remains a bottleneck toward its study and automation. Also, distinguishing and identifying various mental disorders is a challenging task as these are expressed using similar patterns in terms of language usage and overall sentiment polarity. The need for classifying different mental illnesses is rather intuitive as their true diagnosis is imperative for determining their path to recovery. As a first step toward creating automated virtual agent (VA), in this article, we curate a large-scale dataset for multiple mental illnesses comprising of dyadic conversations between a mentally ill user (typically a support seeker) and the VA (typically a peer supporter imparting hope and motivation), collected from a peer-to-peer support platform. We present a hierarchical attention-based deep neural network classifier for modeling conversations to detect different mental disorders as the dialog progresses. The proposed network is equipped with lexicon-based sentiment scores in order to prioritize certain contributing features. The proposed model attained an accuracy of 83.91% and outperformed several strong baselines.

Emojis or emoticons are not just a modern trend but have become an essential part of our day-to-day interactions. Predicting a suitable emoji for a given tweet is a challenging task because a wrong emoji prediction for a tweet can change the meaning of the message or can amplify the emotion of the message. This task is particularly challenging since it requires selecting an appropriate emoji from a huge list of prospective emojis that may or may not be equivalent to one another. Humans use multiple emojis to convey their emotions, thereby making the task a multilabel classification problem. In this article, we propose a multilabel emoji prediction system that predicts the appropriate emoji for a given tweet by using different state-of-the-art baselines. Due to the unavailability of a multi-emoji dataset, we create a large-scale multilabel emoji dataset named Mu-Emoji that comprises of more than 0.6 million tweets having varied emojis belonging to both positive and negative sentiments. For our proposed task, we employ graph attention network along with bidirectional encoder representations from transformer encoder for the accurate prediction of emojis. Qualitative and quantitative analyses show that our multilabel emoji prediction baselines perform well compared with the single-emoji prediction baselines for our proposed Mu-Emoji dataset. Our proposed framework also outperforms all the baselines for both single and multilabel emoji prediction tasks.

Gender plays a crucial role in improving the performance quality of personalized systems. Privacy and anonymity allow users to hide their details. Based on the intuition that post contents of male and female users differ, we can predict the gender of the social media account holder via their corresponding posts. These posts can be multimodal (text + image) in nature. We investigate various emotion-assisted multimodal gender prediction models in this article. The developed models use gated recurrent units (GRUs) and ResNets for extracting features from the tweets and the images, respectively. The distribution of emotion categories is related to the gender of the target person. In response to these findings, this article describes the first attempt to use multimodal (image and text posted) information for gender prediction in a multitask setting with emotion recognition as an auxiliary task. The enriched PAN-2018 dataset with gender and emotion labels is used to train gender and emotion networks. Several models were developed to improve the gender prediction task by generating better emotion-aware features. The results show that the proposed multimodal emotion detection model outperforms single-modal (text and image)-based models and state-of-the-art systems on the benchmark PAN-2018 dataset.

With the advent of internet technologies, it has created different ways of writing anonymously, which has lead to criminal and malicious activities over social media platforms. Thus, the automatic authentication checking of the available contents is the need of the hour. Social media sites, such as Facebook, Twitter, and so on, are used heavily by the users for sharing of information about their day-to-day activities. The identity of the suspect user is matched against tweets written by the specific user in tweet-user verification process. Writing styles of different users differ from each other, due to unique word choices, emoji selection, sentence formation, and punctuation usage. We have developed a multimodal Siamese-based architecture, which uses attention between the text and emoji parts of the tweet for generating a combined representation for the tweet. Attention helps in selecting the relevant information from different modalities. Modality attention is used for fusing the two modalities (text and emoji). We have used a newly developed multimodal Twitter dataset for evaluating the performance of the proposed model. We achieved an average accuracy, precision, recall, and$F$-measure values of 68.50%, 78.52%, 69.47%, and 67.05%, respectively. The results show an increase of 2.14% in$F$-measure in comparison with the current state-of-the-art (SOTA) models for this dataset.

Artificial Intelligence-based clinical decision support is gaining ever-growing popularity and demand in both the research and industry communities. One such manifestation is automatic disease diagnosis, which aims to assist clinicians in conducting symptom investigations and disease diagnoses. When we consult with doctors, we often report and describe our health conditions with visual aids. Moreover, many people are unacquainted with several symptoms and medical terms, such as mouth ulcer and skin growth. Therefore, visual form of symptom reporting is a necessity. Motivated by the efficacy of visual form of symptom reporting, we propose and build a novel end-to-end Multi-modal Disease Diagnosis Virtual Assistant (MDD-VA) using reinforcement learning technique. In conversation, users' responses are heavily influenced by the ongoing dialogue context, and multi-modal responses appear to be of no difference. We also propose and incorporate a Context-aware Symptom Image Identification module that leverages discourse context in addition to the symptom image for identifying symptoms effectively. Furthermore, we first curate a multi-modal conversational medical dialogue corpus in English that is annotated with intent, symptoms, and visual information. The proposed MDD-VA outperforms multiple uni-modal baselines in both automatic and human evaluation, which firmly establishes the critical role of symptom information provided by visuals . The dataset and code are available at https://github.com/NLP-RL/DrCanSee

In this paper, we hypothesize that humor is closely related to sentiment and emotions. Also, due to the tremendous growth in multilingual content, there is a great demand for building models and systems that support multilingual information access. To end this, we first extend the recently released Multimodal Multiparty Hindi Humor (M2H2) dataset by adding parallel English utterances corresponding to Hindi utterances and then annotating each utterance with sentiment and emotion classes. We name it Sentiment, Humor, and Emotion aware Multilingual Multimodal Multiparty Dataset (SHEMuD). Therefore, we propose a multitask framework wherein the primary task is humor detection, and the auxiliary tasks are sentiment and emotion identification. We design a multitasking framework wherein we first propose a Context Transformer to capture the deep contextual relationships with the input utterances. We then propose a Sentiment and Emotion aware Embedding (SE-Embedding) to get the overall representation of a particular emotion and sentiment w.r.t. the specific humor situation. Experimental results on the SHEMuD show the efficacy of our approach and shows that multitask learning offers an improvement over the single-task framework for both monolingual (4.86 points in Hindi and 5.9 points in English in F1-score) and multilingual (5.17 points in F1-score) setting.

The interaction between a consumer and the customer service representative greatly contributes to the overall customer experience. Therefore, to ensure customers’ comfort and retention, it is important that customer service agents and chatbots connect with users on social, cordial, and empathetic planes. In the current work, we automatically identify the sentiment of the user and transform the neutral responses into polite responses conforming to the sentiment and the conversational history. Our technique is basically a reinforced multi-task network- the primary task being ‘polite response generation’ and the secondary task being ‘sentiment analysis’- that uses a Transformer based encoder-decoder. We use sentiment annotated conversations from Twitter as the training data. The detailed evaluation shows that our proposed approach attains superior performance compared to the baseline models.

The World Health Organization has emphasised the need of stepping up suicide prevention efforts to meet the United Nation’s Sustainable Development Goal target of 2030 (Goal 3: Good health and well-being). We address the challenging task of personality subtyping from suicide notes. Most research on personality subtyping has relied on statistical analysis and feature engineering. Moreover, state-of-the-art transformer models in the automated personality subtyping problem have received relatively less attention. We develop a novel EMotion-assisted PERSONAlity Detection Framework (EM-PERSONA). We annotate the benchmark CEASE-v2.0 suicide notes dataset with personality traits across four dichotomies: Introversion (I)-Extraversion (E), Intuition (N)-Sensing (S), Thinking (T)-Feeling (F), Judging (J)–Perceiving (P). Our proposed method outperforms all baselines on comprehensive evaluation using multiple state-of-the-art systems. Across the four dichotomies, EM-PERSONA improved accuracy by 2.04%, 3.69%, 4.52%, and 3.42%, respectively, over the highest-performing single-task systems.

Mental health is a critical component of the United Nations’ Sustainable Development Goals (SDGs), particularly Goal 3, which aims to provide “good health and well-being”. The present mental health treatment gap is exacerbated by stigma, lack of human resources, and lack of research capability for implementation and policy reform. We present and discuss a novel task of detecting emotional reasoning (ER) and accompanying emotions in conversations. In particular, we create a first-of-its-kind multimodal mental health conversational corpus that is manually annotated at the utterance level with emotional reasoning and related emotion. We develop a multimodal multitask framework with a novel multimodal feature fusion technique and a contextuality learning module to handle the two tasks. Leveraging multimodal sources of information, commonsense reasoning, and through a multitask framework, our proposed model produces strong results. We achieve performance gains of 6% accuracy and 4.62% F1 on the emotion detection task and 3.56% accuracy and 3.31% F1 on the ER detection task, when compared to the existing state-of-the-art model.

Disfluencies that appear in the transcriptions from automatic speech recognition systems tend to impair the performance of downstream NLP tasks. Disfluency correction models can help alleviate this problem. However, the unavailability of labeled data in low-resource languages impairs progress. We propose using a pretrained multilingual model, finetuned only on English disfluencies, for zero-shot disfluency detection in Indian languages. We present a detailed pipeline to synthetically generate disfluent text and create evaluation datasets for four Indian languages: Bengali, Hindi, Malayalam, and Marathi. Even in the zero-shot setting, we obtain F1 scores of 75 and higher on five disfluency types across all four languages. We also show the utility of synthetically generated disfluencies by evaluating on real disfluent text in Bengali, Hindi, and Marathi. Finetuning the multilingual model on additional synthetic Hindi disfluent text nearly doubles the number of exact matches and yields a 20-point boost in F1 scores when evaluated on real Hindi disfluent text, compared to training with only English disfluent text.

Word embeddings learned using the distributional hypothesis (e.g., GloVe, Word2vec) are good at encoding various lexical-semantic relations. However, they do not capture the emotion aspects of words. We present a novel retrofitting method for updating the vectors of emotion bearing words like fun, offence, angry, etc. The retrofitted embeddings achieve better inter-cluster and intra-cluster distance for words having the same emotions, e.g., the joy cluster containing words like fun, happiness, etc., and the anger cluster with words like offence, rage, etc., as evaluated through different cluster quality metrics. For the downstream tasks on sentiment analysis and sarcasm detection, simple classification models, such as SVM and Attention Net, learned using our retrofitted embeddings perform better than their pre-trained counterparts (about 1.5 % improvement in F1-score) as well as other benchmarks. Furthermore, the difference in performance is more pronounced in the limited data setting.

Warning: This paper has contents which may be offensive, or upsetting however this cannot be avoided owing to the nature of the work.With the rise of online hate speech, automatic detection of Hate Speech, Offensive texts as a natural language processing task is getting popular.However, very little research has been done to detect unintended social bias from these toxic language datasets.This paper introduces a new dataset ToxicBias curated from the existing dataset of Kaggle competition named "Jigsaw Unintended Bias in Toxicity Classification".We aim to detect social biases, their categories, and targeted groups.The dataset contains instances annotated for five different bias categories, viz., gender, race/ethnicity, religion, political, and LGBTQ.We train transformerbased models using our curated datasets and report baseline performance for bias identification, target generation, and bias implications.Model biases and their mitigation are also discussed in detail.Our study motivates a systematic extraction of social bias data from toxic language datasets.All the codes and dataset used for experiments in this work are publicly available 1 .

In this paper, we show that the combination of Phrase Pair Injection and Corpus Filtering boosts the performance of Neural Machine Translation (NMT) systems.We extract parallel phrases and sentences from the pseudoparallel corpus and augment it with the parallel corpus to train the NMT models.With the proposed approach, we observe an improvement in the Machine Translation (MT) system for 3 low-resource language pairs, Hindi-Marathi, English-Marathi, and English-Pashto, and 6 translation directions by up to 2.7 BLEU points, on the FLORES test data.These BLEU score improvements are over the models trained using the whole pseudo-parallel corpus augmented with the parallel corpus.

Existing bug text reports are widely used by software engineers to assist them in understanding important components of individual defects and adjustments done to resolve the fault. However, bug reports are typically long and require significant effort to comprehend. Summarization of bug reports appears to be beneficial in this regard, covering essential and diverse information. We frame the Bug report summarization problem as a clustering-based optimization problem and solve it using a multi-view multi-objective optimization framework. To represent the bug reports, semantic and syntactic representations, which are regarded as separate views, are taken into account. Several cluster quality measures computed on partitionings obtained using distinct views are optimized simultaneously using a multi-objective optimization-based approach known as archived multi-objective simulated annealing. To determine the consensus between the partitionings generated using different views, an agreement index is computed, which is also optimized simultaneously along with other cluster quality measures. The proposed methodology automatically determines the number of clusters. The experiments are carried out using the two benchmark datasets (SDS and ADS) and evaluated using the well-known ROUGE, Precision, Recall, and F-measure evaluation metrics. The obtained results show that the proposed methodology outperforms state-of-the-art methods.

The advent of the Internet is a boon to society. However, many of its banes cannot be undermined, and cyberbullying is one of them. In this work, we have created a benchmark corpus for cyberbullying detection in code-mixed languages. In India, most communications on different social media platforms are based on Hindi and English languages, and language switching is a common practice in digital communication. To investigate how code-mixed data can be handled effectively, BERT language model and VecMap based bilingual embedding along with a two-channel convolutional neural network model, namely BERT+VecMap-CNN, have been used. The input to one channel is the BERT language model and that to the other is the bilingual word embedding based on VecMap. As a baseline, we used standard machine learning models, as well as deep neural network models like CNN and LSTM. Our proposed model outperforms the baselines with overall accuracy and F1-measure values of 81.12%, and 81.03%, respectively. Furthermore, a different benchmark code-mixed dataset has been considered to show the robustness of our proposed model.1

Plagiarism detection, and identifying the author(s) of scholarly articles are important for determining the actual credit of author(s). Authorship attribution of long texts mostly depends on manual feature engineering for the extraction of relevant features. In this paper, we have developed a capsule-based convolutional neural network (CNN) architecture, which takes term frequency-inverse document frequency (TF-IDF) vector as input, for solving the task of authorship attribution. The TF-IDF vectors are used to represent the articles. The generated TF-IDF vector is fed to a CNN framework, which uses a capsule for learning the spatial features. Our proposed approach reduces the extra effort of hand-crafted feature extraction. The developed model is tested on the created scholarly abstract-based dataset. From the obtained results, it can be concluded that TF-IDF based capsule-CNN method outperforms previous approaches which had utilized hand-engineered features with the neural architectures. Different multi-tasking-based models are also developed to analyze the effect of category information in the authorship prediction task. From the results, it is revealed that multi-tasking-based models yield comparative results as compared to the single-tasking-based system. To illustrate the relevance of our developed system, different qualitative analyses have also been carried out, which clearly reveals the effectiveness of utilizing category information, and capsule networks. The source codes for the proposed approach will be shared after the acceptance.

The World Health Organization (WHO) has emphasized the importance of significantly accelerating suicide prevention efforts to fulfill the United Nations' Sustainable Development Goal (SDG) objective of 2030. In this paper, we present an end-to-end multitask system to address a novel task of detection of two interpersonal risk factors of suicide, Perceived Burdensomeness (PB) and Thwarted Belongingness (TB) from suicide notes. We also introduce a manually translated code-mixed suicide notes corpus, CoMCEASE-v2.0, based on the benchmark CEASE-v2.0 dataset, annotated with temporal orientation, PB and TB labels. We exploit the temporal orientation and emotion information in the suicide notes to boost overall performance. For comprehensive evaluation of our proposed method, we compare it to several state-of-the-art approaches on the existing CEASE-v2.0 dataset and the newly announced CoMCEASE-v2.0 dataset. Empirical evaluation suggests that temporal and emotional information can substantially improve the detection of PB and TB.

With the rise of personalized online services, a huge opportunity for user profiling has developed. Gender plays a very important role for services that rely on information about a user's background. Although, due to anonymity and privacy, the gender information of a user is usually unavailable for other users. Social Networking sites have provided users with a lot of features to express their thoughts and emotions either using pictures or emojis or writing texts. Based on the idea that female and male users have some differences in their post and message contents, social media accounts can be analyzed using their textual posts for finding the user's gender. In this work, we explore different emotion-aided multi-modal gender prediction models. The basic intuition behind our proposed approach is to predict the gender of a user based on the emotional clues present in their multimodal posts, which includes texts as well as images. PAN 2018 dataset is enriched with emotion labels, for the experimentation. Different multi-tasking based architectures have been developed for gender prediction. Obtained results on the benchmark PAN-2018 dataset illustrate that the proposed multimodal emotion-aided system performs better than the single modal (with only text and only image) based models and the state of the art system too.

In the commercial aviation domain, there are a large number of documents, like accident reports of NTSB and ASRS, and regulatory directives ADs. There is a need for a system to efficiently access these diverse repositories to serve the demands of the aviation industry, such as maintenance, compliance, and safety. In this paper, we propose a Knowledge Graph (KG) guided Deep Learning (DL) based Question Answering (QA) system to cater to these requirements. We construct a KG from aircraft accident reports and contribute this resource to the community of researchers. The efficacy of this resource is tested and proved by the proposed QA system. Questions in Natural Language are converted into SPARQL (the interface language of the RDF graph database) queries and are answered from the KG. On the DL side, we examine two different QA models, BERT-QA and GPT3-QA, covering the two paradigms of answer formulation in QA. We evaluate our system on a set of handcrafted queries curated from the accident reports. Our hybrid KG + DL QA system, KGQA + BERT-QA, achieves 7% and 40.3% increase in accuracy over KGQA and BERT-QA systems respectively. Similarly, the other combined system, KGQA + GPT3-QA, achieves 29.3% and 9.3% increase in accuracy over KGQA and GPT3-QA systems respectively. Thus, we infer that the combination of KG and DL is better than either KG or DL individually for QA, at least in our chosen domain.

Social media platforms such as Twitter have evolved into a vast information sharing platform, allowing people from a variety of backgrounds and expertise to share their opinions on numerous events such as terrorism, narcotics and many other social issues. People sometimes misuse the power of social media for their agendas, such as illegal trades and negatively influencing others. Because of this, sentiment analysis has won the interest of a lot of researchers to widely analyze public opinion for social media monitoring. Several benchmark datasets for sentiment analysis across a range of domains have been made available, especially for high-resource languages. A few datasets are available for low-resource Indian languages like Hindi, such as movie reviews and product reviews, which do not address the current need for social media monitoring. In this paper, we address the challenges of sentiment analysis in Hindi and socially relevant domains by introducing a balanced corpus annotated with the sentiment classes, viz. positive, negative and neutral. To show the effective usage of the dataset, we build several deep learning based models and establish them as the baselines for further research in this direction.

Named Entity Recognition (NER) is a foundational NLP task that aims to provide class labels like Person, Location, Organisation, Time, and Number to words in free text. Named Entities can also be multi-word expressions where the additional I-O-B annotation information helps label them during the NER annotation process. While English and European languages have considerable annotated data for the NER task, Indian languages lack on that front- both in terms of quantity and following annotation standards. This paper releases a significantly sized standard-abiding Hindi NER dataset containing 109,146 sentences and 2,220,856 tokens, annotated with 11 tags. We discuss the dataset statistics in all their essential detail and provide an in-depth analysis of the NER tag-set used with our data. The statistics of tag-set in our dataset shows a healthy per-tag distribution especially for prominent classes like Person, Location and Organisation. Since the proof of resource-effectiveness is in building models with the resource and testing the model on benchmark data and against the leader-board entries in shared tasks, we do the same with the aforesaid data. We use different language models to perform the sequence labelling task for NER and show the efficacy of our data by performing a comparative evaluation with models trained on another dataset available for the Hindi NER task. Our dataset helps achieve a weighted F1 score of 88.78 with all the tags and 92.22 when we collapse the tag-set, as discussed in the paper. To the best of our knowledge, no available dataset meets the standards of volume (amount) and variability (diversity), as far as Hindi NER is concerned. We fill this gap through this work, which we hope will significantly help NLP for Hindi. We release this dataset with our code and models for further research at https://github.com/cfiltnlp/HiNER

While sentiment and emotion analysis have been studied extensively, the relationship between sarcasm and emotion has largely remained unexplored. A sarcastic expression may have a variety of underlying emotions. For example, “I love being ignored” belies sadness, while “my mobile is fabulous with a battery backup of only 15 minutes!” expresses frustration. Detecting the emotion behind a sarcastic expression is non-trivial yet an important task. We undertake the task of detecting the emotion in a sarcastic statement, which to the best of our knowledge, is hitherto unexplored. We start with the recently released multimodal sarcasm detection dataset (MUStARD) pre-annotated with 9 emotions. We identify and correct 343 incorrect emotion labels (out of 690). We double the size of the dataset, label it with emotions along with valence and arousal which are important indicators of emotional intensity. Finally, we label each sarcastic utterance with one of the four sarcasm types-Propositional, Embedded, Likeprefixed and Illocutionary, with the goal of advancing sarcasm detection research. Exhaustive experimentation with multimodal (text, audio, and video) fusion models establishes a benchmark for exact emotion recognition in sarcasm and outperforms the state-of-art sarcasm detection. We release the dataset enriched with various annotations and the code for research purposes: https://github.com/apoorva-nunna/MUStARD_Plus_Plus

The long-standing goal of Artificial Intelligence (AI) has been to create human-like conversational systems. Such systems should have the ability to develop an emotional connection with the users, consequently, emotion recognition in dialogues has gained popularity. Emotion detection in dialogues is a challenging task because humans usually convey multiple emotions with varying degrees of intensities in a single utterance. Moreover, emotion in an utterance of a dialogue may be dependent on previous utterances making the task more complex. Recently, emotion recognition in low-resource languages like Hindi has been in great demand. However, most of the existing datasets for multi-label emotion and intensity detection in conversations are in English. To this end, we propose a large conversational dataset in Hindi named EmoInHindi for multi-label emotion and intensity recognition in conversations containing 1,814 dialogues with a total of 44,247 utterances. We prepare our dataset in a Wizard-of-Oz manner for mental health and legal counselling of crime victims. Each utterance of dialogue is annotated with one or more emotion categories from 16 emotion labels including neutral and their corresponding intensity. We further propose strong contextual baselines that can detect the emotion(s) and corresponding emotional intensity of an utterance given the conversational context.

Movies reflect society and also hold power to transform opinions. Social biases and stereotypes present in movies can cause extensive damage due to their reach. These biases are not always found to be the need of storyline but can creep in as the author’s bias. Movie production houses would prefer to ascertain that the bias present in a script is the story’s demand. Today, when deep learning models can give human-level accuracy in multiple tasks, having an AI solution to identify the biases present in the script at the writing stage can help them avoid the inconvenience of stalled release, lawsuits, etc. Since AI solutions are data intensive and there exists no domain specific data to address the problem of biases in scripts, we introduce a new dataset of movie scripts that are annotated for identity bias. The dataset contains dialogue turns annotated for (i) bias labels for seven categories, viz., gender, race/ethnicity, religion, age, occupation, LGBTQ, and other, which contains biases like body shaming, personality bias, etc. (ii) labels for sensitivity, stereotype, sentiment, emotion, emotion intensity, (iii) all labels annotated with context awareness, (iv) target groups and reason for bias labels and (v) expert-driven group-validation process for high quality annotations. We also report various baseline performances for bias identification and category detection on our dataset.

Rahul Kumar, Sandeep Mathias, Sriparna Saha, Pushpak Bhattacharyya. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022.

Tulika Saha, Saichethan Reddy, Anindya Das, Sriparna Saha, Pushpak Bhattacharyya. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022.

The effective fusion of multiple modalities (i.e., text, acoustic, and visual) is a non-trivial task, as these modalities often carry specific and diverse information and do not contribute equally. The fusion of different modalities could even be more challenging under the low-resource setting, where we have fewer samples for training. This paper proposes a multi-representative fusion mechanism that generates diverse fusions with multiple modalities and then chooses the best fusion among them. To achieve this, we first apply convolution filters on multimodal inputs to generate different and diverse representations of modalities. We then fuse pairwise modalities with multiple representations to get the multiple fusions. Finally, we propose an attention mechanism that only selects the most appropriate fusion, which eventually helps resolve the noise problem by ignoring the noisy fusions. We evaluate our proposed approach on three low-resource multimodal sentiment analysis datasets, i.e., YouTube, MOUD, and ICT-MMMO. Experimental results show the effectiveness of our proposed approach with the accuracies of 59.3%, 83.0%, and 84.1% for the YouTube, MOUD, and ICT-MMMO datasets, respectively.

Detecting cyberbullying from memes is highly challenging, because of the presence of the implicit affective content which is also often sarcastic, and multi-modality (image + text). The current work is the first attempt, to the best of our knowledge, in investigating the role of sentiment, emotion and sarcasm in identifying cyberbullying from multi-modal memes in a code-mixed language setting. As a contribution, we have created a benchmark multi-modal meme dataset called MultiBully annotated with bully, sentiment, emotion and sarcasm labels collected from open-source Twitter and Reddit platforms. Moreover, the severity of the cyberbullying posts is also investigated by adding a harmfulness score to each of the memes. The created dataset consists of two modalities, text and image. Most of the texts in our dataset are in code-mixed form, which captures the seamless transitions between languages for multilingual users. Two different multimodal multitask frameworks (BERT+ResNET-Feedback and CLIP-CentralNet) have been proposed for cyberbullying detection (CD), the three auxiliary tasks being sentiment analysis (SA), emotion recognition (ER) and sarcasm detection (SAR). Experimental results indicate that compared to uni-modal and single-task variants, the proposed frameworks improve the performance of the main task, i.e., CD, by 3.18% and 3.10% in terms of accuracy and F1 score, respectively.

Over the last few years, dozens of healthcare surveys have shown a shortage of doctors and an alarming doctor-population ratio. With the motivation of assisting doctors and utilizing their time efficiently, automatic disease diagnosis using artificial intelligence is experiencing an ever-growing demand and popularity. Humans are known by the company they keep; similarly, symptoms also exhibit the association property, i.e., one symptom may strongly suggest another symptom's existence/non-existence, and their association provides crucial information about the suffering condition. The work investigates the role of symptom association in symptom investigation and disease diagnosis process. We propose and build a virtual assistant called Association guided Symptom Investigation and Diagnosis Assistant (A-SIDA) using hierarchical reinforcement learning. The proposed A-SIDDA converses with patients and extracts signs and symptoms as per patients' chief complaints and ongoing dialogue context. We infused association-based recommendations and critic into the assistant, which reinforces the assistant for conducting context-aware, symptom-association guided symptom investigation. Following the symptom investigation, the assistant diagnoses a disease based on the extracted signs and symptoms. The assistant then diagnoses a disease based on the extracted signs and symptoms. In addition to diagnosis accuracy, the relevance of inspected symptoms is critical to the usefulness of a diagnosis framework. We also propose a novel evaluation metric called Investigation Relevance Score (IReS), which measures the relevance of symptoms inspected during symptom investigation. The obtained improvements (Diagnosis success rate-5.36%, Dialogue length-1.16, Match rate-2.19%, Disease classifier-6.36%, IReS-0.3501, and Human score-0.66) over state-of-the-art methods firmly establish the crucial role of symptom association that gets uncovered by the virtual agent. Furthermore, we found that the association guided symptom investigation greatly increases human satisfaction, owing to its seamless topic (symptom) transition.

Abstract The quest for new information is an inborn human trait and has always been quintessential for human survival and progress. Novelty drives curiosity, which in turn drives innovation. In Natural Language Processing (NLP), Novelty Detection refers to finding text that has some new information to offer with respect to whatever is earlier seen or known. With the exponential growth of information all across the Web, there is an accompanying menace of redundancy. A considerable portion of the Web contents are duplicates, and we need efficient mechanisms to retain new information and filter out redundant information. However, detecting redundancy at the semantic level and identifying novel text is not straightforward because the text may have less lexical overlap yet convey the same information. On top of that, non-novel/redundant information in a document may have assimilated from multiple source documents, not just one. The problem surmounts when the subject of the discourse is documents, and numerous prior documents need to be processed to ascertain the novelty/non-novelty of the current one in concern. In this work, we build upon our earlier investigations for document-level novelty detection and present a comprehensive account of our efforts toward the problem. We explore the role of pre-trained Textual Entailment (TE) models to deal with multiple source contexts and present the outcome of our current investigations. We argue that a multipremise entailment task is one close approximation toward identifying semantic-level non-novelty. Our recent approach either performs comparably or achieves significant improvement over the latest reported results on several datasets and across several related tasks (paraphrasing, plagiarism, rewrite). We critically analyze our performance with respect to the existing state of the art and show the superiority and promise of our approach for future investigations. We also present our enhanced dataset TAP-DLND 2.0 and several baselines to the community for further research on document-level novelty detection.

We propose a multi-task ensemble framework that jointly learns multiple related problems. The ensemble model aims to leverage the learned representations of three deep learning models (i.e., CNN, LSTM and GRU) and a hand-crafted feature representation for the predictions. Through multi-task framework, we address four problems of emotion and sentiment analysis, i.e., “emotionclassification&intensity”, “valence,arousal&dominancefor emotion”, “valence&arousalfor sentiment”, and “3-class categorical&5-class ordinal classificationfor sentiment”. The underlying problems cover two granularity (i.e.,coarse-grainedandfine-grained) and a diverse range of domains (i.e.,tweets,Facebook posts,news headlines,blogs,lettersetc.). Experimental results suggest that the proposed multi-task framework outperforms the single-task frameworks in all experiments.

An essential skill for effective communication is the ability to express specific sentiment and emotion in a conversation. Any robust dialogue system should handle the combined effect of both sentiment and emotion while generating responses. This is expected to provide a better experience and concurrently increase users’ satisfaction. Previously, research on either emotion or sentiment controlled dialogue generation has shown great promise in developing the next generation conversational agents, but the simultaneous effect of both is still unexplored. The existing dialogue systems are majorly based on unimodal sources, predominantly the text, and thereby cannot utilize the information present in the other sources, such as video, audio, image, etc. In this article, we present at first a large scale benchmark Sentiment Emotion aware Multimodal Dialogue (SEMD) dataset for the task of sentiment and emotion controlled dialogue generation. The SEMD dataset consists of 55k conversations from 10 TV shows having text, audio, and video information. To utilize multimodal information, we propose multimodal attention based conditional variational autoencoder (M-CVAE) that outperforms several baselines. Quantitative and qualitative analyses show that multimodality, along with contextual information, plays an essential role in generating coherent and diverse responses for any given emotion and sentiment.

Temporal distance (TD) is a type of psychological distance which shows how an individual construes past and future. It is not explored with empirical research as to how an individual’sfocuson temporal distance (near-past, far-past, near-future, and far-future) can be measured from human-written text and further used for studying human tendencies. Traditionally, focus on a Temporal Distance is studied by self-report measurements. In this article, we present a study on human focus on a temporal distance from their Twitter posts (English tweets). We first identify the tweet-level temporal focus by deep neural classifiers which make use of linguistic knowledge for classification. The model classifies each tweet into one ofnear-past,far-past,near-futureorfar-future. Classified tweets are then grouped by users to obtain the user-level temporal focus. Finally, we correlate the user’s focus on temporal distance (near-past, far-past, near-future, and far-future) with his/her demographic (age, gender, education, and relationship status) and psychological attributes (intelligence, optimism, joy, sadness, disgust, anger, surprise, and fear). Our empirical analysis reveals that users’near-pastfocus is more positively correlated to their age. We also observe that users’near-futurefocus is correlated to joy while users’ focus onfar-pastis associated with negative emotions like sadness, disgust, anger, and fear.

Deep learning has become most prominent in solving various Natural Language Processing (NLP) tasks including sentiment analysis. However, these techniques require a considerably large amount of annotated corpus, which is not easy to obtain for most of the languages, especially under the scenario of low-resource settings. In this article, we propose a deep multi-task multi-lingual adversarial framework to solve the resource-scarcity problem of sentiment analysis by leveraging the useful and relevant knowledge from a high-resource language. To transfer the knowledge between the different languages, both the languages are mapped to the shared semantic space using cross-lingual word embeddings. We evaluate our proposed architecture on a low-resource language, Hindi, using English as the high-resource language. Experiments show that our proposed model achieves an accuracy of 60.09% for the movie review dataset and 72.14% for the product review dataset. The effectiveness of our proposed approach is demonstrated with significant performance gains over the state-of-the-art systems and translation-based baselines.

Image captioning refers to the process of generating a textual description that describes objects and activities present in a given image. It connects two fields of artificial intelligence, computer vision, and natural language processing. Computer vision and natural language processing deal with image understanding and language modeling, respectively. In the existing literature, most of the works have been carried out for image captioning in the English language. This article presents a novel method for image captioning in the Hindi language using encoder–decoder based deep learning architecture with efficient channel attention. The key contribution of this work is the deployment of an efficient channel attention mechanism with bahdanau attention and a gated recurrent unit for developing an image captioning model in the Hindi language. Color images usually consist of three channels, namely red, green, and blue. The channel attention mechanism focuses on an image’s important channel while performing the convolution, which is basically to assign higher importance to specific channels over others. The channel attention mechanism has been shown to have great potential for improving the efficiency of deep convolution neural networks (CNNs). The proposed encoder–decoder architecture utilizes the recently introduced ECA-NET CNN to integrate the channel attention mechanism. Hindi is the fourth most spoken language globally, widely spoken in India and South Asia; it is India’s official language. By translating the well-known MSCOCO dataset from English to Hindi, a dataset for image captioning in Hindi is manually created. The efficiency of the proposed method is compared with other baselines in terms of Bilingual Evaluation Understudy (BLEU) scores, and the results obtained illustrate that the method proposed outperforms other baselines. The proposed method has attained improvements of 0.59%, 2.51%, 4.38%, and 3.30% in terms of BLEU-1, BLEU-2, BLEU-3, and BLEU-4 scores, respectively, with respect to the state-of-the-art. Qualities of the generated captions are further assessed manually in terms of adequacy and fluency to illustrate the proposed method’s efficacy.

People suffering from stress and various mental health problems find it easier to express and share their feelings on online platforms, such as Twitter. However, the imposed character limit (280 characters) by Twitter and infrequent online activities of a section of users poses a serious setback in using computational methods for mental health analysis or emotion research. Twitter provides rich metadata information about its users (such as user’s description, geolocation, and profile image URL), which can provide valuable information regarding the mental state of the users. We hypothesize that Twitter’s rich metadata information about their users can provide some valuable depression cues, which may help in an early low-profile evaluation. In this article, we investigate this hypothesis by developing an end-to-end multimodal multitask (MT) system for depression detection (primary task) and emotion recognition (auxiliary task), where the variation of emotion information based on different user descriptions assists the learning of the primary task. The proposed system attains 70% accuracy on the depression detection task outperforming several single-task (ST) baselines built on the various combination of input features. Our findings indicate that Twitters’s rich metadata information can be leveraged to detect depression among users with significant confidence.

Speech act classification determining the communicative intent of an utterance has been studied widely over the years as an independent task. This holds true for discussion in any for a, including social media platforms such as Twitter. However, the tweeter’s emotional state has a huge impact on its pragmatic content because communication is fundamentally characterized and mediated by direct emotions. Sentiment as a human behavior often has a strong relation to emotion, and one helps to understand the other better. We hypothesize that the association between emotion and sentiment will provide a clearer understanding of the tweeter’s state of mind, aiding the identification of tweet acts (speech acts in Twitter, TAs). As the first step, we create a new multimodal, emotion-TA,EmoTAdataset collected from the open-source Twitter dataset. To incorporate these multiple aspects, we propose a multitask ensemble adversarial learning framework for multimodal TA classification (TAC). In addition, we also incorporate a joint embedding network, with bidirectional constraints to capture and efficiently integrate the shared semantic relationships across modalities and learn generalized features across multiple tasks. Experimental results indicate that the proposed framework boosts the performance of the primary task, TAC, by benefiting from the two secondary tasks, i.e., sentiment and emotion analyses compared to its unimodal and single-task TAC variants.

Microblog summarization systems are gaining importance during natural disasters. A lot of tweets are posted along with multimedia content during the occurrence of any natural disaster event. Extracting relevant information/summary from these tweets is important for the smooth functioning of the rescue operation. Moreover, because of the limited size of the tweets, in many cases, tweets are associated with images. The current work is the first of its kind where both the image and the tweet text are utilized simultaneously to generate a summary from microblog data generated during a disaster event. Different aspects, such as syntactic similarity, the maximum length of the tweets, retweet score, and antiredundancy, are considered as objective functions and those are simultaneously optimized using a metaheuristic population-based evolutionary strategy to select a good set of tweets to form a good quality summary. In order to extract information from images, a dense captioning model is utilized and the dense captions are further utilized for calculating the antiredundancy measure. We employed word mover distance to capture the semantic similarity between two tweets. Due to the unavailability of the dataset for multimodal microblog summarization tasks in a disaster-event scenario, datasets are created and made openly available to the community. The obtained summarization results are evaluated using the well-known ROUGE measure.

Authorship attribution (AA) is an important task, as it identifies the author of a written text from a set of suspect authors. Different methodologies of anonymous writing have been discovered with the rising usage of social media. This anonymous writing leads to an increase in malicious and suspicious activities, and anonymity makes it difficult to find the suspect. AA helps to find the writer of a suspect text from a set of suspects. Different social media platforms, such as Twitter, Facebook, and Instagram, are used regularly by the users for sharing their daily life activities. Finding the writer of microtexts is considered the toughest task due to the shorter length of the suspect piece of text. We present a Capsule-based convolutional neural network (CNN) model over character$n$-grams for performing the AA task. Capsule with kervolutional neural networks (KNNs) has also been utilized for this task. We also present different analyses of our developed system, which improves the interpretability of our developed system. Heat-maps for different models illustrate the relevant text fragments for the prediction task. A standard Twitter data set is used for evaluating the performance of the developed systems. The experimental evaluation shows that capsule-based CNNs and capsule-based KNNs perform competitively and are able to outperform previous methods. The source codes and the supplementary file are available herehttps://github.com/chanchalIITP/AuthorIdentification.

The focus on conversational systems has recently shifted towards creating engaging agents by inculcating emotions into them. Human emotions are highly complex as humans can express multiple emotions with varying intensity in a single utterance, whereas the conversational agents convey only one emotion in their responses. To infuse human-like behaviour in the agents, we introduce the task of multi-emotion controllable response generation with the ability to express different emotions with varying levels of intensity in an open-domain dialogue system. We introduce a Multiple Emotion Intensity aware Multi-party Dialogue (MEIMD) dataset having 34k conversations taken from 8 different TV Series. We finally propose a Multiple Emotion with Intensity-based Dialogue Generation (MEI-DG) framework. The system employs two novel mechanisms: viz. (i) determining the trade-off between the emotion and generic words, while focusing on the intensity of the desired emotions; and (ii) computing the amount of emotion left to be expressed, thereby regulating the generation accordingly. The detailed evaluation shows that our proposed approach attains superior performance compared to the baseline models.

Diptesh Kanojia, Prashant Sharma, Sayali Ghodekar, Pushpak Bhattacharyya, Gholamreza Haffari, Malhar Kulkarni. Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. 2021.

Nikhil Saini, Drumil Trivedi, Shreya Khare, Tejas Dhamecha, Preethi Jyothi, Samarth Bharadwaj, Pushpak Bhattacharyya. Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. 2021.

A recent topic of research in natural language generation has been the development of automatic response generation modules that can automatically respond to a user's utterance in an empathetic manner.Previous research has tackled this task using neural generative methods by augmenting emotion classes with the input sequences.However, the outputs by these models may be inconsistent.We employ multitask learning to predict the emotion label and to generate a viable response for a given utterance using a common encoder with multiple decoders.Our proposed encoder-decoder model consists of a self-attention based encoder and a decoder with dot product attention mechanism to generate response with a specified emotion.We use the focal loss to handle imbalanced data distribution, and utilize the consistency loss to allow coherent decoding by the decoders.Human evaluation reveals that our model produces more emotionally pertinent responses.In addition, our model outperforms multiple strong baselines on automatic evaluation measures such as F1 and BLEU scores, thus resulting in more fluent and adequate responses.

We explore the impact of leveraging the relatedness of languages that belong to the same family in NLP models using multilingual fine-tuning.We hypothesize and validate that multilingual fine-tuning of pre-trained language models can yield better performance on downstream NLP applications, compared to models fine-tuned on individual languages.A first of its kind detailed study is presented to track performance change as languages are added to a base language in a graded and greedy (in the sense of best boost of performance) manner; which reveals that careful selection of subset of related languages can significantly improve performance than utilizing all related languages.The Indo-Aryan (IA) language family is chosen for the study, the exact languages being Bengali, Gujarati, Hindi, Marathi, Oriya, Punjabi and Urdu.The script barrier is crossed by simple rule-based transliteration of the text of all languages to Devanagari.Experiments are performed on mBERT, IndicBERT, MuRIL and two RoBERTa-based LMs, the last two being pre-trained by us.Low resource languages, such as Oriya and Punjabi, are found to be the largest beneficiaries of multilingual fine-tuning.Textual Entailment, Entity Classification, Section Title Prediction, tasks of IndicGLUE and POS tagging form our test bed.Compared to monolingual fine tuning we get relative performance improvement of up to 150% in the downstream tasks.The surprise take-away is that for any language there is a particular combination of other languages which yields the best performance, and any additional language is in fact detrimental.

Computational Humour (CH) has attracted the interest of Natural Language Processing and Computational Linguistics communities.Creating datasets for automatic measurement of humour quotient is difficult due to multiple possible interpretations of the content.In this work, we create a multi-modal humourannotated dataset (∼40 hours) using stand-up comedy clips.We devise a novel scoring mechanism to annotate the training data with a humour quotient score using the audience's laughter.The normalized duration (laughter duration divided by the clip duration) of laughter in each clip is used to compute this humour coefficient score on a five-point scale (0-4).This method of scoring is validated by comparing with manually annotated scores, wherein a quadratic weighted kappa of 0.6 is obtained.We use this dataset to train a model that provides a "funniness" score, on a five-point scale, given the audio and its corresponding text.We compare various neural language models for the task of humour-rating and achieve an accuracy of 0.813 in terms of Quadratic Weighted Kappa (QWK).Our "Open Mic" dataset is released for further research along with the code.

Humor recognition in conversations is a challenging task that has recently gained popularity due to its importance in dialogue understanding, including in multimodal settings (i.e., text, acoustics, and visual). The few existing datasets for humor are mostly in English. However, due to the tremendous growth in multilingual content, there is a great demand to build models and systems that support multilingual information access. To this end, we propose a dataset for Multimodal Multiparty Hindi Humor (M2H2) recognition in conversations containing 6,191 utterances from 13 episodes of a very popular TV series ”Shrimaan Shrimati Phir Se”. Each utterance is annotated with humor/non-humor labels and encompasses acoustic, visual, and textual modalities. We propose several strong multimodal baselines and show the importance of contextual and multimodal information for humor recognition in conversations. The empirical results on M2H2 dataset demonstrate that multimodal information complements unimodal information for humor recognition. The dataset and the baselines are available at http://www.iitp.ac.in/~ai-nlp-ml/resources.html and https://github.com/declare-lab/M2H2-dataset.

In this paper, we propose an unsupervised approach for knowledge graph (KG) creation from conversational data. We make use of intent classification and slot-filling, the two important components of any dialogue agent, exploit their interconnectedness, and finally construct a KG. We build a supervised intent classifier to extract the intent classes, and then on top of this we run our occlusion based slot-information extraction algorithm. Our algorithm is able to make use of supervised training of intent classifiers for extracting the relevant slot-information in an unsupervised way. To test the effectiveness of our system, we perform both automatic and manual evaluation of our intent-classifier and slot-filling system on three dialog datasets. Finally, we construct a knowledge graph from the dialogue conversation using an algorithm that makes use of our occlusion based slot-information extraction module. Empirical evaluation shows that our occlusion based method is able to successfully extract slot information from conversations, resulting in a high-quality KG.

Emotion analysis from texts has emerged as an important area of research in the field of Natural Language Processing (NLP) in the past few years. Several benchmark datasets have been released for this task, however most of the datasets are open domain in nature and for resource-rich language like English. These datasets may not always be sufficient for capturing domain specific emotions, and may tend to skew towards an emotion based on the domain. In this paper, we provide a framework for multilingual emotion detection to deal with the crisis situations. We collect and annotate disaster domain social media tweets and news data in two languages, namely English and Hindi. We derive 6 emotions from Plutchik's wheel of emotions suitable for disaster domain, and annotate the data using these disaster specific emotions. In total we create four emotionally enriched datasets i.e. 2 tweets datasets (English and Hindi) and 2 news dataset (English and Hindi). We also establish strong baselines on the dataset using two popular deep learning algorithms, stacked Bi-LSTM+CNN and BERT. Evaluation shows that the best model achieves the averaged accuracy of 68% across the four different datasets.

Emotion Recognition in Conversation (ERC) is becoming increasingly popular due to the accessibility of an enormous measure of openly accessible conversational information. Moreover, it has potential applications in opinion mining, social media and the health care domain. In this paper, we propose a novel Context and Knowledge Enriched Transformer Framework (CKETF) in which we interpret the contextual information from the utterances using a pre-trained Bidirectional Encoder Representations from Transformers (BERT) model and leverage additive attention based hierarchical transformer for encoding the knowledge sentences. Experiments on the knowledge-grounded Topical Chat dataset shows that both context and external knowledge are important for conversational emotion recognition. We demonstrate through extensive experiments and analysis that our proposed model significantly outperforms the current state-of-the-art methods.

There has been significant progress in image captioning in recent years. The generation of video descriptions is still in its early stages; this is due to the complex nature of videos in comparison to images. Generating paragraph descriptions of a video is even more challenging. Amongst the main issues are temporal object dependencies and complex object-object relationships. Recently, many works are proposed on the generation of multi-sentence video descriptions. The majority of the proposed works are based on a two-step approach: 1) event proposals and 2) caption generation. While these approaches produce good results, they miss out on globally available information. Here we propose the use of global object proposals while generating the video captions. Experimental results on ActivityNet dataset illustrate that the use of global object proposals can produce more informative and correct captions. We also propose three scores to evaluate the object detection capacity of the generator. Comparison of captions generated by the proposed method and the state-of-the-art techniques proves the efficacy of the proposed method.

Hate speech has become a critical problem in all social media, leading to many hate crimes alongside affecting the mental and emotional well-being of affected individuals. This calls out for methods to detect online hate speech more than ever. While numerous architectures exist for hate speech detection in unimodal setup (i.e., either textual or visual) we have targeted the problem in the context of both text and images inspired by the real-world raw data which involves several modalities. We propose a multimodal hate speech classifier, called as Character Text Image Classifier (CTIC), which builds upon Bidirectional Encoder Representations from Transformers (BERT), Capsule Network, and EfficientNet involving four modalities, namely word embeddings, character embeddings, sentence embeddings, and images. We report the experiments performed upon our proposed model and several other models which have been tested in the process. We train our model with different sampling techniques, and selective training, upon a Twitter dataset, called MMHS150K, consisting of both texts and associated images. Our proposed multimodal approach attains better performance than the previous models constructed upon the dataset.

Major Depressive Disorder is one of the most serious mental health concerns haunting human kind. While there have been enough global efforts to reduce clinical depression, it is still not sufficient considering the grim statistics in terms of mental health. With the lopsided distribution of Mental Health Professionals (MHPs) to patients, initiatives should be taken to develop automated systems to assist MHPs in combating this grievous mental illness. In this paper, we propose a Virtual Assistant (VA) acting as a first point of contact for depressed or discouraged users. The VA should be capable of generating motivational and affirmative responses, provide users with a safe environment to share thoughts and seek help and advice anonymously. For addressing these multiple aspects, we curate a large-scale dataset, MotiVAte, consisting of dyadic conversations between the depressed user and the VA (imparting hope and motivation). The proposed dataset is evaluated on state of the art generative models. Empirical results both automatic and human evaluation based are reported for the same. To the best of our knowledge, this is the first application of Natural Language Generation to mental health.

With the ever increasing complexity of the user queries in a multi-domain based task-oriented dialogue system, it is imperative to facilitate robust Spoken Language Understanding (SLU) modules that perform multiple tasks in an unified way. In this paper, we present a novel multi-task approach for the joint modelling of three tasks together, namely, Domain Classification, Intent Detection and Slot-Filling. We hypothesize with the intuition that the cross dependencies of all these three tasks mutually help each other towards their representations and classifications which further simplify the SLU module in a multi-domain scenario. Towards this end, we propose a BERT language model based multi-task framework utilizing capsule networks and conditional random fields for addressing the classification and sequence labeling problems, respectively, for different tasks. Experimental results indicate that the proposed multi-task model outperformed several strong baselines and its single task counterparts on three benchmark datasets of different domains and attained state-of-the-art results on different tasks.

The gender of a user plays a very important role in the development of the personalized online services. However, due to privacy, and anonymity, gender information is usually not available for many users. Since male and female users have differences in their message contents, the messages posted by users can be analyzed for finding their genders. Users with different genders express their emotions differently too. In the current paper, we introduce an emotion-aided gender prediction system. The intuition behind our approach is to predict the gender of a user based on emotional clues. The proposed approach consists of two neural network branches, one for gender and the other for emotion. Both networks share their bottom feature extraction module and are optimized within a multi-task learning framework. Gender and emotion networks are trained over our own annotated dataset having gender and emotion labels. Experimental results show the effectiveness of using emotion for predicting the gender of a user.

Developing an adequate and human-like virtual agent has been one of the primary applications of artificial intelligence. In the last few years, task-oriented dialogue systems have gained huge popularity because of their upsurging relevance and positive outcomes. In real-world, users may not always have a predefined and rigid task goal beforehand; they upgrade/downgrade/change their goal component dynamically depending upon their utility value and agent's serving capability. However, existing virtual agents fail to incorporate this dynamic behavior, leading to either unsuccessful task completion or an ungratified user experience. The paper presents an end to end multimodal dialogue system for dynamic and co-operative goal setting, which incorporates i) a multi-modal semantic state representation in policy learning to deal with multi-modal inputs, ii) a goal manager module in a traditional dialogue manager for handling dynamic and goal unavailability scenarios effectively, iii) an accumulative reward (task/persona/sentiment) for task success, personalized persuasion and user-adaptive behavior, respectively. The obtained experimental results and the comparisons with baselines firmly establish the need and efficacy of the proposed system.

Social chatbots have gained immense popularity, and their appeal lies not just in their capacity to respond to the diverse requests from users, but also in the ability to develop an emotional connection with users.To further develop and promote social chatbots, we need to concentrate on increasing user interaction and take into account both the intellectual and emotional quotient in the conversational agents.Therefore, in this work, we propose the task of sentiment aware emotion controlled personalized dialogue generation giving the machine the capability to respond emotionally and in accordance with the persona of the user.As sentiment and emotions are highly co-related, we use the sentiment knowledge of the previous utterance to generate the correct emotional response in accordance with the user persona.We design a Transformer based Dialogue Generation framework, that generates responses that are sensitive to the emotion of the user and corresponds to the persona and sentiment as well.Moreover, the persona information is encoded by a different Transformer encoder, along with the dialogue history, is fed to the decoder for generating responses.We annotate the PersonaChat dataset with sentiment information to improve the response quality.Experimental results on the PersonaChat dataset show that the proposed framework significantly outperforms the existing baselines, thereby generating personalized emotional responses in accordance with the sentiment that provides better emotional connection and user satisfaction as desired in a social chatbot.

In this paper and we explore different techniques of overcoming the challenges of low-resource in Neural Machine Translation (NMT) and specifically focusing on the case of English-Marathi NMT. NMT systems require a large amount of parallel corpora to obtain good quality translations. We try to mitigate the low-resource problem by augmenting parallel corpora or by using transfer learning. Techniques such as Phrase Table Injection (PTI) and back-translation and mixing of language corpora are used for enhancing the parallel data; whereas pivoting and multilingual embeddings are used to leverage transfer learning. For pivoting and Hindi comes in as assisting language for English-Marathi translation. Compared to baseline transformer model and a significant improvement trend in BLEU score is observed across various techniques. We have done extensive manual and automatic and qualitative evaluation of our systems. Since the trend in Machine Translation (MT) today is post-editing and measuring of Human Effort Reduction (HER) and we have given our preliminary observations on Translation Edit Rate (TER) vs. BLEU score study and where TER is regarded as a measure of HER.

Recent advances in Unsupervised Neural Machine Translation (UNMT) has minimized the gap between supervised and unsupervised machine translation performance for closely related language-pairs. However and the situation is very different for distant language pairs. Lack of overlap in lexicon and low syntactic similarity such as between English and IndoAryan languages leads to poor translation quality in existing UNMT systems. In this paper and we show that initialising the embedding layer of UNMT models with cross-lingual embeddings leads to significant BLEU score improvements over existing UNMT models where the embedding layer weights are randomly initialized. Further and freezing the embedding layer weights leads to better gains compared to updating the embedding layer weights during training. We experimented using Masked Sequence to Sequence (MASS) and Denoising Autoencoder (DAE) UNMT approaches for three distant language pairs. The proposed cross-lingual embedding initialization yields BLEU score improvement of as much as ten times over the baseline for English-Hindi and English-Bengali and English-Gujarati. Our analysis shows that initialising embedding layer with static cross-lingual embedding mapping is essential for training of UNMT models for distant language-pairs.

In this paper and we identify an interesting kind of error in the output of Unsupervised Neural Machine Translation (UNMT) systems like Undreamt1. We refer to this error type as Scrambled Translation problem. We observe that UNMT models which use word shuffle noise (as in case of Undreamt) can generate correct words and but fail to stitch them together to form phrases. As a result and words of the translated sentence look scrambled and resulting in decreased BLEU. We hypothesise that the reason behind scrambled translation problem is ’shuffling noise’ which is introduced in every input sentence as a denoising strategy. To test our hypothesis and we experiment by retraining UNMT models with a simple retraining strategy. We stop the training of the Denoising UNMT model after a pre-decided number of iterations and resume the training for the remaining iterations- which number is also pre-decided- using original sentence as input without adding any noise. Our proposed solution achieves significant performance improvement UNMT models that train conventionally. We demonstrate these performance gains on four language pairs and viz. and English-French and English-German and English-Spanish and Hindi-Punjabi. Our qualitative and quantitative analysis shows that the retraining strategy helps achieve better alignment as observed by attention heatmap and better phrasal translation and leading to statistically significant improvement in BLEU scores.

Interactive-predictive translation is a collaborative iterative process and where human translators produce translations with the help of machine translation (MT) systems interactively. Various sampling techniques in active learning (AL) exist to update the neural MT (NMT) model in the interactive-predictive scenario. In this paper and we explore term based (named entity count (NEC)) and quality based (quality estimation (QE) and sentence similarity (Sim)) sampling techniques – which are used to find the ideal candidates from the incoming data – for human supervision and MT model’s weight updation. We carried out experiments with three language pairs and viz. German-English and Spanish-English and Hindi-English. Our proposed sampling technique yields 1.82 and 0.77 and 0.81 BLEU points improvements for German-English and Spanish-English and Hindi-English and respectively and over random sampling based baseline. It also improves the present state-of-the-art by 0.35 and 0.12 BLEU points for German-English and Spanish-English and respectively. Human editing effort in terms of number-of-words-changed also improves by 5 and 4 points for German-English and Spanish-English and respectively and compared to the state-of-the-art.

Kumar Saunack, Kumar Saurav, Pushpak Bhattacharyya. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021.

Tulika Saha, Apoorva Upadhyaya, Sriparna Saha, Pushpak Bhattacharyya. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021.

Image captioning is a multi-modal problem linking computer vision and natural language processing, which combines image analysis and text generation challenges. In the literature, most of the image captioning works have been accomplished in the English language only. This paper proposes a new approach for image captioning in the Hindi language using deep learning-based encoder-decoder architecture. Hindi, widely spoken in India and South Asia, is the fourth most spoken language globally; it is India’s official language. In recent years, significant advancement has been made in image captioning, utilizing encoder-decoder architectures based on convolutional neural networks (CNNs) and recurrent neural networks (RNNs). Encoder CNN extracts features from input images, whereas decoder RNN performs language modeling. The proposed encoder-decoder architecture utilizes information multiplexing in the encoder CNN to achieve a performance gain in feature extraction. Extensive experimentation is carried out on the benchmark MSCOCO Hindi dataset, and significant improvements in BLEU score are reported compared to the baselines. Manual human evaluation in terms of adequacy and fluency of the generated captions further establishes the proposed method’s efficacy in generating good quality captions.

Abstract Detecting, whether a document contains sufficient new information to be deemed as novel , is of immense significance in this age of data duplication. Existing techniques for document-level novelty detection mostly perform at the lexical level and are unable to address the semantic-level redundancy. These techniques usually rely on handcrafted features extracted from the documents in a rule-based or traditional feature-based machine learning setup. Here, we present an effective approach based on neural attention mechanism to detect document-level novelty without any manual feature engineering. We contend that the simple alignment of texts between the source and target document(s) could identify the state of novelty of a target document. Our deep neural architecture elicits inference knowledge from a large-scale natural language inference dataset, which proves crucial to the novelty detection task. Our approach is effective and outperforms the standard baselines and recent work on document-level novelty detection by a margin of $\sim$ 3% in terms of accuracy.

Abstract Neural machine translation (NMT) has recently shown promising results on publicly available benchmark datasets and is being rapidly adopted in various production systems. However, it requires high-quality large-scale parallel corpus, and it is not always possible to have sufficiently large corpus as it requires time, money, and professionals. Hence, many existing large-scale parallel corpus are limited to the specific languages and domains. In this paper, we propose an effective approach to improve an NMT system in low-resource scenario without using any additional data. Our approach aims at augmenting the original training data by means of parallel phrases extracted from the original training data itself using a statistical machine translation (SMT) system. Our proposed approach is based on the gated recurrent unit (GRU) and transformer networks. We choose the Hindi–English, Hindi–Bengali datasets for Health, Tourism, and Judicial (only for Hindi–English) domains. We train our NMT models for 10 translation directions, each using only 5–23k parallel sentences. Experiments show the improvements in the range of 1.38–15.36 BiLingual Evaluation Understudy points over the baseline systems. Experiments show that transformer models perform better than GRU models in low-resource scenarios. In addition to that, we also find that our proposed method outperforms SMT—which is known to work better than the neural models in low-resource scenarios—for some translation directions. In order to further show the effectiveness of our proposed model, we also employ our approach to another interesting NMT task, for example, old-to-modern English translation, using a tiny parallel corpus of only 2.7K sentences. For this task, we use publicly available old-modern English text which is approximately 1000 years old. Evaluation for this task shows significant improvement over the baseline NMT.

The preservation of domain knowledge from source to the target is crucial in any translation workflows. Hence, translation service providers that use machine translation (MT) in production could reasonably expect that the translation process should transfer both the underlying pragmatics and the semantics of the source-side sentences into the target language. However, recent studies suggest that the MT systems often fail to preserve such crucial information (e.g., sentiment, emotion, gender traits) embedded in the source text in the target. In this context, the raw automatic translations are often directly fed to other natural language processing (NLP) applications (e.g., sentiment classifier) in a cross-lingual platform. Hence, the loss of such crucial information during the translation could negatively affect the performance of such downstream NLP tasks that heavily rely on the output of the MT systems. In our current research, we carefully balance both the sides (i.e., sentiment and semantics) during translation, by controlling a global-attention-based neural MT (NMT), to generate translations that encode the underlying sentiment of a source sentence while preserving its non-opinionated semantic content. Toward this, we use a state-of-the-art reinforcement learning method, namely, actor-critic , that includes a novel reward combination module, to fine-tune the NMT system so that it learns to generate translations that are best suited for a downstream task, viz. sentiment classification while ensuring the source-side semantics is intact in the process. Experimental results for Hindi–English language pair show that our proposed method significantly improves the performance of the sentiment classifier and alongside results in an improved NMT system.

Image captioning is the process of generating a textual description of an image that aims to describe the salient parts of the given image. It is an important problem, as it involves computer vision and natural language processing, where computer vision is used for understanding images, and natural language processing is used for language modeling. A lot of works have been done for image captioning for the English language. In this article, we have developed a model for image captioning in the Hindi language. Hindi is the official language of India, and it is the fourth most spoken language in the world, spoken in India and South Asia. To the best of our knowledge, this is the first attempt to generate image captions in the Hindi language. A dataset is manually created by translating well known MSCOCO dataset from English to Hindi. Finally, different types of attention-based architectures are developed for image captioning in the Hindi language. These attention mechanisms are new for the Hindi language, as those have never been used for the Hindi language. The obtained results of the proposed model are compared with several baselines in terms of BLEU scores, and the results show that our model performs better than others. Manual evaluation of the obtained captions in terms of adequacy and fluency also reveals the effectiveness of our proposed approach. Availability of resources : The codes of the article are available at https://github.com/santosh1821cs03/Image_Captioning_Hindi_Language ; The dataset will be made available: http://www.iitp.ac.in/∼ai-nlp-ml/resources.html .

Building Virtual Agents capable of carrying out complex queries of the user involving multiple intents of a domain is quite a challenge, because it demands that the agent manages several subtasks simultaneously. This article presents a universal Deep Reinforcement Learning framework that can synthesize dialogue managers capable of working in a task-oriented dialogue system encompassing various intents pertaining to a domain. The conversation between agent and user is broken down into hierarchies, to segregate subtasks pertinent to different intents. The concept of Hierarchical Reinforcement Learning, particularly options , is used to learn policies in different hierarchies that operates in distinct time steps to fulfill the user query successfully. The dialogue manager comprises top-level intent meta-policy to select among subtasks or options and a low-level controller policy to pick primitive actions to communicate with the user to complete the subtask provided to it by the top-level policy in varying intents of a domain. The proposed dialogue management module has been trained in a way such that it can be reused for any language for which it has been developed with little to no supervision. The developed system has been demonstrated for “Air Travel” and “Restaurant” domain in English and Hindi languages. Empirical results determine the robustness and efficacy of the learned dialogue policy as it outperforms several baselines and a state-of-the-art system.

The rising usage of social media has motivated to invent different methodologies of anonymous writing, which leads to an increase in malicious and suspicious activities. This anonymity has created difficulty in finding the suspect. Author profiling deals with the characterization of an author through some key attributes such as gender, age, language, dialect region variety, personality, and so on. Identifying the gender of the author of a suspect document is a salient task of author-profiling. The linguistic profile of a user can help in determining his/her demographics. Different social media platforms, such as Twitter, Facebook, and Instagram, are used regularly by users for sharing their daily life activities. Moreover, users often post images along with text on different social media platforms; thus, the usage of multimodal information is very common nowadays. In this article, the task of automatic gender prediction from multimodal Twitter data is posed as a classification problem and an efficient multimodal neural framework is proposed for solving this. The popularly used BERT_base is utilized for learning the encoded representation for the text part of the tweet, and recently introduced EfficientNet is used for extracting the features from images. Finally, a direct product-based fusion strategy is applied for fusing the text and image representations, followed by a fully connected layer for predicting the gender of a Twitter user. Plagiarism detection authorship analysis near end duplicate detection (PAN)-2018 author profiling data are used for evaluating the performance of our proposed approach. Our proposed model achieved accuracies of 82.05%, 86.22%, and 89.53% for pure-image, pure-text, and multimodal setting, respectively; outperforming the previous state-of-the-art works in all the cases. Moreover, a deep analysis is carried out to interpret the produced results; different words that serve as clues for gender classification are identified characterizing different gender classes. The supplementary file and the source codes for the proposed approach are available athttps://github.com/chanchalIITP/GenderTCSS.

In this paper, we hypothesize that sarcasm is closely related to sentiment and emotion, and thereby propose a multi-task deep learning framework to solve all these three problems simultaneously in a multi-modal conversational scenario. We, at first, manually annotate the recently released multi-modal MUStARD sarcasm dataset with sentiment and emotion classes, both implicit and explicit. For multi-tasking, we propose two attention mechanisms, viz. Inter-segment Inter-modal Attention (Ie-Attention) and Intra-segment Inter-modal Attention (Ia-Attention). The main motivation of Ie-Attention is to learn the relationship between the different segments of the sentence across the modalities. In contrast, Ia-Attention focuses within the same segment of the sentence across the modalities. Finally, representations from both the attentions are concatenated and shared across the five classes (i.e., sarcasm, implicit sentiment, explicit sentiment, implicit emotion, explicit emotion) for multi-tasking. Experimental results on the extended version of the MUStARD dataset show the efficacy of our proposed approach for sarcasm detection over the existing state-of-the-art systems. The evaluation also shows that the proposed multi-task framework yields better performance for the primary task, i.e., sarcasm detection, with the help of two secondary tasks, emotion and sentiment analysis.

The task of Dialogue Act Classification (DAC) that purports to capture communicative intent has been studied extensively.But these studies limit themselves to text.Non-verbal features (change of tone, facial expressions etc.) can provide cues to identify DAs, thus stressing the benefit of incorporating multi-modal inputs in the task.Also, the emotional state of the speaker has a substantial effect on the choice of the dialogue act, since conversations are often influenced by emotions.Hence, the effect of emotion too on automatic identification of DAs needs to be studied.In this work, we address the role of both multi-modality and emotion recognition (ER) in DAC.DAC and ER help each other by way of multi-task learning.One of the major contributions of this work is a new dataset-multimodal Emotion aware Dialogue Act dataset called EMOTyDA, collected from open-sourced dialogue datasets.To demonstrate the utility of EMOTyDA, we build an attention based (self, inter-modal, inter-task) multi-modal, multi-task Deep Neural Network (DNN) for joint learning of DAs and emotions.We show empirically that multimodality and multi-tasking achieve better performance of DAC compared to uni-modal and single task DAC variants.

Nowadays, micro-blogging sites are getting popular due to the involvement of a large number of users. In the case of natural disasters, a significant amount of relevant information (giving crucial information) are present amongst the tweets. Therefore, there is a need to develop a system that summarizes relevant tweets by extracting informative tweets. In the current paper, we have proposed an unsupervised approach for summarizing the relevant tweets namely, MOOTweetSumm+, which automatically selects the informative tweets. Several tweet-scoring measures: (a) anti-redundancy measuring the dissimilarity between tweets; (b) similarity with outputs provided by LexRank (a graph-based method measuring tweet importance based on the concept of eigen-vector centrality in a graph); (c) BM25 based ranking function; (d) tf-idf based ranking function; (e) length of the tweet; (f) re-tweet count, are simultaneously optimized utilizing a binary differential evolution algorithm. Further, two different versions of the LexRank, utilizing syntactic and semantic similarity, have also been explored. For evaluation, four different disaster-event related datasets are used, and performance is measured in terms of ROUGE scores. An ablation study is also performed to determine which set of measures is best suited for different datasets. From the results obtained, it is clearly evident that our approach improves by 13.2% and 5.8% in terms of ROUGE-2 and ROUGE-L scores, over the existing approaches, respectively.

Emotion and sentiment classification in dialogues is a challenging task that has gained popularity in recent times.Humans tend to have multiple emotions with varying intensities while expressing their thoughts and feelings.Emotions in an utterance of dialogue can either be independent or dependent on the previous utterances, making the task complex and interesting.Multi-label emotion detection in conversations is a significant task that provides the ability to the system to understand the various emotions of the users interacting.On the other hand, sentiment analysis in dialogue or conversation helps in understanding the perspective of the user with respect to the ongoing conversation.Besides text, additional information in the form of audio and video assists in identifying the correct emotions with the appropriate intensity and sentiments in an utterance of a dialogue.Lately, quite a few datasets have been made available for emotion and sentiment classification in dialogues.Still, these datasets are imbalanced in representing different emotions and consist of only a single emotion.Hence, we present at first a large-scale balanced Multimodal Multi-label Emotion, Intensity, and Sentiment Dialogue dataset (MEISD) collected from different TV series that has textual, audio, and visual features, and then establish a baseline setup for further research.

Question generation (QG) attempts to solve the inverse of question answering (QA) problem by generating a natural language question given a document and an answer.While sequence to sequence neural models surpass rule-based systems for QG, they are limited in their capacity to focus on more than one supporting fact.For QG, we often require multiple supporting facts to generate high-quality questions.Inspired by recent works on multi-hop reasoning in QA, we take up Multi-hop question generation, which aims at generating relevant questions based on supporting facts in the context.We employ multitask learning with the auxiliary task of answeraware supporting fact prediction to guide the question generator.In addition, we also proposed a question-aware reward function in a Reinforcement Learning (RL) framework to maximize the utilization of the supporting facts.We demonstrate the effectiveness of our approach through experiments on the multi-hop question answering dataset, HotPotQA.Empirical evaluation shows our model to outperform the single-hop neural question generation models on both automatic evaluation metrics such as BLEU, METEOR, and ROUGE, and human evaluation metrics for quality and coverage of the generated questions.

Cognates are variants of the same lexical form across different languages; for example "fonema" in Spanish and "phoneme" in English are cognates, both of which mean "a unit of sound".The task of automatic detection of cognates among any two languages can help downstream NLP tasks such as Cross-lingual Information Retrieval, Computational Phylogenetics, and Machine Translation.In this paper, we demonstrate the use of cross-lingual word embeddings for detecting cognates among fourteen Indian Languages.Our approach introduces the use of context from a knowledge graph to generate improved feature representations for cognate detection.We then evaluate the impact of our cognate detection mechanism on neural machine translation (NMT), as a downstream task.We evaluate our methods to detect cognates on a challenging dataset of twelve Indian languages, namely, Sanskrit, Hindi, Assamese, Oriya, Kannada, Gujarati, Tamil, Telugu, Punjabi, Bengali, Marathi, and Malayalam.Additionally, we create evaluation datasets for two more Indian languages, Konkani and Nepali 1 .We observe an improvement of up to 18% points, in terms of F-score, for cognate detection.Furthermore, we observe that cognates extracted using our method help improve NMT quality by up to 2.76 BLEU.We also release 2 our code, newly constructed datasets and cross-lingual models publicly.

Unsupervised neural machine translation (NMT) utilizes only monolingual data for training.The quality of back-translated data plays an important role in the performance of NMT systems.In back-translation, all generated pseudo parallel sentence pairs are not of the same quality.Taking inspiration from domain adaptation where in-domain sentences are given more weight in training, in this paper we propose an approach to filter back-translated data as part of the training process of unsupervised NMT.Our approach gives more weight to good pseudo parallel sentence pairs in the back-translation phase.We calculate the weight of each pseudo parallel sentence pair using sentence-wise round-trip BLEU score which is normalized batch-wise.We compare our approach with the current state of the art approaches for unsupervised NMT.

Lemmatization aims to reduce the sparse data problem by relating the inflected forms of a word to its dictionary form.However, most of the prior work on this topic has focused on high resource languages.In this paper, we evaluate cross-lingual approaches for low resource languages, especially in the context of morphologically rich Indian languages.We test our model on six languages from two different families and develop linguistic insights into each model's performance.

We present a novel retrofitting model that can leverage relational knowledge available in a knowledge resource to improve word embeddings. The knowledge is captured in terms of relation inequality constraints that compare similarity of related and unrelated entities in the context of an anchor entity. These constraints are used as training data to learn a non-linear transformation function that maps original word vectors to a vector space respecting these constraints. The transformation function is learned in a similarity metric learning setting using Triplet network architecture. We applied our model to synonymy, antonymy and hypernymy relations in WordNet and observed large gains in performance over original distributional models as well as other retrofitting approaches on word similarity task and significant overall improvement on lexical entailment detection task.

In interactive machine translation (MT), human translators correct errors in automatic translations in collaboration with the MT systems, which is seen as an effective way to improve the productivity gain in translation. In this study, we model source-language syntactic constituency parse and target-language syntactic descriptions in the form of supertags as conditional context for interactive prediction in neural MT (NMT). We found that the supertags significantly improve productivity gain in translation in interactive-predictive NMT (INMT), while syntactic parsing somewhat found to be effective in reducing human effort in translation. Furthermore, when we model this source- and target-language syntactic information together as the conditional context, both types complement each other and our fully syntax-informed INMT model statistically significantly reduces human efforts in a French–to–English translation task, achieving 4.30 points absolute (corresponding to 9.18% relative) improvement in terms of word prediction accuracy (WPA) and 4.84 points absolute (corresponding to 9.01% relative) reduction in terms of word stroke ratio (WSR) over the baseline.

The rate of publishing scientific articles is increasing day by day which has created difficulty for the researchers to learn about the recent advancements in a faster way. Also, relying on the abstract of these published articles is not a good idea as they cover only broad ideas of the article. The summarization of scientific documents (SDS) addresses this challenge. In this paper, we propose a system for SDS having two components: identifying the relevant sentences in the article using citation context; generation of the summary by posing SDS as a binary optimization problem. For the purpose of optimization, a metaheuristic evolutionary algorithm is utilized. In order to improve the quality of summary, various aspects measuring the relevance of sentences are simultaneously optimized using the concept of multi-objective optimization. Inspired by the popularity of graph-based algorithms like LexRank which is popularly used in solving summarization problems of different real-life applications, its impact is studied in fusion with our optimization framework. An ablation study is also performed to identify the most contributing aspects for the summary generation. We investigated the performance of our proposed framework on two datasets related to the computational linguistic domain, CL-SciSumm 2016 and CL-SciSumm 2017, in terms of ROUGE measures. The results obtained illustrate that our framework effectively improves other existing methods. Further, results are validated using the statistical paired t-test.

The unprecedented growth of Internet users has resulted in an abundance of unstructured information on social media including health forums, where patients request health-related information or opinions from other users. Previous studies have shown that online peer support has limited effectiveness without expert intervention. Therefore, a system capable of assessing the severity of health state from the patients' social media posts can help health professionals (HP) in prioritizing the user's post. In this study, we inspect the efficacy of different aspects of Natural Language Understanding (NLU) to identify the severity of the user's health state in relation to two perspectives(tasks) (a) Medical Condition (i.e., Recover, Exist, Deteriorate, Other) and (b) Medication (i.e., Effective, Ineffective, Serious Adverse Effect, Other) in online health communities. We propose a multiview learning framework that models both the textual content as well as contextual-information to assess the severity of the user's health state. Specifically, our model utilizes the NLU views such as sentiment, emotions, personality, and use of figurative language to extract the contextual information. The diverse NLU views demonstrate its effectiveness on both the tasks and as well as on the individual disease to assess a user's health.

Gaze behaviour has been used as a way to gather cognitive information for a number of years. In this paper, we discuss the use of gaze behaviour in solving different tasks in natural language processing (NLP) without having to record it at test time. This is because the collection of gaze behaviour is a costly task, both in terms of time and money. Hence, in this paper, we focus on research done to alleviate the need for recording gaze behaviour at run time. We also mention different eye tracking corpora in multiple languages, which are currently available and can be used in natural language processing. We conclude our paper by discussing applications in a domain - education - and how learning gaze behaviour can help in solving the tasks of complex word identification and automatic essay grading.

Text generator systems have become extremely popular with the advent of recent deep learning models such as encoder-decoder. Controlling the information and style of the generated output without supervision is an important and challenging Natural Language Processing (NLP) task. In this paper we define the task of constructing a coherent paragraph from a set of disaster domain tweets, without any parallel data. We tackle the problem by building two systems in pipeline. The first system focuses on unsupervised style transfer to convert the individual tweets into news sentences. The second system stitches together the outputs from the first system to form a coherent news paragraph. We also propose a novel training mechanism, by splitting the sentences into propositions and training the second system to merge the sentences. We create a validation and test set consisting of tweet-sets and their equivalent news paragraphs to perform empirical evaluation. We also perform human evaluations on our model. In a completely unsupervised setting our model was able to achieve a BLEU score of 19.32, while successfully transferring styles and joining tweets to form a meaningful news paragraph.

Conversational systems are the perfect examples of human-machine interactions. The conversational agents while interacting with humans lack the ability to express emotions and behave inconsistently, making the conversations boring and non-interactive. In this work, we propose the task of persona aware emotional response generation in which the system can generate specific and consistent responses in accordance to the provided personality information and the conversational history. To make the responses interactive and interesting we intend to infuse the emotions in the responses that help in making the responses more human-like. We propose a persona aware attention framework employing an encoder-decoder approach. We investigate different ways to include the desired emotions in the responses. Experimental results on the PersonaChat dataset shows that our proposed framework outperforms the baseline models and can generate interactive and emotional responses.

In interactive machine translation (MT), human translators correct errors in automatic translations in collaboration with the MT systems, and this is an effective way to improve productivity gain in translation. Phrase-based statistical MT (PB-SMT) has been the mainstream approach to MT for the past 30 years, both in academia and industry. Neural MT (NMT), an end-to-end learning approach to MT, represents the current state-of-the-art in MT research. The recent studies on interactive MT have indicated that NMT can significantly outperform PB-SMT. In this work, first we investigate the possibility of integrating lexical syntactic descriptions in the form of supertags into the state-of-the-art NMT model, Transformer. Then, we explore whether integration of supertags into Transformer could indeed reduce human efforts in translation in an interactive-predictive platform. From our investigation we found that our syntax-aware interactive NMT (INMT) framework significantly reduces simulated human efforts in the French-to-English and Hindi- to-English translation tasks, achieving a 2.65 point absolute corresponding to 5.65% relative improvement and a 6.55 point absolute corresponding to 19.1% relative improvement, respectively, in terms of word prediction accuracy (WPA) over the respective baselines.

The purpose of an image paragraph captioning model is to produce detailed descriptions of the source images. Generally, paragraph captioning models use encoder-decoder based architectures similar to the standard image captioning models. The encoder is a CNN based model, and the decoder is a LSTM or GRU. The standard image captioning models produce unsatisfactory results for the paragraph captioning task due to the lack of diversity in the generated outputs [9]. The paragraphs generated from standard image captioning models lack in language diversity and contain redundant information. In this work, we have proposed an approach with language discriminator for increasing the diversity in language, and dissimilarity score using word mover's distance [4] for reducing redundant information. Using this approach with a state-of-the-art model at testing time, we have improved the METEOR score from 13.63 to 19.01 for the Visual Genome dataset.

Development of Virtual Agents (VAs) for Goal/Task-oriented conversations capable of handling complex tasks pertaining to multiple domains and its various intents is quite an onerous task. Lack of high quality, domain specific conversational data required to train policies is one of the biggest challenges for the success of any dialogue system. In this paper, we present a multi-domain, multi-intent based task-oriented dialogue system by successfully combining Hierarchical Deep Reinforcement Learning and Transfer Learning paradigms. The notion is to exploit or take advantage of the resemblance between domains as various domains share considerable amount of overlapping data or slots. Thus, Options framework along with Transfer Learning is employed to curate VAs with better and faster learning performance. Our proposed approach reduced the data requirement to train multi-domain VAs by atleast 20% for distant domains and almost 38% for close domains. It also significantly curtailed the learning time and aided faster learning for transfer learning based policies.

Speech acts help in uncovering and understanding the communicative intention and behavior of a speaker utterance. This is pertinent for communication on any platform that embodies social media platforms like Twitter. This paper presents a supervised speech act (tweet act in our case) classifier for tweets for assessing the content and intent of tweets, thereby, exploring the valuable communication amongst the tweeters. With the recent success of Bi-directional Encoder Representations from Transformers (BERT), a newly introduced language representation model that provides pre-trained deep bi-directional representations of vast unlabeled data, we introduce BERT-extended that is built on top of BERT. Our model is based on calculating attention weights over the representations of tokens of a sequence to identify tweet acts. The proposed model attained a benchmark accuracy of 75.97% and outperformed several strong baselines and state-of-the-art approaches on an open-source, tweet act annotated Twitter dataset.

Fake or incorrect or miss-information detection has nowadays attracted attention to the researchers and developers because of the huge information overloaded in the web. This problem can be considered as equivalent to lie detection, truthfulness identification or stance detection. In our particular work, we focus on deciding whether the title of a news is consistent with its body text- a problem equivalent to fake information identification. In this paper, we propose a deep transfer learning approach where the problem of detecting title-body consistency is posed from the viewpoint of Textual Entailment (TE) where the title is considered as a hypothesis and news body is treated as a premise. The idea is to decide whether the body infers the title or not. Evaluation on the existing benchmark datasets, namely Fake News Challenge (FNC) dataset (released in Fake News Challenge Stage 1 (FNC-I): Stance Detection) show the efficacy of our proposed approach in comparison to the state-of-the-art systems.

Witness testimonies are important constituents of a court case description and play a significant role in the final decision. We propose two techniques to identify sentences representing witness testimonies. The first technique employs linguistic rules whereas the second technique applies distant supervision where training set is constructed automatically using the output of the first technique. We then represent the identified witness testimonies in a more meaningful structure – event verb (predicate) along with its arguments corresponding to semantic roles A0 and A1 [1]. We demonstrate effectiveness of such representation in retrieving semantically similar prior relevant cases. To the best of our knowledge, this is the first paper to apply NLP techniques to extract witness information from court judgements and use it for retrieving prior court cases.

Customer satisfaction is an essential aspect of customer care systems. It is imperative for such systems to be polite while handling customer requests/demands. In this paper, we present a large multi-lingual conversational dataset for English and Hindi. We choose data from Twitter having both generic and courteous responses between customer care agents and aggrieved users. We also propose strong baselines that can induce courteous behaviour in generic customer care response in a multi-lingual scenario. We build a deep learning framework that can simultaneously handle different languages and incorporate polite behaviour in the customer care agent’s responses. Our system is competent in generating responses in different languages (here, English and Hindi) depending on the customer’s preference and also is able to converse with humans in an empathetic manner to ensure customer satisfaction and retention. Experimental results show that our proposed models can converse in both the languages and the information shared between the languages helps in improving the performance of the overall system. Qualitative and quantitative analysis shows that the proposed method can converse in an empathetic manner by incorporating courteousness in the responses and hence increasing customer satisfaction.

A suicide note is usually written shortly before the suicide and it provides a chance to comprehend the self-destructive state of mind of the deceased. From a psychological point of view, suicide notes have been utilized for recognizing the motive behind the suicide. To the best of our knowledge, there is no openly accessible suicide note corpus at present, making it challenging for the researchers and developers to deep dive into the area of mental health assessment and suicide prevention. In this paper, we create a fine-grained emotion annotated corpus (CEASE) of suicide notes in English and develop various deep learning models to perform emotion detection on the curated dataset. The corpus consists of 2393 sentences from around 205 suicide notes collected from various sources. Each sentence is annotated with a particular emotion class from a set of 15 fine-grained emotion labels, namely (forgiveness, happiness_peacefulness, love, pride, hopefulness, thankfulness, blame, anger, fear, abuse, sorrow, hopelessness, guilt, information, instructions). For the evaluation, we develop an ensemble architecture, where the base models correspond to three supervised deep learning models, namely Convolutional Neural Network (CNN), Gated Recurrent Unit (GRU) and Long Short Term Memory (LSTM). We obtain the highest test accuracy of 60.17% and cross-validation accuracy of 60.32%

Cognates are present in multiple variants of the same text across different languages (e.g., “hund” in German and “hound” in the English language mean “dog”). They pose a challenge to various Natural Language Processing (NLP) applications such as Machine Translation, Cross-lingual Sense Disambiguation, Computational Phylogenetics, and Information Retrieval. A possible solution to address this challenge is to identify cognates across language pairs. In this paper, we describe the creation of two cognate datasets for twelve Indian languages namely Sanskrit, Hindi, Assamese, Oriya, Kannada, Gujarati, Tamil, Telugu, Punjabi, Bengali, Marathi, and Malayalam. We digitize the cognate data from an Indian language cognate dictionary and utilize linked Indian language Wordnets to generate cognate sets. Additionally, we use the Wordnet data to create a False Friends’ dataset for eleven language pairs. We also evaluate the efficacy of our dataset using previously available baseline cognate detection approaches. We also perform a manual evaluation with the help of lexicographers and release the curated gold-standard dataset with this paper.

Due to the phenomenal growth of online content in recent time, sentiment analysis has attracted attention of the researchers and developers. A number of benchmark annotated corpora are available for domains like movie reviews, product reviews, hotel reviews, etc. The pervasiveness of social media has also lead to a huge amount of content posted by users who are misusing the power of social media to spread false beliefs and to negatively influence others. This type of content is coming from the domains like terrorism, cybersecurity, technology, social issues, etc. Mining of opinions from these domains is important to create a socially intelligent system to provide security to the public and to maintain the law and order situations. To the best of our knowledge, there is no publicly available tweet corpora for such pervasive domains. Hence, we firstly create a multi-domain tweet sentiment corpora and then establish a deep neural network based baseline framework to address the above mentioned issues. Annotated corpus has Cohen’s Kappa measurement for annotation quality of 0.770, which shows that the data is of acceptable quality. We are able to achieve 84.65% accuracy for sentiment analysis by using an ensemble of Convolutional Neural Network (CNN), Long Short Term Memory (LSTM), and Gated Recurrent Unit(GRU).

Event Extraction is an important task in the widespread field of Natural Language Processing (NLP). Though this task is adequately addressed in English with sufficient resources, we are unaware of any benchmark setup in Indian languages. Hindi is one of the most widely spoken languages in the world. In this paper, we present an Event Extraction framework for Hindi language by creating an annotated resource for benchmarking, and then developing deep learning based models to set as the baselines. We crawl more than seventeen hundred disaster related Hindi news articles from the various news sources. We also develop deep learning based models for Event Trigger Detection and Classification, Argument Detection and Classification and Event-Argument Linking.

We present ScholarlyRead, span-of-word-based scholarly articles’ Reading Comprehension (RC) dataset with approximately 10K manually checked passage-question-answer instances. ScholarlyRead was constructed in semi-automatic way. We consider the articles from two popular journals of a reputed publishing house. Firstly, we generate questions from these articles in an automatic way. Generated questions are then manually checked by the human annotators. We propose a baseline model based on Bi-Directional Attention Flow (BiDAF) network that yields the F1 score of 37.31%. The framework would be useful for building Question-Answering (QA) systems on scientific articles.

Cross-domain sentiment analysis (CDSA) helps to address the problem of data scarcity in scenarios where labelled data for a domain (known as the target domain) is unavailable or insufficient. However, the decision to choose a domain (known as the source domain) to leverage from is, at best, intuitive. In this paper, we investigate text similarity metrics to facilitate source domain selection for CDSA. We report results on 20 domains (all possible pairs) using 11 similarity metrics. Specifically, we compare CDSA performance with these metrics for different domain-pairs to enable the selection of a suitable source domain, given a target domain. These metrics include two novel metrics for evaluating domain adaptability to help source domain selection of labelled data and utilize word and sentence-based embeddings as metrics for unlabelled data. The goal of our experiments is a recommendation chart that gives the K best source domains for CDSA for a given target domain. We show that the best K source domains returned by our similarity metrics have a precision of over 50%, for varying values of K.

In this article, we propose a unified deep neural network framework for multilingual question answering (QA). The proposed network deals with the multilingual questions and answers snippets. The input to the network is a pair of factoid question and snippet in the multilingual environment (English and Hindi), and output is the relevant answer from the snippet. We begin by generating the snippet using a graph-based language-independent algorithm, which exploits the lexico-semantic similarity between the sentences. The soft alignment of the question words from the English and Hindi languages has been used to learn the shared representation of the question. The learned shared representation of question and attention-based snippet representation are passed as an input to the answer extraction layer of the network, which extracts the answer span from the snippet. Evaluation on a standard multilingual QA dataset shows the state-of-the-art performance with 39.44 Exact Match (EM) and 44.97 F1 values. Similarly, we achieve the performance of 50.11 Exact Match (EM) and 53.77 F1 values on Translated SQuAD dataset.

Identification of speech acts provides essential cues in understanding the pragmatics of a user utterance. It typically helps in comprehending the communicative intention of a speaker. This holds true for conversations or discussions on any fora, including social media platforms, such as Twitter. This article presents a novel tweet act classifier (speech act for Twitter) for assessing the content and intent of tweets, thereby exploring the valuable communication among the tweeters. With the recent success of Bidirectional Encoder Representations from Transformers (BERT), a newly introduced language representation model that provides pretrained deep bidirectional representations of vast unlabeled data, we introduce BERT-Caps that is built on top of BERT. The proposed model tends to learn traits and attributes by leveraging from the joint optimization of features from the BERT and capsule layer to develop a robust classifier for the task. Some Twitter-specific symbols are also included in the model to observe its influence and importance. The proposed model attained a benchmark accuracy of 77.52% and outperformed several strong baselines and state-of-the-art approaches.

This article proposes a novel unsupervised approach (FigSum++) for automatic figure summarization in biomedical scientific articles using a multi-objective evolutionary algorithm. The problem is treated as an optimization problem where relevant sentences in the summary for a given figure are selected based on various sentence scoring features (or objective functions), such as the textual entailment score between sentences in the summary and a figure’s caption, the number of sentences referring to that figure, semantic similarity between sentences and a figure’s caption, and the number of overlapping words between sentences and a figure’s caption. These objective functions are optimized simultaneously using multi-objective binary differential evolution (MBDE). MBDE consists of a set of solutions, and each solution represents a subset of sentences to be selected in the summary. MBDE generally uses a single differential evolution variant, but in the current study, an ensemble of two different differential evolution variants measuring diversity among solutions and convergence toward global optimal solution, respectively, is employed for efficient search. Usually, in any summarization system, diversity among sentences (called anti-redundancy ) in the summary is a very critical feature, and it is calculated in terms of similarity (like cosine similarity) among sentences. In this article, a new way of measuring diversity in terms of textual entailment is proposed. To represent the sentences of the article in the form of numeric vectors, the recently proposed BioBERT pre-trained language model in biomedical text mining is utilized. An ablation study has also been presented to determine the importance of different objective functions. For evaluation of the proposed technique, two benchmark biomedical datasets containing 91 and 84 figures are considered. Our proposed system obtains 5% and 11% improvements in terms of the F -measure metric over two datasets, compared to the state-of-the-art unsupervised methods.

Event extraction is one of the crucial tasks in biomedical text mining that aims to extract specific information concerning incidents embedded in the texts. In this article, we propose a deep learning framework that aims to identify the attributes (severity, course, temporal expression, and document creation time) associated with the medical concepts extracted from electronic medical records. The bi-directional long short-term memory network assisted by the attention mechanism is utilized to uncover the important aspects of the patient’s medical conditions. The attention mechanism specific to the medical disorder mention can focus on various parts of the sentence when different disorders are considered as input. The proposed methodology is evaluated on benchmark ShARe/CLEF eHealth Evaluation Lab 2014 shared task 2 datasets. In addition to the CLEF dataset, we also used the social media text, especially the medical blog posts. Experimental results of the proposed approach illustrate that our proposed approach achieves significant performance improvements over the state-of-the-art techniques and the highly competitive deep learning--based baseline methods.

Multimodal dialogue systems have opened new frontiers in the traditional goal-oriented dialogue systems. The state-of-the-art dialogue systems are primarily based on unimodal sources, predominantly the text, and hence cannot capture the information present in the other sources such as videos, audios, images etc. With the availability of large scale multimodal dialogue dataset (MMD) (Saha et al., 2018) on the fashion domain, the visual appearance of the products is essential for understanding the intention of the user. Without capturing the information from both the text and image, the system will be incapable of generating correct and desirable responses. In this paper, we propose a novel position and attribute aware attention mechanism to learn enhanced image representation conditioned on the user utterance. Our evaluation shows that the proposed model can generate appropriate responses while preserving the position and attribute information. Experimental results also prove that our proposed approach attains superior performance compared to the baseline models, and outperforms the state-of-the-art approaches on text similarity based evaluation metrics.

Automatically validating a research artefact is one of the frontiers in Artificial Intelligence (AI) that directly brings it close to competing with human intellect and intuition.Although criticized sometimes, the existing peer review system still stands as the benchmark of research validation.The present-day peer review process is not straightforward and demands profound domain knowledge, expertise, and intelligence of human reviewer(s), which is somewhat elusive with the current state of AI.However, the peer review texts, which contains rich sentiment information of the reviewer, reflecting his/her overall attitude towards the research in the paper, could be a valuable entity to predict the acceptance or rejection of the manuscript under consideration.Here in this work, we investigate the role of reviewers sentiments embedded within peer review texts to predict the peer review outcome.Our proposed deep neural architecture takes into account three channels of information: the paper, the corresponding reviews, and the review polarity to predict the overall recommendation score as well as the final decision.We achieve significant performance improvement over the baselines (∼ 29% error reduction) proposed in a recently released dataset of peer reviews.An AI of this kind could assist the editors/program chairs as an additional layer of confidence in the final decision making, especially when non-responding/missing reviewers are frequent in present day peer review.

In this paper, we propose a multilingual unsupervised NMT scheme which jointly trains multiple languages with a shared encoder and multiple decoders.Our approach is based on denoising autoencoding of each language and back-translating between English and multiple non-English languages.This results in a universal encoder which can encode any language participating in training into an interlingual representation, and language-specific decoders.Our experiments using only monolingual corpora show that multilingual unsupervised model performs better than the separately trained bilingual models achieving improvement of up to 1.48 BLEU points on WMT test sets.We also observe that even if we do not train the network for all possible translation directions, the network is still able to translate in a many-to-many fashion leveraging encoder's ability to generate interlingual representation.

The mining of adverse drug reaction (ADR) has a crucial role in the pharmacovigilance.The traditional ways of identifying ADR are reliable but time-consuming, non-scalable and offer a very limited amount of ADR relevant information.With the unprecedented growth of information sources in the forms of social media texts (Twitter, Blogs, Reviews etc.), biomedical literature, and Electronic Medical Records (EMR), it has become crucial to extract the most pertinent ADR related information from these free-form texts.In this paper, we propose a neural network inspired multitask learning framework that can simultaneously extract ADRs from various sources.We adopt a novel adversarial learning-based approach to learn features across multiple ADR information sources.Unlike the other existing techniques, our approach is capable to extracting fine-grained information (such as 'Indications', 'Symptoms', 'Finding', 'Disease', 'Drug') which provide important cues in pharmacovigilance.We evaluate our proposed approach on three publicly available realworld benchmark pharmacovigilance datasets, a Twitter dataset from PSB 2016 Social Media Shared Task, CADEC corpus and Medline ADR corpus.Experiments show that our unified framework achieves state-of-the-art performance on individual tasks associated with the different benchmark datasets.This establishes the fact that our proposed approach is generic, which enables it to achieve high performance on the diverse datasets.The source code is available here 1 .

Dushyant Singh Chauhan, Md Shad Akhtar, Asif Ekbal, Pushpak Bhattacharyya. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

Detecting the novelty or freshness of an entire document is essential in this age of data duplication and semantic-level redundancy all across the web. Current techniques for the problem mostly root on handcrafted similarity and divergence based measures to classify a document as novel or non-novel. However, document-level novelty detection is relatively less explored in literature if compared to its sentence-level counterpart. In this work, we present a deep neural architecture to automatically predict the amount of new information contained in a document in the form of a novelty score. Along with, we offer a dataset of more than 7500 documents, annotated at the sentence-level to facilitate further research. Our approach which learns the notion of novelty and redundancy only from the data achieves significant performance improvement over the existing methods and adopted baselines (~17% error reduction). Also, our approach complies with the Two-Stage theory of human recall essential to comprehend new information.

Speech act provides valuable insight for understanding the communicative intention and behaviour of a user utterance. This applies for conversation on any platform which includes social media network such as Twitter. This paper presents a deep learning based tweet act classifier (speech acts for Twitter) for understanding the content and intention of tweets and for discovering the valuable communications amongst the tweeters. A generic taxonomy of seven speech acts has been created for the task. A Convolution Neural Network (CNN) based Deep Learning (DL) architecture is integrated with different classifiers at the final layer such as softmax or linear Support Vector Machine (SVM) to develop a tweet act classifier. To make the classifier even more robust, various hand-crafted features are added to the system to boost its effectiveness towards improving the performance of the system. Experimental results indicate that the proposed model achieved good results with an average F1-score of 0.71 and outperformed state of the art approaches.

Dialogue Act (DA) Classification plays a signifi-cant role in the understanding of an utterance in a dialogue. Components of Spoken Dialogue System (SDS) such as Natural Language Understanding (NLU) and Dialogue Management (DM) modules can significantly exploit the output of the DA classification. In this paper, we propose a task-oriented DA classifier based on both traditional supervised Machine Learning (ML) as well as Deep Learning (DL) techniques. The type and nature of dialogues basically depend on the domain and the DA itself. So, in order to make the model task-oriented, a new tag-set has been designed by studying the properties of the target domain. On the benchmark SwitchBoard (SWBD) and TRAINS corpus, our proposed models have performed exceptionally well with the new tag-set. Experimental results indicate that our proposed models have achieved good accuracy on both the datasets and outperformed several state of the art approaches and the new tag-set is well suited for task-oriented applications.

Automatic Cognate Detection (ACD) is a challenging task which has been utilized to help NLP applications like Machine Translation, Information Retrieval and Computational Phylogenetics.Unidentified cognate pairs can pose a challenge to these applications and result in a degradation of performance.In this paper, we detect cognate word pairs among ten Indian languages with Hindi and use deep learning methodologies to predict whether a word pair is cognate or not.We identify IndoWordnet as a potential resource to detect cognate word pairs based on orthographic similarity-based methods and train neural network models using the data obtained from it.We identify parallel corpora as another potential resource and perform the same experiments for them.We also validate the contribution of Wordnets through further experimentation and report improved performance of up to 26%.We discuss the nuances of cognate detection among closely related Indian languages and release the lists of detected cognates as a dataset.We also observe the behaviour of, to an extent, unrelated Indian language pairs and release the lists of detected cognates among them as well.

In the era of deep learning-based systems, efficient input representation is one of the primary requisites in solving various problems related to Natural Language Processing (NLP), data mining, text mining, and the like. Absence of adequate representation for an input introduces the problem of data sparsity, and it poses a great challenge to solve the underlying problem. The problem is more intensified with resource-poor languages due to the absence of a sufficiently large corpus required to train a word embedding model. In this work, we propose an effective method to improve the word embedding coverage in less-resourced languages by leveraging bilingual word embeddings learned from different corpora. We train and evaluate deep Long Short Term Memory (LSTM)-based architecture and show the effectiveness of the proposed approach for two aspect-level sentiment analysis tasks (i.e., aspect term extraction and sentiment classification). The neural network architecture is further assisted by hand-crafted features for prediction. We apply the proposed model in two experimental setups: multi-lingual and cross-lingual. Experimental results show the effectiveness of the proposed approach against the state-of-the-art methods.

Temporality has significantly contributed to various Natural Language Processing and Information Retrieval applications. In this article, we first create a lexical knowledge-base in Hindi by identifying the temporal orientation of word senses based on their definition and then use this resource to detect underlying temporal orientation of the sentences. To create the resource, we propose a semi-supervised learning framework, where each synset of the Hindi WordNet is classified into one of the five categories, namely, past , present , future , neutral , and atemporal . The algorithm initiates learning with a set of seed synsets and then iterates following different expansion strategies, viz. probabilistic expansion based on classifier’s confidence and semantic distance based measures. We manifest the usefulness of the resource that we build on an external task, viz. sentence-level temporal classification. The underlying idea is that a temporal knowledge-base can help in classifying the sentences according to their inherent temporal properties. Experiments on two different domains, viz. general and Twitter, show interesting results.

Existing supervised solutions for Named Entity Recognition (NER) typically rely on a large annotated corpus. Collecting large amounts of NER annotated corpus is time-consuming and requires considerable human effort. However, collecting small amounts of annotated corpus for any language is feasible, but the performance degrades due to data sparsity. We address the data sparsity by borrowing features from the data of a closely related language. We use hierarchical neural networks to train a supervised NER system. The feature borrowing from a closely related language happens via the shared layers of the network. The neural network is trained on the combined dataset of the low-resource language and a closely related language, also termed Multilingual Learning. Unlike existing systems, we share all layers of the network between the two languages. We apply multilingual learning for NER in Indian languages and empirically show the benefits over a monolingual deep learning system and a traditional machine-learning system with some feature engineering. Using multilingual learning, we show that the low-resource language NER performance increases mainly due to (1) increased named entity vocabulary, (2) cross-lingual subword features, and (3) multilingual learning playing the role of regularization.

In recent years, social networking sites such as Twitter have become the primary sources for real-time information of ongoing events such as political rallies, natural disasters, and so on. At the time of occurrence of natural disasters, it has been seen that relevant information collected from tweets can help in different ways. Therefore, there is a need to develop an automated microblog/tweet summarization system to automatically select relevant tweets. In this article, we employ the concept of multiobjective optimization in microblog summarization to produce good quality summaries. Different statistical quality measures namely, length, tf-idf score of the tweets, antiredundancy, measuring different aspects of summary, are optimized simultaneously using the search capability of a multiobjective differential evolution technique. Different types of genetic operators including recently developed self-organizing map (a type of neural network) based operator, are explored in the proposed framework. To measure the similarity between tweets, word mover distance is utilized which is capable of capturing the semantic similarity between tweets. For evaluation, four benchmark data sets related to disaster events are used, and the results obtained are compared with various state-of-the-art techniques using ROUGE measures. It has been found that our algorithm improves by 62.37% and 5.65% in terms of ROUGE-2 and ROUGE-L scores, respectively, over the state-of-the-art techniques. Results are also validated using statistical significance t-test. At the end of the article, extension of the proposed approach to solve the multidocument summarization task is also illustrated.

Getting manually labeled data in each domain is always an expensive and a time consuming task.Cross-domain sentiment analysis has emerged as a demanding concept where a labeled source domain facilitates a sentiment classifier for an unlabeled target domain.However, polarity orientation (positive or negative) and the significance of a word to express an opinion often differ from one domain to another domain.Owing to these differences, crossdomain sentiment classification is still a challenging task.In this paper, we propose that words that do not change their polarity and significance represent the transferable (usable) information across domains for cross-domain sentiment classification.We present a novel approach based on χ 2 test and cosine-similarity between context vector of words to identify polarity preserving significant words across domains.Furthermore, we show that a weighted ensemble of the classifiers enhances the cross-domain classification performance.

Sandeep Mathias, Diptesh Kanojia, Kevin Patel, Samarth Agrawal, Abhijit Mishra, Pushpak Bhattacharyya. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2018.

The rapid growth of documents across the web has necessitated finding means of discarding redundant documents and retaining novel ones. Capturing redundancy is challenging as it may involve investigating at a deep semantic level. Techniques for detecting such semantic redundancy at the document level are scarce. In this work we propose a deep Convolutional Neural Networks (CNN) based model to classify a document as novel or redundant with respect to a set of relevant documents already seen by the system. The system is simple and do not require any manual feature engineering. Our novel scheme encodes relevant and relative information from both source and target texts to generate an intermediate representation which we coin as the Relative Document Vector (RDV). The proposed method outperforms the existing state-of-the-art on a document-level novelty detection dataset by a margin of ∼5% in terms of accuracy. We further demonstrate the effectiveness of our approach on a standard paraphrase detection dataset where paraphrased passages closely resemble to semantically redundant documents.

In this paper, we propose a hybrid technique for semantic question matching. It uses a proposed two-layered taxonomy for English questions by augmenting state-of-the-art deep learning models with question classes obtained from a deep learning based question classifier. Experiments performed on three open-domain datasets demonstrate the effectiveness of our proposed approach. We achieve state-of-the-art results on partial ordering question ranking (POQR) benchmark dataset. Our empirical analysis shows that coupling standard distributional features (provided by the question encoder) with knowledge from taxonomy is more effective than either deep learning or taxonomy-based knowledge alone.

Interpreting noun compounds is a challenging task. It involves uncovering the underlying predicate which is dropped in the formation of the compound. In most cases, this predicate is of the form VERB+PREP. It has been observed that uncovering the preposition is a significant step towards uncovering the predicate. In this paper, we attempt to paraphrase noun compounds using prepositions. We consider noun compounds and their corresponding prepositional paraphrases as parallelly aligned sequences of words. This enables us to adapt different architectures from cross-lingual embedding literature. We choose the architecture where we create representations of both noun compound (source sequence) and its corresponding prepositional paraphrase (target sequence), such that their sim- ilarity is high. We use LSTMs to learn these representations. We use these representations to decide the correct preposition. Our experiments show that this approach performs considerably well on different datasets of noun compounds that are manually annotated with prepositions.

Existing research on question answering (QA) and comprehension reading (RC) are mainly focused on the resource-rich language like English.In recent times, there has been a rapid growth of multi-lingual contents on the web, and this has posed several challenges to the existing QA systems.Code-mixing is one such challenge that makes the task even more complex.In this paper, we propose a linguistically motivated technique for codemixed question generation (CMQG) and a neural network based architecture for code-mixed question answering (CMQA).For evaluation, we manually create the code-mixed questions for Hindi-English language pair.In order to show the effectiveness of our neural network based CMQA technique, we utilize two benchmark datasets, viz.SQuAD and MMQA.Experiments show that our proposed model achieves encouraging performance on CMQG and CMQA.

Multi-modal sentiment analysis offers various challenges, one being the effective combination of different input modalities, namely text, visual and acoustic.In this paper, we propose a recurrent neural network based multi-modal attention framework that leverages the contextual information for utterance-level sentiment prediction.The proposed approach applies attention on multi-modal multi-utterance representations and tries to learn the contributing features amongst them.We evaluate our proposed approach on two multi-modal sentiment analysis benchmark datasets, viz.CMU Multi-modal Opinion-level Sentiment Intensity (CMU-MOSI) corpus and the recently released CMU Multi-modal Opinion Sentiment and Emotion Intensity (CMU-MOSEI) corpus.Evaluation results show the effectiveness of our proposed approach with the accuracies of 82.31% and 79.80% for the MOSI and MO-SEI datasets, respectively.These are approximately 2 and 1 points performance improvement over the state-of-the-art models for the datasets.

The self-organizing map (SOM) helps in exploratory phase of data mining by projecting the input data into a lower dimensional map. In recent years SOM has also been applied for classification of data points. The prominent utility of SOM based classification is evident from the use of no labeled data during training. In a multi-class classification problem where classes have high degree of overlap, it would be difficult to design a single-level SOM based classification system which can perform well for all the classes. In order to deal with the multi-class classification efficiently, the current paper proposes to develop a Cascaded SOM based architecture where classes are handled in a hierarchical way. Also, it can be applied for solving any multi-class classification problem where labeled data is limited. As a case study, in the first part of the paper results are shown for single label version of complex email classification problem where classes are highly overlapping to each other and in the second part, results are shown for some multi-labeled data sets. Different representation schemas for emails and a large set of features are also adopted for the purpose of experiment. Proposed Cascaded SOM based classification model performs well in email-classification compared to standard classification approaches and classical SOM based model.

Md Shad Akhtar, Palaash Sawant, Sukanta Sen, Asif Ekbal, Pushpak Bhattacharyya. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). 2018.

Sabyasachi Kamila, Mohammed Hasanuzzaman, Asif Ekbal, Pushpak Bhattacharyya, Andy Way. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). 2018.

In this paper, we describe our work on the creation of a voice model using a speech synthesis system for the Hindi Language.We use preexisting "voices", use publicly available speech corpora to create a "voice" using the Festival Speech Synthesis System (Black, 1997).Our contribution is two-fold: (1) We scrutinize multiple speech synthesis systems and provide an extensive report on the currently available stateof-the-art systems.We also develop voices using the existing implementations of the aforementioned systems, and (2) We use these voices to generate sample audios for randomly chosen words; manually evaluate the audio generated, and produce audio for all WordNet words using the winner voice model.We also produce audios for the Hindi WordNet Glosses and Example sentences.We describe our efforts to use preexisting implementations for WaveNet -a model to generate raw audio using neural nets (Oord et al., 2016) and generate speech for Hindi.Our lexicographers perform a manual evaluation of the audio generated using multiple voices.A qualitative and quantitative analysis reveals that the voice model generated by us performs the best with an accuracy of 0.44.

Indian language WordNets have their individual web-based browsing interfaces along with a common interface for In-doWordNet.These interfaces prove to be useful for language learners and in an educational domain, however, they do not provide the functionality of connecting to them and browsing their data through a lucid application programming interface or an API.In this paper, we present our work on creating such an easy-to-use framework which is bundled with the data for Indian language WordNets and provides NLTK WordNet interface like core functionalities in Python.Additionally, we use a pre-built speech synthesis system for Hindi language and augment Hindi data with audios for words, glosses, and example sentences.We provide a detailed usage of our API and explain the functions for ease of the user.Also, we package the IndoWord-Net data along with the source code and provide it openly for the purpose of research.We aim to provide all our work as an open source framework for further development.

Given a word, what is the most frequent sense in which it occurs in a given corpus?Most Frequent Sense (MFS) is a strong baseline for unsupervised word sense disambiguation.If we have large amounts of sense-annotated corpora, MFS can be trivially created.However, senseannotated corpora are a rarity.In this paper, we propose a method which can compute MFS from raw corpora.Our approach iteratively exploits the semantic congruity among related words in corpus.Our method performs better compared to another similar work.

Wordnets are rich lexico-semantic resources.Linked wordnets are extensions of wordnets, which link similar concepts in wordnets of different languages.Such resources are extremely useful in many Natural Language Processing (NLP) applications, primarily those based on knowledge-based approaches.In such approaches, these resources are considered as gold standard/oracle.Thus, it is crucial that these resources hold correct information.Thereby, they are created by human experts.However, manual maintenance of such resources is a tedious and costly affair.Thus techniques that can aid the experts are desirable.In this paper, we propose an approach to link wordnets.Given a synset of the source language, the approach returns a ranked list of potential candidate synsets in the target language from which the human expert can choose the correct one(s).Our technique is able to retrieve a winner synset in the top 10 ranked list for 60% of all synsets and 70% of noun synsets.

Hanumant Redkar, Rajita Shukla, Sandhya Singh, Jaya Saraswati, Laxmi Kashyap, Diptesh Kanojia, Preethi Jyothi, Malhar Kulkarni, Pushpak Bhattacharyya. Proceedings of the 9th Global Wordnet Conference. 2018.

We address the task of joint training of transliteration models for multiple language pairs ( multilingual transliteration). This is an instance of multitask learning, where individual tasks (language pairs) benefit from sharing knowledge with related tasks. We focus on transliteration involving related tasks i.e., languages sharing writing systems and phonetic properties ( orthographically similar languages). We propose a modified neural encoder-decoder model that maximizes parameter sharing across language pairs in order to effectively leverage orthographic similarity. We show that multilingual transliteration significantly outperforms bilingual transliteration in different scenarios (average increase of 58% across a variety of languages we experimented with). We also show that multilingual transliteration models can generalize well to languages/language pairs not encountered during training and hence perform well on the zeroshot transliteration task. We show that further improvements can be achieved by using phonetic feature input.

Sarcasm Suite is a browser-based engine that deploys five of our past papers in sarcasm detection and generation. The sarcasm detection modules use four kinds of incongruity: sentiment incongruity, semantic incongruity, historical context incongruity and conversational context incongruity. The sarcasm generation module is a chatbot that responds sarcastically to user input. With a visually appealing interface that indicates predictions using `faces' of our co-authors from our past papers, Sarcasm Suite is our first demonstration of our work in computational sarcasm.

Measuring reading effort is useful for practical purposes such as designing learning material and personalizing text comprehension environment. We propose a quantification of reading effort by measuring the complexity of eye-movement patterns of readers. We call the measure Scanpath Complexity. Scanpath complexity is modeled as a function of various properties of gaze fixations and saccades- the basic parameters of eye movement behavior. We demonstrate the effectiveness of our scanpath complexity measure by showing that its correlation with different measures of lexical and syntactic complexity as well as standard readability metrics is better than popular baseline measures based on fixation alone.

Cognitive NLP systems-i.e., NLP systems that make use of behavioral data -augment traditional text-based features with cognitive features extracted from eye-movement patterns, EEG signals, brain-imaging etc..Such extraction of features is typically manual.We contend that manual extraction of features may not be the best way to tackle text subtleties that characteristically prevail in complex classification tasks like sentiment analysis and sarcasm detection, and that even the extraction and choice of features should be delegated to the learning system.We introduce a framework to automatically extract cognitive features from the eye-movement / gaze data of human readers reading the text and use them as features along with textual features for the tasks of sentiment polarity and sarcasm detection.Our proposed framework is based on Convolutional Neural Network (CNN).The CNN learns features from both gaze and text and uses them to classify the input text.We test our technique on published sentiment and sarcasm labeled datasets, enriched with gaze information, to show that using a combination of automatically learned text and gaze features often yields better classification performance over (i) CNN based systems that rely on text input alone and (ii) existing systems that rely on handcrafted gaze and textual features.

Shweta Yadav, Asif Ekbal, Sriparna Saha, Pushpak Bhattacharyya. Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers. 2017.

End-to-end relation extraction refers to identifying boundaries of entity mentions, entity types of these mentions and appropriate semantic relation for each pair of mentions.Traditionally, separate predictive models were trained for each of these tasks and were used in a "pipeline" fashion where output of one model is fed as input to another.But it was observed that addressing some of these tasks jointly results in better performance.We propose a single, joint neural network based model to carry out all the three tasks of boundary identification, entity type classification and relation type classification.This model is referred to as "All Word Pairs" model (AWP-NN) as it assigns an appropriate label to each word pair in a given sentence for performing end-to-end relation extraction.We also propose to refine output of the AWP-NN model by using inference in Markov Logic Networks (MLN) so that additional domain knowledge can be effectively incorporated.We demonstrate effectiveness of our approach by achieving better end-to-end relation extraction performance than all 4 previous joint modelling approaches, on the standard dataset of ACE 2004.

In this paper, we propose a novel method for combining deep learning and classical feature based models using a Multi-Layer Perceptron (MLP) network for financial sentiment analysis.We develop various deep learning models based on Convolutional Neural Network (CNN), Long Short Term Memory (LSTM) and Gated Recurrent Unit (GRU).These are trained on top of pre-trained, autoencoder-based, financial word embeddings and lexicon features.An ensemble is constructed by combining these deep learning models and a classical supervised model based on Support Vector Regression (SVR).We evaluate our proposed technique on a benchmark dataset of SemEval-2017 shared task on financial sentiment analysis.The propose model shows impressive results on two datasets, i.e. microblogs and news headlines datasets.Comparisons show that our proposed model performs better than the existing state-of-the-art systems for the above two datasets by 2.0 and 4.1 cosine points, respectively.

Identification of intensity ordering among polar (positive or negative) words which have the same semantics can lead to a finegrained sentiment analysis.For example, master, seasoned and familiar point to different intensity levels, though they all convey the same meaning (semantics), i.e., expertise: having a good knowledge of.In this paper, we propose a semisupervised technique that uses sentiment bearing word embeddings to produce a continuous ranking among adjectives that share common semantics.Our system demonstrates a strong Spearman's rank correlation of 0.83 with the gold standard ranking.We show that sentiment bearing word embeddings facilitate a more accurate intensity ranking system than other standard word embeddings (word2vec and GloVe).Word2vec is the state-of-the-art for intensity ordering task.

Phrase-based Statistical Machine Translation (PBSMT) is commonly used for automatic translation. However, PBSMT runs into difficulty when either or both of the source and target languages are morphologically rich. Factored models are found to be useful for such cases, as they consider word as a vector of factors. These factors can contain any information about the surface word and use it while translating. The objective of the current work is to handle morphological inflections in Hindi, Marathi, and Malayalam using Factored translation models when translating from English. Statistical MT approaches face the problem of data sparsity when translating to a morphologically rich language. It is very unlikely for a parallel corpus to contain all morphological forms of words. We propose a solution to generate these unseen morphological forms and inject them into the original training corpus. We propose a simple and effective solution based on enriching the input with various morphological forms of words. We observe that morphology injection improves the quality of translation in terms of both adequacy and fluency. We verify this with experiments on three morphologically rich languages when translating from English. From the detailed evaluations, we observed an order of magnitude improvement in translation quality.

Sarcasm understandability or the ability to understand textual sarcasm depends upon readers' language proficiency, social knowledge, mental state and attentiveness. We introduce a novel method to predict the sarcasm understandability of a reader. Presence of incongruity in textual sarcasm often elicits distinctive eye-movement behavior by human readers. By recording and analyzing the eye-gaze data, we show that eye-movement patterns vary when sarcasm is understood vis-à-vis when it is not. Motivated by our observations, we propose a system for sarcasm understandability prediction using supervised machine learning. Our system relies on readers' eye-movement parameters and a few textual features, thence, is able to predict sarcasm understandability with an F-score of 93%, which demonstrates its efficacy. The availability of inexpensive embedded-eye-trackers on mobile devices creates avenues for applying such research which benefits web-content creators, review writers and social media analysts alike.

WordNets are useful resources for natural language processing. Various WordNets for different languages have been developed by different groups. Recently, World WordNet Database Structure (WWDS) was proposed by Redkar et. al (2015) as a common platform to store these different WordNets. However, it is underutilized due to lack of programming interface. In this paper, we present WWDS APIs, which are designed to address this shortcoming. These WWDS APIs, in conjunction with WWDS, act as a wrapper that enables developers to utilize WordNets without worrying about the underlying storage structure. The APIs are developed in PHP, Java, and Python, as they are the preferred programming languages of most developers and researchers working in language technologies. These APIs can help in various applications like machine translation, word sense disambiguation, multilingual information retrieval, etc.

In this paper, we propose a novel mechanism for enriching the feature vector, for the task of sarcasm detection, with cognitive features extracted from eye-movement patterns of human readers.Sarcasm detection has been a challenging research problem, and its importance for NLP applications such as review summarization, dialog systems and sentiment analysis is well recognized.Sarcasm can often be traced to incongruity that becomes apparent as the full sentence unfolds.This presence of incongruity-implicit or explicit-affects the way readers eyes move through the text.We observe the difference in the behaviour of the eye, while reading sarcastic and non sarcastic sentences.Motivated by this observation, we augment traditional linguistic and stylistic features for sarcasm detection with the cognitive features obtained from readers eye movement data.We perform statistical classification using the enhanced feature set so obtained.The augmented cognitive features improve sarcasm detection by 3.7% (in terms of Fscore), over the performance of the best reported system.

In this paper, we propose a novel hybrid deep learning archtecture which is highly efficient for sentiment analysis in resource-poor languages. We learn sentiment embedded vectors from the Convolutional Neural Network (CNN). These are augmented to a set of optimized features selected through a multi-objective optimization (MOO) framework. The sentiment augmented optimized vector obtained at the end is used for the training of SVM for sentiment classification. We evaluate our proposed approach for coarse-grained (i.e. sentence level) as well as fine-grained (i.e. aspect level) sentiment analysis on four Hindi datasets covering varying domains. In order to show that our proposed method is generic in nature we also evaluate it on two benchmark English datasets. Evaluation shows that the results of the proposed method are consistent across all the datasets and often outperforms the state-of-art systems. To the best of our knowledge, this is the very first attempt where such a deep learning model is used for less-resourced languages such as Hindi.

In this paper, we provide a solution to multilingual sentiment classification using deep learning. Given input text in a language, we use word translation into English and then the embeddings of these English words to train a classifier. This projection into the English space plus word embeddings gives a simple and uniform framework for multilingual sentiment analysis. A novel idea is augmentation of the training data with polar words, appearing in these sentences, along with their polarities. This approach leads to a performance gain of 7-10% over traditional classifiers on many languages, irrespective of text genre, despite the scarcity of resources in most languages.

This paper is a novel study that views sarcasm detection in dialogue as a sequence labeling task, where a dialogue is made up of a sequence of utterances.We create a manuallylabeled dataset of dialogue from TV series 'Friends' annotated with sarcasm.Our goal is to predict sarcasm in each utterance, using sequential nature of a scene.We show performance gain using sequence labeling as compared to classification-based approaches.Our experiments are based on three sets of features, one is derived from information in our dataset, the other two are from past works.Two sequence labeling algorithms (SVM-HMM and SEARN) outperform three classification algorithms (SVM, Naive Bayes) for all these feature sets, with an increase in F-score of around 4%.Our observations highlight the viability of sequence labeling techniques for sarcasm detection of dialogue.

Sentiments expressed in user-generated short text and sentences are nuanced by subtleties at lexical, syntactic, semantic and pragmatic levels. To address this, we propose to augment traditional features used for sentiment analysis and sarcasm detection, with cognitive features derived from the eye-movement patterns of readers. Statistical classification using our enhanced feature set improves the performance (F-score) of polarity detection by a maximum of 3.7% and 9.3% on two datasets, over the systems that use only traditional features. We perform feature significance analysis, and experiment on a held-out dataset, showing that cognitive features indeed empower sentiment analyzers to handle complex constructs.

This paper makes a simple increment to state-of-the-art in sarcasm detection research. Existing approaches are unable to capture subtle forms of context incongruity which lies at the heart of sarcasm. We explore if prior work can be enhanced using semantic similarity/discordance between word embeddings. We augment word embedding-based features to four feature sets reported in the past. We also experiment with four types of word embeddings. We observe an improvement in sarcasm detection, irrespective of the word embedding used or the original feature set to which our features are augmented. For example, this augmentation results in an improvement in F-score of around 4\% for three out of these four feature sets, and a minor degradation in case of the fourth, when Word2Vec embeddings are used. Finally, a comparison of the four embeddings shows that Word2Vec and dependency weight-based features outperform LSA and GloVe, in terms of their benefit to sarcasm detection.

We explore the use of the orthographic syllable, a variable-length consonant-vowel sequence, as a basic unit of translation between related languages which use abugida or alphabetic scripts.We show that orthographic syllable level translation significantly outperforms models trained over other basic units (word, morpheme and character) when training over small parallel corpora.

In recent past there has been phenomenal growth in biomedical literature and health care records. Robust text mining techniques are essential in order to properly organize the documents as well as to extract relevant information. Traditional techniques for document classification focus on machine learning algorithms where learning of classifier is decided on the basis of labeled data and the features that are prominent. In this paper we focus on developing an automated technique for classifying biomedical articles containing protein-protein interaction related information against the others. Our proposed approach is based on deep neural network framework. We investigate the role of convolution neural network (CNN) and propose two model variants. We evaluate the proposed approach on the benchmark datasets of BioCreative-II Interaction Article Subtask (IAS) data sets. Effectiveness of our proposed model is evident with the significant performance gains, 2.8 % in terms of F-measure and 5 % in terms of accuracy over the traditional models.

Since the inception of the SENSEVAL evaluation exercises there has been a great deal of recent research into Word Sense Disambiguation (WSD).Over the years, various supervised, unsupervised and knowledge based WSD systems have been proposed.Beating the first sense heuristics is a challenging task for these systems.In this paper, we present our work on Most Frequent Sense (MFS) detection using Word Embeddings and BabelNet features.The semantic features from BabelNet viz., synsets, gloss, relations, etc. are used for generating sense embeddings.We compare word embedding of a word with its sense embeddings to obtain the MFS with the highest similarity.The MFS is detected for six languages viz., English, Spanish, Russian, German, French and Italian.However, this approach can be applied to any language provided that word embeddings are available for that language.

Semantic similarity and relatedness measures play an important role in natural language processing applications.In this paper, we present the IndoWordNet::Similarity tool and interface, designed for computing the semantic similarity and relatedness between two words in IndoWordNet.A java based tool and a web interface have been developed to compute this semantic similarity and relatedness.Also, Java API has been developed for this purpose.This tool, web interface and the API are made available for the research purpose.

India is a country with 22 officially recognized languages and 17 of these have WordNets, a crucial resource.Web browser based interfaces are available for these WordNets, but are not suited for mobile devices which deters people from effectively using this resource.We present our initial work on developing mobile applications and browser extensions to access WordNets for Indian Languages.Our contribution is two fold: (1) We develop mobile applications for the Android, iOS and Windows Phone OS platforms for Hindi, Marathi and Sanskrit WordNets which allow users to search for words and obtain more information along with their translations in English and other Indian languages.(2) We also develop browser extensions for English, Hindi, Marathi, and Sanskrit WordNets, for both Mozilla Firefox, and Google Chrome.We believe that such applications can be quite helpful in a classroom scenario, where students would be able to access the WordNets as dictionaries as well as lexical knowledge bases.This can help in overcoming the language barrier along with furthering language understanding.

WordNet has proved to be immensely useful for Word Sense Disambiguation, and thence Machine translation, Information Retrieval and Question Answering.It can also be used as a dictionary for educational purposes.The semantic nature of concepts in a Word-Net motivates one to try to express this meaning in a more visual way.In this paper, we describe our work of enriching IndoWordNet with image acquisitions from the OpenClipArt library.We describe an approach used to enrich WordNets for eighteen Indian languages.Our contribution is three fold: (1) We develop a system, which, given a synset in English, finds an appropriate image for the synset.The system uses the OpenclipArt library (OCAL) to retrieve images and ranks them.(2) After retrieving the images, we map the results along with the linkages between Princeton WordNet and Hindi Word-Net, to link several synsets to corresponding images.We choose and sort top three images based on our ranking heuristic per synset.(3) We develop a tool that allows a lexicographer to manually evaluate these images.The top images are shown to a lexicographer by the evaluation tool for the task of choosing the best image representation.The lexicographer also selects the number of relevant images.Using our system, we obtain an Average Precision (P @ 3) score of 0.30.

WordNet plays a significant role in Linked Open Data (LOD) cloud.It has numerous application ranging from ontology annotation to ontology mapping.IndoWord-Net is a linked WordNet connecting 18 Indian language WordNets with Hindi as a source WordNet.The Hindi WordNet was initially developed by linking it to English WordNet.In this paper, we present a data representation of IndoWordNet in Web Ontology Language (OWL).The schema of Princeton WordNet has been enhanced to support the representation of IndoWordNet.This IndoWordNet representation in OWL format is now available to link other web resources.This representation is implemented for eight Indian languages.

Samāsa or compounds are a regular feature of Indian Languages.They are also found in other languages like German, Italian, French, Russian, Spanish, etc. Compound word is constructed from two or more words to form a single word.The meaning of this word is derived from each of the individual words of the compound.To develop a system to generate, identify and interpret compounds, is an important task in Natural Language Processing.This paper introduces a web based tool -Samāsa-Kartā for producing compound words.Here, the focus is on Sanskrit language due to its richness in usage of compounds; however, this approach can be applied to any Indian language as well as other languages.IndoWordNet is used as a resource for words to be compounded.The motivation behind creating compound words is to create, to improve the vocabulary, to reduce sense ambiguity, etc. in order to enrich the WordNet.The Samāsa-Kartā can be used for various applications viz., compound categorization, sandhi creation, morphological analysis, paraphrasing, synset creation, etc.

For fine-grained sentiment analysis, we need to go beyond zero-one polarity and find a way to compare adjectives (synonyms) that share the same sense.Choice of a word from a set of synonyms, provides a way to select the exact polarityintensity.For example, choosing to describe a person as benevolent rather than kind 1 changes the intensity of the expression.In this paper, we present a sense based lexical resource, where synonyms are assigned intensity levels, viz., high, medium and low.We show that the measure P (s|w) (probability of a sense s given the word w) can derive the intensity of a word within the sense.We observe a statistically significant positive correlation between P (s|w) and intensity of synonyms for three languages, viz., English, Marathi and Hindi.The average correlation scores are 0.47 for English, 0.56 for Marathi and 0.58 for Hindi.

Detection of MultiWord Expressions (MWEs) is one of the fundamental problems in Natural Language Processing.In this paper, we focus on two categories of MWEs -Compound Nouns and Light Verb Constructions.These two categories can be tackled using knowledge bases, rather than pure statistics.We investigate usability of IndoWordNet for the detection of MWEs.Our IndoWordNet based approach uses semantic and ontological features of words that can be extracted from IndoWordNet.This approach has been tested on Indian languages viz., Assamese, Bengali, Hindi, Konkani, Marathi, Odia and Punjabi.Results show that ontological features are found to be very useful for the detection of light verb constructions, while use of semantic properties for the detection of compound nouns is found to be satisfactory.This approach can be easily adapted by other Indian languages.Detected MWEs can be interpolated into WordNets as they help in representing semantic knowledge.

This paper reports the work of creating bilingual mappings in English for certain synsets of Hindi wordnet, the need for doing this, the methods adopted and the tools created for the task.Hindi wordnet, which forms the foundation for other Indian language wordnets, has been linked to the English WordNet.To maximize linkages, an important strategy of using direct and hypernymy linkages has been followed.However, the hypernymy linkages were found to be inadequate in certain cases and posed a challenge due to sense granularity of language.Thus, the idea of creating bilingual mappings was adopted as a solution.A bilingual mapping means a linkage between a concept in two different languages, with the help of translation and/or transliteration.Such mappings retain meaningful representations, while capturing semantic similarity at the same time.This has also proven to be a great enhancement of Hindi wordnet and can be a crucial resource for multilingual applications in natural language processing, including machine translation and cross language information retrieval.

Retrievals in response to queries to search engines in resource-scarce languages often produce no results, which annoys the user. In such cases, at least partially relevant documents must be retrieved. We propose a novel multilingual framework, MultiStructPRF , which expands the query with related terms by (i) using a resource-rich assisting language and (ii) giving varied importance to the expansion terms depending on their position of occurrence in the document. Our system uses the help of an assisting language to expand the query in order to improve system recall. We propose a systematic expansion model for weighting the expansion terms coming from different parts of the document. To combine the expansion terms from query language and assisting language, we propose a heuristics-based fusion model . Our experimental results show an improvement over other PRF techniques in both precision and recall for multiple resource-scarce languages like Marathi, Bengali, Odia, Finnish, and the like. We study the effect of different assisting languages on precision and recall for multiple query languages. Our experiments reveal an interesting fact: Precision is positively correlated with the typological closeness of query language and assisting language, whereas recall is positively correlated with the resource richness of the assisting language.

Word Sense Disambiguation is a difficult problem to solve in the unsupervised setting. This is because in this setting inference becomes more dependent on the interplay between different senses in the context due to unavailability of learning resources. Using two basic ideas, sense dependency and selective dependency, we model the WSD problem as a Maximum A Posteriori (MAP) Inference Query on a Markov Random Field (MRF) built using WordNet and Link Parser or Stanford Parser. To the best of our knowledge this combination of dependency and MRF is novel, and our graph-based unsupervised WSD system beats state-of-the-art system on SensEval-2, SensEval-3 and SemEval-2007 English all-words datasets while being over 35 times faster.

WordNet is an online lexical resource which expresses unique concepts in a language. English WordNet is the first WordNet which was developed at Princeton University. Over a period of time, many language WordNets were developed by various organizations all over the world. It has always been a challenge to store the WordNet data. Some WordNets are stored using file system and some WordNets are stored using different database models. In this paper, we present the World WordNet Database Structure which can be used to efficiently store the WordNet information of all languages of the World. This design can be adapted by most language WordNets to store information such as synset data, semantic and lexical relations, ontology details, language specific features, linguistic information, etc. An attempt is made to develop Application Programming Interfaces to manipulate the data from these databases. This database structure can help in various Natural Language Processing applications like Multilingual Information Retrieval, Word Sense Disambiguation, Machine Translation, etc.

Opinion summarization is the task of producing the summary of a text, such that the summary also preserves the sentiment of the text. Opinion Summarization is thus a trade-off between summarization and sentiment analysis. The demand of compression may drop sentiment bearing sentences, and the demand of sentiment detection may bring in redundant sentences. We harness the power of submodularity to strike a balance between two conflicting requirements. We investigate an incipient class of submodular functions for the problem, and a partial enumeration based greedy algorithm that has performance guarantee of 63%. Our functions generate summaries such that there is good correlation between document sentiment and summary sentiment along with good ROUGE score, which outperforms thestate-of-the-art algorithms.

For fine-grained sentiment analysis, we need to go beyond zero-one polarity and find a way to compare adjectives that share a common semantic property. In this paper, we present a semi-supervised ap-proach to assign intensity levels to adjec-tives, viz. high, medium and low, where adjectives are compared when they belong

Sudha Bhingardive, Dhirendra Singh, Rudramurthy V, Hanumant Redkar, Pushpak Bhattacharyya. Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2015.

Raj Dabre, Fabien Cromieres, Sadao Kurohashi, Pushpak Bhattacharyya. Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2015.

We present Brahmi-Net- an online system for transliteration and script conversion for all ma-jor Indian language pairs (306 pairs). The sys-tem covers 13 Indo-Aryan languages, 4 Dra-vidian languages and English. For training the transliteration systems, we mined paral-lel transliteration corpora from parallel trans-lation corpora using an unsupervised method and trained statistical transliteration systems using the mined corpora. Languages which do not have parallel corpora are supported by transliteration through a bridge language. Our script conversion system supports con-version between all Brahmi-derived scripts as well as ITRANS romanization scheme. For this, we leverage co-ordinated Unicode ranges between Indic scripts and use an extended ITRANS encoding for transliterating between English and Indic scripts. The system also pro-vides top-k transliterations and simultaneous transliteration into multiple output languages. We provide a Python as well as REST API to access these services. The API and the mined transliteration corpus are made available for research use under an open source license. 1

We present a graph based algorithm for automatic domain segmentation of Wordnet.We pose the problem as a Markov Random Field Classification problem and show how existing graph based algorithms for Image Processing can be used to solve the problem.Our approach is unsupervised and can be easily adopted for any language.We conduct our experiments for two domains, health and tourism.We achieve F-Score more than .70 in both domains.This work can be useful for many critical problems like word sense disambiguation, domain specific ontology extraction etc.

Sense marked corpora is essential for supervised word sense disambiguation (WSD).The marked sense ids come from wordnets.However, words in corpora appear in morphed forms, while wordnets store lemma.This situation calls for accurate lemmatizers.The lemma is the gateway to the wordnet.However, the problem is that for many languages, lemmatizers do not exist, and this problem is not easy to solve, since rule based lemmatizers take time and require highly skilled linguists.Satistical stemmers on the other hand do not return legitimate lemma.

In this paper, we report our methods and results of using, for the first time, semi-automatic approach to enhance an Indian language Wordnet.We apply our methods to enhancing an already existing Sanskrit Wordnet created from Hindi Wordnet (which is created from Princeton Wordnet) using expansion approach.We base our experiment on an existing bilingual Sanskrit English Dictionary and show how lemma in this dictionary can be mapped to Princeton Wordnet through which corresponding Sanskrit synsets can be populated by Sanskrit lexemes.This our method will also show how absence of resources of a pair of languages need not be an obstacle, if another resource of one of them is available.Sanskrit being historically related to languages of Indo-European family, we believe that this semi-automatic approach will help enhance Wordnets of other Indian languages of the same family.

In this paper, we are presenting a graphical user interface to browse and explore the In-doWordnet lexical database for various Indian languages.IndoWordnet visualizer extracts the related concepts for a given word and displays a sub graph containing those concepts.The interface is enhanced with different features in order to provide flexibility to the user.IndoWordnet visualizer is made publically available.Though it was initially constructed for making the wordnet validation process easier, it is proving to be very useful in analyzing various Natural Language Processing tasks, viz.,

The task of Word Sense Disambiguation (WSD) incorporates in its definition the role of 'context'.We present our work on the development of a tool which allows for automatic acquisition and ranking of 'context clues' for WSD.These clue words are extracted from the contexts of words appearing in a large monolingual corpus.These mined collection of contextual clues form a discrimination net in the sense that for targeted WSD, navigation of the net leads to the correct sense of a word given its context.Utilizing this resource we intend to develop efficient and light weight WSD based on look up and navigation of memoryresident knowledge base, thereby avoiding heavy computation which often prevents incorporation of any serious WSD in MT and search.The need for large quantities of sense marked data too can be reduced.

In this paper, we present TwiSent, a sentiment analysis system for Twitter. Based on the topic searched, TwiSent collects tweets pertaining to it and categorizes them into the different polarity classes positive, negative and objective. However, analyzing micro-blog posts have many inherent challenges compared to the other text genres. Through TwiSent, we address the problems of 1) Spams pertaining to sentiment analysis in Twitter, 2) Structural anomalies in the text in the form of incorrect spellings, nonstandard abbreviations, slangs etc., 3) Entity specificity in the context of the topic searched and 4) Pragmatics embedded in text. The system performance is evaluated on manually annotated gold standard data and on an automatically annotated tweet set based on hashtags. It is a common practise to show the efficacy of a supervised system on an automatically annotated dataset. However, we show that such a system achieves lesser classification accurcy when tested on generic twitter dataset. We also show that our system performs much better than an existing system.

This paper describes a weakly supervised system for sentiment analysis in the movie review domain. The objective is to classify a movie review into a polarity class, positive or negative, based on those sentences bearing opinion on the movie alone, leaving out other irrelevant text. Wikipedia incorporates the world knowledge of movie-specific features in the system which is used to obtain an extractive summary of the review, consisting of the reviewer’s opinions about the specific aspects of the movie. This filters out the concepts which are irrelevant or objective with respect to the given movie. The proposed system, WikiSent, does not require any labeled data for training. It achieves a better or comparable accuracy to the existing semi-supervised and unsupervised systems in the domain, on the same dataset. We also perform a general movie review trend analysis using WikiSent.

We address the problem of mining name transliterations from comparable corpora in languages P and Q in the following resource-poor scenario:Parallel names in PQ are not available for training. Parallel names in PR and RQ are available for training.We propose a novel solution for the problem by computing a common geometric feature space for P,Q and R where name transliterations are mapped to similar vectors. We employ Canonical Correlation Analysis (CCA) to compute the common geometric feature space using only parallel names in PR and RQ and without requiring parallel names in PQ. We test our algorithm on data sets in several languages and show that it gives results comparable to the state-of-the-art transliteration mining algorithms that use parallel names in PQ for training.

In this paper, we present a novel approach to Pseudo-Relevance Feedback (PRF) called Multilingual PRF (MultiPRF). The key idea is to harness multilinguality. Given a query in a language, we take the help of another language to ameliorate the well known problems of PRF, viz. (a) The expansion terms from PRF are primarily based on co-occurrence relationships with query terms, and thus other terms which are lexically and semantically related, such as morphological variants and synonyms, are not explicitly captured, and (b) PRF is quite sensitive to the quality of the initially retrieved top k documents and is thus not robust. In MultiPRF, given a query in language L1, it is translated into language L2 and PRF is performed on a collection in language L2 and the resultant feedback model is translated from L2 back into L1. The final feedback model is obtained by combining the translated model with the original feedback model of the query in L1.

Machine transliteration is an important problem in an increasingly multilingual world, as it plays a critical role in many downstream applications, such as machine translation or crosslingual information retrieval systems. In this article, we propose compositional machine transliteration systems, where multiple transliteration components may be composed either to improve existing transliteration quality, or to enable transliteration functionality between languages even when no direct parallel names corpora exist between them. Specifically, we propose two distinct forms of composition: serial and parallel. Serial compositional system chains individual transliteration components, say, X → Y and Y → Z systems, to provide transliteration functionality, X → Z. In parallel composition evidence from multiple transliteration paths between X → Z are aggregated for improving the quality of a direct system. We demonstrate the functionality and performance benefits of the compositional methodology using a state-of-the-art machine transliteration framework in English and a set of Indian languages, namely, Hindi, Marathi, and Kannada. Finally, we underscore the utility and practicality of our compositional approach by showing that a CLIR system integrated with compositional transliteration systems performs consistently on par with, and sometimes better than, that integrated with a direct transliteration system.

We introduce a Bayesian model, BayesANIL, that is capable of estimating uncertainties associated with the labeling process. Given a labeled or partially labeled training corpus of text documents, the model estimates the joint distribution of training documents and class labels by using a generalization of the Expectation Maximization algorithm. The estimates can be used in standard classification models to reduce error rates. Since uncertainties in the labeling are taken into account, the model provides an elegant mechanism to deal with noisy labels. We provide an intuitive modification to the EM iterations by re-estimating the empirical. distribution in order to reinforce feature values in unlabeled data and to reduce the influence of noisily labeled examples. Considerable improvement in the classification accuracies of two popular classification algorithms on standard labeled data-sets with and without artificially introduced noise, as well as in the presence and absence of unlabeled data, indicates that this may be a promising method to reduce the burden of manual labeling.

We present a question answering (QA) system which learns how to detect and rank answer passages by analyzing questions and their answers (QA pairs) provided as training data. We built our system in only a few person-months using o#- the-shelf components: a part-of-speech tagger, a shallow parser, a lexical network, and a few well-known supervised learning algorithms. In contrast, many of the top TREC QA systems are large group e#orts, using customized ontologies, question classifiers, and highly tuned ranking functions. Our ease of deployment arises from using generic, trainable algorithms that exploit simple feature extractors on QA pairs. With TREC QA data, our system achieves mean reciprocal rank (MRR) that compares favorably with the best scores in recent years, and generalizes from one corpus to another. Our key technique is to recover, from the question, fragments of what might have been posed as a structured query, had a suitable schema been available. One fragment comprises selectors: tokens that are likely to appear (almost) unchanged in an answer passage. The other fragment contains question tokens which give clues about the answer type, and are expected to be replaced in the answer passage by tokens which specialize or instantiate the desired answer type. Selectors are like constants in where-clauses in relational queries, and answer types are like column names. We present new algorithms for locating selectors and answer type clues and using them in scoring passages with respect to a question.

The authors propose an orthogonal polynomial based color texture model for color texture segmentation. The proposed model is based on a complete set of difference operators which are easily configurable from a set of orthogonal polynomials. The operators are employed to represent a color image region as a linear combination of the operator's responses towards texture and noise. A simple statistical design of experiments paradigm is used for separating out the responses towards color texture from the responses towards noise. A local descriptor for color texture called cpronum is computed by encoding only the significant responses towards texture. Finally a global descriptor for color texture called cprospectrum is obtained by measuring the frequency of occurrences of the cpronums in the color image. The usage of the proposed color texture representation scheme is highlighted in unsupervised color texture segmentation.