In Psychotherapy, Early Maladaptive Schemas (EMS) are entrenched negative perceptions of self or others that perpetuate mental health challenges, contribute to treatment resistance and relapse, and obstruct therapeutic progress. Addressing EMS using appropriate psychotherapeutic support (PS) strategies helps resolve core emotional deficits, mitigate resistance, and improve client engagement. Moreover, adapting polite and empathetic communication based on clients’ emotional states fosters trust, emotional safety, and a conducive therapeutic environment, which is critical for addressing EMS and achieving positive outcomes. Motivated by these insights, we introduce MATE - a novel EMS-guided polite and empAthetic dialogue sysTem for psychothErapeutic support. MATE integrates a Large Language Model (LLM) with a Mixture of Experts-based Reinforcement Learning (MoE-RL) approach to overcome the limitations of traditional RL methods, such as large action spaces and generic responses. The LLM captures diverse semantic patterns from dialogue context. MoE-RL leverages dedicated psychotherapeutic, politeness, and empathy experts, along with a new reward function, comprising PS, politeness, empathy, contextual consistency, and diversity rewards to guide policy learning for effective response generation. Evaluations on the HOPE and PSYCON datasets demonstrate MATE’s efficacy in generating polite and empathetic psychotherapeutic responses based on clients’ EMS and emotional cues while ensuring contextual consistency and diversity.

Emotion plays a pivotal role in shaping negotiation outcomes, influencing trust, cooperation, and long-term relationships.Developing negotiation dialog systems that can recognize and respond strategically to emotions is, therefore, essential to create more effective human-centered interactions.Beyond generating emotionally appropriate responses, interpretability -understanding how a system generates a particular emotion-aware response, is critical for fostering reliability and building rapport.Driven by these aspects, in this work, we introduce PRISMA, an interpretable emotionally intelligent negotiation dialogue system targeting two application domains, viz.job interviews and resource allocation.To enable interpretability, we propose an Emotion-aware Negotiation Strategyinformed Chain-of-Thought (ENS-CoT) reasoning mechanism, which mimics human negotiation by perceiving, understanding, using, and managing emotions.Leveraging ENS-CoT, we curate two new datasets: JobNego (for job interview negotiation) and ResNego (for resource allocation negotiation).We then leverage these datasets to develop PRISMA by augmenting self-training with Direct Preference Optimization (DPO), guiding agents toward more accurate, interpretable, and emotionally appropriate negotiation responses.Automatic and human evaluation on JobNego and ResNego datasets demonstrate that PRISMA substantially enhances interpretability and generates appropriate emotion-aware responses, while improving overall negotiation effectiveness 1 .

Automatic Speech Recognition (ASR) transcripts often contain disfluencies, such as fillers, repetitions, and false starts, which reduce readability and hinder downstream applications like chatbots and voice assistants.If left unaddressed, such disfluencies can significantly degrade the reliability of downstream systems.Most existing approaches rely on classical models that focus on identifying disfluent tokens for removal.While this strategy is effective to some extent, it often disrupts grammatical structure and semantic coherence, leading to incomplete or unnatural sentences.Recent literature explored the use of large language models (LLMs); however, these efforts have primarily focused on disfluency detection or data augmentation, rather than performing comprehensive correction.We propose a multilingual correction pipeline where a sequence tagger first marks disfluent tokens, and these signals guide instruction fine-tuning of an LLM to rewrite transcripts into fluent text.To further improve reliability, we add a contrastive learning objective that penalizes the reproduction of disfluent tokens, encouraging the model to preserve grammar and meaning while removing disfluent artifacts.Our experiments across three Indian languages, namely Hindi, Bengali, and Marathi show consistent improvements over strong baselines, including multilingual sequence-to-sequence models.These results highlight that detection-only strategies are insufficient.Combining token-level cues with instruction tuning and contrastive learning provides a practical and scalable solution for multilingual disfluency correction in speechdriven NLP systems.We make the codes publicly available at https://github.com/ deepak-kumar-98/Mind-the-Pause.

Movie subtitle translation is inherently multimodal, yet text-only systems often miss visual cues needed to convey emotion, action, and social nuance, especially for low-resource Indic languages (English to Hindi, Bengali, Telugu, Tamil and Kannada). We present a case study on five full-length films and compare two lightweight visual grounding strategies: structured attribute summaries from a 5-minute sliding window and free-text summaries of inter-subtitle visual gaps. Our analysis shows that temporal misalignment between subtitles and frames is a major obstacle in long-form video, often rendering indiscriminate visual grounding ineffective. However, oracle selective grounding, which replaces only the lowest-quality 20-30

We investigate whether reasoning information can enhance machine translation when incorporated as supportive context during training and inference. Using Hindi-Bengali translation as a case study, we define five reasoning components: Key Terms, Syntactic, Semantic, Pragmatic, and Paraphrase. We conduct a complete ablation across all 31 possible combinations using Gemma-3-1B-Instruct and evaluate on multi-domain benchmark with BLEU, chrF, and TER. Evaluation results show that reasoning effectiveness depends on its type and composition rather than quantity. Combining multiple heterogeneous signals causes objective diffusion, degrading performance. The compact Semantic and Paraphrase combination proves optimal, and providing it during inference yields 23.86 BLEU compared to 22.12 from standard fine-tuning a +1.74 BLEU gain across eight domains. These findings demonstrate that targeted semantic guidance consistently and meaningfully improves the compact translation models.

Federated learning (FL) allows multiple medical institutions to collaboratively train machine learning models without sharing sensitive patient data, preserving privacy. However, as medical guidelines and disease classifications change over time, existing models can become outdated and may need updates to stay relevant. We propose a novel approach that efficiently updates federated models by selectively removing outdated knowledge without requiring full retraining. Our approach uses gradient-based Shapley value approximations to identify and modify the most important model parameters linked to obsolete diagnostic categories. This enables precise unlearning of outdated information while preserving performance on current diagnoses. We validate our method on the PathMNIST and COVID-19 Radiography datasets, showing that it can effectively eliminate specific diagnostic classes with minimal loss in accuracy for relevant conditions. Our method only requires a single communication round among clients and offers better control than previous techniques by targeting individual parameters instead of whole convolutional feature channels (in Convolutional Neural Networks (CNN)). This makes it especially useful for keeping federated medical models aligned with evolving medical knowledge.1

In medical dialogue systems, recent advancements underscore the critical role of incorporating relevant medical knowledge to enhance performance. However, existing knowledge bases often lack completeness, posing a challenge in sourcing pertinent information. We present MedProm, a novel generative model tailored for medical dialogue generation to address this gap. Motivated by the need for comprehensive and contextually relevant responses, MedProm leverages state-of-the-art language models such as BioGPT. Our model is designed to integrate extensive medical knowledge into conversations, facilitating effective communication between patients and healthcare providers. At the core of MedProm lies the MediConnect Graph, a meticulously constructed knowledge graph capturing intricate relationships among medical entities extracted from dialogue contexts. By employing a KnowFusion encoder with a pretraining objective and masked multi-head self-attention, MedProm effectively processes the MediConnect graph, enabling precise control over information flow to capture its underlying structure. Furthermore, MedProm incorporates a sophisticated Curriculum Knowledge Decoder, leveraging transformer-based decoding to generate response utterances conditioned on input representations from the KnowFusion Encoder. The training process is guided through curriculum learning, gradually increasing optimization difficulty based on a coherence-based criterion. Experimental results on two datasets demonstrate the efficacy of MedProm in generating accurate and contextually relevant responses compared to state-of-the-art models.

Abstract Multimodal classifiers function as opaque black box models. While several techniques exist to interpret their predictions, very few of them are as intuitive and accessible as natural language explanations (NLEs). To build trust, such explanations must faithfully capture the classifier’s internal decision making behavior, a property known as faithfulness. In this paper, we propose CAuSE (Causal Abstraction under Simulated Explanations), a novel framework to generate faithful NLEs for any pretrained multimodal classifier. We demonstrate that CAuSE generalizes across datasets and models through extensive empirical evaluation. Theoretically, we show that CAuSE, trained via interchange intervention, forms a causal abstraction of the underlying classifier. We further validate this through a redesigned metric for measuring causal faithfulness in multimodal settings. CAuSE surpasses other methods on this metric, with qualitative analysis reinforcing its advantages. We also perform detailed error analysis to pinpoint the failure cases of CAuSE1.

Individuals living with disabilities often face challenges in their daily lives, from managing physical tasks to coping with emotional needs. It is imperative to provide them with personalized, courteous, and empathetic support that can address their unique needs. To bridge this gap, we propose an Empathetic Disability Support System (EDiSS), designed to offer personalized support tailored with correct politeness and empathetic strategies as per individual users’ OCEAN traits, gender, and age. To train EDiSS, first, a specialized personalized disability support dialogue dataset (PDCARE) is created encompassing a wide spectrum of disabilities, such as Spinal Cord Injuries, Neurological Disorders, Orthopedic Disabilities, etc, and support areas like Physical Therapy Exercises, Pain Management, Emotional Support, etc. EDiSS employs a reinforcement learning-based dialogue model with a novel reward function. It adapts its tone and content based on the user’s persona, gender, and age to provide respectful and empathetic assistance across various aspects of daily living. Our experiments and evaluation demonstrate the effectiveness of EDiSS in improving the quality of life of individuals with disabilities, marking a significant advancement in leveraging technology to provide much-needed support and assistance in their daily challenges.

Developing intelligent negotiation dialogue systems that resolve conflicts and promote equitable, inclusive, and sustainable outcomes is at the forefront of advancing automated negotiation technology for social good. Negotiation involves balancing cooperation and competition to maximize value without causing offense. Using polite language fosters mutual understanding and creates a respectful and collaborative environment essential for successful negotiations in various domains. Considering this, in this paper, we propose a polite negotiation dialogue system, GENTEEL-NEGOTIATOR for social good applications to boost the overall quality of negotiation outcomes. We focus on developing a negotiation dialogue system for two key application areas, namely tourism and e-commerce. We begin by curating a unique negotiation dialogue dataset, NEGOCHAT for tourism. We further enrich the NEGOCHAT and Integrative Negotiation Dataset (IND) for e-commerce with various negotiation strategies. These datasets are then used to develop the GENTEEL-NEGOTIATOR, leveraging the Large Language Model (LLM) and mixture-of-expert (MoE)-based reinforcement learning approach. The proposed MoE-based method employs heuristic experts dedicated to negotiation, politeness, and dialogue coherence to facilitate the learning of diverse semantics by analyzing the dialogue context. A novel reward function with negotiation strategy congruence, politeness, dialogue coherence, and engagingness rewards is designed to guide the policy’s learning for generating responses. Automatic and human evaluations on NEGOCHAT and IND datasets validate the effectiveness of GENTEEL-NEGOTIATOR in generating polite responses during negotiation while maintaining conversation goals, including coherence and engagingness.

Soumitra Ghosh, Gopendra Vikram Singh, Shambhavi Shambhavi, Sabarna Choudhury, Asif Ekbal. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

In medical question-answering, traditional knowledge triples often fail due to superfluous data and their inability to capture complex relationships between symptoms and treatments across diseases. This limits models’ ability to provide accurate, contextually relevant responses. To overcome this, we introduce MedEx, which employs First-Order Logic (FOL)-based reasoning to model intricate relationships between diseases and treatments. We construct FOL-based triplets that encode the interplay of symptoms, diseases, and treatments, capturing not only surface-level data but also the logical constraints of the medical domain. MedEx encodes the discourse (questions and context) using a transformer-based unit, enhancing context comprehension. These encodings are processed by a Knowledge Injection Cell that integrates knowledge graph triples via a Graph Attention Network. The Logic Fusion Cell then combines medical-specific logical rule triples (e.g., co-occurrence, causation, diagnosis) with knowledge triples and extracts answers through a feed-forward layer. Our analysis demonstrates MedEx’s effectiveness and generalization across medical question-answering tasks. By merging logical reasoning with knowledge, MedEx provides precise medical answers and adapts its logical rules based on training data nuances.

The widespread adoption of Large Language Models (LLMs) and publicly available Chat-GPT have marked a significant turning point in the integration of Artificial Intelligence (AI) into people's everyday lives.This study examines the ability of Large Language Models (LLMs) to generate future research ideas from scientific papers.Unlike tasks such as summarization or translation, idea generation lacks a clearly defined reference set or structure, making manual evaluation the default standard.However, human evaluation in this setting is extremely challenging -it requires substantial domain expertise, contextual understanding of the paper, and awareness of the current research landscape.This makes it time-consuming, costly, and fundamentally non-scalable, particularly as new LLMs are being released at a rapid pace.Currently, there is no automated evaluation metric specifically designed for this task.To address this gap, we propose two automated evaluation metrics: Idea Alignment Score (IAScore) and Idea Distinctness Index.We further conducted human evaluation to assess the novelty, relevance, and feasibility of the generated future research ideas.This investigation offers insights into the evolving role of LLMs in idea generation, highlighting both its capability and limitations.Our work contributes to the ongoing efforts in evaluating and utilizing language models for generating future research ideas.We make our datasets and codes publicly available 1 ."Innovation is seeing what everybody has seen and thinking what nobody has thought" -Dr.

Memes are popular in the modern world and are distributed primarily for entertainment. However, harmful ideologies such as misogyny can be propagated through innocent-looking memes. The detection and understanding of why a meme is misogynous is a research challenge due to its multimodal nature (image and text) and its nuanced manifestations across different societal contexts. We introduce a novel multimodal approach, namely, MM-Misogyny to detect, categorize, and explain misogynistic content in memes. MM-Misogyny processes text and image modalities separately and unifies them into a multimodal context through a cross-attention mechanism. The resulting multimodal context is then easily processed for labeling, categorization, and explanation via a classifier and Large Language Model (LLM). The evaluation of the proposed model is performed on a newly curated dataset (What’s Beneath Misogynous Stereotyping (WBMS)) created by collecting misogynous memes from cyberspace and categorizing them into four categories, namely, Kitchen, Leadership, Working, and Shopping. The model not only detects and classifies misogyny, but also provides a granular understanding of how misogyny operates in operates in domains of life. The results demonstrate the superiority of our approach compared to existing methods. The code and dataset are available at https://github.com/Misogyny.

ABSTRACT This paper studies neural machine translation (NMT) of code‐mixed (CM) text. Specifically, we generate synthetic CM data and how it can be used to improve the translation performance of NMT through the data augmentation strategy. We conduct experiments on three data augmentation approaches viz. CM‐Augmentation, CM‐Concatenation, and Multi‐Encoder approaches, and the latter two approaches are inspired by document‐level NMT, where we use synthetic CM data as context to improve the performance of the NMT models. We conduct experiments on three language pairs, viz. Hindi–English, Telugu–English and Czech–English. Experimental results demonstrate that the proposed approaches significantly improve performance over the baseline model trained without data augmentation and over the existing data augmentation strategies. The CM‐Concatenation model attains the best performance.

Neural Machine Translation (NMT) has made remarkable progress using large-scale textual data, but the potential of incorporating multimodal inputs, especially visual information, remains underexplored in high-resource settings. While prior research has focused on using multimodal data in low-resource scenarios, this study examines how image features impact translation when added to a large-scale, pre-trained unimodal NMT system. Surprisingly, the study finds that images might be redundant in this context. Additionally, the research introduces synthetic noise to assess whether images help the model handle textual noise. Multimodal models slightly outperform text-only models in noisy settings, even when random images are used. The study’s experiments translate from English to Hindi, Bengali, and Malayalam, significantly outperforming state-of-the-art benchmarks. Interestingly, the effect of visual context varies with the level of source text noise: no visual context works best for non-noisy translations, cropped image features are optimal for low noise, and full image features perform better in high-noise scenarios. This sheds light on the role of visual context, especially in noisy settings, and opens up a new research direction for Noisy Neural Machine Translation in multimodal setups. The research emphasizes the importance of combining visual and textual information to improve translation across various environments. Our code is publicly available at https://github.com/babangain/indicMMT .

Social media platforms have become an open door for users to share their views, resulting in a growing trend of offensive content being shared on social media. Detecting and addressing offensive content is crucial due to its significant impact on society. Although there has been extensive research on the detection of offensive content in the English language, there is a notable gap in detecting offensive content in multimodal settings involving code-mixed languages. In this article, we propose a large scale multimodal code-mixed dataset for Hinglish (Hindi+English)MultiSEAO-Mixfocusing on women and children. TheMultiSEAO-Mixis annotated with offensiveness, sentiment, emotion, and their respective intensities. Additionally, it is also annotated with author support. A multimodal, multitask framework is proposed that considers offensive detection, intensity prediction, and author support as the primary tasks and improves their performance using sentiment, emotion, and corresponding intensities as the auxiliary tasks. Further, we propose a fusion technique that captures the enhanced multimodal representation to improve the performance of our model. Experimental results demonstrate that the proposed multitask framework improves the model performance by more than 4.5 points compared to multitask system without sentiment and emotion as the auxiliary tasks.

In the digital age, cybercrimes, particularly cyber harassment, have become pressing issues, targeting vulnerable individuals like children, teenagers, and women. Understanding the experiences and needs of the victims is crucial for effective support and intervention. Online conversations between victims and virtual harassment counselors (chatbots) offer valuable insights into cyber harassment manifestations (CHMs) and determinants (CHDs). However, the distinction between CHMs and CHDs remains unclear. This research is the first to introduce concrete definitions for CHMs and CHDs, investigating their distinction through automated methods to enable efficient cyber-harassment dialogue comprehension. We present a novel dataset, Cyber-MaD that contains Cyber harassment dialogues manually annotated with Manifestations and Determinants. Additionally, we design an Emotion-informed Contextual Dual attention Convolution Transformer (E-ConDuCT) framework to extract CHMs and CHDs from cyber harassment dialogues. The framework primarily: a) utilizes inherent emotion features through adjective-noun pairs modeled by an autoencoder, b) employs a unique Contextual Dual attention Convolution Transformer to learn contextual insights; and c) incorporates a demarcation module leveraging task-specific emotional knowledge and a discriminator loss function to differentiate manifestations and determinants. E-ConDuCT outperforms the state-of-the-art systems on the Cyber-MaD corpus, showcasing its potential in the extraction of CHMs and CHDs. Furthermore, its robustness is demonstrated on the emotion cause extraction task using the CARES_CEASE-v2.0 dataset of suicide notes, confirming its efficacy across diverse cause extraction objectives. Access the code and data at 1. https://www.iitp.ac.in/~ai-nlp-ml/resources.html#E-ConDuCT-on-Cyber-MaD, 2. https://github.com/Soumitra816/Manifestations-Determinants.

Sarcasm is a widespread linguistic phenomenon that poses a considerable challenge to explain due to its subjective nature, absence of contextual cues, and rooted personal perspectives. Even though the identification of sarcasm has been extensively studied in dialogue analysis, merely detecting sarcasm falls short of enabling conversational systems to genuinely comprehend the underlying meaning of a conversation and generate fitting responses. It is imperative to not only detect sarcasm but also pinpoint its origination and the rationale behind the sarcastic expressions to capture its authentic essence. In this paper, we delve into the discourse structure of conversations infused with sarcasm and introduce a novel task - Sarcasm Initiation and Reasoning in Conversations (SIRC). Embedded in a multimodal environment and involving a combination of both English and code-mixed interactions, the objective of the task is to discern the trigger or starting point of sarcasm. Additionally, the task involves producing a natural language explanation that rationalizes the satirical dialogues. To this end, we introduce Sarcasm Initiation and Reasoning Dataset (SIRD) to facilitate our task and provide sarcasm initiation annotations and reasoning. We develop a comprehensive model named Sarcasm Initiation and Reasoning Generation (SIRG), which is designed to encompass textual, audio, and visual representations. To achieve this, we introduce a unique shared fusion method that employs cross-attention mechanisms to seamlessly integrate these diverse modalities. Our experimental outcomes, conducted on the SIRC dataset, demonstrate that our proposed framework establishes a new benchmark for both sarcasm initiation and its reasoning generation in the context of multimodal conversations. The code and dataset can be accessed from https://www.iitp.ac.in/∼ai-nlp-ml resources.html#sarcasm-explain and https://github.com/GussailRaat/SIRG-Sarcasm-Initiation-and-Reasoning-Generation.

Science communication, in layperson’s terms, is essential to reach the general population and also maximize the impact of underlying scientific research. Hence, good science blogs and journalistic reviews of research articles are so well-read and critical to conveying science. Scientific blogging goes beyond traditional research summaries, offering experts a platform to articulate findings in layperson’s terms. It bridges the gap between intricate research and its comprehension by the general public, policymakers, and other researchers. Amid the rapid expansion of scientific data and the accelerating pace of research, credible science blogs serve as vital artifacts for evidence-based information to the general non-expert audience. However, writing a scientific blog or even a short lay summary requires significant time and effort. Here, we are intrigued what if the process of writing a scientific blog based on a given paper could be semi-automated to produce the first draft? In this paper, we introduce a novel task of Artificial Intelligence (AI)-based science blog generation from a research article. We leverage the idea that presentations and science blogs share a symbiotic relationship in their aim to clarify and elucidate complex scientific concepts. Both rely on visuals, such as figures, to aid comprehension. With this motivation, we create a new dataset of science blogs using the presentation transcript and the corresponding slides. We create a dataset containing a paper’s presentation transcript and figures annotated from nearly 3000 papers. We then propose a multimodal attention model to generate a blog text and select the most relevant figures to explain a research article in layperson’s terms, essentially a science blog. Our experimental results with respect to both automatic and human evaluation metrics show the effectiveness of our proposed approach and the usefulness of our proposed dataset.

Detecting offensive content in internet memes is challenging as it needs additional contextual knowledge. While previous works have only focused on detecting offensive memes, classifying them further into implicit and explicit categories depending on their severity is still a challenging and underexplored area. In this work, we present an end-to-end multitask model for addressing this challenge by empirically investigating two correlated tasks simultaneously: (i) offensive meme detection and (ii) explicit-implicit offensive meme detection by leveraging the two self-supervised pre-trained models. The first pre-trained model, referred to as the “knowledge encoder,” incorporates contextual knowledge of the meme. On the other hand, the second model, referred to as the “fine-grained information encoder”, is trained to understand the obscure psycho-linguistic information of the meme. Our proposed model utilizes contrastive learning to integrate these two pre-trained models, resulting in a more comprehensive understanding of the meme and its potential for offensiveness. To support our approach, we create a large-scale dataset, CM-Off-Meme, as there is no publicly available such dataset for the code-mixed Hindi-English (Hinglish) domain. Empirical evaluation, including both qualitative and quantitative analysis, on the CM-Off-Meme dataset demonstrates the effectiveness of the proposed model in terms of cross-domain generalization.

Online sexism has become a concerning issue in recent years, especially conveyed through memes.Although this alarming phenomenon has triggered many studies from computational linguistic and natural language processing points of view, less effort has been spent analyzing if those misogyny detection models are affected by an unintended bias.Such biases can lead models to incorrectly label non-misogynous memes misogynous due to specific identity terms, perpetuating harmful stereotypes and reinforcing negative attitudes.This paper presents the first and most comprehensive approach to measure and mitigate unintentional bias in the misogynous memes detection model, aiming to develop effective strategies to counter their harmful impact.Our proposed model, the Contextualized Scene Graphbased Multimodal Network (CTXSGMNet), is an integrated architecture that combines Vi-sualBERT, a CLIP-LSTM-based memory network, and an unbiased scene graph module with supervised contrastive loss, achieves state-ofthe-art performance in mitigating unintentional bias in misogynous memes.Empirical evaluation, including both qualitative and quantitative analysis, demonstrates the effectiveness of our CTXSGMNet framework on the SemEval-2022 Task 5 (MAMI task) dataset, showcasing its promising performance in terms of Equity of Odds and F1 score.Additionally, we assess the generalizability of the proposed model by evaluating their performance on a few benchmark meme datasets, providing a comprehensive understanding of our approach's efficacy across diverse datasets 1 .

In document-level neural machine translation (DocNMT), multi-encoder approaches are common in encoding context and source sentences. Recent studies (CITATION) have shown that the context encoder generates noise and makes the model robust to the choice of context. This paper further investigates this observation by explicitly modelling context encoding through multi-task learning (MTL) to make the model sensitive to the choice of context. We conduct experiments on cascade MTL architecture, which consists of one encoder and two decoders. Generation of the source from the context is considered an auxiliary task, and generation of the target from the source is the main task. We experimented with German–English language pairs on News, TED, and Europarl corpora. Evaluation results show that the proposed MTL approach performs better than concatenation-based and multi-encoder DocNMT models in low-resource settings and is sensitive to the choice of context. However, we observe that the MTL models are failing to generate the source from the context. These observations align with the previous studies, and this might suggest that the available document-level parallel corpora are not context-aware, and a robust sentence-level model can outperform the context-aware models.

The integrity of the peer-review process is vital for maintaining scientific rigor and trust within the academic community.With the steady increase in the usage of large language models (LLMs) like ChatGPT in academic writing, there is a growing concern that AI-generated texts could compromise scientific publishing, including peer-reviews.Previous works have focused on generic AI-generated text detection or have presented an approach for estimating the fraction of peer-reviews that can be AIgenerated.Our focus here is to solve a realworld problem by assisting the editor or chair in determining whether a review is written by ChatGPT or not.To address this, we introduce the Term Frequency (TF) model, which posits that AI often repeats tokens, and the Review Regeneration (RR) model, which is based on the idea that ChatGPT generates similar outputs upon re-prompting.We stress test these detectors against token attack and paraphrasing.Finally, we propose an effective defensive strategy to reduce the effect of paraphrasing on our models.Our findings suggest both our proposed methods perform better than the other AI text detectors.Our RR model is more robust, although our TF model performs better than the RR model without any attacks.We make our code, dataset, and model public 12 .

In recent years, there has been a significant rise in the phenomenon of hate against women on social media platforms, particularly through the use of misogynous memes. These memes often target women with subtle and obscure cues, making their detection a challenging task for automated systems. Recently, Large Language Models (LLMs) have shown promising results in reasoning using Chain-of-Thought (CoT) prompting to generate the intermediate reasoning chains as the rationale to facilitate multimodal tasks, but often neglect cultural diversity and key aspects like emotion and contextual knowledge hidden in the visual modalities. To address this gap, we introduce a Multimodal Multi-hop CoT (M3Hop-CoT) framework for Misogynous meme identification, combining a CLIP-based classifier and a multimodal CoT module with entity-object-relationship integration. M3Hop-CoT employs a three-step multimodal prompting principle to induce emotions, target awareness, and contextual knowledge for meme analysis. Our empirical evaluation, including both qualitative and quantitative analysis, validates the efficacy of the M3Hop-CoT framework on the SemEval-2022 Task 5 (MAMI task) dataset, highlighting its strong performance in the macro-F1 score. Furthermore, we evaluate the model’s generalizability by evaluating it on various benchmark meme datasets, offering a thorough insight into the effectiveness of our approach across different datasets. Codes are available at this link: https://github.com/Gitanjali1801/LLM_CoT

Toxic content detection plays a vital role in addressing the misuse of social media platforms to harm people or groups due to their race, gender or ethnicity.However, due to the nature of the datasets, systems develop an unintended bias due to the over-generalization of the model to the training data.This compromises the fairness of the systems, which can impact certain groups due to their race, gender, etc. Existing methods mitigate bias using data augmentation, adversarial learning, etc., which require re-training and adding extra parameters to the model.In this work, we present a robust and generalizable technique BiasWipe to mitigate unintended bias in language models.BiasWipe utilizes model interpretability using Shapley values, which achieve fairness by pruning the neuron weights responsible for unintended bias.It first identifies the neuron weights responsible for unintended bias and then achieves fairness by pruning them without loss of original performance.It does not require re-training or adding extra parameters to the model.To show the effectiveness of our proposed technique for bias unlearning, we perform extensive experiments for Toxic content detection for BERT, RoBERTa, and GPT models. 1 .

In today's dynamic world, providing inclusive and personalized support for individuals with physical disabilities is imperative.With diverse needs and preferences, tailored assistance according to user personas is crucial.In this paper, we introduce ABLE (Adaptive, Bespoke, Listen and Empathetic), a Conversational Support System for Physical Disabilities.By tracking user personas, including gender, age, and personality traits based on the OCEAN model, ABLE ensures that support interactions are uniquely tailored to each user's characteristics and preferences.Moreover, integrating politeness and empathy levels in responses enhances user satisfaction and engagement, fostering a supportive and respectful environment.The development of ABLE involves compiling a comprehensive conversational dataset enriched with user profile annotations.Leveraging reinforcement learning techniques and diverse reward mechanisms, ABLE trains a model to generate responses aligned with individual user profiles while maintaining appropriate levels of politeness and empathy.Based on rigorous empirical analysis encompassing automatic and human evaluation metrics based on personaconsistency, politeness accuracy, empathy accuracy, perplexity, and conversation coherence, the efficacy of ABLE is assessed.Our findings underscore ABLE's success in delivering tailored support to individuals grappling with physical disabilities.To the best of our knowledge, this is the very first attempt towards building a user's persona-oriented physical disability support system 1 .

Previous studies on question generation from videos have mostly focused on generating questions about common objects and attributes and hence are not entity-centric.In this work, we focus on the generation of entity-centric information-seeking questions from videos.Such a system could be useful for videobased learning, recommending "People Also Ask" questions, video-based chatbots, and factchecking.Our work addresses three key challenges: identifying question-worthy information, linking it to entities, and effectively utilizing multimodal signals.Further, to the best of our knowledge, there does not exist a large-scale dataset for this task.Most video question generation datasets are on TV shows, movies, or human activities or lack entitycentric information-seeking questions.Hence, we contribute a diverse dataset of YouTube videos, VIDEOQUESTIONS, consisting of 411 videos with 2265 manually annotated questions.We further propose a model architecture combining Transformers, rich context signals (titles, transcripts, captions, embeddings), and a combination of cross-entropy and contrastive loss function to encourage entity-centric question generation.Our best method yields BLEU, ROUGE, CIDEr, and METEOR scores of 71.3, 78.6, 7.31, and 81.9, respectively, demonstrating practical usability.We make the code and dataset publicly available 1 .How is the public ministry of Jesus?

Cognitive distortion research holds increasing significance as it sheds light on pervasive errors in thinking patterns, providing crucial insights into mental health challenges and fostering the development of targeted interventions and therapies.This paper delves into the complex domain of cognitive distortions which are prevalent distortions in cognitive processes often associated with mental health issues.Focusing on patient-doctor dialogues, we introduce a pioneering method for detecting and reasoning about cognitive distortions utilizing Large Language Models (LLMs).Operating within a multimodal context encompassing audio, video, and textual data, our approach underscores the critical importance of integrating diverse modalities for a comprehensive understanding of cognitive distortions.By leveraging multimodal information, including audio, video, and textual data, our method offers a nuanced perspective that enhances the accuracy and depth of cognitive distortion detection and reasoning in a zero-shot manner.Our proposed hierarchical framework adeptly tackles both detection and reasoning tasks, showcasing significant performance enhancements compared to current methodologies.Through comprehensive analysis, we elucidate the efficacy of our approach, offering promising insights into the diagnosis and understanding of cognitive distortions in multimodal settings.The code and dataset can be found here: https://www.iitp.ac. in/~ai-nlp-ml/resources.html#ZS-CoDR.CoD Reasoning: The patient's final words, "They are always commenting on everything that I'm doing."could be seen as an example of cognitive distortion.This distortion occurs when someone assumes others are always scrutinizing them, despite lacking evidence.The patient's belief that others are constantly monitoring and critiquing their actions is exaggerated and unsupported, demonstrating a distorted perception of external attention.D: Ok, ok.And can you hear what they are actually saying?P: Yeah, they are talking about me.

The method of translation from one language to another without human intervention is known as Machine Translation (MT). Multilingual neural machine translation (MNMT) is a technique for MT that builds a single model for multiple languages. It is preferred over other approaches, since it decreases training time and improves translation in low-resource contexts, i.e., for languages that have insufficient corpus. However, good-quality MT models are yet to be built for many scenarios such as for Indic-to-Indic Languages (IL-IL). Hence, this article is an attempt to address and develop the baseline models for low-resource languages i.e., IL-IL (for 11 Indic Languages (ILs)) in a multilingual environment. The models are built on the Samanantar corpus and analyzed on the Flores-200 corpus. All the models are evaluated using standard evaluation metrics i.e., Bilingual Evaluation Understudy (BLEU) score (with the range of 0 to 100). This article examines the effect of the grouping of related languages, namely, East Indo-Aryan (EI), Dravidian (DR), and West Indo-Aryan (WI) on the MNMT model. From the experiments, the results reveal that related language grouping is beneficial for the WI group only while it is detrimental for the EI group and it shows an inconclusive effect on the DR group. The role of pivot-based MNMT models in enhancing translation quality is also investigated in this article. Owing to the presence of large good-quality corpora from English (EN) to ILs, MNMT IL-IL models using EN as a pivot are built and examined. To achieve this, English-Indic Language (EN-IL) models are developed with and without the usage of related languages. Results show that the use of related language grouping is advantageous specifically for EN to ILs. Thus, related language groups are used for the development of pivot MNMT models. It is also observed that the usage of pivot models greatly improves MNMT baselines. Furthermore, the effect of transliteration on ILs is also analyzed in this article. To explore transliteration, the best MNMT models from the previous approaches (in most of cases pivot model using related groups) are determined and built on corpus transliterated from the corresponding scripts to a modified Indian language Transliteration script (ITRANS). The outcome of the experiments indicates that transliteration helps the models built for lexically rich languages, with the best increment of BLEU scores observed in Malayalam (ML) and Tamil (TA), i.e., 6.74 and 4.72, respectively. The BLEU score using transliteration models ranges from 7.03 to 24.29. The best model obtained is the Punjabi (PA)-Hindi (HI) language pair trained on PA-WI transliterated corpus.

In a persuasive conversation forsocial good, even the most compelling and persuasive argument may fail to persuade a persuadee resisting the persuasion. Whereas use of polite tone, apologetic expressions, or deferential modes of reference such as, ‘thank you’, ‘Please’ etc. can make the conversation more interesting, engaging, and persuading to the persuadee. We propose a resistance handling polite persuasive dialogue system (Re-Po-PDS), harnessing an efficient reward function consisting of persuasiveness, politeness, coherence, and non-repetitiveness rewards in a reinforcement learning framework and a politeness transfer model. Due to the lack of polite annotated persuasive data, we first annotate thePersuaionForGooddataset with different politeness labels and name it as PP4G dataset. Then we train two transformer-based persuasive and politeness classifiers to receive persuasive and politeness feedback for our RL-agent. Further, we train a politeness transfer model which is used at the inference time as per persuadee's resistive strategy encountered to form a more polite response. Our experimental results confirm that our proposed model increases the rate of generating polite persuasive responses as compared to the available state-of-the-art dialogue models while also making the dialogues more engaging and retaining.

Recognizing humor in meme data is a challenging task in natural language processing (NLP) and computer vision (CV) due to the complexity and variability of humor. With the explosive growth of Internet memes on social media platforms such as Facebook, Twitter, and Instagram, this task has become more important. However, there have been few studies that investigate humor recognition from memes, particularly in languages other than English. In this work, we hypothesize that humor is closely related to the valence and arousal dimensions of sentiment. We make the first attempt to release a new meme dataset for humor recognition in Hindi and propose a multitask deep learning framework to simultaneously solve three problems: humor recognition (the primary task) and valence and arousal classification (the two secondary tasks) for Internet memes. Empirical results on the Hindi meme dataset demonstrate the efficacy of our multitask learning approach over traditional pretrained models such as BERT and VGG19. The complete resources and codes will be made available for further research after acceptance of the manuscript.

Code-mixing refers to switching between two or more languages within the same utterance, which is very prevalent in multilingual societies. The amount of code-mixed content has increased due to the spike in multilingual users on review platforms. Analyzing these reviews can be beneficial for both consumers and service providers. Aspect category (AC) sentiment analysis (ACSA) provides a fine-grained analysis of reviews. ACSA identifies the AC and measures the sentiment expressed toward a given AC. The research in this direction has mostly focused on monolingual languages, which are insufficient for analyzing code-mixed reviews. To expedite research in this direction, we propose new tasks in the code-mixed language (ACSA-Mix). We develop a benchmark setup to create a code-mixed Hinglish (i.e., mixing of Hindi and English) dataset for ACSA-Mix, annotated with AC and sentiment values. To demonstrate the practical usage of the dataset, we solve ACSA-Mix tasks in the Seq2Seq framework, where natural language sentences are generated to represent the outputs that allow pretrained language models to be used effectively. Further, a multilingual multitask joint learning framework is proposed that transfers knowledge between Hinglish (ACSA-Mix) and English (ACSA) tasks. We consider ACSA-Mix tasks the primary tasks and enhance their performance by ACSA tasks (auxiliary) by sharing knowledge between them. We observe improvement over the single task ACSA-Mix models.11The dataset has been made available onhttps://www.iitp.ac.in/ai-nlp-ml/resources.htmland at Github repository:https://github.com/20118/ACSA-Mix

Sarcasm detection in unimodal or multimodal setting is a very complex task. Sarcasm, emotion, and sentiment are related to each other, and hence any multitask model could be an effective way to leverage the interdependence among these tasks. In order to better represent these clandestine associations, we avoid solely relying on traditional machine learning methods to encode the relationships between the modalities. In this article, we propose a hybrid quantum model that banks upon the low computational complexity and robust representational power of a variational quantum circuit (VQC) and the tried and tested dense neural network to tackle sentiment, emotion, and sarcasm classification simultaneously. We empirically establish that the quantum properties like superposition, entanglement, and interference will better capture and replicate not only the cross-modal interactions between text, acoustics, and visuals but also the correlations between the three responses. We consider the extended MUStARD dataset to evaluate our proposed hybrid model. The results show that our proposed hybrid quantum framework yields more promising results for the primary task of sarcasm detection with the help of the two secondary classification tasks,viz.sentiment and emotion.

Politeness is key to successful conversations. It depicts the behavior that is socially valued and is often accompanied by emotions. Previously, researchers have focused on detecting politeness in goal-oriented conversations in high-resource English language. The existing studies do not focus on identifying politeness in a resource-scared Indian languages such as Hindi, primarily due to the lack of labeled data. To overcome this limitation, in this article, we propose a novel emotion-aware contrastive learning (CL) method for zero-shot cross-lingual politeness identification (XeroPol) task in dialogues. We introduceContrastiveAligner, a CL-based alignment method for zero-shot cross-lingual transfer.ContrastiveAligneremploys translated data and pushes the model to generate similar utterance embeddings for different languages. As politeness and emotion are interrelated, hence, as the conversation progresses, the variation in emotions tends to pose challenges in identifying politeness in dialogues. Thus, in this work, we also design an auxiliary emotion-aware CL objective using sentiment information, namely theEmoSenti objective, which is expected to implicitly model the emotion change across utterances and help in the primary task of politeness identification. Experiments on MultiDoGo and EmoWOZ datasets demonstrate that the proposed approach significantly outperforms the baselines. Further analysis such as human evaluation on the EmoInHindi dataset validates the efficacy of the entire approach.

The potential for conversational agents offering mental health and legal counseling in an autonomous, interactive, and vitally accessible environment is getting highlighted due to the increased access to information through the internet and mobile devices. A counseling conversational agent should be able to offer higher engagement mimicking the real-time counseling sessions. The ability to empathize or comprehend and feel another person’s emotions and experiences is a crucial quality that promotes effective therapeutic bonding and rapport-building. Further, the use of polite encoded language in the counseling reflects the nobility and creates a familiar, warm, and comfortable atmosphere to resolve human issues. Therefore, focusing on these two aspects, we propose a Polite and Empathetic Mental Health and Legal Counseling Dialogue System (Po-Em-MHLCDS) for the victims of crimes. To build Po-Em-MHLCDS, we first create a Mental Health and Legal Counseling Dataset (MHLCD) by recruiting six employees who are asked to converse with each other, acting as a victim and the agent interchangeably following a fixed stated guidelines. Second, the MHLCD dataset is annotated with three informative labels, viz. counseling strategies, politeness, and empathy. Lastly, we train the Po-Em-MHLCDS in a reinforcement learning framework by designing an efficient and effective reward function to reinforce correct counseling strategy, politeness and empathy while maintaining contextual-coherence and non-repetitiveness in the generated responses. Our extensive automatic and human evaluation demonstrate the strength of the proposed system. Codes and Data can be accessed at https://www.iitp.ac.in/ ai-nlp-ml/resources.html#MHLCD or https://github.com/Mishrakshitij/Po-Em-MHLCDS

The World Health Organization (WHO) has significantly emphasized the need for mental health care.The social stigma associated with mental illness prevents individuals from addressing their issues and getting assistance.In such a scenario, the relevance of online counseling has increased dramatically.The feelings and attitudes that a client and a counselor express towards each other result in a higher or lower counseling experience.A counselor should be friendly and gain clients' trust to make them share their problems comfortably.Thus, it is essential for the counselor to adequately comprehend the client's emotions and ensure client's welfare, i.e. s/he should adapt and deal with the clients politely and empathetically to provide a pleasant, cordial and personalized experience.Motivated by this, in this work, we attempt to build a novel Polite and empAthetic counseLing conversational agent PAL.To have client's emotion-based polite and empathetic responses, two counseling datasets laying down the counseling support to substance addicts and crime victims are annotated.These annotated datasets are used to build PAL in a reinforcement learning framework.A novel reward function is formulated to ensure correct politeness and empathy preferences as per client's emotions with naturalness and non-repetitiveness in responses.Thorough automatic and human evaluation showcases the usefulness and strength of the designed novel reward function.Our proposed system is scalable and can be easily modified with different modules of preference models as per need 1 .

Conversational systems have shown immense growth in their ability to communicate like humans. With the emergence of large pre-trained language models (PLMs) the ability to provide informative responses have improved significantly. Despite the success of PLMs, the ability to identify and generate engaging and empathetic responses is largely dependent on labelled-data. In this work, we design a prompting approach that identifies the emotion of a given utterance and uses the emotion information for generating the appropriate responses for conversational systems. We propose a two-step prompting method that first recognises the emotion in the dialogue utterance and in the second-step uses the predicted emotion to prompt the PLM to generate the corresponding em- pathetic response in a few-shot setting. Experimental results on three publicly available datasets show that our proposed approach outperforms the state-of-the-art approaches for both automatic and manual evaluation.

Although there has been a plethora of work on open-domain conversational systems, most of these lack the mechanism of controlling the concept transitions in a dialogue.For activities like switching from casual chit-chat to taskoriented conversation, an agent with the ability to manage the flow of concepts in a conversation might be helpful.The user would find the dialogue more fascinating and engaging and be more receptive to such transitions if these concept transitions were made while taking into account the user's persona.Focusing on personaaware concept transitions, we propose a Reinforced Persona-aware Topic-guiding Conversational System (RPTCS).Due to the lack of a persona-aware topic transition dataset, we propose a novel conversation dataset creation mechanism in which the conversational agent leads the discourse to drift to a set of target concepts depending on the persona of the speaker and the context of the conversation.To avoid scarcely available expensive human resources, the entire data-creation process is mostly automatic with human-in-loop only for quality checks.This created conversational dataset named PTCD is used to develop the RPTCS in two steps.First, a maximum likelihood estimation loss-based dialogue model is trained on PTCD.The trained model is then fine-tuned in a Reinforcement Learning (RL) framework by employing novel reward functions to assure persona, topic, and context consistency with non-repetitiveness in generated responses.Our experimental results demonstrate the strength of the proposed system with respect to strong baselines 1 .

To this date, the efficacy of the scientific publishing enterprise fundamentally rests on the strength of the peer review process.The journal editor or the conference chair primarily relies on the expert reviewers' assessment, identify points of agreement and disagreement and try to reach a consensus to make a fair and informed decision on whether to accept or reject a paper.However, with the escalating number of submissions requiring review, especially in toptier Artificial Intelligence (AI) conferences, the editor/chair, among many other works, invests a significant, sometimes stressful effort to mitigate reviewer disagreements.Here in this work, we introduce a novel task of automatically identifying contradictions among reviewers on a given article.To this end, we introduce Con-traSciView, a comprehensive review-pair contradiction dataset on around 8.5k papers (with around 28k review pairs containing nearly 50k review pair comments) from the open reviewbased ICLR and NeurIPS conferences.We further propose a baseline model that detects contradictory statements from the review pairs.To the best of our knowledge, we make the first attempt to identify disagreements among peer reviewers automatically.We make our dataset and code public for further investigations 1 .

With the growing popularity of code-mixed data, there is an increasing need for better handling of this type of data, which poses a number of challenges, such as dealing with spelling variations, multiple languages, different scripts, and a lack of resources.Current language models face difficulty in effectively handling code-mixed data as they primarily focus on the semantic representation of words and ignore the auditory phonetic features.This leads to difficulties in handling spelling variations in code-mixed text.In this paper, we propose an effective approach for creating language models for handling code-mixed textual data using auditory information of words from SOUNDEX.Our approach includes a pre-training step based on masked-language-modelling, which includes SOUNDEX representations (SAMLM) and a new method of providing input data to the pretrained model.Through experimentation on various code-mixed datasets (of different languages) for sentiment, offensive and aggression classification tasks, we establish that our novel language modeling approach (SAMLM) results in improved robustness towards adversarial attacks on code-mixed classification tasks.Additionally, our SAMLM based approach also results in better classification results over the popular baselines for code-mixed tasks.We use the explainability technique, SHAP (SHapley Additive exPlanations) to explain how the auditory features incorporated through SAMLM assist the model to handle the code-mixed text effectively and increase robustness against adversarial attacks 1 .

The shortage of therapists for mental health patients emphasizes the importance of globally accessible dialogue systems alleviating their issues.To have effective interpersonal psychotherapy, these systems must exhibit politeness and empathy when needed.However, these factors may vary as per the user's gender, age, persona, and sentiment.Hence, in order to establish trust and provide a personalized cordial experience, it is essential that generated responses should be tailored to individual profiles and attributes.Focusing on this objective, we propose e-THERAPIST, a novel polite interpersonal psychotherapy dialogue system to address issues like depression, anxiety, schizophrenia, etc.We begin by curating a unique conversational dataset for psychotherapy, called PSYCON.It is annotated at two levels: (i) dialogue-level -including user's profile information (gender, age, persona) and therapist's psychotherapeutic approach; and (ii) utterance-level -encompassing user's sentiment and therapist's politeness, and interpersonal behaviour.Then, we devise a novel reward model to adapt correct polite interpersonal behaviour and use it to train e-THERAPIST on PSYCON employing NLPO loss.Our extensive empirical analysis validates the effectiveness of each component of the proposed e-THERAPIST demonstrating its potential impact in psychotherapy settings 1 .

Due to its growing impact on public opinion, hate speech on social media has garnered increased attention.While automated methods for identifying hate speech have been presented in the past, they have mostly been limited to analyzing textual content.The interpretability of such models has received very little attention, despite the social and legal consequences of erroneous predictions.In this work, we present a novel problem of Distress Identification and Cause Extraction (DICE) from multimodal online posts.We develop a multi-task deep framework for the simultaneous detection of distress content and identify connected causal phrases from the text using emotional information.The emotional information is incorporated into the training process using a zero-shot strategy, and a novel mechanism is devised to fuse the features from the multimodal inputs.Furthermore, we introduce the first-ofits-kind Distress and Cause annotated Multimodal (DCaM) dataset of 20,764 social media posts.We thoroughly evaluate our proposed method by comparing it to several existing benchmarks.Empirical assessment and comprehensive qualitative analysis demonstrate that our proposed method works well on distress detection and cause extraction tasks, improving F1 and ROS scores by 1.95% and 3%, respectively, relative to the best-performing baseline.The code and the dataset can be accessed from the following link: https://www.iitp. ac.in/~ai-nlp-ml/resources.html#DICE.

The World Health Organization has underlined the significance of expediting the preventive measures for crime against women and children to attain the United Nations Sustainable Development Goals 2030 (promoting well-being, gender equality, and equal access to justice). The crime victims typically need mental health and legal counselling support for their ultimate well-being and sometimes they need to be persuaded to seek desired support. Further, counselling interactions should adopt correct politeness and empathy strategies so that a warm, amicable, and respectful environment can be built to better understand the victims’ situations. To this end, we propose PARTNER, a Politeness and empAthy strategies-adaptive peRsuasive dialogue sysTem for meNtal health and LEgal counselling of cRime victims. For this, first, we create a novel mental HEalth and legAl counseLling conversational dataset HEAL, annotated with three distinct aspects, viz. counselling act, politeness strategy, and empathy strategy. Then, by formulating a novel reward function, we train a counselling dialogue system in a reinforcement learning setting to ensure correct counselling act, politeness strategy, and empathy strategy in the generated responses. Extensive empirical analysis and experimental results show that the proposed reward function ensures persuasive counselling responses with correct polite and empathetic tone in the generated responses. Further, PARTNER proves its efficacy to engage the victim by generating diverse and natural responses.

Gender bias is a pervasive issue that impacts women's and marginalized groups' ability to fully participate in social, economic, and political spheres. This study introduces a novel problem of Gender-biased Language Identification and Extraction (GLIdE) from social media interactions and develops a multi-task deep framework that detects gender-biased content and identifies connected causal phrases from the text using emotional information that is present in the input. The method uses a zero-shot strategy with emotional information and a mechanism to represent gender-stereotyped information as a knowledge graph. In this work, we also introduce the first-of-its-kind Gender-biased Analysis Corpus (GAC) of 12,432 social media posts and improve the best-performing baseline for gender-biased language identification and extraction tasks by margins of 4.88% and 5 ROS points, demonstrating this through empirical evaluation and extensive qualitative analysis. By improving the accuracy of identifying and analyzing gender-biased language, this work can contribute to achieving gender equality and promoting inclusive societies, in line with the United Nations Sustainable Development Goals (UN SDGs) and the Leave No One Behind principle (LNOB). We adhere to the principles of transparency and collaboration in line with the UN SDGs by openly sharing our code and dataset.

Gitanjali Kumari, Pranali Shinde, Asif Ekbal. Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.

Sign language translation (SLT) is usually seen as a two-step process of continuous sign language recognition (CSLR) and gloss-to-text translation. We propose a novel, Transformer-based architecture to jointly perform CSLR and sign-translation in an end-to-end fashion. We extend the ordinary Transformer decoder with two channels to support multitasking, where each channel is devoted to solving a particular problem. To control the memory footprint of our model, channels are designed to share most of their parameters with each other. However, each channel still has a dedicated set of parameters that is fine-tuned with respect to the channel's task. In order to evaluate the proposed architecture, we focus on translating German signs into English sequences and use the RWTH-PHOENIX-Weather 2014 T corpus in our experiments. Evaluation results along with detailed quantitative and qualitative analyses indicate that the mixture of information provided by the multitask decoder was successful and enabled us to achieve superior performance in comparison to other SLT models.

Recent studies have shown that the multi-encoder models are agnostic to the choice of context and the context encoder generates noise which helps in the improvement of the models in terms of BLEU score. In this paper, we further explore this idea by evaluating with context-aware pronoun translation test set by training multi-encoder models trained on three different context settings viz, previous two sentences, random two sentences, and a mix of both as context. Specifically, we evaluate the models on the ContraPro test set to study how different contexts affect pronoun translation accuracy. The results show that the model can perform well on the ContraPro test set even when the context is random. We also analyze the source representations to study whether the context encoder is generating noise or not. Our analysis shows that the context encoder is providing sufficient information to learn discourse-level information. Additionally, we observe that mixing the selected context (the previous two sentences in this case) and the random context is generally better than the other settings.

Abstract Moderators often face a double challenge regarding reducing offensive and harmful content in social media. Despite the need to prevent the free circulation of such content, strict censorship on social media cannot be implemented due to a tricky dilemma – preserving free speech on the Internet while limiting them and how not to overreact. Existing systems do not essentially exploit the correlatedness of hate-offensive content and aggressive posts; instead, they attend to the tasks individually. As a result, the need for cost-effective, sophisticated multi-task systems to effectively detect aggressive and offensive content on social media is highly critical in recent times. This work presents a novel multifaceted transformer-based framework to identify aggressive and hate posts on social media. Through an end-to-end transformer-based multi-task network, our proposed approach addresses the following array of tasks: (a) aggression identification, (b) misogynistic aggression identification, (c) identifying hate-offensive and non-hate-offensive content, (d) identifying hate, profane, and offensive posts, (e) type of offense. We further investigate the role of emotion in improving the system’s overall performance by learning the task of emotion detection jointly with the other tasks. We evaluate our approach on two popular benchmark datasets of aggression and hate speech, covering four languages, and compare the system performance with various state-of-the-art methods. Results indicate that our multi-task system performs significantly well for all the tasks across multiple languages, outperforming several benchmark methods. Moreover, the secondary task of emotion detection substantially improves the system performance for all the tasks, indicating strong correlatedness among the tasks of aggression, hate, and emotion, thus opening avenues for future research.

Social chatbots have gained immense popularity, and their appeal lies in their capacity to respond to diverse requests, but also in their ability to develop an emotional connection with users. To develop and promote social chatbots, we need to concentrate on increasing user interaction and consider both the intellectual and emotional quotient in conversational agents. In this work, we propose the task of empathetic, personalized dialogue generation giving the machine the capability to respond emotionally and in accordance with the persona of the user. We design a generative adversarial framework, named EP-GAN (Empathy and Persona aware Generative Adversarial Network) to generate responses that are sensitive to the emotion of the user and corresponds to the persona. The persona information is encoded as memory vectors that, along with the dialogue history, are fed to the decoder for generation. An interactive adversarial learning framework is implemented to verify whether the generated responses elicit the emotion and persona in dialogues. Experimental results show that the EP-GAN framework significantly outperforms the existing baselines for both automatic and manual evaluation. We achieve an improvement of 1 point BLEU-4 and 2 points in the emotion accuracy metric compared to the best performing baseline.

In the natural language processing community, open-domain conversational agents, also known as chatbots, are gaining popularity. One of the difficulties is getting them to communicate in an emotionally intelligent manner. To generate dialogues, current neural response generation methods depend solely on end-to-end learning from large scale conversation data. Therefore, we introduce a large-scale multi Emotion and Intent guided Multimodal Dialogue (EmoInt-MD) dataset labelled with 32 emotions and 15 empathetic intents having 32 k dialogues taken from different movie genres. We propose a novel multi-task multimodal contextual Transformer framework for simultaneously identifying the emotions and intents in a given utterance utilizing audio and visual features in addition to the textual information. Experimental analysis proves that the proposed framework outperforms several unimodal and multimodal baselines on theEmoInt-MDdataset. This dataset along with our baseline and proposed framework implementations will be made publicly available for research purposes.

Systems meant for tackling hate speech have been increasing in demand with the rapid growth of social media platforms. One way of controlling hate speech in texts is to transform the text into its non-hate version while preserving the rest of the contents. Without the use of parallel data, unsupervised back-translation-based text style transfer is a common method of tackling such problems. In this article, we propose a zero-shot style-transfer technique that does effective unsupervised hate to non-hate conversion without using any hate domain text for training. While decoding the outputs produced by the system, we define an additional step of introducing lexical constraints, for better preservation of contents. Detailed empirical evaluation shows that the zero-shot method outperforms classical unsupervised style-transfer methods while at the same time reducing the data required while training.

Politeness enhances interactions by improving relations between the participants. If there is a display of rudeness, even the finest conversation can fall through. In addition, if lathered with kindness, even the most angst-prone circumstance can be expressed with far less suffering. Previously, researchers have focused upon including politeness in conversations. But the existing research does not focus on variations in politeness according to the user profile. Therefore, in this article, we propose a novel task of generating polite personalized dialog responses in accordance with the user profile and consistent with the conversational history. We design a novel Polite Personalized Dialog Generation (PoPe-DG) framework that employs a reinforced deliberation network. We create human-annotated politeness templates according to user profiles to induce politeness variation in the generated responses for the proposed task. Precisely, the personality profile is transformed and normalized into a vector using the fusion attention combined with dialogue utterances to build context. Furthermore, the context modules and the annotated templates are appended to initialize the deliberation decoder. Experimental analysis validates that our proposed approach inculcates politeness in responses in accordance with the user profile and the conversational history.

Thanks to the digital age, online speech and information may now be disseminated anonymously without regard for repercussions. Regulators face a unique problem with social media platforms because of the speed and volume of material and the lack of editorial supervision. The existing datasets on hate speech or offensive language identification lack diversity in the dataset’s content. In this article, we create a multi-domain hate speech corpus (MHC) of English tweets that includes hate speech against religion, nationality, ethnicity, and gender in general and cover diverse domains, such as current affairs, politics, terrorism, technology, natural disasters, and human/drugs trafficking. Each instance in our dataset is manually annotated as hate or non-hate. We use the existing state-of-the-art models and present a stacked-ensemble-based hate speech classifier (SEHC) to identify hate speech from Twitter data. Our results indicate that the proposed method may serve as a strong baseline for future studies using this dataset. (The dataset is available athttps://www.iitp.ac.in/~ai-nlp-ml/resources.html#MHC.)

Politeness is an expression in language that eases the conversation toward a positive undertone. If there is a display of rudeness, even the finest communication can fall through. In addition, if lathered with politeness, even the most angst-prone scenario can be expressed with far less hurt. In this article, we address the task of identifying politeness in goal-oriented dialog systems. In this regard, we create politeness-annotated conversational data (PACD) utilizing Microsoft Dialogue Challenge and DSTC1 datasets. For correctly identifying the politeness, we employ a hierarchical transformer network that effectively captures the contextual information (i.e., previous utterances) and current input for predicting the politeness in a given utterance of a dialog. Empirical results demonstrate that our proposed approach outperforms all the defined baselines. Furthermore, through in- and cross-domain experiments, we show the necessity of a PACD to mitigate acts such as rude requests or insults for both socially interactive and task-oriented dialog systems.

Emojis or emoticons are not just a modern trend but have become an essential part of our day-to-day interactions. Predicting a suitable emoji for a given tweet is a challenging task because a wrong emoji prediction for a tweet can change the meaning of the message or can amplify the emotion of the message. This task is particularly challenging since it requires selecting an appropriate emoji from a huge list of prospective emojis that may or may not be equivalent to one another. Humans use multiple emojis to convey their emotions, thereby making the task a multilabel classification problem. In this article, we propose a multilabel emoji prediction system that predicts the appropriate emoji for a given tweet by using different state-of-the-art baselines. Due to the unavailability of a multi-emoji dataset, we create a large-scale multilabel emoji dataset named Mu-Emoji that comprises of more than 0.6 million tweets having varied emojis belonging to both positive and negative sentiments. For our proposed task, we employ graph attention network along with bidirectional encoder representations from transformer encoder for the accurate prediction of emojis. Qualitative and quantitative analyses show that our multilabel emoji prediction baselines perform well compared with the single-emoji prediction baselines for our proposed Mu-Emoji dataset. Our proposed framework also outperforms all the baselines for both single and multilabel emoji prediction tasks.

In this paper, we hypothesize that humor is closely related to sentiment and emotions. Also, due to the tremendous growth in multilingual content, there is a great demand for building models and systems that support multilingual information access. To end this, we first extend the recently released Multimodal Multiparty Hindi Humor (M2H2) dataset by adding parallel English utterances corresponding to Hindi utterances and then annotating each utterance with sentiment and emotion classes. We name it Sentiment, Humor, and Emotion aware Multilingual Multimodal Multiparty Dataset (SHEMuD). Therefore, we propose a multitask framework wherein the primary task is humor detection, and the auxiliary tasks are sentiment and emotion identification. We design a multitasking framework wherein we first propose a Context Transformer to capture the deep contextual relationships with the input utterances. We then propose a Sentiment and Emotion aware Embedding (SE-Embedding) to get the overall representation of a particular emotion and sentiment w.r.t. the specific humor situation. Experimental results on the SHEMuD show the efficacy of our approach and shows that multitask learning offers an improvement over the single-task framework for both monolingual (4.86 points in Hindi and 5.9 points in English in F1-score) and multilingual (5.17 points in F1-score) setting.

The interaction between a consumer and the customer service representative greatly contributes to the overall customer experience. Therefore, to ensure customers’ comfort and retention, it is important that customer service agents and chatbots connect with users on social, cordial, and empathetic planes. In the current work, we automatically identify the sentiment of the user and transform the neutral responses into polite responses conforming to the sentiment and the conversational history. Our technique is basically a reinforced multi-task network- the primary task being ‘polite response generation’ and the secondary task being ‘sentiment analysis’- that uses a Transformer based encoder-decoder. We use sentiment annotated conversations from Twitter as the training data. The detailed evaluation shows that our proposed approach attains superior performance compared to the baseline models.

The World Health Organization has emphasised the need of stepping up suicide prevention efforts to meet the United Nation’s Sustainable Development Goal target of 2030 (Goal 3: Good health and well-being). We address the challenging task of personality subtyping from suicide notes. Most research on personality subtyping has relied on statistical analysis and feature engineering. Moreover, state-of-the-art transformer models in the automated personality subtyping problem have received relatively less attention. We develop a novel EMotion-assisted PERSONAlity Detection Framework (EM-PERSONA). We annotate the benchmark CEASE-v2.0 suicide notes dataset with personality traits across four dichotomies: Introversion (I)-Extraversion (E), Intuition (N)-Sensing (S), Thinking (T)-Feeling (F), Judging (J)–Perceiving (P). Our proposed method outperforms all baselines on comprehensive evaluation using multiple state-of-the-art systems. Across the four dichotomies, EM-PERSONA improved accuracy by 2.04%, 3.69%, 4.52%, and 3.42%, respectively, over the highest-performing single-task systems.

Mental health is a critical component of the United Nations’ Sustainable Development Goals (SDGs), particularly Goal 3, which aims to provide “good health and well-being”. The present mental health treatment gap is exacerbated by stigma, lack of human resources, and lack of research capability for implementation and policy reform. We present and discuss a novel task of detecting emotional reasoning (ER) and accompanying emotions in conversations. In particular, we create a first-of-its-kind multimodal mental health conversational corpus that is manually annotated at the utterance level with emotional reasoning and related emotion. We develop a multimodal multitask framework with a novel multimodal feature fusion technique and a contextuality learning module to handle the two tasks. Leveraging multimodal sources of information, commonsense reasoning, and through a multitask framework, our proposed model produces strong results. We achieve performance gains of 6% accuracy and 4.62% F1 on the emotion detection task and 3.56% accuracy and 3.31% F1 on the ER detection task, when compared to the existing state-of-the-art model.

Persuasive conversations for a social cause often require influencing other person’s attitude or intention that may fail even with compelling arguments. The use of emotions and different types of polite tones as needed with facts may enhance the persuasiveness of a message. To incorporate these two aspects, we propose a polite, empathetic persuasive dialogue system (PEPDS). First, in a Reinforcement Learning setting, a Maximum Likelihood Estimation loss based model is fine-tuned by designing an efficient reward function consisting of five different sub rewards viz. Persuasion, Emotion, Politeness-Strategy Consistency, Dialogue-Coherence and Non-repetitiveness. Then, to generate empathetic utterances for non-empathetic ones, an Empathetic transfer model is built upon the RL fine-tuned model. Due to the unavailability of an appropriate dataset, by utilizing the PERSUASIONFORGOOD dataset, we create two datasets, viz. EPP4G and ETP4G. EPP4G is used to train three transformer-based classification models as per persuasiveness, emotion and politeness strategy to achieve respective reward feedbacks. The ETP4G dataset is used to train an empathetic transfer model. Our experimental results demonstrate that PEPDS increases the rate of persuasive responses with emotion and politeness acknowledgement compared to the current state-of-the-art dialogue models, while also enhancing the dialogue’s engagement and maintaining the linguistic quality.

Availability of the user reviews in vernacular languages is helpful for the users to get information regarding the products. Since most of the e-commerce websites allow the reviews in English language only, it is important to provide the translated versions of the reviews to the non-English speaking users. Translation of the user reviews from English to vernacular languages is a challenging task, predominantly due to the lack of sufficient in-domain datasets. In this paper, we present a pre-training based efficient technique which is used to adapt and improve the single multilingual neural machine translation (NMT) model for the low-resource language pairs. The pre-trained model contains a special synthetic cross-lingual decoder. The decoder for the pre-training is trained over the cross-lingual target samples where the phrases are replaced with their translated counterparts. After pre-training, the model is adapted to multiple samples of the low-resource language pairs using incremental learning that does not require full training from the very scratch. We perform the experiments over eight low-resource and three high resource language pairs from the generic domain, and two language pairs from the product review domains. Through our synthetic multilingual decoder based pre-training, we achieve improvements of upto 4.35 BLEU points compared to the baseline and 2.13 BLEU points compared to the previous code-switched pre-trained models. The review domain outputs from the proposed model are evaluated in real time by human evaluators in the e-commerce company Flipkart.

The development of conversational agents to interact with patients and deliver clinical advice has attracted the interest of many researchers, particularly in light of the COVID-19 pandemic.The training of an end-to-end neural based dialog system, on the other hand, is hampered by a lack of multi-turn medical dialog corpus.We make the very first attempt to release a highquality multi-turn Medical Dialog dataset relating to Covid-19 disease named CDialog, with over 1K conversations collected from the online medical counselling websites.We annotate each utterance of the conversation with seven different categories of medical entities, including diseases, symptoms, medical tests, medical history, remedies, medications and other aspects as additional labels.Finally, we propose a novel neural medical dialog system based on the CDialog dataset to advance future research on developing automated medical dialog systems.We use pre-trained language models for dialogue generation, incorporating annotated medical entities, to generate a virtual doctor's response that addresses the patient's query.Experimental results show that the proposed dialog models perform comparably better when supplemented with entity information and hence can improve the response quality.

The World Health Organization (WHO) has emphasized the importance of significantly accelerating suicide prevention efforts to fulfill the United Nations' Sustainable Development Goal (SDG) objective of 2030. In this paper, we present an end-to-end multitask system to address a novel task of detection of two interpersonal risk factors of suicide, Perceived Burdensomeness (PB) and Thwarted Belongingness (TB) from suicide notes. We also introduce a manually translated code-mixed suicide notes corpus, CoMCEASE-v2.0, based on the benchmark CEASE-v2.0 dataset, annotated with temporal orientation, PB and TB labels. We exploit the temporal orientation and emotion information in the suicide notes to boost overall performance. For comprehensive evaluation of our proposed method, we compare it to several state-of-the-art approaches on the existing CEASE-v2.0 dataset and the newly announced CoMCEASE-v2.0 dataset. Empirical evaluation suggests that temporal and emotional information can substantially improve the detection of PB and TB.

In the commercial aviation domain, there are a large number of documents, like accident reports of NTSB and ASRS, and regulatory directives ADs. There is a need for a system to efficiently access these diverse repositories to serve the demands of the aviation industry, such as maintenance, compliance, and safety. In this paper, we propose a Knowledge Graph (KG) guided Deep Learning (DL) based Question Answering (QA) system to cater to these requirements. We construct a KG from aircraft accident reports and contribute this resource to the community of researchers. The efficacy of this resource is tested and proved by the proposed QA system. Questions in Natural Language are converted into SPARQL (the interface language of the RDF graph database) queries and are answered from the KG. On the DL side, we examine two different QA models, BERT-QA and GPT3-QA, covering the two paradigms of answer formulation in QA. We evaluate our system on a set of handcrafted queries curated from the accident reports. Our hybrid KG + DL QA system, KGQA + BERT-QA, achieves 7% and 40.3% increase in accuracy over KGQA and BERT-QA systems respectively. Similarly, the other combined system, KGQA + GPT3-QA, achieves 29.3% and 9.3% increase in accuracy over KGQA and GPT3-QA systems respectively. Thus, we infer that the combination of KG and DL is better than either KG or DL individually for QA, at least in our chosen domain.

Natural Language Inference (NLI), also known as Recognizing Textual Entailment (RTE), has been one of the central tasks in Artificial Intelligence (AI) and Natural Language Processing (NLP). RTE between the two pieces of texts is a crucial problem, and it adds further challenges when involving two different languages, i.e., in the cross-lingual scenario. This paper proposes an effective transfer learning approach for cross-lingual NLI. We perform experiments on English-Hindi language pairs in the cross-lingual setting to find out that our novel loss formulation could enhance the performance of the baseline model by up to 2%. To assess the effectiveness of our method further, we perform additional experiments on every possible language pair using four European languages, namely French, German, Bulgarian, and Turkish, on top of XNLI dataset. Evaluation results yield up to 10% performance improvement over the respective baseline models, in some cases surpassing the state-of-the-art (SOTA). It is also to be noted that our proposed model has 110M parameters which is much lesser than the SOTA model having 220M parameters. Finally, we argue that our transfer learning-based loss objective is model agnostic and thus can be used with other deep learning-based architectures for cross-lingual NLI.

Social media platforms such as Twitter have evolved into a vast information sharing platform, allowing people from a variety of backgrounds and expertise to share their opinions on numerous events such as terrorism, narcotics and many other social issues. People sometimes misuse the power of social media for their agendas, such as illegal trades and negatively influencing others. Because of this, sentiment analysis has won the interest of a lot of researchers to widely analyze public opinion for social media monitoring. Several benchmark datasets for sentiment analysis across a range of domains have been made available, especially for high-resource languages. A few datasets are available for low-resource Indian languages like Hindi, such as movie reviews and product reviews, which do not address the current need for social media monitoring. In this paper, we address the challenges of sentiment analysis in Hindi and socially relevant domains by introducing a balanced corpus annotated with the sentiment classes, viz. positive, negative and neutral. To show the effective usage of the dataset, we build several deep learning based models and establish them as the baselines for further research in this direction.

The long-standing goal of Artificial Intelligence (AI) has been to create human-like conversational systems. Such systems should have the ability to develop an emotional connection with the users, consequently, emotion recognition in dialogues has gained popularity. Emotion detection in dialogues is a challenging task because humans usually convey multiple emotions with varying degrees of intensities in a single utterance. Moreover, emotion in an utterance of a dialogue may be dependent on previous utterances making the task more complex. Recently, emotion recognition in low-resource languages like Hindi has been in great demand. However, most of the existing datasets for multi-label emotion and intensity detection in conversations are in English. To this end, we propose a large conversational dataset in Hindi named EmoInHindi for multi-label emotion and intensity recognition in conversations containing 1,814 dialogues with a total of 44,247 utterances. We prepare our dataset in a Wizard-of-Oz manner for mental health and legal counselling of crime victims. Each utterance of dialogue is annotated with one or more emotion categories from 16 emotion labels including neutral and their corresponding intensity. We further propose strong contextual baselines that can detect the emotion(s) and corresponding emotional intensity of an utterance given the conversational context.

Grounding dialogue on external knowledge and interpreting linguistic patterns in dialogue history context, such as ellipsis, anaphora, and co-references is critical for dialogue comprehension and generation.In this paper, we present a novel open-domain dialogue generation model which effectively utilizes the large-scale commonsense and named entity based knowledge in addition to the unstructured topic-specific knowledge associated with each utterance.We enhance the commonsense knowledge with named entity-aware structures using co-references.Our proposed model utilizes a multi-hop attention layer to preserve the most accurate and critical parts of the dialogue history and the associated knowledge.In addition, we employ a Commonsense and Named Entity Enhanced Attention Module, which starts with the extracted triples from various sources and gradually finds the relevant supporting set of triples using multi-hop attention with the query vector obtained from the interactive dialogue-knowledge module.Empirical results on two benchmark dataset demonstrate that our model significantly outperforms the state-of-the-art methods in terms of both automatic evaluation metrics and human judgment.Our code is publicly available at https://github.com/deekshaVarshney/CNTF;https://www.iitp.ac.in/-ai-nlp-ml/resources/ codes/CNTF.zip.

The effective fusion of multiple modalities (i.e., text, acoustic, and visual) is a non-trivial task, as these modalities often carry specific and diverse information and do not contribute equally. The fusion of different modalities could even be more challenging under the low-resource setting, where we have fewer samples for training. This paper proposes a multi-representative fusion mechanism that generates diverse fusions with multiple modalities and then chooses the best fusion among them. To achieve this, we first apply convolution filters on multimodal inputs to generate different and diverse representations of modalities. We then fuse pairwise modalities with multiple representations to get the multiple fusions. Finally, we propose an attention mechanism that only selects the most appropriate fusion, which eventually helps resolve the noise problem by ignoring the noisy fusions. We evaluate our proposed approach on three low-resource multimodal sentiment analysis datasets, i.e., YouTube, MOUD, and ICT-MMMO. Experimental results show the effectiveness of our proposed approach with the accuracies of 59.3%, 83.0%, and 84.1% for the YouTube, MOUD, and ICT-MMMO datasets, respectively.

Abstract The quest for new information is an inborn human trait and has always been quintessential for human survival and progress. Novelty drives curiosity, which in turn drives innovation. In Natural Language Processing (NLP), Novelty Detection refers to finding text that has some new information to offer with respect to whatever is earlier seen or known. With the exponential growth of information all across the Web, there is an accompanying menace of redundancy. A considerable portion of the Web contents are duplicates, and we need efficient mechanisms to retain new information and filter out redundant information. However, detecting redundancy at the semantic level and identifying novel text is not straightforward because the text may have less lexical overlap yet convey the same information. On top of that, non-novel/redundant information in a document may have assimilated from multiple source documents, not just one. The problem surmounts when the subject of the discourse is documents, and numerous prior documents need to be processed to ascertain the novelty/non-novelty of the current one in concern. In this work, we build upon our earlier investigations for document-level novelty detection and present a comprehensive account of our efforts toward the problem. We explore the role of pre-trained Textual Entailment (TE) models to deal with multiple source contexts and present the outcome of our current investigations. We argue that a multipremise entailment task is one close approximation toward identifying semantic-level non-novelty. Our recent approach either performs comparably or achieves significant improvement over the latest reported results on several datasets and across several related tasks (paraphrasing, plagiarism, rewrite). We critically analyze our performance with respect to the existing state of the art and show the superiority and promise of our approach for future investigations. We also present our enhanced dataset TAP-DLND 2.0 and several baselines to the community for further research on document-level novelty detection.

We propose a multi-task ensemble framework that jointly learns multiple related problems. The ensemble model aims to leverage the learned representations of three deep learning models (i.e., CNN, LSTM and GRU) and a hand-crafted feature representation for the predictions. Through multi-task framework, we address four problems of emotion and sentiment analysis, i.e., “emotionclassification&intensity”, “valence,arousal&dominancefor emotion”, “valence&arousalfor sentiment”, and “3-class categorical&5-class ordinal classificationfor sentiment”. The underlying problems cover two granularity (i.e.,coarse-grainedandfine-grained) and a diverse range of domains (i.e.,tweets,Facebook posts,news headlines,blogs,lettersetc.). Experimental results suggest that the proposed multi-task framework outperforms the single-task frameworks in all experiments.

An essential skill for effective communication is the ability to express specific sentiment and emotion in a conversation. Any robust dialogue system should handle the combined effect of both sentiment and emotion while generating responses. This is expected to provide a better experience and concurrently increase users’ satisfaction. Previously, research on either emotion or sentiment controlled dialogue generation has shown great promise in developing the next generation conversational agents, but the simultaneous effect of both is still unexplored. The existing dialogue systems are majorly based on unimodal sources, predominantly the text, and thereby cannot utilize the information present in the other sources, such as video, audio, image, etc. In this article, we present at first a large scale benchmark Sentiment Emotion aware Multimodal Dialogue (SEMD) dataset for the task of sentiment and emotion controlled dialogue generation. The SEMD dataset consists of 55k conversations from 10 TV shows having text, audio, and video information. To utilize multimodal information, we propose multimodal attention based conditional variational autoencoder (M-CVAE) that outperforms several baselines. Quantitative and qualitative analyses show that multimodality, along with contextual information, plays an essential role in generating coherent and diverse responses for any given emotion and sentiment.

Temporal distance (TD) is a type of psychological distance which shows how an individual construes past and future. It is not explored with empirical research as to how an individual’sfocuson temporal distance (near-past, far-past, near-future, and far-future) can be measured from human-written text and further used for studying human tendencies. Traditionally, focus on a Temporal Distance is studied by self-report measurements. In this article, we present a study on human focus on a temporal distance from their Twitter posts (English tweets). We first identify the tweet-level temporal focus by deep neural classifiers which make use of linguistic knowledge for classification. The model classifies each tweet into one ofnear-past,far-past,near-futureorfar-future. Classified tweets are then grouped by users to obtain the user-level temporal focus. Finally, we correlate the user’s focus on temporal distance (near-past, far-past, near-future, and far-future) with his/her demographic (age, gender, education, and relationship status) and psychological attributes (intelligence, optimism, joy, sadness, disgust, anger, surprise, and fear). Our empirical analysis reveals that users’near-pastfocus is more positively correlated to their age. We also observe that users’near-futurefocus is correlated to joy while users’ focus onfar-pastis associated with negative emotions like sadness, disgust, anger, and fear.

Fake news classification is one of the most interesting problems that has attracted huge attention to the researchers of artificial intelligence, natural language processing, and machine learning (ML). Most of the current works on fake news detection are in the English language, and hence this has limited its widespread usability, especially outside the English literate population. Although there has been a growth in multilingual web content, fake news classification in low-resource languages is still a challenge due to the non-availability of an annotated corpus and tools. This article proposes an effective neural model based on the multilingual Bidirectional Encoder Representations from Transformer (BERT) for domain-agnostic multilingual fake news classification. Large varieties of experiments, including language-specific and domain-specific settings, are conducted. The proposed model achieves high accuracy in domain-specific and domain-agnostic experiments, and it also outperforms the current state-of-the-art models. We perform experiments on zero-shot settings to assess the effectiveness of language-agnostic feature transfer across different languages, showing encouraging results. Cross-domain transfer experiments are also performed to assess language-independent feature transfer of the model. We also offer a multilingual multidomain fake news detection dataset of five languages and seven different domains that could be useful for the research and development in resource-scarce scenarios.

Deep learning has become most prominent in solving various Natural Language Processing (NLP) tasks including sentiment analysis. However, these techniques require a considerably large amount of annotated corpus, which is not easy to obtain for most of the languages, especially under the scenario of low-resource settings. In this article, we propose a deep multi-task multi-lingual adversarial framework to solve the resource-scarcity problem of sentiment analysis by leveraging the useful and relevant knowledge from a high-resource language. To transfer the knowledge between the different languages, both the languages are mapped to the shared semantic space using cross-lingual word embeddings. We evaluate our proposed architecture on a low-resource language, Hindi, using English as the high-resource language. Experiments show that our proposed model achieves an accuracy of 60.09% for the movie review dataset and 72.14% for the product review dataset. The effectiveness of our proposed approach is demonstrated with significant performance gains over the state-of-the-art systems and translation-based baselines.

MOTIVATION: To minimize the accelerating amount of time invested on the biomedical literature search, numerous approaches for automated knowledge extraction have been proposed. Relation extraction is one such task where semantic relations between the entities are identified from the free text. In the biomedical domain, extraction of regulatory pathways, metabolic processes, adverse drug reaction or disease models necessitates knowledge from the individual relations, for example, physical or regulatory interactions between genes, proteins, drugs, chemical, disease or phenotype. RESULTS: In this paper, we study the relation extraction task from three major biomedical and clinical tasks, namely drug-drug interaction, protein-protein interaction, and medical concept relation extraction. Towards this, we model the relation extraction problem in a multi-task learning (MTL)framework, and introduce for the first time the concept of structured self-attentive network complemented with the adversarial learning approach for the prediction of relationships from the biomedical and clinical text. The fundamental notion of MTL is to simultaneously learn multiple problems together by utilizing the concepts of the shared representation. Additionally, we also generate the highly efficient single task model which exploits the shortest dependency path embedding learned over the attentive gated recurrent unit to compare our proposed MTL models. The framework we propose significantly improves over all the baselines (deep learning techniques)and single-task models for predicting the relationships, without compromising on the performance of all the tasks.

People suffering from stress and various mental health problems find it easier to express and share their feelings on online platforms, such as Twitter. However, the imposed character limit (280 characters) by Twitter and infrequent online activities of a section of users poses a serious setback in using computational methods for mental health analysis or emotion research. Twitter provides rich metadata information about its users (such as user’s description, geolocation, and profile image URL), which can provide valuable information regarding the mental state of the users. We hypothesize that Twitter’s rich metadata information about their users can provide some valuable depression cues, which may help in an early low-profile evaluation. In this article, we investigate this hypothesis by developing an end-to-end multimodal multitask (MT) system for depression detection (primary task) and emotion recognition (auxiliary task), where the variation of emotion information based on different user descriptions assists the learning of the primary task. The proposed system attains 70% accuracy on the depression detection task outperforming several single-task (ST) baselines built on the various combination of input features. Our findings indicate that Twitters’s rich metadata information can be leveraged to detect depression among users with significant confidence.

The focus on conversational systems has recently shifted towards creating engaging agents by inculcating emotions into them. Human emotions are highly complex as humans can express multiple emotions with varying intensity in a single utterance, whereas the conversational agents convey only one emotion in their responses. To infuse human-like behaviour in the agents, we introduce the task of multi-emotion controllable response generation with the ability to express different emotions with varying levels of intensity in an open-domain dialogue system. We introduce a Multiple Emotion Intensity aware Multi-party Dialogue (MEIMD) dataset having 34k conversations taken from 8 different TV Series. We finally propose a Multiple Emotion with Intensity-based Dialogue Generation (MEI-DG) framework. The system employs two novel mechanisms: viz. (i) determining the trade-off between the emotion and generic words, while focusing on the intensity of the desired emotions; and (ii) computing the amount of emotion left to be expressed, thereby regulating the generation accordingly. The detailed evaluation shows that our proposed approach attains superior performance compared to the baseline models.

A recent topic of research in natural language generation has been the development of automatic response generation modules that can automatically respond to a user's utterance in an empathetic manner.Previous research has tackled this task using neural generative methods by augmenting emotion classes with the input sequences.However, the outputs by these models may be inconsistent.We employ multitask learning to predict the emotion label and to generate a viable response for a given utterance using a common encoder with multiple decoders.Our proposed encoder-decoder model consists of a self-attention based encoder and a decoder with dot product attention mechanism to generate response with a specified emotion.We use the focal loss to handle imbalanced data distribution, and utilize the consistency loss to allow coherent decoding by the decoders.Human evaluation reveals that our model produces more emotionally pertinent responses.In addition, our model outperforms multiple strong baselines on automatic evaluation measures such as F1 and BLEU scores, thus resulting in more fluent and adequate responses.

Humor recognition in conversations is a challenging task that has recently gained popularity due to its importance in dialogue understanding, including in multimodal settings (i.e., text, acoustics, and visual). The few existing datasets for humor are mostly in English. However, due to the tremendous growth in multilingual content, there is a great demand to build models and systems that support multilingual information access. To this end, we propose a dataset for Multimodal Multiparty Hindi Humor (M2H2) recognition in conversations containing 6,191 utterances from 13 episodes of a very popular TV series ”Shrimaan Shrimati Phir Se”. Each utterance is annotated with humor/non-humor labels and encompasses acoustic, visual, and textual modalities. We propose several strong multimodal baselines and show the importance of contextual and multimodal information for humor recognition in conversations. The empirical results on M2H2 dataset demonstrate that multimodal information complements unimodal information for humor recognition. The dataset and the baselines are available at http://www.iitp.ac.in/~ai-nlp-ml/resources.html and https://github.com/declare-lab/M2H2-dataset.

In this paper, we propose an unsupervised approach for knowledge graph (KG) creation from conversational data. We make use of intent classification and slot-filling, the two important components of any dialogue agent, exploit their interconnectedness, and finally construct a KG. We build a supervised intent classifier to extract the intent classes, and then on top of this we run our occlusion based slot-information extraction algorithm. Our algorithm is able to make use of supervised training of intent classifiers for extracting the relevant slot-information in an unsupervised way. To test the effectiveness of our system, we perform both automatic and manual evaluation of our intent-classifier and slot-filling system on three dialog datasets. Finally, we construct a knowledge graph from the dialogue conversation using an algorithm that makes use of our occlusion based slot-information extraction module. Empirical evaluation shows that our occlusion based method is able to successfully extract slot information from conversations, resulting in a high-quality KG.

Emotion analysis from texts has emerged as an important area of research in the field of Natural Language Processing (NLP) in the past few years. Several benchmark datasets have been released for this task, however most of the datasets are open domain in nature and for resource-rich language like English. These datasets may not always be sufficient for capturing domain specific emotions, and may tend to skew towards an emotion based on the domain. In this paper, we provide a framework for multilingual emotion detection to deal with the crisis situations. We collect and annotate disaster domain social media tweets and news data in two languages, namely English and Hindi. We derive 6 emotions from Plutchik's wheel of emotions suitable for disaster domain, and annotate the data using these disaster specific emotions. In total we create four emotionally enriched datasets i.e. 2 tweets datasets (English and Hindi) and 2 news dataset (English and Hindi). We also establish strong baselines on the dataset using two popular deep learning algorithms, stacked Bi-LSTM+CNN and BERT. Evaluation shows that the best model achieves the averaged accuracy of 68% across the four different datasets.

Dialogue system has become a prominent platform for human-machine interactions. The ongoing research in vision and language has opened new frontiers for building multimodal dialogue systems that incorporate information from the various complementary sources such as text, images, audios, and videos. For interactive systems, multimodal knowledge in the form of different modalities need to be presented to the user for effective communication. Recent research has mainly focused upon textual generation in a multimodal setting, thereby providing incomplete information to the users. Hence, in our current work, we present an attribute centered image generation framework in a multimodal system that is capable of generating images, conditioned upon the textual features with the help of the taxonomy-attribute combined tree. The visual features interact with multi-head attended textual features through attention based factorized bilinear pooling approach for fine-grained representation. Further, the multimodal representation is extended to generate images using the adversarial network. The loss functions encourage the network to generate images that are very similar to the natural image. We perform our experiments on the Multimodal Dialog Dataset (MMD) to create contextualized images using the fashion attribute features. Empirical studies show that the generated attribute centered images help in making the dialogue more engaging to the users.

Multimodality in dialogues has become crucial for a thorough understanding of the intent of the user to provide better responses to fulfill user's demands. Existing dialogue systems suffer from the issue of inconsistency and dull responses. Many goal-oriented conversational systems lack the different aspect information of the products or services to present an informative and exciting response. Aspects such as the price, color, pattern, rating are essential for deciding whether to purchase/order. To alleviate the issues in the existing systems, we propose the novel task of multi-aspect guided dialogue generation. This task is introduced to focus on making the responses focused and consistent with the different aspects mentioned in the current dialogue. In our present work, we design a hierarchical transformer network to capture the dialogue context for generating the responses. For creating the responses with multiple aspects, we explicitly give the aspect vectors at the time of decoding for generation. The information of the aspects specified to the decoder controls the overall generation process. We evaluate our proposed hierarchical framework on the newly created multi-domain multi-modal dialogue (MDMMD) dataset consisting of both text and images. Experimental results show that the proposed system outperforms all the existing and baseline approaches. The aspect controlled responses are immensely consistent with the ongoing dialogue and highly diverse resolving the issues of the current systems.

Multilingual chat systems are the need of the hour for organizations who render online conversational services to their customers. This can effectively be facilitated using the robust machine translation (MT) systems. Translating user-generated contents is regarded as one of the challenging tasks for MT. As for translating conversations, it is more challenging since the meaning of any particular utterance in a conversation usually depends on its context. Moreover, chats in conversational systems are usually informal, contain code-mixed (mix of more than one language), and other grammatical inconsistencies. In this paper, we use state-of-the-art Transformer models to build our MT systems and to translate conversations between the customer service agents and customers. We propose a novel method which effectively selects contextual information from the source text of conversation to be translated. We also employ a terminology-based pseudo in-domain corpus mining strategy for fine-tuning our translation model. We evaluate our methods on the German-English WMT20 Shared Task on Chat Translation dataset, and obtain 61.1 and 63.9 BLEU points on evaluation test sets for English-to-German and German-to-English, respectively, surpassing the present state-of-the-art MT systems by 0.7 BLEU and 1.5 BLEU points, respectively. In this paper, we use state-of-the-art Transformer models to build our MT systems and to translate conversations between the customer service agents and customers. We propose a novel method which effectively selects contextual information from the source text of conversation to be translated. We also employ a terminology-based pseudo in-domain corpus mining strategy for fine-tuning our translation model. We evaluate our methods on the German-English WMT20 Shared Task on Chat Translation dataset, and obtain 61.1 and 63.9 BLEU points on evaluation test sets for English-to-German and German-to-English, respectively, surpassing the present state-of-the-art MT systems by 0.7 BLEU and 1.5 BLEU points, respectively.

Emotion Recognition in Conversation (ERC) is becoming increasingly popular due to the accessibility of an enormous measure of openly accessible conversational information. Moreover, it has potential applications in opinion mining, social media and the health care domain. In this paper, we propose a novel Context and Knowledge Enriched Transformer Framework (CKETF) in which we interpret the contextual information from the utterances using a pre-trained Bidirectional Encoder Representations from Transformers (BERT) model and leverage additive attention based hierarchical transformer for encoding the knowledge sentences. Experiments on the knowledge-grounded Topical Chat dataset shows that both context and external knowledge are important for conversational emotion recognition. We demonstrate through extensive experiments and analysis that our proposed model significantly outperforms the current state-of-the-art methods.

Naive dialogue generation systems do not have the ability to generate distinguishable utterances while replying to different speakers, as they do not take into consideration whom the speaker is. To overcome this situation, we present an end-to-end deep learning model that relies on a person-specific embedding (persona) to generate adequate responses in multi-party conversations. In particular, the persona contains information about how a person behaves in the conversational multi-party setting. Empirical results on the Multi-Domain Wizard-of-Oz (MultiWoz) data set show the efficacy of our approach over the existing state-of-the-art systems, and show that our approach efficiently generates person-specific utterances.

The recent explosion in false information on social media has led to intensive research on automatic fake news detection models and fact-checkers. Fake news and misinformation, due to its peculiarity and rapid dissemination, have posed many interesting challenges to the Natural Language Processing (NLP) and Machine Learning (ML) community. Admissible literature shows that novel information includes the element of surprise, which is the principal characteristic for the amplification and virality of misinformation. Novel and emotional information attracts immediate attention in the reader. Emotion is the presentation of a certain feeling or sentiment. Sentiment helps an individual to convey his emotion through expression and hence the two are co-related. Thus, Novelty of the news item and thereafter detecting the Emotional state and Sentiment of the reader appear to be three key ingredients, tightly coupled with misinformation. In this paper we propose a deep multitask learning model that jointly performs novelty detection, emotion recognition, sentiment prediction, and misinformation detection. Our proposed model achieves the state-of-the-art(SOTA) performance for fake news detection on three benchmark datasets, viz. ByteDance, Fake News Challenge(FNC), and Covid-Stance with 11.55%, 1.58%, and 21.76% improvement in accuracy, respectively. The proposed approach also shows the efficacy over the single-task framework with an accuracy gain of 11.53, 28.62, and 14.31 percentage points for the above three datasets. The source code is available at https://github.com/Nish-19/Multitask-Fake-News-NES.

Verifying fact of multimodal (text, image and/or videos) reports is emerging as an important challenge to prevent circulation of fake news reports in various online platform. Moreover, such reports often being posted in local languages penetrate even faster, making the problem more complex. To tackle such problem many fact verifying web sites have emerged, which deploys humans to manually find the truthfulness of such reports with the help of multimodal contents that are being used in such report. But in recent times, due to diversity of the problem owing to various political, and malicious motives, the existing (manual) system fails to handle the enormity. Existing content based approaches rely only on textual content of such reports and fail to handle the multimodal nature. On the other hand, existing multimodal based approaches rely only on the report based and user based features, and fail to interpret the fake contents with evidences. In this work, we propose a novel end-to-end automated multimodal content based multilingual fact verification system, which automates the task of fact verifying web sites, and provides evidences for every judgment. Evaluation results on three benchmark datasets show the robustness and effectiveness of our approach.

Social chatbots have gained immense popularity, and their appeal lies not just in their capacity to respond to the diverse requests from users, but also in the ability to develop an emotional connection with users.To further develop and promote social chatbots, we need to concentrate on increasing user interaction and take into account both the intellectual and emotional quotient in the conversational agents.Therefore, in this work, we propose the task of sentiment aware emotion controlled personalized dialogue generation giving the machine the capability to respond emotionally and in accordance with the persona of the user.As sentiment and emotions are highly co-related, we use the sentiment knowledge of the previous utterance to generate the correct emotional response in accordance with the user persona.We design a Transformer based Dialogue Generation framework, that generates responses that are sensitive to the emotion of the user and corresponds to the persona and sentiment as well.Moreover, the persona information is encoded by a different Transformer encoder, along with the dialogue history, is fed to the decoder for generating responses.We annotate the PersonaChat dataset with sentiment information to improve the response quality.Experimental results on the PersonaChat dataset show that the proposed framework significantly outperforms the existing baselines, thereby generating personalized emotional responses in accordance with the sentiment that provides better emotional connection and user satisfaction as desired in a social chatbot.

Interactive-predictive translation is a collaborative iterative process and where human translators produce translations with the help of machine translation (MT) systems interactively. Various sampling techniques in active learning (AL) exist to update the neural MT (NMT) model in the interactive-predictive scenario. In this paper and we explore term based (named entity count (NEC)) and quality based (quality estimation (QE) and sentence similarity (Sim)) sampling techniques – which are used to find the ideal candidates from the incoming data – for human supervision and MT model’s weight updation. We carried out experiments with three language pairs and viz. German-English and Spanish-English and Hindi-English. Our proposed sampling technique yields 1.82 and 0.77 and 0.81 BLEU points improvements for German-English and Spanish-English and Hindi-English and respectively and over random sampling based baseline. It also improves the present state-of-the-art by 0.35 and 0.12 BLEU points for German-English and Spanish-English and respectively. Human editing effort in terms of number-of-words-changed also improves by 5 and 4 points for German-English and Spanish-English and respectively and compared to the state-of-the-art.

Product reviews provide valuable feedback of the customers and however and they are available today only in English on most of the e-commerce platforms. The nature of reviews provided by customers in any multilingual country poses unique challenges for machine translation such as code-mixing and ungrammatical sentences and presence of colloquial terms and lack of e-commerce parallel corpus etc. Given that 44% of Indian population speaks and operates in Hindi language and we address the above challenges by presenting an English–to–Hindi neural machine translation (NMT) system to translate the product reviews available on e-commerce websites by creating an in-domain parallel corpora and handling various types of noise in reviews via two data augmentation techniques and viz. (i). a novel phrase augmentation technique (PhrRep) where the syntactic noun phrases in sentences are replaced by the other noun phrases carrying different meanings but in similar context; and (ii). a novel attention guided noise augmentation (AttnNoise) technique to make our NMT model robust towards various noise. Evaluation shows that using the proposed augmentation techniques we achieve a 6.67 BLEU score improvement over the baseline model. In order to show that our proposed approach is not language-specific and we also perform experiments for two other language pairs and viz. En-Fr (MTNT18 corpus) and En-De (IWSLT17) that yield the improvements of 2.55 and 0.91 BLEU points and respectively and over the baselines.

Machine Translation (MT) systems often fail to preserve different stylistic and pragmatic properties of the source text (e.g. sentiment and emotion and gender traits and etc.) to the target and especially in a low-resource scenario. Such loss can affect the performance of any downstream Natural Language Processing (NLP) task and such as sentiment analysis and that heavily relies on the output of the MT systems. The susceptibility to sentiment polarity loss becomes even more severe when an MT system is employed for translating a source content that lacks a legitimate language structure (e.g. review text). Therefore and we must find ways to minimize the undesirable effects of sentiment loss in translation without compromising with the adequacy. In our current work and we present a deep re-inforcement learning (RL) framework in conjunction with the curriculum learning (as per difficulties of the reward) to fine-tune the parameters of a pre-trained neural MT system so that the generated translation successfully encodes the underlying sentiment of the source without compromising the adequacy unlike previous methods. We evaluate our proposed method on the English–Hindi (product domain) and French–English (restaurant domain) review datasets and and found that our method brings a significant improvement over several baselines in the machine translation and and sentiment classification tasks.

Text style transfer is an important issue for conversational agents as it may adapt utterance production to specific dialogue situations. It consists in introducing a given style within a sentence while preserving its semantics. Within this scope, different strategies have been proposed that either rely on parallel data or take advantage of non-supervised techniques. In this paper, we follow the latter approach and show that the sequential introduction of different loss functions into the learning process can boost the performance of a standard model. We also evidence that combining different style classifiers that either focus on global or local textual information improves sentence generation. Experiments on the Yelp dataset show that our methodology strongly competes with the current state-of-the-art models across style accuracy, grammatical correctness, and content preservation.

Abstract Detecting, whether a document contains sufficient new information to be deemed as novel , is of immense significance in this age of data duplication. Existing techniques for document-level novelty detection mostly perform at the lexical level and are unable to address the semantic-level redundancy. These techniques usually rely on handcrafted features extracted from the documents in a rule-based or traditional feature-based machine learning setup. Here, we present an effective approach based on neural attention mechanism to detect document-level novelty without any manual feature engineering. We contend that the simple alignment of texts between the source and target document(s) could identify the state of novelty of a target document. Our deep neural architecture elicits inference knowledge from a large-scale natural language inference dataset, which proves crucial to the novelty detection task. Our approach is effective and outperforms the standard baselines and recent work on document-level novelty detection by a margin of $\sim$ 3% in terms of accuracy.

Abstract Neural machine translation (NMT) has recently shown promising results on publicly available benchmark datasets and is being rapidly adopted in various production systems. However, it requires high-quality large-scale parallel corpus, and it is not always possible to have sufficiently large corpus as it requires time, money, and professionals. Hence, many existing large-scale parallel corpus are limited to the specific languages and domains. In this paper, we propose an effective approach to improve an NMT system in low-resource scenario without using any additional data. Our approach aims at augmenting the original training data by means of parallel phrases extracted from the original training data itself using a statistical machine translation (SMT) system. Our proposed approach is based on the gated recurrent unit (GRU) and transformer networks. We choose the Hindi–English, Hindi–Bengali datasets for Health, Tourism, and Judicial (only for Hindi–English) domains. We train our NMT models for 10 translation directions, each using only 5–23k parallel sentences. Experiments show the improvements in the range of 1.38–15.36 BiLingual Evaluation Understudy points over the baseline systems. Experiments show that transformer models perform better than GRU models in low-resource scenarios. In addition to that, we also find that our proposed method outperforms SMT—which is known to work better than the neural models in low-resource scenarios—for some translation directions. In order to further show the effectiveness of our proposed model, we also employ our approach to another interesting NMT task, for example, old-to-modern English translation, using a tiny parallel corpus of only 2.7K sentences. For this task, we use publicly available old-modern English text which is approximately 1000 years old. Evaluation for this task shows significant improvement over the baseline NMT.

The preservation of domain knowledge from source to the target is crucial in any translation workflows. Hence, translation service providers that use machine translation (MT) in production could reasonably expect that the translation process should transfer both the underlying pragmatics and the semantics of the source-side sentences into the target language. However, recent studies suggest that the MT systems often fail to preserve such crucial information (e.g., sentiment, emotion, gender traits) embedded in the source text in the target. In this context, the raw automatic translations are often directly fed to other natural language processing (NLP) applications (e.g., sentiment classifier) in a cross-lingual platform. Hence, the loss of such crucial information during the translation could negatively affect the performance of such downstream NLP tasks that heavily rely on the output of the MT systems. In our current research, we carefully balance both the sides (i.e., sentiment and semantics) during translation, by controlling a global-attention-based neural MT (NMT), to generate translations that encode the underlying sentiment of a source sentence while preserving its non-opinionated semantic content. Toward this, we use a state-of-the-art reinforcement learning method, namely, actor-critic , that includes a novel reward combination module, to fine-tune the NMT system so that it learns to generate translations that are best suited for a downstream task, viz. sentiment classification while ensuring the source-side semantics is intact in the process. Experimental results for Hindi–English language pair show that our proposed method significantly improves the performance of the sentiment classifier and alongside results in an improved NMT system.

Multimodality in dialogue systems has opened up new frontiers for the creation of robust conversational agents. Any multimodal system aims at bridging the gap between language and vision by leveraging diverse and often complementary information from image, audio, and video, as well as text. For every task-oriented dialog system, different aspects of the product or service are crucial for satisfying the user’s demands. Based upon the aspect, the user decides upon selecting the product or service. The ability to generate responses with the specified aspects in a goal-oriented dialogue setup facilitates user satisfaction by fulfilling the user’s goals. Therefore, in our current work, we propose the task of aspect controlled response generation in a multimodal task-oriented dialog system. We employ a multimodal hierarchical memory network for generating responses that utilize information from both text and images. As there was no readily available data for building such multimodal systems, we create a Multi-Domain Multi-Modal Dialog (MDMMD++) dataset. The dataset comprises the conversations having both text and images belonging to the four different domains, such as hotels, restaurants, electronics, and furniture. Quantitative and qualitative analysis on the newly created MDMMD++ dataset shows that the proposed methodology outperforms the baseline models for the proposed task of aspect controlled response generation.

In this paper, we hypothesize that sarcasm is closely related to sentiment and emotion, and thereby propose a multi-task deep learning framework to solve all these three problems simultaneously in a multi-modal conversational scenario. We, at first, manually annotate the recently released multi-modal MUStARD sarcasm dataset with sentiment and emotion classes, both implicit and explicit. For multi-tasking, we propose two attention mechanisms, viz. Inter-segment Inter-modal Attention (Ie-Attention) and Intra-segment Inter-modal Attention (Ia-Attention). The main motivation of Ie-Attention is to learn the relationship between the different segments of the sentence across the modalities. In contrast, Ia-Attention focuses within the same segment of the sentence across the modalities. Finally, representations from both the attentions are concatenated and shared across the five classes (i.e., sarcasm, implicit sentiment, explicit sentiment, implicit emotion, explicit emotion) for multi-tasking. Experimental results on the extended version of the MUStARD dataset show the efficacy of our proposed approach for sarcasm detection over the existing state-of-the-art systems. The evaluation also shows that the proposed multi-task framework yields better performance for the primary task, i.e., sarcasm detection, with the help of two secondary tasks, emotion and sentiment analysis.

Emotion and sentiment classification in dialogues is a challenging task that has gained popularity in recent times.Humans tend to have multiple emotions with varying intensities while expressing their thoughts and feelings.Emotions in an utterance of dialogue can either be independent or dependent on the previous utterances, making the task complex and interesting.Multi-label emotion detection in conversations is a significant task that provides the ability to the system to understand the various emotions of the users interacting.On the other hand, sentiment analysis in dialogue or conversation helps in understanding the perspective of the user with respect to the ongoing conversation.Besides text, additional information in the form of audio and video assists in identifying the correct emotions with the appropriate intensity and sentiments in an utterance of a dialogue.Lately, quite a few datasets have been made available for emotion and sentiment classification in dialogues.Still, these datasets are imbalanced in representing different emotions and consist of only a single emotion.Hence, we present at first a large-scale balanced Multimodal Multi-label Emotion, Intensity, and Sentiment Dialogue dataset (MEISD) collected from different TV series that has textual, audio, and visual features, and then establish a baseline setup for further research.

Question generation (QG) attempts to solve the inverse of question answering (QA) problem by generating a natural language question given a document and an answer.While sequence to sequence neural models surpass rule-based systems for QG, they are limited in their capacity to focus on more than one supporting fact.For QG, we often require multiple supporting facts to generate high-quality questions.Inspired by recent works on multi-hop reasoning in QA, we take up Multi-hop question generation, which aims at generating relevant questions based on supporting facts in the context.We employ multitask learning with the auxiliary task of answeraware supporting fact prediction to guide the question generator.In addition, we also proposed a question-aware reward function in a Reinforcement Learning (RL) framework to maximize the utilization of the supporting facts.We demonstrate the effectiveness of our approach through experiments on the multi-hop question answering dataset, HotPotQA.Empirical evaluation shows our model to outperform the single-hop neural question generation models on both automatic evaluation metrics such as BLEU, METEOR, and ROUGE, and human evaluation metrics for quality and coverage of the generated questions.

In interactive machine translation (MT), human translators correct errors in automatic translations in collaboration with the MT systems, which is seen as an effective way to improve the productivity gain in translation. In this study, we model source-language syntactic constituency parse and target-language syntactic descriptions in the form of supertags as conditional context for interactive prediction in neural MT (NMT). We found that the supertags significantly improve productivity gain in translation in interactive-predictive NMT (INMT), while syntactic parsing somewhat found to be effective in reducing human effort in translation. Furthermore, when we model this source- and target-language syntactic information together as the conditional context, both types complement each other and our fully syntax-informed INMT model statistically significantly reduces human efforts in a French–to–English translation task, achieving 4.30 points absolute (corresponding to 9.18% relative) improvement in terms of word prediction accuracy (WPA) and 4.84 points absolute (corresponding to 9.01% relative) reduction in terms of word stroke ratio (WSR) over the baseline.

The unprecedented growth of Internet users has resulted in an abundance of unstructured information on social media including health forums, where patients request health-related information or opinions from other users. Previous studies have shown that online peer support has limited effectiveness without expert intervention. Therefore, a system capable of assessing the severity of health state from the patients' social media posts can help health professionals (HP) in prioritizing the user's post. In this study, we inspect the efficacy of different aspects of Natural Language Understanding (NLU) to identify the severity of the user's health state in relation to two perspectives(tasks) (a) Medical Condition (i.e., Recover, Exist, Deteriorate, Other) and (b) Medication (i.e., Effective, Ineffective, Serious Adverse Effect, Other) in online health communities. We propose a multiview learning framework that models both the textual content as well as contextual-information to assess the severity of the user's health state. Specifically, our model utilizes the NLU views such as sentiment, emotions, personality, and use of figurative language to extract the contextual information. The diverse NLU views demonstrate its effectiveness on both the tasks and as well as on the individual disease to assess a user's health.

Text generator systems have become extremely popular with the advent of recent deep learning models such as encoder-decoder. Controlling the information and style of the generated output without supervision is an important and challenging Natural Language Processing (NLP) task. In this paper we define the task of constructing a coherent paragraph from a set of disaster domain tweets, without any parallel data. We tackle the problem by building two systems in pipeline. The first system focuses on unsupervised style transfer to convert the individual tweets into news sentences. The second system stitches together the outputs from the first system to form a coherent news paragraph. We also propose a novel training mechanism, by splitting the sentences into propositions and training the second system to merge the sentences. We create a validation and test set consisting of tweet-sets and their equivalent news paragraphs to perform empirical evaluation. We also perform human evaluations on our model. In a completely unsupervised setting our model was able to achieve a BLEU score of 19.32, while successfully transferring styles and joining tweets to form a meaningful news paragraph.

Conversational systems are the perfect examples of human-machine interactions. The conversational agents while interacting with humans lack the ability to express emotions and behave inconsistently, making the conversations boring and non-interactive. In this work, we propose the task of persona aware emotional response generation in which the system can generate specific and consistent responses in accordance to the provided personality information and the conversational history. To make the responses interactive and interesting we intend to infuse the emotions in the responses that help in making the responses more human-like. We propose a persona aware attention framework employing an encoder-decoder approach. We investigate different ways to include the desired emotions in the responses. Experimental results on the PersonaChat dataset shows that our proposed framework outperforms the baseline models and can generate interactive and emotional responses.

In interactive machine translation (MT), human translators correct errors in automatic translations in collaboration with the MT systems, and this is an effective way to improve productivity gain in translation. Phrase-based statistical MT (PB-SMT) has been the mainstream approach to MT for the past 30 years, both in academia and industry. Neural MT (NMT), an end-to-end learning approach to MT, represents the current state-of-the-art in MT research. The recent studies on interactive MT have indicated that NMT can significantly outperform PB-SMT. In this work, first we investigate the possibility of integrating lexical syntactic descriptions in the form of supertags into the state-of-the-art NMT model, Transformer. Then, we explore whether integration of supertags into Transformer could indeed reduce human efforts in translation in an interactive-predictive platform. From our investigation we found that our syntax-aware interactive NMT (INMT) framework significantly reduces simulated human efforts in the French-to-English and Hindi- to-English translation tasks, achieving a 2.65 point absolute corresponding to 5.65% relative improvement and a 6.55 point absolute corresponding to 19.1% relative improvement, respectively, in terms of word prediction accuracy (WPA) over the respective baselines.

Fake or incorrect or miss-information detection has nowadays attracted attention to the researchers and developers because of the huge information overloaded in the web. This problem can be considered as equivalent to lie detection, truthfulness identification or stance detection. In our particular work, we focus on deciding whether the title of a news is consistent with its body text- a problem equivalent to fake information identification. In this paper, we propose a deep transfer learning approach where the problem of detecting title-body consistency is posed from the viewpoint of Textual Entailment (TE) where the title is considered as a hypothesis and news body is treated as a premise. The idea is to decide whether the body infers the title or not. Evaluation on the existing benchmark datasets, namely Fake News Challenge (FNC) dataset (released in Fake News Challenge Stage 1 (FNC-I): Stance Detection) show the efficacy of our proposed approach in comparison to the state-of-the-art systems.

Sarcasm detection is, inherently, a non-trivial problem where people express negative sentiment using positive insinuation words. Traditional approaches, in general, rely on the textual information to detect the incongruity between the surface meaning and the actual meaning. However, textual information is not always sufficient, and, often, other sources of information (e.g., visual) provides an important clue for sarcasm detection. In this paper, we propose an effective method based on deep learning that utilizes both textual and visual information for multi-modal sarcasm detection. Our proposed approach is based on the recurrent neural network that aims to exploit the interaction among the input modalities for the prediction. Experimental results suggest that the incorporation of visual modalities plays a decisive role in performance improvement.

Customer satisfaction is an essential aspect of customer care systems. It is imperative for such systems to be polite while handling customer requests/demands. In this paper, we present a large multi-lingual conversational dataset for English and Hindi. We choose data from Twitter having both generic and courteous responses between customer care agents and aggrieved users. We also propose strong baselines that can induce courteous behaviour in generic customer care response in a multi-lingual scenario. We build a deep learning framework that can simultaneously handle different languages and incorporate polite behaviour in the customer care agent’s responses. Our system is competent in generating responses in different languages (here, English and Hindi) depending on the customer’s preference and also is able to converse with humans in an empathetic manner to ensure customer satisfaction and retention. Experimental results show that our proposed models can converse in both the languages and the information shared between the languages helps in improving the performance of the overall system. Qualitative and quantitative analysis shows that the proposed method can converse in an empathetic manner by incorporating courteousness in the responses and hence increasing customer satisfaction.

A suicide note is usually written shortly before the suicide and it provides a chance to comprehend the self-destructive state of mind of the deceased. From a psychological point of view, suicide notes have been utilized for recognizing the motive behind the suicide. To the best of our knowledge, there is no openly accessible suicide note corpus at present, making it challenging for the researchers and developers to deep dive into the area of mental health assessment and suicide prevention. In this paper, we create a fine-grained emotion annotated corpus (CEASE) of suicide notes in English and develop various deep learning models to perform emotion detection on the curated dataset. The corpus consists of 2393 sentences from around 205 suicide notes collected from various sources. Each sentence is annotated with a particular emotion class from a set of 15 fine-grained emotion labels, namely (forgiveness, happiness_peacefulness, love, pride, hopefulness, thankfulness, blame, anger, fear, abuse, sorrow, hopelessness, guilt, information, instructions). For the evaluation, we develop an ensemble architecture, where the base models correspond to three supervised deep learning models, namely Convolutional Neural Network (CNN), Gated Recurrent Unit (GRU) and Long Short Term Memory (LSTM). We obtain the highest test accuracy of 60.17% and cross-validation accuracy of 60.32%

Due to the phenomenal growth of online content in recent time, sentiment analysis has attracted attention of the researchers and developers. A number of benchmark annotated corpora are available for domains like movie reviews, product reviews, hotel reviews, etc. The pervasiveness of social media has also lead to a huge amount of content posted by users who are misusing the power of social media to spread false beliefs and to negatively influence others. This type of content is coming from the domains like terrorism, cybersecurity, technology, social issues, etc. Mining of opinions from these domains is important to create a socially intelligent system to provide security to the public and to maintain the law and order situations. To the best of our knowledge, there is no publicly available tweet corpora for such pervasive domains. Hence, we firstly create a multi-domain tweet sentiment corpora and then establish a deep neural network based baseline framework to address the above mentioned issues. Annotated corpus has Cohen’s Kappa measurement for annotation quality of 0.770, which shows that the data is of acceptable quality. We are able to achieve 84.65% accuracy for sentiment analysis by using an ensemble of Convolutional Neural Network (CNN), Long Short Term Memory (LSTM), and Gated Recurrent Unit(GRU).

Event Extraction is an important task in the widespread field of Natural Language Processing (NLP). Though this task is adequately addressed in English with sufficient resources, we are unaware of any benchmark setup in Indian languages. Hindi is one of the most widely spoken languages in the world. In this paper, we present an Event Extraction framework for Hindi language by creating an annotated resource for benchmarking, and then developing deep learning based models to set as the baselines. We crawl more than seventeen hundred disaster related Hindi news articles from the various news sources. We also develop deep learning based models for Event Trigger Detection and Classification, Argument Detection and Classification and Event-Argument Linking.

We present ScholarlyRead, span-of-word-based scholarly articles’ Reading Comprehension (RC) dataset with approximately 10K manually checked passage-question-answer instances. ScholarlyRead was constructed in semi-automatic way. We consider the articles from two popular journals of a reputed publishing house. Firstly, we generate questions from these articles in an automatic way. Generated questions are then manually checked by the human annotators. We propose a baseline model based on Bi-Directional Attention Flow (BiDAF) network that yields the F1 score of 37.31%. The framework would be useful for building Question-Answering (QA) systems on scientific articles.

In this article, we propose a unified deep neural network framework for multilingual question answering (QA). The proposed network deals with the multilingual questions and answers snippets. The input to the network is a pair of factoid question and snippet in the multilingual environment (English and Hindi), and output is the relevant answer from the snippet. We begin by generating the snippet using a graph-based language-independent algorithm, which exploits the lexico-semantic similarity between the sentences. The soft alignment of the question words from the English and Hindi languages has been used to learn the shared representation of the question. The learned shared representation of question and attention-based snippet representation are passed as an input to the answer extraction layer of the network, which extracts the answer span from the snippet. Evaluation on a standard multilingual QA dataset shows the state-of-the-art performance with 39.44 Exact Match (EM) and 44.97 F1 values. Similarly, we achieve the performance of 50.11 Exact Match (EM) and 53.77 F1 values on Translated SQuAD dataset.

Multi-modal affect analysis (e.g., sentiment and emotion analysis) is an interdisciplinary study and has been an emerging and prominent field in Natural Language Processing and Computer Vision. The effective fusion of multiple modalities (e.g., text , acoustic, or visual frames ) is a non-trivial task, as these modalities, often, carry distinct and diverse information, and do not contribute equally. The issue further escalates when these data contain noise. In this article, we study the concept of multi-task learning for multi-modal affect analysis and explore a contextual inter-modal attention framework that aims to leverage the association among the neighboring utterances and their multi-modal information. In general, sentiments and emotions have inter-dependence on each other (e.g., anger → negative or happy → positive ). In our current work, we exploit the relatedness among the participating tasks in the multi-task framework. We define three different multi-task setups, each having two tasks, i.e., sentiment 8 emotion classification, sentiment classification 8 sentiment intensity prediction, and emotion classificati on 8 emotion intensity prediction. Our evaluation of the proposed system on the CMU-Multi-modal Opinion Sentiment and Emotion Intensity benchmark dataset suggests that, in comparison with the single-task learning framework, our multi-task framework yields better performance for the inter-related participating tasks. Further, comparative studies show that our proposed approach attains state-of-the-art performance for most of the cases.

Event extraction is one of the crucial tasks in biomedical text mining that aims to extract specific information concerning incidents embedded in the texts. In this article, we propose a deep learning framework that aims to identify the attributes (severity, course, temporal expression, and document creation time) associated with the medical concepts extracted from electronic medical records. The bi-directional long short-term memory network assisted by the attention mechanism is utilized to uncover the important aspects of the patient’s medical conditions. The attention mechanism specific to the medical disorder mention can focus on various parts of the sentence when different disorders are considered as input. The proposed methodology is evaluated on benchmark ShARe/CLEF eHealth Evaluation Lab 2014 shared task 2 datasets. In addition to the CLEF dataset, we also used the social media text, especially the medical blog posts. Experimental results of the proposed approach illustrate that our proposed approach achieves significant performance improvements over the state-of-the-art techniques and the highly competitive deep learning--based baseline methods.

Multimodal dialogue systems have opened new frontiers in the traditional goal-oriented dialogue systems. The state-of-the-art dialogue systems are primarily based on unimodal sources, predominantly the text, and hence cannot capture the information present in the other sources such as videos, audios, images etc. With the availability of large scale multimodal dialogue dataset (MMD) (Saha et al., 2018) on the fashion domain, the visual appearance of the products is essential for understanding the intention of the user. Without capturing the information from both the text and image, the system will be incapable of generating correct and desirable responses. In this paper, we propose a novel position and attribute aware attention mechanism to learn enhanced image representation conditioned on the user utterance. Our evaluation shows that the proposed model can generate appropriate responses while preserving the position and attribute information. Experimental results also prove that our proposed approach attains superior performance compared to the baseline models, and outperforms the state-of-the-art approaches on text similarity based evaluation metrics.

Automatically validating a research artefact is one of the frontiers in Artificial Intelligence (AI) that directly brings it close to competing with human intellect and intuition.Although criticized sometimes, the existing peer review system still stands as the benchmark of research validation.The present-day peer review process is not straightforward and demands profound domain knowledge, expertise, and intelligence of human reviewer(s), which is somewhat elusive with the current state of AI.However, the peer review texts, which contains rich sentiment information of the reviewer, reflecting his/her overall attitude towards the research in the paper, could be a valuable entity to predict the acceptance or rejection of the manuscript under consideration.Here in this work, we investigate the role of reviewers sentiments embedded within peer review texts to predict the peer review outcome.Our proposed deep neural architecture takes into account three channels of information: the paper, the corresponding reviews, and the review polarity to predict the overall recommendation score as well as the final decision.We achieve significant performance improvement over the baselines (∼ 29% error reduction) proposed in a recently released dataset of peer reviews.An AI of this kind could assist the editors/program chairs as an additional layer of confidence in the final decision making, especially when non-responding/missing reviewers are frequent in present day peer review.

In this paper, we propose a multilingual unsupervised NMT scheme which jointly trains multiple languages with a shared encoder and multiple decoders.Our approach is based on denoising autoencoding of each language and back-translating between English and multiple non-English languages.This results in a universal encoder which can encode any language participating in training into an interlingual representation, and language-specific decoders.Our experiments using only monolingual corpora show that multilingual unsupervised model performs better than the separately trained bilingual models achieving improvement of up to 1.48 BLEU points on WMT test sets.We also observe that even if we do not train the network for all possible translation directions, the network is still able to translate in a many-to-many fashion leveraging encoder's ability to generate interlingual representation.

The mining of adverse drug reaction (ADR) has a crucial role in the pharmacovigilance.The traditional ways of identifying ADR are reliable but time-consuming, non-scalable and offer a very limited amount of ADR relevant information.With the unprecedented growth of information sources in the forms of social media texts (Twitter, Blogs, Reviews etc.), biomedical literature, and Electronic Medical Records (EMR), it has become crucial to extract the most pertinent ADR related information from these free-form texts.In this paper, we propose a neural network inspired multitask learning framework that can simultaneously extract ADRs from various sources.We adopt a novel adversarial learning-based approach to learn features across multiple ADR information sources.Unlike the other existing techniques, our approach is capable to extracting fine-grained information (such as 'Indications', 'Symptoms', 'Finding', 'Disease', 'Drug') which provide important cues in pharmacovigilance.We evaluate our proposed approach on three publicly available realworld benchmark pharmacovigilance datasets, a Twitter dataset from PSB 2016 Social Media Shared Task, CADEC corpus and Medline ADR corpus.Experiments show that our unified framework achieves state-of-the-art performance on individual tasks associated with the different benchmark datasets.This establishes the fact that our proposed approach is generic, which enables it to achieve high performance on the diverse datasets.The source code is available here 1 .

Dushyant Singh Chauhan, Md Shad Akhtar, Asif Ekbal, Pushpak Bhattacharyya. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

Detecting the novelty or freshness of an entire document is essential in this age of data duplication and semantic-level redundancy all across the web. Current techniques for the problem mostly root on handcrafted similarity and divergence based measures to classify a document as novel or non-novel. However, document-level novelty detection is relatively less explored in literature if compared to its sentence-level counterpart. In this work, we present a deep neural architecture to automatically predict the amount of new information contained in a document in the form of a novelty score. Along with, we offer a dataset of more than 7500 documents, annotated at the sentence-level to facilitate further research. Our approach which learns the notion of novelty and redundancy only from the data achieves significant performance improvement over the existing methods and adopted baselines (~17% error reduction). Also, our approach complies with the Two-Stage theory of human recall essential to comprehend new information.

In this paper, we propose a two-layered multi-task attention based neural network that performs sentiment analysis through emotion analysis. The proposed approach is based on Bidirectional Long Short-Term Memory and uses Distributional Thesaurus as a source of external knowledge to improve the sentiment and emotion prediction. The proposed system has two levels of attention to hierarchically build a meaningful representation. We evaluate our system on the benchmark dataset of SemEval 2016 Task 6 and also compare it with the state-of-the-art systems on Stance Sentiment Emotion Corpus. Experimental results show that the proposed system improves the performance of sentiment analysis by 3.2 F-score points on SemEval 2016 Task 6 dataset. Our network also boosts the performance of emotion analysis by 5 F-score points on Stance Sentiment Emotion Corpus.

Dialogue Act (DA) Classification plays a signifi-cant role in the understanding of an utterance in a dialogue. Components of Spoken Dialogue System (SDS) such as Natural Language Understanding (NLU) and Dialogue Management (DM) modules can significantly exploit the output of the DA classification. In this paper, we propose a task-oriented DA classifier based on both traditional supervised Machine Learning (ML) as well as Deep Learning (DL) techniques. The type and nature of dialogues basically depend on the domain and the DA itself. So, in order to make the model task-oriented, a new tag-set has been designed by studying the properties of the target domain. On the benchmark SwitchBoard (SWBD) and TRAINS corpus, our proposed models have performed exceptionally well with the new tag-set. Experimental results indicate that our proposed models have achieved good accuracy on both the datasets and outperformed several state of the art approaches and the new tag-set is well suited for task-oriented applications.

In the era of deep learning-based systems, efficient input representation is one of the primary requisites in solving various problems related to Natural Language Processing (NLP), data mining, text mining, and the like. Absence of adequate representation for an input introduces the problem of data sparsity, and it poses a great challenge to solve the underlying problem. The problem is more intensified with resource-poor languages due to the absence of a sufficiently large corpus required to train a word embedding model. In this work, we propose an effective method to improve the word embedding coverage in less-resourced languages by leveraging bilingual word embeddings learned from different corpora. We train and evaluate deep Long Short Term Memory (LSTM)-based architecture and show the effectiveness of the proposed approach for two aspect-level sentiment analysis tasks (i.e., aspect term extraction and sentiment classification). The neural network architecture is further assisted by hand-crafted features for prediction. We apply the proposed model in two experimental setups: multi-lingual and cross-lingual. Experimental results show the effectiveness of the proposed approach against the state-of-the-art methods.

Temporality has significantly contributed to various Natural Language Processing and Information Retrieval applications. In this article, we first create a lexical knowledge-base in Hindi by identifying the temporal orientation of word senses based on their definition and then use this resource to detect underlying temporal orientation of the sentences. To create the resource, we propose a semi-supervised learning framework, where each synset of the Hindi WordNet is classified into one of the five categories, namely, past , present , future , neutral , and atemporal . The algorithm initiates learning with a set of seed synsets and then iterates following different expansion strategies, viz. probabilistic expansion based on classifier’s confidence and semantic distance based measures. We manifest the usefulness of the resource that we build on an external task, viz. sentence-level temporal classification. The underlying idea is that a temporal knowledge-base can help in classifying the sentences according to their inherent temporal properties. Experiments on two different domains, viz. general and Twitter, show interesting results.

The rapid growth of documents across the web has necessitated finding means of discarding redundant documents and retaining novel ones. Capturing redundancy is challenging as it may involve investigating at a deep semantic level. Techniques for detecting such semantic redundancy at the document level are scarce. In this work we propose a deep Convolutional Neural Networks (CNN) based model to classify a document as novel or redundant with respect to a set of relevant documents already seen by the system. The system is simple and do not require any manual feature engineering. Our novel scheme encodes relevant and relative information from both source and target texts to generate an intermediate representation which we coin as the Relative Document Vector (RDV). The proposed method outperforms the existing state-of-the-art on a document-level novelty detection dataset by a margin of ∼5% in terms of accuracy. We further demonstrate the effectiveness of our approach on a standard paraphrase detection dataset where paraphrased passages closely resemble to semantically redundant documents.

In this paper, we propose a hybrid technique for semantic question matching. It uses a proposed two-layered taxonomy for English questions by augmenting state-of-the-art deep learning models with question classes obtained from a deep learning based question classifier. Experiments performed on three open-domain datasets demonstrate the effectiveness of our proposed approach. We achieve state-of-the-art results on partial ordering question ranking (POQR) benchmark dataset. Our empirical analysis shows that coupling standard distributional features (provided by the question encoder) with knowledge from taxonomy is more effective than either deep learning or taxonomy-based knowledge alone.

Existing research on question answering (QA) and comprehension reading (RC) are mainly focused on the resource-rich language like English.In recent times, there has been a rapid growth of multi-lingual contents on the web, and this has posed several challenges to the existing QA systems.Code-mixing is one such challenge that makes the task even more complex.In this paper, we propose a linguistically motivated technique for codemixed question generation (CMQG) and a neural network based architecture for code-mixed question answering (CMQA).For evaluation, we manually create the code-mixed questions for Hindi-English language pair.In order to show the effectiveness of our neural network based CMQA technique, we utilize two benchmark datasets, viz.SQuAD and MMQA.Experiments show that our proposed model achieves encouraging performance on CMQG and CMQA.

Multi-modal sentiment analysis offers various challenges, one being the effective combination of different input modalities, namely text, visual and acoustic.In this paper, we propose a recurrent neural network based multi-modal attention framework that leverages the contextual information for utterance-level sentiment prediction.The proposed approach applies attention on multi-modal multi-utterance representations and tries to learn the contributing features amongst them.We evaluate our proposed approach on two multi-modal sentiment analysis benchmark datasets, viz.CMU Multi-modal Opinion-level Sentiment Intensity (CMU-MOSI) corpus and the recently released CMU Multi-modal Opinion Sentiment and Emotion Intensity (CMU-MOSEI) corpus.Evaluation results show the effectiveness of our proposed approach with the accuracies of 82.31% and 79.80% for the MOSI and MO-SEI datasets, respectively.These are approximately 2 and 1 points performance improvement over the state-of-the-art models for the datasets.

Sentiment analysis has immense implications in modern businesses through user-feedback mining.Large product-based enterprises like Samsung and Apple make crucial business decisions based on the large quantity of user reviews and suggestions available in different e-commerce websites and social media platforms like Amazon and Facebook.Sentiment analysis caters to these needs by summarizing user sentiment behind a particular object.In this paper, we present a novel approach of incorporating the neighboring aspects related information into the sentiment classification of the target aspect using memory networks.Our method outperforms the state of the art by 1.6% on average in two distinct domains.

Md Shad Akhtar, Palaash Sawant, Sukanta Sen, Asif Ekbal, Pushpak Bhattacharyya. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). 2018.

Sabyasachi Kamila, Mohammed Hasanuzzaman, Asif Ekbal, Pushpak Bhattacharyya, Andy Way. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). 2018.

Shweta Yadav, Asif Ekbal, Sriparna Saha, Pushpak Bhattacharyya. Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers. 2017.

In this paper, we propose a novel method for combining deep learning and classical feature based models using a Multi-Layer Perceptron (MLP) network for financial sentiment analysis.We develop various deep learning models based on Convolutional Neural Network (CNN), Long Short Term Memory (LSTM) and Gated Recurrent Unit (GRU).These are trained on top of pre-trained, autoencoder-based, financial word embeddings and lexicon features.An ensemble is constructed by combining these deep learning models and a classical supervised model based on Support Vector Regression (SVR).We evaluate our proposed technique on a benchmark dataset of SemEval-2017 shared task on financial sentiment analysis.The propose model shows impressive results on two datasets, i.e. microblogs and news headlines datasets.Comparisons show that our proposed model performs better than the existing state-of-the-art systems for the above two datasets by 2.0 and 4.1 cosine points, respectively.

The phenomenal growth in the healthcare data has inspired us in investigating robust and scalable models for data mining. For classification problems Information Gain(IG) based Decision Tree is one of the popular choices. However, depending upon the nature of the dataset, IG based Decision Tree may not always perform well as it prefers the attribute with more number of distinct values as the splitting attribute. Healthcare datasets generally have many attributes and each attribute generally has many distinct values. In this paper, we have tried to focus on this characteristics of the datasets while analysing the performance of our proposed approach which is a variant of Decision Tree model and uses the concept of Correlation Ratio(CR). Unlike IG based approach, this CR based approach has no biasness towards the attribute with more number of distinct values. We have applied our model on some benchmark healthcare datasets to show the effectiveness of the proposed technique.

In this paper, we propose a novel hybrid deep learning archtecture which is highly efficient for sentiment analysis in resource-poor languages. We learn sentiment embedded vectors from the Convolutional Neural Network (CNN). These are augmented to a set of optimized features selected through a multi-objective optimization (MOO) framework. The sentiment augmented optimized vector obtained at the end is used for the training of SVM for sentiment classification. We evaluate our proposed approach for coarse-grained (i.e. sentence level) as well as fine-grained (i.e. aspect level) sentiment analysis on four Hindi datasets covering varying domains. In order to show that our proposed method is generic in nature we also evaluate it on two benchmark English datasets. Evaluation shows that the results of the proposed method are consistent across all the datasets and often outperforms the state-of-art systems. To the best of our knowledge, this is the very first attempt where such a deep learning model is used for less-resourced languages such as Hindi.

In recent past there has been phenomenal growth in biomedical literature and health care records. Robust text mining techniques are essential in order to properly organize the documents as well as to extract relevant information. Traditional techniques for document classification focus on machine learning algorithms where learning of classifier is decided on the basis of labeled data and the features that are prominent. In this paper we focus on developing an automated technique for classifying biomedical articles containing protein-protein interaction related information against the others. Our proposed approach is based on deep neural network framework. We investigate the role of convolution neural network (CNN) and propose two model variants. We evaluate the proposed approach on the benchmark datasets of BioCreative-II Interaction Article Subtask (IAS) data sets. Effectiveness of our proposed model is evident with the significant performance gains, 2.8 % in terms of F-measure and 5 % in terms of accuracy over the traditional models.

Studying the patterns hidden in gene-expression data helps to understand the functionality of genes. In general, clustering techniques are widely used for the identification of natural partitionings from the gene expression data. In order to put constraints on dimensionality, feature selection is the key issue because not all features are important from clustering point of view. Moreover some limited amount of supervised information can help to fine tune the obtained clustering solution. In this paper, the problem of simultaneous feature selection and semisupervised clustering is formulated as a multiobjective optimization (MOO) task. A modern simulated annealing-based MOO technique namely AMOSA is utilized as the background optimization methodology. Here, features and cluster centers are represented in the form of a string and the assignment of genes to different clusters is done using a point symmetry-based distance. Six optimization criteria based on several internal and external cluster validity indices are utilized. In order to generate the supervised information, a popular clustering technique, Fuzzy C-mean, is utilized. Appropriate subset of features, proper number of clusters and the proper partitioning are determined using the search capability of AMOSA. The effectiveness of this proposed semisupervised clustering technique, Semi-FeaClustMOO, is demonstrated on five publicly available benchmark gene-expression datasets. Comparison results with the existing techniques for gene-expression data clustering again reveal the superiority of the proposed technique. Statistical and biological significance tests have also been carried out.

Evaluating a given clustering result is a very difficult problem in real world. Cluster validity indices are developed for this purpose. There are two different types of cluster validity indices available : External and Internal. External cluster validity indices utilize some supervised information and internal cluster validity indices utilize the intrinsic structure of the data. In this paper a new external cluster validity index, MMI has been implemented based on Max-Min distance among data points and prior information based on structure of the data. A new probabilistic approach has been implemented to find the correct correspondence between the true and obtained clustering. Genetic K-means algorithm (GAK-means) and single linkage have been used as the underlying clustering techniques. Results of the proposed index for identifying the appropriate number of clusters is shown for five artificial and two real-life data sets. GAK-means and single linkage clustering techniques are used as the underlying partitioning techniques with the number of clusters varied over a range. The MMI index is then used to determine the appropriate number of clusters. The performance of MMI is compared with existing external cluster validity indices, adjusted rand index (ARI) and rand index (RI). It works well for two class and multi class data sets.

Plagiarism denotes the act of copying someone else's idea (or, works) and claiming it as his/her own. Plagiarism detection is the procedure to detect the texts of a given document which are plagiarized, i.e. copied from from some other documents. Potential challenges are due to the facts that plagiarists often obfuscate the copied texts; might shuffle, remove, insert, or replace words or short phrases; might also restructure the sentences replacing words with synonyms; and changing the order of appearances of words in a sentence. In this paper we propose a technique based on textual similarity for external plagiarism detection. For a given suspicious document we have to identify the set of source documents from which the suspicious document is copied. The method we propose comprises of four phases. In the first phase, we process all the documents to generate tokens, lemmas, finding Part-of-Speech (PoS) classes, character-offsets, sentence numbers and named-entity (NE) classes. In the second phase we select a subset of documents that may possibly be the sources of plagiarism. We use an approach based on the traditional Vector Space Model (VSM) for this candidate selection. In the third phase we use a graph-based approach to find out the similar passages in suspicious document and selected source documents. Finally we filter out the false detections1.

Semi-supervised clustering uses the information of unsupervised and supervised learning to overcome the problems associated with them. Extracted information are given in the form of class labels and data distribution during clustering process. In this paper the problem of semi-supervised clustering is formulated under the framework of multiobjective optimization (MOO). Thereafter, a multiobjective based clustering technique is extended to solve the semi-supervised clustering problem. The newly developed semi-supervised multiobjective clustering algorithm (Semi-GenClustMOO), is used for appropriate partitioning of data into appropriate number of clusters. Four objective functions are optimized, out of which first three use some unsupervised information and the last one uses supervised information. These four objective functions represent, respectively, the, total compactness of the partitioning, total symmetry present in the clusters, cluster connectedness and Adjust Rand Index. These four objective functions are optimized simultaneously using AMOSA, a newly developed simulated annealing based multiobjective optimization method. Results show that it can easily detect the appropriate number of clusters as well as the appropriate partitioning from data sets having either well-separated clusters of any shape or symmetrical clusters with or without overlaps. Seven artificial and four real-life data sets have been used for evaluation to show the effectiveness of the Semi-GenClustMOO technique. In each case class information of 10% randomly chosen data point is known to us1.

In this article, we report the search capability of Genetic Algorithm (GA) to construct a weighted vote-based classifier ensemble for Named Entity Recognition (NER). Our underlying assumption is that the reliability of predictions of each classifier differs among the various named entity (NE) classes. Thus, it is necessary to quantify the amount of voting of a particular classifier for a particular output class. Here, an attempt is made to determine the appropriate weights of voting for each class in each classifier using GA. The proposed technique is evaluated for four leading Indian languages, namely Bengali, Hindi, Telugu, and Oriya, which are all resource-poor in nature. Evaluation results yield the recall, precision and F-measure values of 92.08%, 92.22%, and 92.15%, respectively for Bengali; 96.07%, 88.63%, and 92.20%, respectively for Hindi; 78.82%, 91.26%, and 84.59%, respectively for Telugu; and 88.56%, 89.98%, and 89.26%, respectively for Oriya. Finally, we evaluate our proposed approach with the benchmark dataset of CoNLL-2003 shared task that yields the overall recall, precision, and F -measure values of 88.72%, 88.64%, and 88.68%, respectively. Results also show that the vote based classifier ensemble identified by the GA-based approach outperforms all the individual classifiers, three conventional baseline ensembles, and some other existing ensemble techniques. In a part of the article, we formulate the problem of feature selection in any classifier under the single objective optimization framework and show that our proposed classifier ensemble attains superior performance to it.

In this paper, we report a multiobjective optimization (MOO) based classifier ensemble technique to solve the problem of Named Entity Recognition (NER). Our underlying assumption is that rather than searching for the best feature set for a particular classifier, ensembling of several classifiers which are trained using different feature representations could be a more fruitful approach. But, it is very crucial to select the appropriate classifiers that can participate in final ensembling. Here, we propose a new technique for classifier ensembling based on MOO that can simultaneously optimize several different classification measures. Maximum Entropy (ME) framework is used to generate a number of classifiers by considering the various combinations of the available features. The proposed technique is evaluated for two resource constrained languages, namely Bengali and Hindi. Evaluation results yield the recall, precision and F-measure values of 72.34%, 84.94% and 78.13%, respectively for Bengali, and 64.93%, 83.29% and 72.97%, respectively for Hindi. Experiments also show that the classifier ensemble identified by the proposed multiobjective based approach outperforms all the individual classifiers, two different baseline ensembles and a classifier ensemble identified by the single objective genetic algorithm (GA) based approach.

Appropriate feature selection is a very crucial issue in any machine learning framework, specially in Maximum Entropy (ME). In this paper, the selection of appropriate features for constructing a ME based Named Entity Recognition (NER) system is posed as a multiobjective optimization (MOO) problem. Two classification quality measures, namely recall and precision are simultaneously optimized using the search capability of a popular evolutionary MOO technique, NSGA-II. The proposed technique is evaluated to determine suitable feature combinations for NER in two languages, namely Bengali and English that have significantly different characteristics. Evaluation results yield the recall, precision and F-measure values of 70.76%, 81.88% and 75.91%, respectively for Bengali, and 78.38%, 81.27% and 79.80%, respectively for English. Comparison with an existing ME based NER system shows that our proposed feature selection technique is more efficient than the heuristic based feature selection.

In this paper, we present the problem of appropriate feature selection for constructing a Maximum Entropy (ME) based Named Entity Recognition (NER) system under the multiobjective optimization (MOO) framework. Two conflicting objective functions are simultaneously optimized using the search capability of MOO. These objectives are (i). the dimensionality of features, which is tried to be minimized, and (ii). the corresponding F-measure value of the classifier, trained using the features present, is maximized. The features are encoded in the chromosomes. Thereafter, a multiobjective evolutionary algorithm in the steps of a popular MOO technique, NSGA-II, is developed to determine the appropriate feature subset. The proposed technique is evaluated to determine the suitable feature combinations for NER in a resource-constrained language, namely Bengali. Evaluation results yield the recall, precision and F-measure values of 72.45%, 82.39% and 77.11%, respectively.

In this paper, we report a classifier ensemble technique using the search capability of genetic algorithm (GA) for Named Entity Recognition (NER) in biomedical domain. We use Maximum Entropy (ME) framework to build a number of classifiers depending upon the various representations of a set of features. The proposed technique is evaluated with the JNLPBA 2004 data sets that yield the overall recall, precision and F-measure values of 67.98%, 71.68% and 69.78%, respectively.