Healthcare decision-making requires not only accurate predictions but also insights into how factors influence patient outcomes. While traditional machine learning (ML) models excel at predicting outcomes, such as identifying high-risk patients, they are limited in addressing "what if" questions about interventions. This study introduces the Probabilistic Causal Fusion (PCF) framework, which integrates Causal Bayesian Networks (CBNs) and Probability Trees (PTrees) to extend beyond predictions. PCF leverages causal relationships from CBNs to structure PTrees, enabling both the quantification of factor impacts and the simulation of hypothetical interventions. The framework is evaluated on three clinically diverse, real-world datasets, MIMIC-IV, Framingham Heart Study, and BRFSS (Diabetes), demonstrating consistent predictive performance comparable to conventional ML models, while offering enhanced interpretability and causal reasoning capabilities. In contrast to conventional approaches focused solely on prediction, PCF offers a unified framework for prediction, intervention modelling, and counterfactual analysis, forming a holistic toolkit for clinical decision support. To enhance interpretability, PCF incorporates sensitivity analysis and SHapley Additive exPlanations (SHAP). Sensitivity analysis quantifies the influence of causal parameters on outcomes such as Length of Stay (LOS), Coronary Heart Disease (CHD), and Diabetes, while SHAP highlights the importance of individual features in predictive modelling. This dual-layered interpretability offers both macro-level insights into causal pathways and micro-level explanations for individual predictions. By combining causal reasoning with predictive modelling, PCF bridges the gap between clinical intuition and data-driven insights. Its ability to uncover relationships between modifiable factors and simulate hypothetical scenarios provides clinicians with a clearer understanding of causal pathways. This approach supports more informed, evidence-based decision-making, offering a robust framework for addressing complex questions in diverse healthcare settings.

Abstract Multimodal classifiers function as opaque black box models. While several techniques exist to interpret their predictions, very few of them are as intuitive and accessible as natural language explanations (NLEs). To build trust, such explanations must faithfully capture the classifier’s internal decision making behavior, a property known as faithfulness. In this paper, we propose CAuSE (Causal Abstraction under Simulated Explanations), a novel framework to generate faithful NLEs for any pretrained multimodal classifier. We demonstrate that CAuSE generalizes across datasets and models through extensive empirical evaluation. Theoretically, we show that CAuSE, trained via interchange intervention, forms a causal abstraction of the underlying classifier. We further validate this through a redesigned metric for measuring causal faithfulness in multimodal settings. CAuSE surpasses other methods on this metric, with qualitative analysis reinforcing its advantages. We also perform detailed error analysis to pinpoint the failure cases of CAuSE1.

Diabetes, a pervasive and enduring health challenge, imposes significant global implications on health, financial healthcare systems, and societal well-being. This study undertakes a comprehensive exploration of various structural learning algorithms to discern causal pathways amongst potential risk factors influencing diabetes progression. This study evaluates a diverse set of structure learning algorithms to discern causal pathways amongst potential risk factors influencing diabetes progression. The methodology involves the application of these algorithms to relevant diabetes data, followed by the conversion of their output graphs into Causal Bayesian Networks (CBNs), enabling predictive analysis and the evaluation of discrepancies in the effect of hypothetical interventions within our context-specific case study. This study highlights the substantial impact of algorithm selection on intervention outcomes. To consolidate insights from diverse algorithms, we employ a model-averaging technique that helps us obtain a unique causal model for diabetes derived from a varied set of structural learning algorithms.We also investigate how each of those individual graphs, as well as the average graph, compare to the structures elicited by a domain expert who categorised graph edges into high confidence, moderate, and low confidence types, leading into three individual graphs corresponding to the three levels of confidence. The resulting causal model and data are made available online, and serve as a valuable resource and a guide for informed decision-making by healthcare practitioners. Our applied work integrates and evaluates existing causal structure learning methods for decision support in patients with diabetes. It offers a comprehensive understanding of the interactions between relevant risk factors for intervention, and enables us to simulate the effect of hypothetical interventions before implementation. Therefore, this research not only contributes to the academic discussion on diabetes, but also provides practical guidance for healthcare professionals in developing efficient intervention and risk management strategies.

Sign language translation (SLT) is usually seen as a two-step process of continuous sign language recognition (CSLR) and gloss-to-text translation. We propose a novel, Transformer-based architecture to jointly perform CSLR and sign-translation in an end-to-end fashion. We extend the ordinary Transformer decoder with two channels to support multitasking, where each channel is devoted to solving a particular problem. To control the memory footprint of our model, channels are designed to share most of their parameters with each other. However, each channel still has a dedicated set of parameters that is fine-tuned with respect to the channel's task. In order to evaluate the proposed architecture, we focus on translating German signs into English sequences and use the RWTH-PHOENIX-Weather 2014 T corpus in our experiments. Evaluation results along with detailed quantitative and qualitative analyses indicate that the mixture of information provided by the multitask decoder was successful and enabled us to achieve superior performance in comparison to other SLT models.

Temporal distance (TD) is a type of psychological distance which shows how an individual construes past and future. It is not explored with empirical research as to how an individual’sfocuson temporal distance (near-past, far-past, near-future, and far-future) can be measured from human-written text and further used for studying human tendencies. Traditionally, focus on a Temporal Distance is studied by self-report measurements. In this article, we present a study on human focus on a temporal distance from their Twitter posts (English tweets). We first identify the tweet-level temporal focus by deep neural classifiers which make use of linguistic knowledge for classification. The model classifies each tweet into one ofnear-past,far-past,near-futureorfar-future. Classified tweets are then grouped by users to obtain the user-level temporal focus. Finally, we correlate the user’s focus on temporal distance (near-past, far-past, near-future, and far-future) with his/her demographic (age, gender, education, and relationship status) and psychological attributes (intelligence, optimism, joy, sadness, disgust, anger, surprise, and fear). Our empirical analysis reveals that users’near-pastfocus is more positively correlated to their age. We also observe that users’near-futurefocus is correlated to joy while users’ focus onfar-pastis associated with negative emotions like sadness, disgust, anger, and fear.

Depression is a severe psychological disorder that is experienced by a significant number of individuals across the globe. It greatly changes the way one thinks, triggering a constant decline in mood. Studies have shown that gender can act as a good indicator of depression. In this paper, we analyse the effects of gender information in the estimation of depression. We have carried out different experiments on the benchmark data set named Distress Analysis Interview Corpus - a Wizard of Oz (DAIC-WOZ). Concretely, we discovered that a) gender information substantially improves the performance of depression severity estimation, and b) adversarially learning to predict the depression score distributed by gender improves the performance of depression severity estimation.

Large amounts of multi-modal information online make it difficult for users to obtain proper insights. In this paper, we introduce and formally define the concepts of supplementary and complementary multi-modal summaries in the context of the overlap of information covered by different modalities in the summary output. A new problem statement of combined complementary and supplementary multi-modal summarization (CCS-MMS) is formulated. The problem is then solved in several steps by utilizing the concepts of multi-objective optimization by devising a novel unsupervised framework. An existing multi-modal summarization data set is further extended by adding outputs in different modalities to establish the efficacy of the proposed technique. The results obtained by the proposed approach are compared with several strong baselines; ablation experiments are also conducted to empirically justify the proposed techniques. Furthermore, the proposed model is evaluated separately for different modalities quantitatively and qualitatively, demonstrating the superiority of our approach.

Abstract Neural machine translation (NMT) has recently shown promising results on publicly available benchmark datasets and is being rapidly adopted in various production systems. However, it requires high-quality large-scale parallel corpus, and it is not always possible to have sufficiently large corpus as it requires time, money, and professionals. Hence, many existing large-scale parallel corpus are limited to the specific languages and domains. In this paper, we propose an effective approach to improve an NMT system in low-resource scenario without using any additional data. Our approach aims at augmenting the original training data by means of parallel phrases extracted from the original training data itself using a statistical machine translation (SMT) system. Our proposed approach is based on the gated recurrent unit (GRU) and transformer networks. We choose the Hindi–English, Hindi–Bengali datasets for Health, Tourism, and Judicial (only for Hindi–English) domains. We train our NMT models for 10 translation directions, each using only 5–23k parallel sentences. Experiments show the improvements in the range of 1.38–15.36 BiLingual Evaluation Understudy points over the baseline systems. Experiments show that transformer models perform better than GRU models in low-resource scenarios. In addition to that, we also find that our proposed method outperforms SMT—which is known to work better than the neural models in low-resource scenarios—for some translation directions. In order to further show the effectiveness of our proposed model, we also employ our approach to another interesting NMT task, for example, old-to-modern English translation, using a tiny parallel corpus of only 2.7K sentences. For this task, we use publicly available old-modern English text which is approximately 1000 years old. Evaluation for this task shows significant improvement over the baseline NMT.

Significant development of communication technology over the past few years has motivated research in multi-modal summarization techniques. A majority of the previous works on multi-modal summarization focus on text and images. In this paper, we propose a novel extractive multi-objective optimization based model to produce a multi-modal summary containing text, images, and videos. Important objectives such as intra-modality salience, cross-modal redundancy and cross-modal similarity are optimized simultaneously in a multi-objective optimization framework to produce effective multi-modal output. The proposed model has been evaluated separately for different modalities, and has been found to perform better than state-of-the-art approaches.

Understanding the chemical-disease relations (CDR) is a crucial task in various biomedical domains. Manual mining of these information from biomedical literature is costly and time-consuming. To address these issues, various researches have been carried out to design an efficient automatic tool. In this paper, we propose a multi-view based deep neural network model for CDR task. Typically, multiple representations (or views) of the datasets are not available for this task. So, we train multiple conceptually different deep neural network models on the dataset to generate different abstract features, treated as different views. A novel loss function, "Penalized LF", is defined to address the problem of imbalance dataset. The proposed loss function is generic in nature. The model is designed as a combination of Convolution Neural Network (CNN) and Bidirectional Long Short Term Memory (Bi-LSTM) network along with a Multi-Layer Perceptron (MLP). To show the efficacy of our proposed model, we have compared it with six baseline models and other state-of-the-art techniques, on "chemicals-and-disease-DFE" dataset, a free text dataset created by Li et al. from BioCreative V Chemical Disease Relation dataset. Results show that the proposed model attains highest F1-score for individual classes, proving its efficiency in handling class imbalance problem in the dataset. To further demonstrate the efficacy of the proposed model, we have presented results on BioCreative V dataset and two Protein-Protein Interaction Identification (PPI) datasets, viz., AiMed and BioInfer. All these results are also compared with the state-of-the-art models.

There is a large body of works on multi-view clustering that exploit multiple representations (or views) of the same input data for better convergence. These multiple views can come from multiple modalities (image, audio, text) or different feature subsets. Obtaining one consensus partitioning after considering different views is usually a non-trivial task. Recently, multi-objective based multi-view clustering methods have suppressed the performance of single objective based multi-view clustering techniques. One key problem is that it is difficult to select a single solution from a set of alternative partitionings generated by multi-objective techniques on the final Pareto optimal front. In this article, we propose a novel multi-objective based multi-view clustering framework that overcomes the problem of selecting a single solution in multi-objective based techniques. In particular, our proposed framework has three major components as follows: (i) multi-view based multi-objective algorithm, Multiview-AMOSA, for initial clustering of data points; (ii) a generative model for generating a combined solution having probabilistic labels; and (iii) K -means algorithm for obtaining the final labels. As the first component, we have adopted a recently developed multi-view based multi-objective clustering algorithm to generate different possible consensus partitionings of a given dataset taking into account different views. A generative model is coupled with the first component to generate a single consensus partitioning after considering multiple solutions. It exploits the latent subsets of the non-dominated solutions obtained from the multi-objective clustering algorithm and combines them to produce a single probabilistic labeled solution. Finally, a simple clustering algorithm, namely K -means, is applied on the generated probabilistic labels to obtain the final cluster labels. Experimental validation of our proposed framework is carried out over several benchmark datasets belonging to three different domains; UCI datasets, multi-view datasets, search result clustering datasets, and patient stratification datasets. Experimental results show that our proposed framework achieves an improvement of around 2%--4% over different evaluation metrics in all the four domains in comparison to state-of-the art methods.

Temporality has significantly contributed to various Natural Language Processing and Information Retrieval applications. In this article, we first create a lexical knowledge-base in Hindi by identifying the temporal orientation of word senses based on their definition and then use this resource to detect underlying temporal orientation of the sentences. To create the resource, we propose a semi-supervised learning framework, where each synset of the Hindi WordNet is classified into one of the five categories, namely, past , present , future , neutral , and atemporal . The algorithm initiates learning with a set of seed synsets and then iterates following different expansion strategies, viz. probabilistic expansion based on classifier’s confidence and semantic distance based measures. We manifest the usefulness of the resource that we build on an external task, viz. sentence-level temporal classification. The underlying idea is that a temporal knowledge-base can help in classifying the sentences according to their inherent temporal properties. Experiments on two different domains, viz. general and Twitter, show interesting results.

Current paper explores the use of multi-view learning for search result clustering. A web-snippet can be represented using multiple views. Apart from textual view cued by both the semantic and syntactic information, a complimentary view extracted from images contained in the web-snippets is also utilized in the current framework. A single consensus partitioning is finally obtained after consulting these two individual views by the deployment of a multiobjective based clustering technique. Several objective functions including the values of a cluster quality measure measuring the goodness of partitionings obtained using different views and an agreement-disagreement index, quantifying the amount of oneness among multiple views in generating partitionings are optimized simultaneously using AMOSA. In order to detect the number of clusters automatically, concepts of variable length solutions and a vast range of permutation operators are introduced in the clustering process. Finally, a set of alternative partitioning are obtained on the final Pareto front by the proposed multi-view based multiobjective technique. Experimental results by the proposed approach on several benchmark test datasets of SRC with respect to different performance metrics evidently establish the power of visual and text-based views in achieving better search result clustering.

Sabyasachi Kamila, Mohammed Hasanuzzaman, Asif Ekbal, Pushpak Bhattacharyya, Andy Way. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). 2018.

In line with global trends, historical records are increasingly available in forms that computer can process. These ever expanding records (such as scanned books, large-scale corpora, academic papers, maps, photos, audios, videos)---either digitally born or reconstructed through digitization pipelines---are too big to be read or viewed manually. Historians, like other humanities researchers, have a keen interest in computational approaches to process and study digitized historical information for research, writing, and dissemination of historical knowledge. In Computer Science, experimental tools and methods are challenged to be validated regarding their relevance for real-world questions and applications. The HistoInformatics workshop series is focused on the challenges and opportunities of data-driven humanities and brings together scientists and scholars at the forefront of this emerging field, at the interface between History, Anthropology, Archaeology, Computer Science and associated disciplines as well as the cultural heritage sector. The 4th HistoInformatics Workshop was a half day workshop co-located with the 26th ACM International Conference on Information and Knowledge Management (CIKM 2017) in Singapore.

Most social media platforms grant users freedom of speech by allowing them to freely express their thoughts, beliefs, and opinions. Although this represents incredible and unique communication opportunities, it also presents important challenges. Online racism is such an example. In this study, we present a supervised learning strategy to detect racist language on Twitter based on word embedding that incorporate demographic (Age, Gender, and Location) information. Our methodology achieves reasonable classification accuracy over a gold standard dataset (F1=76.3%) and significantly improves over the classification performance of demographic-agnostic models.

We present a two-step strategy that addresses fundamental deficiencies in social media-based event detection and achieves effective local event by taking advantage of geo-located data from Twitter. While previous work has mainly relied on an analysis of tweet text to identify local events, we show how to reliably detect events using meta-data analysis of geo-tagged tweets. The first step of the method identifies several spatio-temporal clusters within the dataset across both space and time using metadata to form potential candidate events. In the second step, it ranks all the candidates by the amount of hashtag/entity inequality. We used crowdsourcing to evaluate the proposed approach on a data set that contains millions of geo-tagged tweets. The results show that our framework performs reasonably well in terms of precision and discovers local events faster.

The ability to capture the time information conveyed in natural language is essential to many natural language processing applications such as information retrieval, question answering, automatic summarization, targeted marketing, loan repayment forecasting, and understanding economic patterns. In this paper, we propose a graph-based semi-supervised classification strategy that makes use of WordNet definitions or ‘glosses’, its conceptual-semantic and lexical relations to supplement WordNet entries with information on the temporality of its word senses. Intrinsic evaluation results show that the proposed approach outperforms prior semi-supervised, non-graph classification approaches to the temporality recognition of word senses, and confirm the soundness of the proposed approach.

Web search query logs can be used to track and, in some cases, anticipate the dynamics of individual behavior which is the smallest building block of the economy. We study AOL query logs and introduce a collective future intent index to measure the degree to which Internet users seek more information about the future than the past and the present. We have asked the question whether there is link between the collective future intent index and financial market fluctuations on a weekly time scale, and found a clear indication that the weekly transaction volume of S&P 500 index is correlated with the collective intent of the public to look forward.

Understanding the temporal orientation of web search queries is an important issue for the success of information access systems. In this paper, we propose a multi-objective ensemble learning solution that (1) allows to accurately classify queries along their temporal intent and (2) identifies a set of performing solutions thus offering a wide range of possible applications. Experiments show that correct representation of the problem can lead to great classification improvements when compared to recent state-of-the-art solutions and baseline ensemble techniques.

Mohammed Hasanuzzaman, Gaël Dias, Stéphane Ferrari, Yann Mathet. Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, volume 2: Short Papers. 2014.

We present in this paper a new architecture of a visual intracortical microstimulator, which is composed of an external controller providing electromagnetic energy and capacitive-link based high data rate link, and a multi-unit energy-efficient implant. The latter is composed of 2 full custom chips. The first one is a multiwaveform stimuli generator dedicated to supply microstimulation-based constant current to the second chip grouping a multichannel high-impedance microelectrode driver (MED). The stimuli generator is featured with several new power-efficient building blocks such as high-performance current mirrors, low-area source/sink current-mode digital-to-analog converters (DACs), and low-power dedicated controller. The highly-configurable MED, which provides the multi-level current (2 to 196 μA), drives an array of microelectrodes through high-voltage switches. The stimuli generator is implemented in 1.2/3.3 V IBM CMOS 0.13 μm technology. However the output MED is fabricated in DALSA 0.8 μm 5V/20V CMOS/DMOS technology. The latter supplies needed compliance voltage of 10V across high impedance (average value of 100kΩ) microelectrode-tissue interface. The silicon areas of the low-voltage and highvoltage chips are 1.75×1.75 mm2and 4.0×4.0 mm2respectively. Post-layout simulation results are provided to show the expected operation of the device.

In this paper, we present the problem of appropriate feature selection for constructing a Maximum Entropy (ME) based Named Entity Recognition (NER) system under the multiobjective optimization (MOO) framework. Two conflicting objective functions are simultaneously optimized using the search capability of MOO. These objectives are (i). the dimensionality of features, which is tried to be minimized, and (ii). the corresponding F-measure value of the classifier, trained using the features present, is maximized. The features are encoded in the chromosomes. Thereafter, a multiobjective evolutionary algorithm in the steps of a popular MOO technique, NSGA-II, is developed to determine the appropriate feature subset. The proposed technique is evaluated to determine the suitable feature combinations for NER in a resource-constrained language, namely Bengali. Evaluation results yield the recall, precision and F-measure values of 72.45%, 82.39% and 77.11%, respectively.

In this paper, we report a classifier ensemble technique using the search capability of genetic algorithm (GA) for Named Entity Recognition (NER) in biomedical domain. We use Maximum Entropy (ME) framework to build a number of classifiers depending upon the various representations of a set of features. The proposed technique is evaluated with the JNLPBA 2004 data sets that yield the overall recall, precision and F-measure values of 67.98%, 71.68% and 69.78%, respectively.

In a number of real-world signal processing applications, signals from various independent sources may get distorted by environmental factors that can be represented as convolutive mixtures of original signals received at the sensors. In this paper, the effects of environmental factors and modeling assumptions on the performance capabilities of independent component analysis-based techniques are investigated. The so-called blind source separation feedback network architecture that is capable of coping with convolutive mixtures of sources is derived using Bell and Sejnowski's information maximization principle. We develop ideal solutions for separation of independent source signals from the convolutive mixtures that is applicable to an arbitrary N /spl times/ N feedback network architecture. A number of simulation case studies corresponding to various types of environment filters are presented using synthetically generated data.

This paper presents a knowledge-based person-centric human-robot interaction system using facial and hand gestures. In the proposed method, face detection and person identification are first made. With the knowledge of the known user face and hand poses are then classified from the later image frame by the subspace method and the gestures are finally recognized. The rules for interpreting the gestures are selected according to each specific user recognized by the facial image. The user's name and gesture commands are sent to the robot through a Software Platform for Agent and Knowledge Management (SPAK) to implement person-centric human-robot interaction. The effectiveness of this method has been demonstrated by interacting with a humanoid robot Robovie.

This paper proposes a knowledge model-based coordinator robot, in order to realize coordination of tasks by robot instead of human being according to human requests in a symbiotic autonomous human-robot system. This coordinator robot is constructed based on a knowledge model of coordination. In this model, a physical domain world on coordination is modeled by a description world, including interpreter for the human request domain, coordination interface for the problem domain, coordination policy for the solution domain, etc. In this paper, a coordinator robot is implemented by means of a software platform and distributed intelligent agents. The experimental work with using actual robots demonstrates the effectiveness of the proposed method.

This paper describes a systematic method for the automatic generation of fabrication processes of thin film devices. The method uses a partially ordered set (poset) representation of device topology describing the order between its various components in the form of a directed acyclic graph. The sequence in which these components are fabricated is determined from the poset linear extensions, and the component sequence is expanded into a corresponding process flow. The graph-theoretic synthesis method is powerful enough to establish existence and multiplicity of flows thus creating a design space D suitable for optimization. The cardinality /spl par/D/spl par/ for a device with N components is large with a worst case /spl par/D/spl par//spl les/(N-1)! yielding in general a combinatorial explosion of solutions. The number of solutions is controlled through a priori estimates of /spl par/D/spl par/ and condensation of the device graph. The method has been implemented in the computer program MISTIC (Michigan Synthesis Tools for Integrated Circuits) which calculates specific process parameters using an internal database of process modules and materials. Currently, MISTIC includes process modules for deposition, lithography, etching, ion implantation, coupled simultaneous diffusions, and reactive growth. The compilation procedure was applied to several device structures. For a double metal twin-well BiCMOS structure, the compiler generated 168 complete process flows.