While Large language models (LLMs) have shown to exhibit remarkable performance in a wide range of NLP tasks, they often struggle to interpret and reason over multi-hop questions in open-domain question answering (ODQA) settings. While popular prompt approaches such as Chain-of-Thought and Plan-and-Solve facilitate more manageable questions for OQDA via task decomposition, these approaches are prone to generating erroneous and redundant intermediate steps in multi-hop queries due to limited capacity for modeling complex entity relationships. In this paper, we introduce a novel prompt approach for multi-hop QA viz., MoDeGraph (Multi-Hop Dependency Graphs), that is designed to steer LLMs to extract and model entity relationships in complex questions. MoDeGraph constructs a dependency graph from LLM-generated entity-relation triples to enable more coherent and human-like multi-step reasoning. Experimental results in knowledge-intensive tasks for multi-hop QA demonstrate our approach produces more coherent and faithful reasoning chains as well as consistent increase in QA performance across several benchmark datasets.

Bold scientific research tasks need multi-disciplinary knowledge and collaborations that require finding scholars from particular domains with relevant knowledge. Given the variety of scholars and diversity, finding the appropriate scholar is an important and challenging problem for scientific communities. In this paper, we propose a “ScholarFinder” framework that uses contextual information (abstracts or publications) for embedding a scholar's knowledge in an unsupervised learning manner. Specifically, we implement an unsupervised embedding technique viz.,Variational AutoEncoder (VAE). For better feature representation learning, we also implement aVariational Deep Embedded Clustering (VDEC)method that further enhances downstream tasks (e.g., clustering, classification) accuracy, scalability, and performance. In addition, we incorporate a multi-task learning scheme into our VDEC model for improving the effectiveness of simultaneously learning both embedding and clustering. Subsequently, the downstream tasks can be built based on pre-trained scholars' knowledge embeddings to predict suitability of a scholar for a research task. Using a dataset involving a 20-year collection of federal grant awards, we have demonstrated how our pre-trained model improved the performance for downstream tasks. We have also investigated how our pre-trained model can be integrated into a knowledge graph to achieve better performance. Lastly, we show that our ScholarFinder model variants outperform state-of-the-art baseline models (i.e., XGBoost, GBDT, AdaBoost, DNN, GraphSAGE, DEC, VaDE) and recent LLM based models (i.e., Bert4Rec, OpenP5) by atleast 18%.

The advent of machine learning (ML) has led to the widespread adoption of developing task-oriented dialog systems for scientific applications (e.g., science gateways) where voluminous information sources are retrieved and curated for domain users. Yet, there still exists a challenge in designing chatbot dialog systems that achieve widespread diffusion among scientific communities. In this article, we propose a novel Vidura advisor design framework (VADF) to develop dialog system designs for information retrieval (IR) and question-answering (QA) tasks, while enabling the quantification of system utility based on human performance in diverse application environments. We adopt a socio-technical approach in our framework for designing dialog systems by utilizing domain expert feedback, which features a sparse retriever for enabling accurate responses in QA settings using linear interpolation smoothing. We apply our VADF for an exemplar science gateway, viz. KnowCOVID-19, to conduct experiments that demonstrate the utility of dialog systems based on IR and QA performance, application utility, and perceived adoption. Experimental results show our VADF approach significantly improves IR performance against retriever baselines (up to 5% increase) and QA performance against large language models (LLMs) such as ChatGPT (up to 43% increase) on scientific literature datasets. In addition, through a usability survey, we observe that measuring application utility and human performance when applying VADF to KnowCOVID-19 translates to an increase in perceived community adoption.

Task-oriented dialog systems have shown to foster effective human-chatbot collaborations for accomplishing goal-specific tasks through intent classification. In a real-world setting, collecting and training over user intents incurs a labeling-cost challenge for human annotators. While existing human-AI collaborative approaches such as active learning (AL) can properly resolve such labeling-cost challenges, most existing AL algorithms assume the unlabeled pool has similar distributions as the in domain (IND) training set. To address the conflict between AL and out-of-domain (OOD) data samples, we present Deep Contrastive Active Learning (DeCAL), a deep novel AL framework that uses contrastive learning techniques for query intent classification in task-oriented dialogs. DeCAL features an acquisition function that filters OOD samples by computing a distance-based confidence score over unlabeled samples using their neighboring features. To validate DeCAL, we compare against deep AL baselines via the performance of acquired IND/OOD samples and using the classification accuracy metric. Experimental results on benchmark datasets demonstrate De-CAL outperforms deep AL baseline algorithms on acquired OOD by 14%, while simultaneously showing competitive performance on IND accuracy.

The advancements in time-efficient data collection techniques such as active learning (AL) have become salient for user intent classification performance in task-oriented dialog systems (TODS).In realistic settings, however, traditional AL techniques often fail to efficiently select targeted in-distribution (IND) data when encountering newly acquired out-ofdistribution (OOD) user intents in the unlabeled pool.In this paper, we introduce a novel adaptive open-set AL framework viz., "AOSAL" for TODS that combines a distance-based OOD detector using an adaptive false positive rate threshold along with an informativeness measure (e.g., entropy) to strategically select informative IND data points in the unlabeled pool.Specifically, we utilize the adaptive OOD detector to classify and filter out OOD samples from the unlabeled pool, then prioritize the acquisition of classified IND instances based on their informativeness scores.To validate our approach, we conduct experiments that display our framework's flexibility and performance over multiple distance-based approaches and informativeness measures against deep AL baselines on benchmark text datasets.The results show that our AOSAL consistently outperforms the baselines on IND classification and percentage of acquired IND samples, demonstrating its ability to improve robustness of task-oriented dialog systems.

Summary There is a growing need for next‐generation science gateways to increase the accessibility of emerging large‐scale datasets for data consumers (e.g., clinicians, researchers) who aim to combat COVID‐19‐related challenges. Such science gateways that enable access to distributed computing resources for large‐scale data management need to be made more programmable, extensible, and scalable. In this article, we propose a novel socio‐technical approach for developing a next‐generation healthcare science gateway, namely, OnTimeEvidence that addresses data consumer challenges surrounding the COVID‐19 pandemic related data analytics. OnTimeEvidence implements an intelligent agent, namely, Vidura Advisor that integrates an evidence‐based filtering method to transform manual practices and improve scalability of data analytics. It also features a plug‐in management middleware that improves the programmability and extensibility of the science gateway capabilities using microservices. Lastly, we present a usability study that shows the important factors from data consumers' perspective to adopt OnTimeEvidence with chatbot‐assisted middleware support to increase their productivity and collaborations to access vast publication archives for rapid knowledge discovery tasks.

Virtual Reality Learning Environments (VRLEs) are a new form of immersive environments which are integrated with wearable devices for delivering distance learning content in a collaborative manner in e.g.,special education,surgical training. Gaining unauthorized access to these connected devices can cause security, privacy attacks (SP) that adversely impacts the user immersive experience (UIX). In this article, we identify potential SP attack surfaces that impact the application usability and immersion experience, and propose a novel anomaly detection method to detect attacks before the UIX can be disrupted. Specifically, we apply: (i) machine learning techniques such as amulti-label KNN classificationalgorithm to detect anomaly events of network-based attacks that include potential threat scenarios ofDoS (packet tampering, packet drop, packet duplication), and (ii) statistical analysis techniques that use a combination of boolean and threshold functions (Z-scores) to detect an anomaly related to application-based attacks (Unauthorized access). We demonstrate the effectiveness of our proposed anomaly detection method using a VRLE application case study viz., vSocial, specifically designed for teaching youth with learning impediments about social cues and interactions. Based on our detection results, we validate the impact of network and application based SP attacks on the VRLE UIX.

Delivering curriculum using desktop-based virtual learning environment (VLE) technologies in a collaborative group setting has been shown to reduce the social skill limitations of students with learning disabilities. However, the lack of the immersiveness and effective generalization of acquiring knowledge and skills among students remains a critical challenge in the interactive tools used in current VLEs. In this paper, we present a networked social virtual reality learning environment (VRLE) system viz., vSocial that has been redesigned based on iterative user feedback and developed in order to leverage the latest advances in integration of smart devices such as VR headsets for virtual content delivery. We describe a comparative study to evaluate technology trade-offs in the development process of transitioning from a VLE to a VRLE, from both technological and user (e.g., student/instructor) perspectives. Lastly, we outline open issues in using VRLEs which include: system complexity, emotion recognition, cybersickness and system sustainability.

Social Virtual Reality based Learning Environments (VRLEs) such as vSocial render instructional content in a three-dimensional immersive computer experience for training youth with learning impediments. There are limited prior works that explored attack vulnerability in VR technology, and hence there is a need for systematic frameworks to quantify risks corresponding to security, privacy, and safety (SPS) threats. The SPS threats can adversely impact the educational user experience and hinder delivery of VRLE content. In this paper, we propose a novel risk assessment framework that utilizes attack trees to calculate a risk score for varied VRLE threats with rate and duration of threats as inputs. We compare the impact of a well-constructed attack tree with an adhoc attack tree to study the trade-offs between overheads in managing attack trees, and the cost of risk mitigation when vulnerabilities are identified. We use a vSocial VRLE testbed in a case study to showcase the effectiveness of our framework and demonstrate how a suitable attack tree formalism can result in a more safer, privacy-preserving and secure VRLE system.