In general, educational support with Large Language Models (LLMs) faces challenges in knowledge organization, expertise integration, and contextual adaptation. So, we present EduMAS, a novel multi-agent framework that coordinates specialized agents with graph-based knowledge navigation. Our framework introduces three key innovations: (1) Specialized Agents that provide expertise in different learning aspects to solve decomposed subtasks professionally; (2) Graph Navigator for graph-based knowledge extraction and selection to improve the quality of responses; (3) The Emotional Awareness mechanism for better contextual adaptation. Through comprehensive experiments on college-level physics education and evaluated by six state-of-the-art LLMs, EduMAS demonstrates significant improvements over the baseline model in complex concept integration, cross-disciplinary understanding, and theory-to-application translation. Ablation studies further validate the contribution of each framework component, Specialized Agents and Graph Navigator play important roles in performance improvement. Our work provides strong support for LLM-powered multi-agent system in AI-assisted education.

E-commerce click-stream data and product catalogs offer critical user behavior insights and product knowledge. This paper propose a multi-modal transformer termed as PINCER, that leverages the above data sources to transform initial user queries into pseudo-product representations. By tapping into these external data sources, our model can infer users’ potential purchase intent from their limited queries and capture query relevant product features. We demonstrate our model’s superior performance over state-of-the-art alternatives on e-commerce online retrieval in both controlled and real-world experiments. Our ablation studies confirm that the proposed transformer architecture and integrated learning strategies enable the mining of key data sources to infer purchase intent, extract product features, and enhance the transformation pipeline from queries to more accurate pseudo-product representations.

Climate change profoundly impacts human health, ecosystems, and food and water security. This review provides a comprehensive analysis of climate change's effects on human health using data science methods, including statistical and machine learning models, to assess current knowledge. It identifies critical gaps in data collection and methodological limitations across different regions and climates. Key recommendations include improving data resolution and duration to enhance model accuracy, as well as fostering collaboration between data scientists, public health experts, and climate specialists. The review underscores the need for innovative, data-driven models to better predict and address the health impacts of climate change.

As the COVID-19 pandemic has subsided, it remains crucial to analyze the vast amount of research produced during this period to advance our understanding of effective medical treatments and therapies. The wealth of biomedical literature generated during the pandemic presents a challenge for researchers who need to quickly assimilate new information. To tackle this issue, we introduce Tree-BERT, a novel representation learning language model designed for the automatic extraction of biomedical relationships from medical publications. Unlike traditional BERT models and their variants, which solely focus on the contextual relationship between words in input sentences, Tree-BERT also incorporates sentence structure information, enhancing word embeddings. Our experiments reveal that Tree-BERT outperforms alternative methods in biomedical relationship extraction. We will also demonstrate the model’s ability to identify new relationships from Covid-19 publications, showcasing its potential to contribute to the ongoing analysis of pandemic research and its broader applications in the field of bio-medicine.

Food deserts are geographic areas where people have limited access to healthy and affordable food. Millions of people in the US are experiencing adverse effects stemming from living in such areas and it becomes an important challenge for federal and local authorities to improve the quality of life of people living in food deserts. The majority of studies in this field explicitly consider income level when analyzing problems of food deserts, thus shadowing other attributes that might provide deeper insights into the problems of the food desert’s population. The current paper assesses the significance of subtler, non-income related characteristics of census tracts by using a new variable attribution technique Appley which approximates Shapley values in linear time. The results show that attributes of census tracts that fall into the set of the so called "social determinants of health" have similar classification power compared to models that explicitly use income variables. The results suggest that each food desert has its own unique set of problems with social determinants of health with varying levels of intensities of each problem and this allows to clearly distinguish not only food deserts from non-food desert areas but also distinguish food deserts from each other. Appley attributes localized importance scores to food deserts’ social determinants of health, thus showing which issues are more severe in a given food desert compared to another one. This allows for devising food desert-specific solutions to the health issues of desert residents thus letting authorities improve the quality of life of people in a targeted and budget-efficient manner.

We have seen complex deep learning models out-performing human benchmarks in many areas (e.g. computer vision, natural language processing). Clever architectures and higher model complexity are two of the major drivers of such outstanding performances. Higher model complexity generally makes the decision-making process of a model opaque to human perception. But understanding the decision-making process is very important for many reasons including enhancing trust in the model’s prediction, improving model robustness, gaining actionable insight from why a model made a particular prediction and discovering new knowledge about a problem. Model explainability has been an active area of research for some time now, but the problem is still far from being solved. An established way of model explanation is to assign a score to each variable (also known as variable attribution), which represents the importance of the variable in a particular prediction of a model. In a lot of techniques, the scoring process involves distributing the output to each variable. This is challenging when the model is complex and consists of a high degree of interaction terms. A coalition game theoretic approach called Shapley Value provides a fair way to tackle the challenge. However, the growth of computation time of the exact Shapley Values is exponential in the number of variables. Hence, many attribution techniques use approximations instead of the exact Shapley Value as attribution. In this manuscript, we propose a novel variable attribution technique called Appley (short for Approximate Shapley) by approximating the Shapley Values in linear time. Moreover, we show that the “Appley” attributions are generally closer to the exact Shapley Values than an existing well-known and comparable attribution technique.

In modern society, epistemic uncertainty limits trust in financial relationships, necessitating transparency and accountability mechanisms for both consumers and lenders. One upshot is that credit risk assessments must be explainable to the consumer. In the United States regulatory milieu, this entails both the identification of key factors in a decision and the provision of consistent actions that would improve standing. The traditionally accepted approach to explainable credit risk modeling involves generating scores with Generalized Linear Models (GLMs) - usually logistic regression, calculating the contribution of each predictor to the total points lost from the theoretical maximum, and generating reason codes based on the 4 or 5 most impactful predictors. The industry standard approach is not directly applicable to a more expressive and flexible class of nonlinear models known as neural networks. This paper demonstrates that an eXplainable AI (XAI) variable attribution technique known as Integrated Gradients (IG) is a natural generalization of the industry standard to neural networks. We also discuss the unique semantics surrounding implementation details in this nonlinear context. While the primary purpose of this paper is to introduce IG to the credit industry and argue for its establishment as an industry standard, a secondary goal is to familiarize academia with the legislative constraints – including their historical and philosophical roots – and sketch the standard approach in the credit industry since there is a dearth of literature on the topic.

Heart disease has become the leading cause of death worldwide. In this paper a feature based on the Tunable Q Wavelet Transform is proposed to detect the abnormal heart sound. The 2016 PhysioNet / CinC heart audio database is used for developing the model. Machine learning models: Random Forest (RF) and Vector Support Machine (SVM) are used to evaluate the performance of the proposed feature. An average accuracy of 86.92% is achieved on the test data using the SVM classifier. The proposed feature showed better performance compared to the existing time and frequency domain features.

Tracking the concentration of students during online learning offers great benefits. For examples, distracted students can be suggested to do a brief exercise to refresh their brains; or the teacher can be notified when too many students have difficulties on concentration so the class could take a short break. Traditionally, mental states like concentration levels can be analyzed using Electroencephalogram (EEG) or Functional Near-Infrared Spectroscopy (fNIRS). However, methods that utilize these data require specialized equipment which is not feasible to deploy on a large scale. On the other hand, recent breakthroughs in deep learning provide possibilities of scalable solutions to detect concentration levels using only webcam. Leveraging this advancement, we investigate the task of tracking students’ concentration levels during online learning using facial data coupled with deep learning based computer vision technologies. More specifically, we examine the performances of different representations of facial data integrated with various deep architectures to empirically determine a solution balanced between prediction accuracy and time efficiency that is suitable for real-time application. Our experimental study shows that the proposed solution achieves over 91% accuracy while keeping execution time low enough for real-time deployment.

This paper proposes a new deep learning architecture that aligns human poses to be used in an exercise/rehabilitation assistant system. In short, the assistant system aims to provide users with visual feedback for their physical exercises. The feedback is generated by first extracting a user's poses while they are performing an exercise through the video feed of the session. The extracted poses are then overlaid with the correct poses and display for the user to observe and fix their errors. This paper focuses on the task of aligning the user's pose with the correct pose so that they can be overlaid on each other with minimal differences, including scales, locations, and perspectives. We design a new deep architecture to accomplish this task, and show that our methods can effectively reduce alignment errors by 70% on average.

This paper explores a new method for representing temporal information found in videos. Dynamic Mode Decomposition (DMD), a method commonly used to reduce the computational effort for other big-data tasks such as flow calculations, is used in this study to aggregate changes between multiple frames. This is applied to the challenge of human action recognition (HAR) tasks using a Two-Stream architecture. Such an architecture takes two convolutional neural networks (CNNs), one analyzing the spatial data and the other analyzing the temporal data as calculated using DMD. This method is compared against others using two common benchmarks, the UCF-101 dataset and the HMDB-51 dataset, achieving 46.0% accuracy on the UCF-101 and 30.8% on the HMDB-51.

Long Short-Term Memory (LSTM) units are a family of Recurrent Neural Network (RNN) architectures that have proven incredibly effective at learning from sequence data. They are also extremely complex, making them expensive to train and difficult to understand. A recent trend towards simplification has produced the Gated Recurrent Unit (GRU) and the Minimal Gated Unit (MGU), both of which perform as well as the LSTM (or better) on a variety of tasks. The MGU is one of the simplest gated recurrent architectures at the moment. Our study demonstrates that it is possible to radically simplify the MGU without significant loss of performance for some tasks and datasets. For the gun violence data used here, an extraordinarily simple Forget Gate (FG) architecture (as well as many other simplified architectures) performs just as well as an MGU on the given task. While more complex architectures such as the MGU, GRU, or LSTM may be needed in some situations, they are likely overkill for many real-world datasets, and the marginal performance benefit may come with a very large price tag.

Net Promotor Score is an important business measurement process where customers are surveyed and asked to rate their likelihood of recommending the company's products and/or services. In many applications, customers are asked to respond on an 11-point ordinal scale of 0 to 10. In developing the score, the data are reformulated into a labelled 3 class scale (0-6: Detractor, 7-8: Passive and 9-10: Promoter). [1] Many companies that choose to use Net Promoter Score as a core management metric integrate the measurement into all phases of the company and seek every opportunity to assess company performance in terms of likelihood to promote the company. In addition to a variety of survey opportunities, the ability to score comments in survey, social media and blogs with promoter rating may provide an additional valuable source of business insight. Even on a three-point scale, Net Promoter is an ordinal classification problem. A number of successful algorithms, that develop ordinal classifiers have been developed. [2] None of the top performing classifiers can be used for applications like text classification or image classification, since they don't employ deep learning. Any appropriate strategy must utilize the ordering information of classes without imposing a strong continuous assumption or fixed spacing assumption on the ordinal classes. In this paper, we use a novel Deep Learning methodology called OHPLnet (Ordinal Hyperplane Loss Network) that is specifically designed for data with ordinal classes. [3] The algorithm is used to develop predictions of the eleven classes, that may be used in the standard Net Promoter Score generation process.

Stock price movement is typically affected by a lot of hidden factors. Predicting stock price direction, especially short-term direction, is very challenging and consistently attracts researches. Deep recurrent neural networks, such as Long Short-Term Memory, typically outperform statistical time series models and traditional machine learning approaches with their mechanisms of learning to vectorize historical information. However, encoding entire history into a vector may unavoidably causes information loss regardless of memory learning and updating mechanisms, especially for those tasks where decisions need to be made on the current time point and similar historical time points are of great references to the decision making. In this paper, we propose a new deep architecture called Recurrent Embedding Kernel (REK) that can learn to make optimal decisions by referring to the entire history instead of just current memory vectors. Experimental results on multiple stock ETFs with different long-term trends show that REK outperforms RNN, LSTM, and GRU, on predicting daily price direction.

Speech impairment is a disability which affects an individuals ability to communicate using speech and hearing. People who are affected by this use other media of communication such as sign language. Although sign language is ubiquitous in recent times, there remains a challenge for non-sign language speakers to communicate with sign language speakers or signers. With recent advances in deep learning and computer vision there has been promising progress in the fields of motion and gesture recognition using deep learning and computer vision based techniques. The focus of this work is to create a visionbased application which offers sign language translation to text thus aiding communication between signers and non-signers. The proposed model takes video sequences and extracts temporal and spatial features from them. We then use Inception, a CNN (Convolutional Neural Network) for recognizing spatial features. We then use a RNN (Recurrent Neural Network) to train on temporal features. The dataset used is the American Sign Language Dataset.

The k Nearest Neighbors (KNN) algorithm has been widely applied in various supervised learning tasks due to its simplicity and effectiveness. However, the quality of KNN decision making is directly affected by the quality of the neighborhoods in the modeling space. Efforts have been made to map data to a better feature space either implicitly with kernel functions, or explicitly through learning linear or nonlinear transformations. However, all these methods use pre-determined distance or similarity functions, which may limit their learning capacity. In this paper, we propose a novel deep learning architecture, which is called the Deep Similarity-Enhanced K Nearest Neighbors (DSE-KNN), to learn an optimized similarity function of the data directly towards the goal of optimizing the KNN decision making. In other words, the type of similarity function that is used in our method is not pre-determined but rather learned to map data to a high-dimensional feature space where the accuracy of the KNN decision making is maximized. Experimental results show that DSE-KNN outperforms other common machine learning methods on classifying different types of disease datasets and predicting daily price direction of different stock ETFs.

The problem of ordinal classification occurs in a large and growing number of areas. Some of the most common source and applications of ordinal data include rating scales, medical classification scales, socio-economic scales, meaningful groupings of continuous data, facial emotional intensity, facial age estimation, etc. The problem of predicting ordinal classes is typically addressed by either performing n-1 binary classification for n ordinal classes or treating ordinal classes as continuous values for regression. However, the first strategy doesn't fully utilize the ordering information of classes and the second strategy imposes a strong continuous assumption to ordinal classes. In this paper, we propose a novel loss function called Ordinal Hyperplane Loss (OHPL) that is particularly designed for data with ordinal classes. The proposal of OHPL is a significant advancement in predicting ordinal class data, since it enables deep learning techniques to be applied to the ordinal classification problem on both structured and unstructured data. By minimizing OHPL, a deep neural network learns to map data to an optimal space where the distance between points and their class centroids are minimized while a nontrivial ordinal relationship among classes are maintained. Experimental results show that deep neural network with OHPL not only outperforms the state-of-the-art alternatives on classification accuracy but also scales well to large ordinal classification problems.

We present a system that automatically discovers important events that have significantly driven people's sentiment changes towards a target using Twitter data (i.e. tweets). This system can also provide the time, importance, and description of events that are associated with people's sentiment changes. In this system, a sentiment classifier is used as the sensor to detect the time points of those changes. It is also used as the filter to effectively eliminate a considerable amount of noisy information and select the most informative tweets to be further analyzed for event descriptions. Discovered events are described from the following aspects, 1) the most important tweets ranked by tweet-based TextRank algorithm, 2) the topics generated by the nonnegative matrix factorization, and 3) the most important keywords generated by word-based TextRank algorithm. Compared with traditional event discovery techniques, the experimental results show that this system can effectively discover important patterns from tweets and unveil 3Ws of an event (i.e. what happens, when it happens, what its effect is), which provides good reference on understanding behavior changes and making strageties. Furthermore, the system was applied to analyze people's sentiment changes towards the two candidates during the 2016 U.S. presidential election. It can also be applied in other scenarios where people's attitude plays an important role like the brand influence marketing and financial investment markets.

Kernel function implicitly maps data from its original space to a higher dimensional feature space. Kernel based machine learning algorithms are typically applied to data that is not linearly separable in its original space. Although kernel methods are among the most elegant part of machine learning, it is challenging for users to define or select a proper kernel function with optimized parameter settings for their data. In this paper, we propose a novel method called Deep Kernel that can automatically learn a kernel function from data using deep learning. The deep kernel is currently utilized in classification, and dimension reduction and visualization. For the classification task, we evaluate the deep kernel method by comparing its performance with the optimized Gaussian kernels, both using support vector machines as the decision model, on different types of datasets. The experimental results show that the proposed deep kernel method outperforms the traditional methods with Gaussian kernels on most of the data sets. For the dimension reduction and visualization task, the deep kernel is used along with kernel PCA. The results are also compared and contrasted with using the RBF kernel with multiple parameters. The deep kernel is shown to be more powerful in dimension reduction and visualization than the RBF kernel.

This paper studies feasibility and scalable computing processes for visualizing big high dimensional data in a 3 dimensional space by using dimension reduction techniques. More specifically, we propose an unsupervised approach to compute a measure that is called visualizability in a 3 dimensional space for a high dimensional data. This measure of visualizability is computed based on the comparison of the clustering structures of the data before and after dimension reduction. The computation of visualizability requires finding an optimal clustering structure for the given data sets. Therefore, we further implement a scalable approach based on K-Means algorithm for finding an optimal clustering structure for the given big data. Then we can reduce the volume of a given big data for dimension reduction and visualization by sampling the big data based on the discovered clustering structure of the data.

Abstract In this presentation, we propose a novel integrated information retrieval approach that provides a unified solution for two challenging problems in the field of information retrieval. The first problem is how to build an optimal vector space corresponding to users' different information needs when applying the vector space model. The second one is how to smoothly incorporate the advantages of machine learning techniques into the language modeling approach. To solve these problems, we designed the language‐modeling kernel function, which has all the modeling powers provided by language modeling techniques. In addition, for each information need, this kernel function automatically determines an optimal vector space, for which a discriminative learning machine, such as the support vector machine, can be applied to find an optimal decision boundary between relevant and nonrelevant documents. Large‐scale experiments on standard test‐beds show that our approach makes significant improvements over other state‐of‐the‐art information retrieval methods.