Despite recognizing that Large Language Models (LLMs) can generate inaccurate or unacceptable responses, universities are increasingly making such models available to their students.Existing university policies defer the responsibility of checking for correctness and appropriateness of LLM responses to students and assume that they will have the required knowledge and skills to do so on their own.In this work, we conducted a series of user studies with students (N=47) from a large North American public research university to understand if and how they critically engage with LLMs.Our participants evaluated an LLM provided by the university in a quasi-experimental setup; first by themselves, and then with a scaffolded design probe that guided them through an end-user auditing exercise.Qualitative analysis of participant think-aloud and LLM interaction data showed that students without basic AI literacy skills struggle to conceptualize and evaluate LLM biases on their own.However, they transition to focused thinking and purposeful interactions when provided with structured guidance.We highlight areas where current university policies may fall short and offer policy and design recommendations to better support students.

Ensuring that Machine Learning (ML) models make correct and meaningful inferences is necessary for the broader adoption of such models into high-stakes decision-making scenarios. Thus, ML model engineers increasingly use eXplainable AI (XAI) tools to investigate the capabilities and limitations of their ML models before deployment. However, explaining sequential ML models, which make a series of decisions at each timestep, remains challenging. We present Visual Interactive Model Explorer (VIME), an XAI toolbox that enables ML model engineers to explain decisions of sequential models in different “what-if” scenarios. Our evaluation with 14 ML experts, who investigated two existing sequential ML models using VIME and a baseline XAI toolbox to explore “what-if” scenarios, showed that VIME made it easier to identify and explain instances when the models made wrong decisions compared to the baseline. Our work informs the design of future interactive XAI mechanisms for evaluating sequential ML-based decision support systems.

Computational models that formalize complex human behaviors enable study and understanding of such behaviors. However, collecting behavior data required to estimate the parameters of such models is often tedious and resource intensive. Thus, estimating dataset size as part of data collection planning (also known as Sample Size Determination) is important to reduce the time and effort of behavior data collection while maintaining an accurate estimate of model parameters. In this article, we present a sample size determination method based on Uncertainty Quantification (UQ) for a specific Inverse Reinforcement Learning (IRL) model of human behavior, in two cases: (1)pre-hocexperiment design—conducted in the planning stage before any data is collected, to guide the estimation of how many samples to collect; and (2)post-hocdataset analysis—performed after data is collected, to decide if the existing dataset has sufficient samples and whether more data is needed. We validate our approach in experiments with a realistic model of behaviors of people with Multiple Sclerosis (MS) and illustrate how to pick a reasonable sample size target. Our work enables model designers to perform a deeper, principled investigation of the effects of dataset size on IRL model parameters.

Action recognition is a very widely explored research area in computer vision and related fields. We propose Kinematics Posture Feature (KPF) extraction from 3D joint positions based on skeleton data for improving the performance of action recognition. In this approach, we consider the skeleton 3D joints as kinematics sensors. We propose Linear Joint Position Feature (LJPF) and Angular Joint Position Feature (AJPF) based on 3D linear joint positions and angles between bone segments. We then combine these two kinematics features for each video frame for each action to create the KPF feature sets. These feature sets encode the variation of motion in the temporal domain as if each body joint represents kinematics position and orientation sensors. In the next stage, we process the extracted KPF feature descriptor by using a low pass filter, and segment them by using sliding windows with optimized length. This concept resembles the approach of processing kinematics sensor data. From the segmented windows, we compute the Position-based Statistical Feature (PSF). These features consist of temporal domain statistical features (e.g., mean, standard deviation, variance, etc.). These statistical features encode the variation of postures (i.e., joint positions and angles) across the video frames. For performing classification, we explore Support Vector Machine (Linear), RNN, CNNRNN, and ConvRNN model. The proposed PSF feature sets demonstrate prominent performance in both statistical machine learning- and deep learning-based models. For evaluation, we explore five benchmark datasets namely UTKinect-Action3D, Kinect Activity Recognition Dataset (KARD), MSR 3D Action Pairs, Florence 3D, and Office Activity Dataset (OAD). To prevent overfitting, we consider the leave-one-subject-out framework as the experimental setup and perform 10-fold cross-validation. Our approach outperforms several existing methods in these benchmark datasets and achieves very promising classification performance.

Smartphone sensor-based activity recognition seeks broad, high-level knowledge about human behaviors from multitudes of low-level sensor readings, and makes considerable headway in healthcare domain. Our primary contribution is to study the effective pre-processing technique and the extraction of robust features for the classification of sensor data for human activity recognition (HAR). In the pre-processing stages, we investigated multiple filtering parameters for reducing waveform delay, smartphone orientation constraint by introducing magnitude and jerk-based features, and optimum window length for analyzing the trade-off between model performance and latency. Besides, we proposed a feature named "Average Height" that summarizes the average peak to trough distance of the activity and encodes any change of motion for classification. We also proposed two feature selection techniques for offline and real-time faster activity recognition, and analyzed the impact of different feature sets on classifying different activities. Moreover, after performing the classification with optimized hyperparameters, we proposed a Static Postural Transition-based Post-Processing (SPTPP) technique. This post-processing approach analyzes the existence of postural transition from previous window activity to current window activity, and helps to improve the model output by analyzing the posture change. The impact of our proposed techniques are demonstrated on three benchmark datasets named HASC, HAR, and HAPT, where we obtained the state-of-the-art results. We used HASC dataset for optimizing model parameters in different stages, and explored HAR and HAPT datasets as test-beds to verify our optimizations and postprocessing technique.

Recognition of daily human activities in various locomotion and transportation modes has numerous applications like coaching users for behavior modification and maintaining a healthy lifestyle. Besides, applications and user interfaces aware of user mobility through their smartphones can also aid in urban transportation planning, smart parking, and vehicular traffic monitoring. In this paper, we explored smartphone sensor-based two benchmark datasets (Sussex Huawei Locomotion (SHL) and Transportation Mode Detection (TMD)). Firstly, we demonstrated preprocesssing of sensor data, window length optimization based on Akaike Information Criteria (AIC), and introduced smartphone orientation independent features. We also provided an in-depth analysis of different smartphone sensors' importance for classifying daily activities and transportation modes. We justified the sensor relevance by showing the variation of performances with the number of sensors explored. For refining classifier predictions, we also proposed a post-processing approach named "Mode technique". This method primarily concentrates on the statistical analysis of transportation modes and improves the activity recognition rate in statistical classifiers: Decision Tree, K-Nearest Neighbors, Linear Discriminant Analysis, Logistic Regression, Support Vectors Machine with RBF kernel, Random Forest, and deep learning-based methods: Artificial Neural Network and Recurrent Neural Network by smoothing the outputs of these classifiers. Besides, we showed the use of magnitude and jerk-based features to overcome the overfitting problem due to smartphone orientation. We obtained 97.2% accuracy in the SHL dataset and 99.13% accuracy in the TMD dataset. These results demonstrate that our approach can profoundly recognize various activities in advanced locomotion and transportation modes compared to existing methods in two large-scale datasets.