Many obstacles stand in the way of modular, extensible code. Some language constructs, such as pattern matching, are not easily extensible. Inherited code may not be type safe in the presence of extended types. The burden of setting up design patterns can discourage users, and parameter clutter can make the code less readable. Given these challenges, it is no wonder that extensibility often gives way to code duplication. We present our solution: Persimmon, a functional system with nested family polymorphism, extensible variant types, and extensible pattern matching. Most constructs in our language are built-in "extensibility hooks," cutting down on the parameter clutter and user burden associated with extensible code. Persimmon preserves the relationships between nested families upon inheritance, enabling extensibility at a large scale. Since nested family polymorphism can express composable extensions, Persimmon supports mixins via an encoding. We show how Persimmon can be compiled into a functional language without extensible variants with our translation to Scala. Finally, we show that our system is sound by proving the properties of progress and preservation.

Soil moisture plays important roles in the land-atmosphere interaction, rainfall–runoff process, climate and weather forecast, and crop yield. An accurate estimation of soil moisture creates opportunities to improve the understanding of soil moisture-related processes in atmospheric sciences, hydrology, and agriculture. Traditional methods to measure soil moisture mainly rely on in-situ measurements which are usually costly and reliant on the use of field instrumentation and point data. A robust remote sensing technique is highly needed to frequently monitor the soil moisture content over a large region or landscape. This paper presents a metric learning-based regression algorithm to learn the mapping function from the input hyperspectral data to the output soil moisture. In particular, the proposed soil moisture estimation method employs the Gaussian process to model the joint distribution of spectral bands over functions. It further incorporates the learning of distance function to find feature subspace for improving soil estimation performance. Experimental results on a real dataset show that the proposed metric learning-based soil moisture estimation method can significantly improve the performance compared to the state-of-the-art soil moisture estimation methods using principle component analysis for feature reduction.

Online video-based job interviews are becoming very popular in the screening of potential employees. In this study, we collected a corpus of 1891 monologue job interview videos (63 hours in duration) from 260 online workers. These videos were annotated for personality traits and hiring recommendation score by experts from a major assessment company. We proposed a unified method of automatic analysis that consists of using clustering to convert continuous audio/video analysis output to discrete pseudoword documents, and then applying modern text classification methods to process speech content, prosody and facial expressions. Our experiments showed that using what the interviewees say (i.e., spoken text), we can predict their personality traits such as openness, conscientiousness, extraversion, agreeableness, and emotional stability with an F-measure of 0.8 or better, while we get an F-measure of 0.6 in predicting hiring recommendation score. Prosody and facial expressions added limited usefulness on interview judgments and need further investigation.

As the popularity of video-based job interviews rises, so does the need for automated tools to evaluate interview performance. Real world hiring decisions are based on assessments of knowledge and skills as well as holistic judgments of person-job fit. While previous research on automated scoring of interview videos shows promise, it lacks coverage of monologue-style responses to structured interview (SI) questions and content-focused interview rating. We report the development of a standardized video interview protocol as well as human rating rubrics focusing on verbal content, personality, and holistic judgment. A novel feature extraction method using ``visual words" automatically learned from video analysis outputs and the Doc2Vec paradigm is proposed. Our promising experimental results suggest that this novel method provides effective representations for the automated scoring of interview videos.

Public speaking, an important type of oral communication, is critical to success in both learning and career development. However, there is a lack of tools to efficiently and economically evaluate presenters' verbal and nonverbal behaviors. The recent advancements in automated scoring and multimodal sensing technologies may address this issue. We report a study on the development of an automated scoring model for public speaking performance using multimodal cues. A multimodal presentation corpus containing 14 subjects' 56 presentations has been recorded using a Microsoft Kinect depth camera. Task design, rubric development, and human rating were conducted according to standards in educational assessment. A rich set of multimodal features has been extracted from head poses, eye gazes, facial expressions, motion traces, speech signal, and transcripts. The model building experiment shows that jointly using both lexical/speech and visual features achieves more accurate scoring, which suggests the feasibility of using multimodal technologies in the assessment of public speaking skills.

Body language plays an important role in learning processes and communication. For example, communication research produced evidence that mathematical knowledge can be embodied in gestures made by teachers and students. Likewise, body postures and gestures are also utilized by speakers in oral presentations to convey ideas and important messages. Consequently, capturing and analyzing non-verbal behaviors is an important aspect in multimodal learning analytics (MLA) research. With regard to sensing capabilities, the introduction of depth sensors such as the Microsoft Kinect has greatly facilitated research and development in this area. However, the rapid advancement in hardware and software capabilities is not always in sync with the expanding set of features reported in the literature. For example, though Anvil is a widely used state-of-the-art annotation and visualization toolkit for motion traces, its motion recording component based on OpenNI is outdated. As part of our research in developing multimodal educational assessments, we began an effort to develop and standardize algorithms for purposes of multimodal feature extraction and creating automated scoring models. This paper provides an overview of relevant work in multimodal research on educational tasks, and proceeds to summarize our work using multimodal sensors in developing assessments of communication skills, with attention on the use of depth sensors. Specifically, we focus on the task of public speaking assessment using Microsoft Kinect. Additionally, we introduce an open-source Python package for computing expressive body language features from Kinect motion data, which we hope will benefit the MLA research community.

We analyze how fusing features obtained from different multimodal data streams such as speech, face, body movement and emotion tracks can be applied to the scoring of multimodal presentations. We compute both time-aggregated and time-series based features from these data streams--the former being statistical functionals and other cumulative features computed over the entire time series, while the latter, dubbed histograms of cooccurrences, capture how different prototypical body posture or facial configurations co-occur within different time-lags of each other over the evolution of the multimodal, multivariate time series. We examine the relative utility of these features, along with curated speech stream features in predicting human-rated scores of multiple aspects of presentation proficiency. We find that different modalities are useful in predicting different aspects, even outperforming a naive human inter-rater agreement baseline for a subset of the aspects analyzed.

We present a technique for automated assessment of public speaking and presentation proficiency based on the analysis of concurrently recorded speech and motion capture data. With respect to Kinect motion capture data, we examine both timeaggregated as well as time-series based features. While the former is based on statistical functionals of body-part position and/or velocity computed over the entire series, the latter feature set, dubbed histograms of cooccurrences, captures how often different broad postural configurations co-occur within different time lags of each other over the evolution of the multimodal time series. We examine the relative utility of these features, along with curated features derived from the speech stream, in predicting human-rated scores of different aspects of public speaking and presentation proficiency. We further show that these features outperform the human inter-rater agreement baseline for a subset of the analyzed aspects.

Eye tracking is a useful tool to understand students' cognitive process during problem solving. This paper offers a unique perspective by applying techniques from social network analysis to eye movement patterns in mathematics problem solving. We construct and visualize transition networks using eye-tracking data collected from 37 8th grade students while solving linear function problems. By applying network analysis on the constructed transition networks, we find general transition patterns between areas of interest (AOIs) for all students, and we also compare patterns for high- and low-performing students. Our results show that even though students share general transition patterns during problem solving, high-performing students made more strategic transitions among AOI triples than low-performing students.

Summary form only given. Control and automation has gone through tremendous developments for the past few decades. After briefly reviewing the history of control this talk will present some Magnus challenges to control. In particular, this talk will take model based fuzzy control as an example to share a personal perspective on developments and challenges in this chosen topic.

Traditional assessments of public speaking skills rely on human scoring. We report an initial study on the development of an automated scoring model for public speaking performances using multimodal technologies. Task design, rubric development, and human rating were conducted according to standards in educational assessment. An initial corpus of 17 speakers with 4 speaking tasks was collected using audio, video, and 3D motion capturing devices. A scoring model based on basic features in the speech content, speech delivery, and hand, body, and head movements significantly predicts human rating, suggesting the feasibility of using multimodal technologies in the assessment of public speaking skills.