We investigate which prosodic features matter most in conveying pragmatic functions. We use the problem of predicting human perceptions of pragmatic similarity among utterance pairs to evaluate the utility of prosodic features of different types. We find evidence that the duration-related features are most important, that pitch-related features are much less important and less adequate, and that complete modeling will require additional acoustic and prosodic features, including nasality and phonetic reduction. These findings can guide future basic research in prosody, and suggest how to improve speech synthesis evaluation, among other applications.

The diagnosis and treatment of individuals with communication disorders offers many opportunities for the application of speech technology, but research so far has not adequately considered: the diversity of conditions, the challenges of limited data, and the role of pragmatic deficits. This paper explores how a general-purpose model of perceived pragmatic similarity may overcome these limitations. It shows that a simple model can capture utterance aspects that are relevant to diagnosis of autism and of specific language impairment, outlines how it might support several use cases for clinicians and clients, and analyzes its performance and limitations.

This study investigates how different forms of input elicitation obtained from crowdsourcing can be utilized to improve the quality of inferred labels for image classification tasks, where an image must be labeled as either positive or negative depending on the presence/absence of a specified object. Three types of input elicitation methods are tested: binary classification (positive or negative); level of confidence in binary response (on a scale from 0-100%); and what participants believe the majority of the other participants' binary classification is. We design a crowdsourcing experiment to test the performance of the proposed input elicitation methods and use data from over 200 participants. Various existing voting and machine learning (ML) methods are applied and others developed to make the best use of these inputs. In an effort to assess their performance on classification tasks of varying difficulty, a systematic synthetic image generation process is developed. Each generated image combines items from the MPEG-7 Core Experiment CE-Shape-1 Test Set into a single image using multiple parameters (e.g., density, transparency, etc.) and may or may not contain a target object. The difficulty of these images is validated by the performance of an automated image classification method. Experimental results suggest that more accurate classifications can be achieved when using the average of the self-reported confidence values as an additional attribute for ML algorithms relative to what is achieved with more traditional approaches. Additionally, they demonstrate that other performance metrics of interest, namely reduced false-negative rates, can be prioritized through special modifications of the proposed aggregation methods that leverage the variety of elicited inputs.

Normalization layers have been shown to improve convergence in deep neural networks, and even add useful inductive biases. In many vision applications the local spatial context of the features is important, but most common normalization schemes including Group Normalization (GN), Instance Normalization (IN), and Layer Normalization (LN) normalize over the entire spatial dimension of a feature. This can wash out important signals and degrade performance. For example, in applications that use satellite imagery, input images can be arbitrarily large; consequently, it is nonsensical to normalize over the entire area. Positional Normalization (PN), on the other hand, only normalizes over a single spatial position at a time. A natural compromise is to normalize features by local context, while also taking into account group level information. In this pa- per, we propose Local Context Normalization (LCN): a normalization layer where every feature is normalized based on a window around it and the filters in its group. We propose an algorithmic solution to make LCN efficient for arbitrary window sizes, even if every point in the image has a unique window. LCN outperforms its Batch Normalization (BN), GN, IN, and LN counterparts for object detection, semantic segmentation, and instance segmentation applications in several benchmark datasets, while keeping performance in- dependent of the batch size and facilitating transfer learning.

The accurate prediction of solar X-ray flux is a difficult problem due to noise and miscalibration of sensors, missing data, and the effects of the Earth's position relative to the line of sight from the sensing satellite to the Sun. Most work on this problem has focused on predicting large sudden increases in flux known as solar flares that can have severe detrimental effects on human activities. Since solar flares are relatively rare and have heterogeneous characteristics, it has been difficult to train machine learning models for their prediction. In this work we approach the problem as regression rather than classification, addressing it as a time series prediction problem. We use X-ray data from the Geostationary Operational Environmental Satellite (GOES) gathered at one-minute intervals to predict the flux after time intervals ranging from one to five hours. We evaluate three deep neural network architectures - a convolutional neural network, Long Short-Term Memory, and a hierarchical dense residual network and compare with two conventional baselines. Our experiments show that all three architectures provide better results than the baseline algorithms, with the hierarchical residual network providing the best results.

Going beyond turn-taking models built to solve specific tasks, such as predicting if a user will hold his/her turn after a pause, there is growing interest in more general models for turn taking that subsume many such tasks, and very good results have recently been obtained [1]. Here we present an improved recurrent network model that outperforms [1] and does so without requiring lexical annotation. Further, we show that this model can be trained for different languages with no modifications, providing good results in turn-taking prediction for English, Spanish, Japanese, Mandarin and French. We also show that our model performs well across genres, including task-oriented dialog and general conversation.

We address the problem of automatically fusing hyperspectral data of a digitized scene with an image-based 3D model, overlapping the same scene, in order to associate material spectra with corresponding height information for improved scene understanding. The datasets have been independently collected at different spatial resolutions by different aerial platforms and the georegistration information about the datasets is assumed to be insufficient or unavailable. We propose a method to solve the fusion problem by associating Scale Invariant Feature Transform (SIFT) descriptors from the hyperspectral data with the corresponding 3D point cloud in a large scale 3D model. We find the correspondences efficiently without affecting matching performance by limiting the initial search space to the centroids obtained after performing k-means clustering. Finally, we apply the Optimal Randomized RANdom Sample Consensus (RANSAC) algorithm to enforce geometric alignment of the hyperspectral images onto the 3D model. We present preliminary results that show the effectiveness of the method using two large datasets collected from drone-based sensors in an urban setting.

Volcanic eruptions can be disastrous; it is therefore important to be able to predict them as accurately as possible. Theoretically, we can use the general machine learning techniques for such predictions. However, in general, without any prior information, such methods require an unrealistic amount of computation time. It is therefore desirable to look for additional information that would enable us to speed up the corresponding computations. In this paper, we provide an empirical evidence that the volcanic system exhibit chaotic and delayed character. We also show that in general (and in volcanic predictions in particular), we can speed up the corresponding predictions if we take into account chaotic and delayed character of the corresponding system.

A neural net can learn to discriminate among a set of classes without explicitly training to do so. It does not even need exposure to any instances of those classes. The learning occurs while the net is being trained to discriminate among a set of related classes. This form of transfer learning is referred to as `Latent Learning' by psychologists, because until specifically elicited, the knowledge remains latent. Evidence that latent learning has occurred lies in the existence of consistent, unique responses to the unseen classes. Standard supervised learning can improve the accuracy of those responses with exceedingly small sets of labeled images. In this paper, we use a convolutional neural net (CNN) to demonstrate not only a method of determining a net's latent responses, but also simple ways to optimize latent learning. Additionally, we take advantage of the fact that CNN's are deep nets in order to show how the latently learned accuracy of the CNN may be greatly improved by allowing only its output layer to train. We compare our results both to those obtained with standard backpropagation training of the CNN on small datasets without any transfer learning and to a related set of current published results.

High-quality mammography is the most effective technology presently available for breast cancer screening. High resolution mammograms usually lead to more accurate diagnoses; however, they require large doses of radiation, which may have harmful effects. In this paper, we present a method to synthesize high-resolution mammograms from low-resolution inputs, which offers the potential of allowing accurate diagnoses while minimizing risks to patients. Our algorithm combines statistical machine learning methods and stochastic search to learn the mapping from low-resolution to high-resolution mammograms using a large dataset of training image pairs. Experimental results show that the super-resolution algorithm can generate high-quality, high-resolution breast mammograms from low-resolution input with no human intervention.

Psychologists define latent learning as learning that occurs without task-specific reinforcement and is not demonstrated until needed. Since this knowledge is acquired while mastering some other task(s), it is a form of transfer learning. We utilize latent learning to enable a deep neural net to distinguish among a set of handwritten numerals. The accuracies obtained are compared to those achievable with a simplistic `group-mean' classification technique, which is explained later in this paper. The deep neural net architecture used was a Le-Net 5 convolutional neural net with only minor differences in the output layer.

Today there are solutions for some specific turn-taking problems, but no general model. We show how turn-taking can be reduced to two more general problems, prediction and selection. We also discuss the value of predicting not only future speech/silence but also prosodic features, thereby handing not only turn-taking but “turn-shaping”. To illustrate how such predictions can be made, we trained a neural network predictor. This was adequate to support some specific turn-taking decisions and was modestly accurate overall. Index Terms: dialog system, predictive, prosody, endpointing, back-channeling, time, interaction control, dialog model

In this paper we present a system for multi-pose face detection. Our system presents three main contributions. First, we introduce the use of asymmetric Haar features. Asymmetric Haar features provide a rich feature space, which allows to build classifiers that are accurate and much simpler than those obtained with other features. The second contribution is the use of a genetic algorithm to search efficiently in the extremely large parameter space of potential features. Using this genetic algorithm, we generate a feature set that allows to exploit the expressive advantage of asymmetric Haar features and is small enough to permit exhaustive evaluation. The third contribution is the application of a skin color-segmentation scheme to reduce the search space. Our system uses specialized detectors in different face poses that are built using AdaBoost and the C4.5 rule induction algorithm. Experimental results using the CMU profile test set and BioID frontal faces test set, in addition to our own multi-pose face test set, show that our system is competitive with other systems presented recently in the literature.

Current surveys from modern astronomical observatories contain a huge amount of data; in particular, the Sloan Digital Sky Survey (SDSS) has reached the order of terabytes of data in images and spectra. Such an amount of information needs to be exploited by sophisticated algorithms that automatically analyze the data in order to extract useful knowledge from the mega databases. In this work we employ evolution strategies (ES) to automatically extract a set of physical parameters corresponding to stellar population synthesis (ages, metallicities, reddening and relative contributions) from a sample of galaxy spectra taken from SDSS. Such parameters are useful in cosmological studies and for understanding galaxy formation, composition, and evolution. We pose this parameter extraction as an optimization problem and then solve it using ES. The idea is to reconstruct each galaxy spectrum from the sample by means of a linear combination of three similar theoretical models, each contributing in a different way to the stellar population synthesis. This linear combination produces a model spectrum that is compared with the original spectrum using a simple difference function. The goal is to find a model that minimizes this difference, using ES as the algorithm to explore the parameter space. We present experimental results using a set of 100 spectra from SDSS Data Release 2 that show that ES are very well suited to extract stellar population parameters from galaxy spectra.

In this work we employ Evolution Strategies (ES) to automatically extract a set of physical parameters (ages, metallicities, reddening and contributions) from a sample of galaxy spectra taken from Sloan Digital Sky Survey (SDSS) for stellar populations studies. We pose this parameter extraction as an optimization problem and then solve it using ES. The idea is to reconstruct each galactic spectrum from the sample by means of a linear combination of three different theoretical models of stellar population synthesis. This combination produces a model spectrum that is compared with the original spectrum using a difference function. The goal is to find a model that minimizes this difference, using ES as the algorithm to explore the parameter space.

Using prosodic information to predict when back-channels are appropriate in spontaneous dialogs has become somewhat of a reference problem for automatic discovery techniques. Here we present experiments with two ideas: the use of features derived from randomly generated pitch and energy filters, and the use of instancebased learning, specifically the Locally Weighted Linear Regression (LWLR) algorithm. For the task of predicting possible backchannel locations in Iraqi Arabic [6], we obtain 22 % precision and 51 % recall, which is as good as that obtained using a laboriously developed and hand-tuned rule. 1.

This paper presents an approach based on the evolution strategy for autonomous learning of dextrous manipulation primitives with a dextrous robot hand. We use heuristics derived from observations made on human hands to reduce the degrees of freedom of the task and make learning possible. Our system does not rely on simulation; all the experimentation is performed the 16-degree-of-freedom Utah/MIT hand. We present experimental results that show that accurate dextrous manipulation skills can be learned in a period of a few minutes. We also show the application of the learned primitives to perform an assembly task.

We present an experimental evaluation of adaptive and non-adaptive visual servoing in 3, 6 and 12 degrees of freedom (DOF), comparing it to traditional joint feedback control. While the purpose of experiments in most other work has been to show that the particular algorithm presented indeed also works in practice, we do not focus on the algorithm but rather on properties important to visual servoing in general. Our main results are: positioning of a 6 axis PUMA 762 arm is up to 5 times more precise under visual control than under joint control; positioning of a Utah/MIT dextrous hand is better under visual control than under joint control by a factor of 2; and a trust-region-based adaptive visual feedback controller is very robust. For m tracked visual features the algorithm can successfully estimate online the m/spl times/3 (m/spl ges/3) image Jacobian (J) without any prior information, while carrying out a 3 DOF manipulation task. For 6 and higher DOF manipulation, a rough initial estimate of J is beneficial. We also verified that redundant visual information is valuable. Errors due to imprecise tracking and goal specification were reduced as the number of visual features, m, was increased. Furthermore highly redundant systems allow us to detect outliers in the feature vector and deal with partial occlusion.

We propose a behavioral construct called a "virtual tool" that allows many of the advantages of high-degree-of-freedom robotic systems to be realized without excessive computational costs or modeling requirements. The basic idea of a virtual tool is drawn from the observation that the most efficient way of performing a task is generally to use a device that is specially designed for it; this suggests that a way to use redundant degrees of freedom in a system is to use them to customize it so that it becomes, in essence, a special purpose tool for the task at hand. We call the resulting instantiation a virtual tool. Formally, this tailoring takes the form of implementing constraints on the degrees of freedom of the system through lower-level control processes. Remaining degrees of freedom appear as control parameters of the virtual tool, and can be used by higher-level processes, or by a human teleoperator, to control the system. We present experimental results showing the application of the virtual tool approach to telemanipulation using a 22 degree-of-freedom hand-arm system. Our chosen task is a fairly complex assembly task requiring the robot to pick up a small object, transport it, and insert it into a hole with matching shape. Instead of attempting to map the configuration of the teleoperator's hand-arm system to the robot using a data-glove or similar device, we provide basic parameterized grasping and manipulation primitives (or virtual tools) that can be invoked by the teleoperator as he progresses through the stages of the complex task.

A mathematical explanation as to why fuzzy control is smoother and more stable than control by experts whose experience was used to design the fuzzy control is presented. The analysis indicates that fuzzy control is always continuous and exhibits better performance, even in complicated control situations.>