Automatic speech recognition systems can benefit from cues in user voice such as hyperarticulation. Traditional approaches typically attempt to define and detect an absolute state of hyperarticulation, which is very difficult, especially on short voice queries. We present a novel approach for hyperarticulation detection using pairwise comparisons and demonstrate its application in a real-world speech recognition system. Our approach uses delta features extracted from a pair of repetitive user utterances. Results show significant improvements in WER (word error rate) by using hyperarticulation information as a feature in a second pass N-best hypotheses rescoring setup.

Web search via voice is becoming increasingly popular, taking advantage of recent advances in automatic speech recognition. Speech recognition systems are trained using audio transcripts, which can be generated by a paid annotator listening to some audio and manually transcribing it. This paper considers an alternative source of training data for speech recognition, called implicit transcription. This is based on Web search clicks and reformulations, which can be interpreted as validating or correcting the recognition done during a real Web search. This can give a large amount of free training data that matches the exact characteristics of real incoming voice searches and the implicit transcriptions can better reflect the needs of real users because they come from the user who generated the audio. On an overall basis we demonstrate that the new training data has value in improving speech recognition. We further show that the in-context feedback from real users can allow the speech recognizer to exploit contextual signals, and reduce the recognition error rate further by up to 23%.

“Happy New Year!” At the beginning of 2016, I would like to take this opportunity to wish everyone a very happy, healthy, and prosperous new year! It is my great honor and privilege to serve as the Editor-in-Chief (EiC) of the IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS (TNNLS), and I am excited to write this Editorial to start a new journey with you all.

Voice interactions are becoming more prevalent as the usage of voice search and intelligent assistants gains more popularity. Users frequently reformulate their requests in hope of getting better results either because the system was unable to recognize what they said or because it was able to recognize it but was unable to return the desired response. Query reformulation has been extensively studied in the context of text input. Many of the characteristics studied in the context of text query reformulation are potentially useful for voice query reformulation. However, voice query reformulation has its unique characteristics in terms of the reasons that lead users to reformulating their queries and how they reformulate them. In this paper, we study the problem of voice query reformulation. We perform a large scale human annotation study to collect thousands of labeled instances of voice reformulation and non-reformulation query pairs. We use this data to compare and contrast characteristics of reformulation and non-reformulation queries over a large a number of dimensions. We then train classifiers to distinguish between reformulation and non-reformulation query pairs and to predict the rationale behind reformulation. We demonstrate through experiments with the human labeled data that our classifiers achieve good performance in both tasks.

Voice-activated intelligent assistants, such as Siri, Google Now, and Cortana, are prevalent on mobile devices. However, it is challenging to evaluate them due to the varied and evolving number of tasks supported, e.g., voice command, web search, and chat. Since each task may have its own procedure and a unique form of correct answers, it is expensive to evaluate each task individually. This paper is the first attempt to solve this challenge. We develop consistent and automatic approaches that can evaluate different tasks in voice-activated intelligent assistants. We use implicit feedback from users to predict whether users are satisfied with the intelligent assistant as well as its components, i.e., speech recognition and intent classification. Using this approach, we can potentially evaluate and compare different tasks within and across intelligent assistants ac-cording to the predicted user satisfaction rates. Our approach is characterized by an automatic scheme of categorizing user-system interaction into task-independent dialog actions, e.g., the user is commanding, selecting, or confirming an action. We use the action sequence in a session to predict user satisfaction and the quality of speech recognition and intent classification. We also incorporate other features to further improve our approach, including features derived from previous work on web search satisfaction prediction, and those utilizing acoustic characteristics of voice requests. We evaluate our approach using data collected from a user study. Results show our approach can accurately identify satisfactory and unsatisfactory sessions.

Current speech-input systems typically use a nonspeech threshold for end-of-utterance detection. While usually sufficient for short utterances, the approach can cut speakers off during pauses in more complex utterances. We elicit personal-assistant speech (reminders, calendar entries, messaging, search) using a recognizer with a dramatically increased endpoint threshold, and find frequent nonfinal pauses. A standard endpointer with a 500 ms threshold (latency) results in a 36% cutoff rate for this corpus. Based on the new data, we develop low-cost acoustic features to discriminate nonfinal from final pauses. Features capture periodicity, speaking rate, spectral constancy, duration/intensity, and pitch of prepausal speech - using no speech recognition, speaker or session information. Classification experiments yield 20% EER at a 100 ms latency, thereby reducing both cutoffs and latency compared with the threshold-only baseline. Additional results on computational cost, feature importance, and speaker differences are discussed.

Users frequently interact with web search systems on their mobile devices via multiple modalities, including touch and speech. These interaction modes are substantially different from the user experience on desktop search. As a result, system designers have new challenges and questions around understanding the intent on these platforms. In this paper, we study the query reformulation patterns in mobile logs. We group query reformulations based on their input method into four categories; text-text, text-voice, voice-text and voice-voice. We discuss the unique characteristics of each of these groups by comparing them against each other and desktop logs. We also compare the distribution of reformulation types (e.g. adding/dropping words) against desktop logs and show that there are new classes of reformulations that are caused by errors in speech recognition. Our results suggest that users do not tend to switch between different input types (e.g. voice and text). Voice to text switches are largely caused by speech recognition errors, and text to voice switches are unlikely to be about the same intent.

We consider the task of suggesting related queries to users after they issue their initial query to a web search engine. We propose a machine learning approach to learn the probability that a user may find a follow-up query both useful and relevant, given his initial query. Our approach is based on a machine learning model which enables us to generalize to queries that have never occurred in the logs as well. The model is trained on co-occurrences mined from the search logs, with novel utility and relevance models, and the machine learning step is done without any labeled data by human judges. The learning step allows us to generalize from the past observations and generate query suggestions that are beyond the past co-occurred queries. This brings significant gains in coverage while yielding modest gains in relevance. Both offline (based on human judges) and online (based on millions of user interactions) evaluations demonstrate that our approach significantly outperforms strong baselines.

Categorization of web-search queries in semantically coherent topics is a crucial task to understand the interest trends of search engine users and, therefore, to provide more intelligent personalization services. Query clustering usually relies on lexical and clickthrough data, while the information originating from the user actions in submitting their queries is currently neglected. In particular, the intent that drives users to submit their requests is an important element for meaningful aggregation of queries. We propose a new intent-centric notion of topical query clusters and we define a query clustering technique that differs from existing algorithms in both methodology and nature of the resulting clusters. Our method extracts topics from the query log by merging missions, i.e., activity fragments that express a coherent user intent, on the basis of their topical affinity. Our approach works in a bottom-up way, without any a-priori knowledge of topical categorization, and produces good quality topics compared to state-of-the-art clustering techniques. It can also summarize topically-coherent missions that occur far away from each other, thus enabling a more compact user profiling on a topical basis. Furthermore, such a topical user profiling discriminates the stream of activity of a particular user from the activity of others, with a potential to predict future user search activity.

Assistance technology is undoubtedly one of the important elements in the commercial search engines, and routing the user towards the right direction throughout the search sessions is of great importance for providing a good search experience. Most search assistance methods in the literature that involve query generation, query expansion and other techniques consider each suggestion candidate individually, which implies an independence assumption. We challenge this independence assumption and give a method to maximize the utility of a given set of suggestions. For this, we will define a measure of conditional utility for query pairs using query-URL bipartite graphs based on the session logs (clicked and viewed URLs). Afterwards, we remove the redundant queries from the suggestion set using a greedy algorithm to be able to replace them with more useful ones. Both offline (based on user studies and session log analysis) and online (based on millions of user interactions) evaluations show that modeling the conditional utility and maximizing the utility of the set of queries (by eliminating redundant ones) significantly increases the effectiveness of the search assistance both for the presubmit and postsubmit modes.

Search engines are continuously looking into methods to alleviate users' effort in finding desired information. For this, all major search engines employ query suggestions methods to facilitate effective query formulation and reformulation. Providing high quality query suggestions is a critical task for search engines and so far most research efforts have focused on tapping various information available in search query logs to identify potential suggestions. By relying on this single source of information, suggestion providing systems often restrict themselves to only previously observed query sessions. Therefore, a critical challenge faced by query suggestions provision mechanism is that of coverage, i.e., the number of unique queries for which users are provided with suggestions, while keeping the suggestion quality high. To address this problem, we propose a novel way of generating suggestions for user search queries by moving beyond the dependency on search query logs and providing synthetic suggestions for web search queries. The key challenges in providing synthetic suggestions include identifying important concepts in a query and systematically exploring related concepts while ensuring that the resulting suggestions are relevant to the user query and of high utility. We present an end-to-end system to generate synthetic suggestions that builds upon novel query-level operations and combines information available from various textual sources. We evaluate our suggestion system over a large-scale real-world dataset of query logs and show that our methods increase the coverage of query-suggestion pairs by up to 39% without compromising the quality or the utility of the suggestions.

We provide a novel method of evaluating search results, which allows us to combine existing editorial judgments with the relevance estimates generated by click-based user browsing models. There are evaluation methods in the literature that use clicks and editorial judgments together, but our approach is novel in the sense that it allows us to predict the impact of unseen search models without online tests to collect clicks and without requesting new editorial data, since we are only re-using existing editorial data, and clicks observed for previous result set configurations. Since the user browsing model and the pre-existing editorial data cannot provide relevance estimates for all documents for the selected set of queries, one important challenge is to obtain this performance estimation where there are a lot of ranked documents with missing relevance values. We introduce a query and rank based smoothing to overcome this problem. We show that a hybrid of these smoothing techniques performs better than both query and position based smoothing, and despite the high percentage of missing judgments, the resulting method is significantly correlated (0.74) with DCG values evaluated using fully judged datasets, and approaches inter-annotator agreement. We show that previously published techniques, applicable to frequent queries, degrade when applied to a random sample of queries, with a correlation of only 0.29. While our experiments focus on evaluation using DCG, our method is also applicable to other commonly used metrics.

The reassignment method is a widespread approach for obtaining high resolution time-frequency representations. Nevertheless, its performance is not always optimal and can deteriorate for low signal-to-noise ratio (SNR) values. In order to overcome these obstacles, a novel method for obtaining high resolution time-frequency representations is proposed in this paper. The new method implements proposed nonparametric snakes in order to obtain accurate locations of the signal ridges in the time-frequency domain. The results of numerical analysis show that the proposed method is capable of achieving significantly higher concentration of signals in the time-frequency domain in comparison to the spectrogram and the traditional reassignment method. Furthermore, the new scheme also maintains good performance for low SNR values, while the performance of the other two considered methods significantly diminishes. It is clear from the results that the proposed method might be of significance in applications where accurate estimation of the signal components is required for low SNR values.

System identification with noisy input-output measurements has been dominantly addressed through the optimization of the mean-squared-error criterion (MSE), especially in adaptive filtering. MSE is known to provide models that approximate the conditional expectation of the target output given the input; however, when the input signal is also contaminated by noise - a frequent occurrence - MSE yields biased estimates of the model parameters with the severity of the bias dependent on the noise power. This drawback has been addressed in various ways, including errors-in-variables techniques. Recently, error whitening criterion (EWC) and associated adaptation algorithms were proposed to address this issue in linear system identification. We extend the applicability of the main concept behind EWC to the unbiased identification of order-2 Volterra series models of nonlinear dynamical systems. The extension does not apply to higher order Volterra models. The main contribution of this letter is a statistical criterion that can be utilized to identify analytically the true parameters of an order-2 Volterra model from noisy input-output data. We also support the theoretical results with simulations; however online learning algorithms that can be derived for the proposed criterion will not be addressed.

Given the knowledge of class probability densities, a priori probabilities, and relative risk levels, Bayes classifier provides the optimal minimum-risk decision rule. Specifically, focusing on the two-class (detection) scenario, under certain symmetry assumptions, matched filters provide optimal results for the detection problem. Noticing that the Bayes classifier is in fact a nonlinear projection of the feature vector to a single-dimensional statistic, in this paper, we develop a smooth nonlinear projection filter constrained to the estimated span of class conditional distributions as does the Bayes classifier. The nonlinear projection filter is designed in a reproducing kernel Hilbert space leading to an analytical solution both for the filter and the optimal threshold. The proposed approach is tested on typical detection problems, such as neural spike detection or automatic target detection in synthetic aperture radar (SAR) imagery. Results are compared with linear and kernel discriminant analysis, as well as classification algorithms such as support vector machine, AdaBoost and LogitBoost.

Principal manifolds are essential underlying structures that manifest canonical solutions for significant problems such as data de- noising and dimensionality reduction. The traditional definition of self-consistent manifolds rely on a least-squares construction error approach that utilizes semi-global expectations across hyperplanes orthogonal to the solution. This definition creates various practical difficulties for algorithmic solutions to identify such manifolds, besides the theoretical shortcoming that self-intersecting or nonsmooth manifolds are not acceptable in this framework. We present local conditions for critical and principal manifolds by introducing the concept of subspace local maxima. The conditions generalize the two conditions that characterize stationary points of a function to stationary surfaces. The proposed framework yields a unique set of principal points which can be partitioned into principal curves and manifolds of any intrinsic dimensionality. A subspace-constrained fixed-point algorithm is proposed to determine the principal graph.

One of the most important problems with current time warping algorithms in the literature is sensitivity to noise. To improve the noise robustness of the current algorithms, we propose a denoising step based on the likelihood maximization of the pairwise signals. This approach is independent of the selection of the particular time warping algorithm, and can be coupled with any algorithm in the literature. Improvement in noise robustness not only brings increased robustness to current time warping applications, but also may trigger new application areas where the signals that need to be compared are buried in noise.

We address the problem of similarity metric selection in pairwise affinity clustering. Traditional techniques employ standard algebraic context-independent sample-distance measures, such as the Euclidean distance. More recent context-dependent metric modifications employ the bottleneck principle to develop path-bottleneck or path- average distances and define similarities based on geodesies determined according to these metrics. This paper develops a principled context-adaptive similarity metric for pairs of feature vectors utilizing the probability density of all data. Specifically, based on the postulate that Euclidean distance is the canonical metric for data drawn from a unit-hypercube uniform density, a density-geodesic distance measure stemming from Riemannian geometry of curved surfaces is derived. Comparisons with alternative metrics demonstrate the superior properties such as robustness.

The insufficiency of using only second-order statistics and premise of exploiting higher order statistics of the data has been well understood, and more advanced objectives including higher order statistics, especially those stemming from information theory, such as error entropy minimization, are now being studied and applied in many contexts of machine learning and signal processing. In the adaptive system training context, the main drawback of utilizing output error entropy as compared to correlation-estimation-based second-order statistics is the computational load of the entropy estimation, which is usually obtained via a plug-in kernel estimator. Sample-spacing estimates offer computationally inexpensive entropy estimators; however, resulting estimates are not differentiable, hence, not suitable for gradient-based adaptation. In this brief paper, we propose a nonparametric entropy estimator that captures the desirable properties of both approaches. The resulting estimator yields continuously differentiable estimates with a computational complexity at the order of those of the sample-spacing techniques. The proposed estimator is compared with the kernel density estimation (KDE)-based entropy estimator in the supervised neural network training framework with computation time and performance comparisons.

Principal curves and surfaces play an important role in dimensionality reduction applications of machine learning and signal processing. Vaguely defined, principal curves are smooth curves that pass through the middle of the data distribution. This intuitive definition is ill posed and to this day researchers have struggled with its practical implications. Two main causes of these difficulties are: (i) the desire to build a self-consistent definition using global statistics (for instance conditional expectations), and (ii) not decoupling the definition of the principal curve from the data samples. In this paper, we introduce the concept of principal sets, which are the union of all principal surfaces with a particular dimensionality. The proposed definition of principal surfaces provides rigorous conditions for a point to satisfy that can be evaluated using only the gradient and Hessian of the probability density at the point of interest. Since the definition is decoupled from the data samples, any density estimator could be employed to obtain a probability distribution expression and identify the principal surfaces of the data under this particular model.

We propose a pairwise mutual information based regularization technique for maximum likelihood sensor fusion in dense distributed sensor networks. The principle is demonstrated in target localization and tracking using a dense binary motion sensor network under a centralized data fusion framework. Simulations demonstrate that the information regularization enables the maximum likelihood localization procedure to provide significantly more accurate target position estimates compared to its unregularized counterpart, which is the current benchmark. The extensions of the information regularization principle to various sensor and data fusion problems such as outlier detection, and sensor failure identification are discussed.

Under the assumptions of non-Gaussian, non-stationary, or non-white independent sources, linear blind source separation can be formulated as a generalized eigenvalue decomposition problem. Here we provide an elegant method of doing this online, instead of waiting for a sufficiently large batch of data. This is done through a recursive generalized eigendecomposition algorithm that tracks the optimal solution, which is obtained using all the data observed. The algorithms proposed in this paper follow the well-known recursive least squares (RLS) algorithm in nature.

Active contours are commonly used in many image segmentation applications. There are different active contour definitions, but all active contour definitions in the literature use parametric forms to determine the shape priors or adjust the weighting of internal and external forces acting on the active contour. However, the evaluation or estimation of the optimal values of these parameters is impossible in a general sense, and the algorithms are run with different parameters until a satisfactory result is obtained. To get rid of this exhaustive parameter search, we approach the same problem in a nonparametric way to translate the problem of seeking good values of these unknown parameters into seeking for a good density estimate. We tested the proposed method and compared with earlier approaches and obtained better results.

Ischemic stroke is the most prevalent catastrophic disease of the brain. Various animal models have been used to study the disease. The majority of the models are based on induction of focal ischemic cerebral necrosis, followed by exhaustive morphometric analysis of the tissues. Despite recent advances in machine learning and image processing, neurological damage evaluations are still based on tedious manual or semi-automatic segmentation of brain images. We demonstrate a method that uses active contours combined with a kernel annealing approach to automatically segment the brain organs of interest, as well as a simple feature that highlights the contrast between normal and infarct brain tissue for automated analysis. The automated segmentation and analysis solution will be useful for increasing the productivity of experimentation and removing investigator bias from the data analysis.

Active contours, or so-called snakes, require some parameters to determine the form of the external force or to adjust the tradeoff between the internal forces and the external forces acting on the active contour. However, the optimal values of these parameters cannot be easily identified in a general sense. The usual way to find these required parameters is to run the algorithm several times for a different set of parameters, until a satisfactory performance is obtained. Our nonparametric formulation translates the problem of seeking these unknown parameters into the problem of seeking a good edge probability density estimate. Density estimation is a well-researched field, and our nonparametric formulation allows using well-known concepts of density estimation to get rid of the exhaustive parameter search. Indeed, with the use of kernel density estimation these parameters can be defined locally, whereas, in the original snake approach, all the shape parameters are defined globally. We tested the proposed method on synthetic and real images and obtained comparatively better results.

The typical cut is a clustering method that is based on the probability pnmthat points xnand xmare in the same cluster over all possible partitions (under the Boltzmann distribution for the mincut cost function). We present two contributions regarding this algorithm. (1) We show that, given a kernel density estimate of the data, minimising the overlap between cluster densities is equivalent to the mincut criterion. This gives a principled way to determine what affinities and scales to use in the typical-cut algorithm, and more generally in clustering and dimensionality reduction algorithms based on pair-wise affinities. (2) We introduce an iterated version of the typical-cut algorithm, where the estimated pnmare used to refine the affinities. We show this procedure is equivalent to finding stationary points of a certain objective function over clusterings; and that at the stationary points the value of pnmis 1 if n and m are in the same cluster and a small value otherwise. Thus, the iterated typical-cut algorithm sharpens the pnmmatrix and makes the cluster structure more obvious.

Segmentation is a fundamental problem in image processing having a wide range of applications. Image segmentation algorithms in the literature range from a cost criterion based optimization techniques to various heuristic methods. In this paper, we propose utilizing mean shift spectral clustering for perceptually better image segmentation results

Feature selection is a critical step for pattern recognition and many other applications. Typically, feature selection strategies can be categorized into wrapper and filter approaches. Filter approach has attracted much attention because of its flexibility and computational efficiency. Previously, we have developed an ICA-MI framework for feature selection, in which the Mutual Information (MI) between features and class labels was used as the criterion. However, since this method depends on the linearity assumption, it is not applicable for an arbitrary distribution. In this paper, exploiting the fact that Gaussian Mixture Model (GMM) is generally a suitable tool for estimating probability densities, we propose GMM-MI method for feature ranking and selection. We will discuss the details of GMM-MI algorithm and demonstrate the experimental results. We will also compare the GMM-MI method with the ICA-MI method in terms of performance and computational efficiency.

From a probabilistic perspective, the question of clustering is 'what is the probability that two data samples belong to the same cluster?'' Accepting the natural preclustering of samples into corresponding modes of the data probability distribution and answering the question posed above for these modes can reduce the problem complexity. Under the maximum entropy principle, a Boltzmann distribution model can be employed to evaluate the required mode-connectivity probabilities. An algorithm is developed using kernel density estimation in this framework. Its performance is demonstrated on benchmark datasets.

Determining the optimal subspace projections, which maintains the best representation of the original data, is an important problem in machine learning and pattern recognition. In this paper, we propose a nonparametric nonlinear subspace projection technique that employs kernel density estimation based information theoretic methods and kernel machines, in order to maintain class separability maximally under the Shannon mutual information criterion.