Although the prognosis of lower-grade glioma (LGG) patients is better than others, outcomes are highly heterogeneous. Isocitrate dehydrogenase (IDH) mutation and 1p/19q codeletion status can identify patient subsets with different prognosis. However, in the era of precision medicine, there is still a lack of biomarkers that can accurately predict the individual prognosis of each patient. In this study, we found that most DNA damage response (DDR) genes were aberrantly expressed in LGG patients and were associated with their prognosis. Consequently, we developed an artificial neural network (ANN) model based on DDR genes to predict outcomes of LGG glioma patients. Then, we validated the predictive ability in an independent external dataset and found that the concordance indexes and area under time-dependent receiver operating characteristic curves of the predict index (PI) calculated based on the model were superior to those of the mutation markers. Subgroup analyses demonstrated that the model could accurately identify patients with the same mutation status but different prognosis. Moreover, the model can also identify patients with favorable prognostic mutation status but poor prognosis or vice versa. Finally, we also found that the PI was associated with the mutation status and with the altered immune microenvironment. These results demonstrated that the ANN model can accurately predict outcomes of LGG patients and will contribute to individualized therapies. In addition, a web-based application program for the model was developed.

This paper investigates the use of multidistribution deep neural networks (DNNs) for mispronunciation detection and diagnosis (MDD), to circumvent the difficulties encountered in an existing approach based on extended recognition networks (ERNs). The ERNs leverage existing automatic speech recognition technology by constraining the search space via including the likely phonetic error patterns of the target words in addition to the canonical transcriptions. MDDs are achieved by comparing the recognized transcriptions with the canonical ones. Although this approach performs reasonably well, it has the following issues: 1) Learning the error patterns of the target words to generate the ERNs remains a challenging task. Phones or phone errors missing from the ERNs cannot be recognized even if we have well-trained acoustic models; and 2) acoustic models and phonological rules are trained independently, and hence, contextual information is lost. To address these issues, we propose an acoustic-graphemic-phonemic model (AGPM) using a multidistribution DNN, whose input features include acoustic features, as well as corresponding graphemes and canonical transcriptions (encoded as binary vectors). The AGPM can implicitly model both grapheme-to-likely-pronunciation and phoneme-to-likely-pronunciation conversions, which are integrated into acoustic modeling. With the AGPM, we develop a unified MDD framework, which works much like free-phone recognition. Experiments show that our method achieves a phone error rate (PER) of 11.1%. The false rejection rate (FRR), false acceptance rate (FAR), and diagnostic error rate (DER) for MDD are 4.6%, 30.5%, and 13.5%, respectively. It outperforms the ERN approach using DNNs as acoustic models, whose PER, FRR, FAR, and DER are 16.8%, 11.0%, 43.6%, and 32.3%, respectively.

This paper presents a two-pass framework with discriminative acoustic modeling for mispronunciation detection and diagnoses (MD&D). The first pass of mispronunciation detection does not require explicit phonetic error pattern modeling. The framework instantiates a set of antiphones and a filler model to augment the original phone model for each canonical phone. This guarantees full coverage of all possible error patterns while maximally exploiting the phonetic information derived from the text prompt. The antiphones can be used to detect substitutions. The filler model can detect insertions, and phone skips are allowed to detect deletions. As such, there is no prior assumption on the possible error patterns that can occur. The second pass of mispronunciation diagnosis expands the detected insertions and substitutions into phone networks, and another recognition pass attempts to reveal the phonetic identities of the detected mispronunciation errors. Discriminative training (DT) is applied respectively to the acoustic models of the mispronunciation detection pass and the mispronunciation diagnosis pass. DT effectively separates the acoustic models of the canonical phones and the antiphones. Overall, with DT in both passes of MD&D, the error rate is reduced by 40.4% relative, compared with the maximum likelihood baseline. After DT, the error rates of the respective passes are also lower than those of a strong single-pass baseline with DT by 1.3% and 5.1% relative which are statistically significant.

Generation of corrective feedback carries significant pedagogical importance in the design of computer-aided pronunciation training systems. Such feedback generation should take into account the severity of detected mispronunciations, in order to prioritize different kinds of corrections to be conveyed to the learner. However, mispronunciation gradation is highly dependent on the phonetic context and acoustic context of the word pronunciation, as well as human perception. We have defined several categories of mispronunciation gradation, ranging from subtle to salient, and collected crowdsourced ratings from a large number of listeners. This work aims to capture the phonetic context of word mispronunciation by phonological rules, which are then augmented with statistical scoring to quantitatively model mispronunciation gradations. The model can thus be used to generate gradation ratings of word mispronunciations, especially those that are previously unseen in the training set. We will report the results of automatic gradation classification, as well as its correlation(s) with human perception.

Deep belief network (DBN) has been shown to be a good generative model in tasks such as hand-written digit image generation. Previous work on DBN in the speech community mainly focuses on using the generatively pre-trained DBN to initialize a discriminative model for better acoustic modeling in speech recognition (SR). To fully utilize its generative nature, we propose to model the speech parameters including spectrum and F0 simultaneously and generate these parameters from DBN for speech synthesis. Compared with the predominant HMM-based approach, objective evaluation shows that the spectrum generated from DBN has less distortion. Subjective results also confirm the advantage of the spectrum from DBN, and the overall quality is comparable to that of context-independent HMM.

This paper investigates lexical stress detection for L2 English speech using Deep Belief Networks (DBNs). The features of the DBN used in this work include the syllable-based prosodic features (assumed to have Gaussian distribution) and their expected lexical stress (assumed to have Bernoulli distribution). As stressed syllables are more prominent than their neighbors, the two preceding and two following syllables are taken into consideration. Experimental results show that the DBN achieves an accuracy of about 80 % in syllable stress classification (primary/secondary/no stress) for words with three or more syllables. It outperforms the conventional Gaussian Mixture Model and our previous Prominence Model by an absolute accuracy of about 8 % and 4%, respectively.

This paper investigates acoustic modeling using the hybrid DBN-HMM framework in mispronunciation detection and diagnosis of L2 English. This is one of the first efforts that compare the performance of DBN-HMM with that of the best-tuned GMM-HMM trained in ML and MWE on the same set of features. Previous work in ASR has also shown the necessity of unsupervised pre-training for DBNs to work well. We explore further the effect of training our ASR engine in an unsupervised manner with additional unannotated L2 data from the test speakers. This is compared with the original ASR that has been trained with annotated data in a supervised manner. Experiments show that DBN-HMM can give significant improvement (between 13-18 % relative in word pronunciation error rate) but is computationally more expensive. Index Terms: mispronunciation detection and diagnosis, restricted boltzmann machine, deep belief network 1.

We propose a novel framework for determining the conceptual difficulty of a domain-specific text document without using any external lexicon. Conceptual difficulty relates to finding the reading difficulty of domain-specific documents. Previous approaches to tackling domain-specific readability problem have heavily relied upon an external lexicon, which limits the scalability to other domains. Our model can be readily applied in domain-specific vertical search engines to re-rank documents according to their conceptual difficulty. We develop an unsupervised and principled approach for computing a term's conceptual difficulty in the latent space. Our approach also considers transitions between the segments generated in sequence. It performs better than the current state-of-the-art comparative methods.

We investigate the use of joint-sequence multigrams to generate L2 mispronunciation lexicons for mispronunciation detection and diagnosis. In the joint-sequence framework, a pair of parallel strings (namely, the input string of either graphemes or phonemes of the canonical pronunciation and the phonetic string of the mispronunciation) are aligned to form joint units for probabilistic estimation. We compare results on lexicons produced by phoneme-to-mispronunciation conversion and those by grapheme-to-mispronunciation conversion. Results reflect the hypothesized advantage (1.1 % reduction in expected miss rate) in unifying phonetic confusion due to L1 negative transfer with those due to grapheme-to-phoneme errors. The impact of mispronunciation by mis-use of analogy is also studied. Recognition results show the benefit of a lexicon with proper priors. Index Terms: mispronunciation detection and diagnosis, lexicon extension, joint-sequence multigrams

In this study, we propose a discriminative training algorithm to jointly minimize mispronunciation detection errors (i.e., false rejections and false acceptances) and diagnosis errors (i.e., correctly pinpointing mispronunciations but incorrectly stating how they are wrong). An optimization procedure, similar to Minimum Word Error (MWE) discriminative training, is developed to refine the ML-trained HMMs. The errors to be minimized are obtained by comparing transcribed training utterances (including mispronunciations) with Extended Recognition Networks [3] which contain both canonical pronunciations and explicitly modeled mispronunciations. The ERN is compiled by handcrafted rules, or data-driven rules. Several conclusions can be drawn from the experiments: (1) data-driven rules are more effective than hand-crafted ones in capturing mispronunciations; (2) compared with the ML training baseline, discriminative training can reduce false rejections and diagnostic errors, though false acceptances increase slightly due to a small number of false-acceptance samples in the training set.

In this paper, we propose an HMM trajectory-guided, real image sample concatenation approach to photo-real talking head synthesis. It renders a smooth and natural video of articulators in sync with given speech signals. An audio-visual database is used to train a statistical Hidden Markov Model (HMM) of lips movement first and the trained model is then used to generate a visual parameter trajectory of lips movement for given speech signals, all in the maximum likelihood sense. The HMM generated trajectory is then used as a guide to select, in the original training database, an optimal sequence of mouth images which are then stitched back to a background head video. The whole procedure is fully automatic and data driven. With an audio/video footage as short as 20 minutes from a speaker, the proposed system can synthesize a highly photo-real video in sync with the given speech signals. This system won the FIRST place in the Audio-Visual match contest in LIPS2009 Challenge, which was perceptually evaluated by recruited human subjects. Index Terms: visual speech synthesis, photo-real, talking head, trajectory-guided