Detecting defects buried inside chips is critical for failure analysis in semiconductor manufacturing. In this paper, we perform 3D voxel segmentation on 2.5D semicon chips to locate and identify defects that may be present in them. We integrate tree based Ensemble method with the Cascaded Anisotropic Convolutional Neural Networks to employ component structural characteristics of semicon 3D object in voxel segmentation process. We fabricate custom 2.5D chips purposely creating defective regions by using a specific fabrication and assembling process. Thereafter, use commercial 3D XRM tools for 3D imaging of these chips. We perform accurate 3D Object localization for each 3D x-ray scan by using a slice and fuse approach. Then, we perform voxel segmentation on logic die (integral component of semicon chip) to detect Cu-pillar, solder, and void regions (if any). The results show that we achieve state-of-the-art voxel segmentation dice scores for all three sub-components.

3D object recognition is a well studied 2D multi-view object classification task that achieves high accuracy if the object textures are distinctive. However, if objects are texture-less and are only differentiable by their shapes but at certain viewpoints. Thus, the problem is still very challenging. Furthermore, the existing methods are mostly based on supervised learning with lots of images per object which are difficult to collect and label them for training. In this letter, we introduced a multi-loss view invariant stochastic prototype embedding to minimize and improve the recognition accuracy of novel objects at different viewpoints by using a progressive multi-view learning approach. An extensive experimental results show that the proposed method outperforms the state-of-the-art methods on different types datasets and also on different backbones.

Defect segmentation is critical in real-wold industrial product quality assessment. There are usually a huge number of normal (defect-free) images but a very limited number of annotated anomalous images. This poses huge challenges to exploiting Fully-Convolutional Networks (FCN), e.g., UNet, as they require sufficient anomalous images with defect annotations during training. To further leverage the information from normal data, a novel CAM-guided U-Net with adversarial regularization (CAM-UNet-AR) is proposed. We first modify the existing CAM-UNet to incorporate the CAMs for both normal and anomalous classes and fine-tune the segmentation network using a combined loss which jointly considers pixel-wise classification, foreground segmentation and boundary segmentation. Secondly, an auxiliary adversarial regularization module (ARM) is proposed to facilitate the segmentation network to encode the “normal components” from training images into consistent representations. Extensive experiments on MVTec AD dataset show the superiority of our proposed network over multiple state-of-the-art U-Net variants.

This paper tackles the task of defect segmentation by exploiting sufficient normal (defect-free) training images and limited annotated anomalous images. We propose a class activation map guided UNet (CAM-UNet) with feedback refinement mechanism for accurate defect segmentation. We first modify and pretrain the encoder of a VGG-16 backboned UNet to classify normal and anomalous training images. Then, for each of the anomalous training images, a CAM is generated as the prior segmentation information. Based on the CAM, we propose a feedback refinement process to train two decoder networks to progressively improve the segmentation output. Extensive experiments conducted on MVTEC AD dataset show that the proposed method significantly outperforms multiple benchmarking UNet methods in terms of mean IOU.

This paper demonstrates the use of 3D Anisotropic Convolutional Neural Network (CNN) with predict-refine mechanism for 3D brain tumor segmentation. We propose two networks that utilize multi-scale feedback and saliency maps respectively to segment three critical regions involved in automated brain tumor segmentation. The proposed networks are formulated to predict feature maps at different resolutions during the prediction phase. These networks perform refinement process using the saliency or feature maps as feedback information for the refinement process. The recurrent architecture allows the network to automatically rectify errors in saliency map of the previous prediction phase resulting in more reliable final predictions. Our experimental results on the BraTS2017 dataset demonstrate the superior performance of our proposed predict-refine architecture than current state of the art approaches improving results by up to 8% without any additional increase in the 1.9M model parameters.

An important problem in artificial intelligence is to develop an efficient system that can adapt to new knowledge in an incremental manner without forgetting previously learned knowledge. Although Convolutional Neural Networks (CNNs) are good at learning strong classifier and discriminative features, CNNs can not perform well in incremental classifier learning due to the catastrophic forgetting problem in the retraining process. In this paper, we propose a novel yet extremely simple approach to enhance the discriminative property of features for incremental classifier learning. We build a network for the universal feature space in which a group of image classes have intra-class compactness and inter-class separability. And, we model each incremental class to have a maximum margin from the rest of the models in universal space. Experiments are conducted on CIFAR-100 dataset and IMage Database for Context Aware Advertisement (IMDB-CAA) we collected. The results demonstrate the superiority of our approach, improving performance on CIFAR-100 dataset over state-of-the-art incremental learning systems. Furthermore, experiments on few-short incremental learning setting show very promising performance although we use only 4% of training samples on CIFAR-100 dataset.

Technology development in wearable sensors and biosignal processing has made it possible to detect human stress from the physiological features. However, the intersubject difference in stress responses presents a major challenge for reliable and accurate stress estimation. This research proposes a novel cluster-based analysis method to measure perceived stress using physiological signals, which accounts for the intersubject differences. The physiological data are collected when human subjects undergo a series of task-rest cycles, incurring varying levels of stress that is indicated by an index of the State Trait Anxiety Inventory. Next, a quantitative measurement of stress is developed by analyzing the physiological features in two steps: 1) a k -means clustering process to divide subjects into different categories (clusters), and 2) cluster-wise stress evaluation using the general regression neural network. Experimental results show a significant improvement in evaluation accuracy as compared to traditional methods without clustering. The proposed method is useful in developing intelligent, personalized products for human stress management.

This paper presents a method for detecting cognitive load levels from speech. When speech is modulated by different lev-els of cognitive load, acoustic characteristics of speech change. In this paper, we measure acoustic distance of a stressed ut-terance from the baseline stress free speech using GMM-SVM kernel with Bhattacharyya based GMM distance. In addition, it is believed that airflow structure of speech production is non-linear. This motivates us to investigate better techniques to cap-ture nonlinear characteristic of stress information in acoustic features. Inspired by the recent success of neural networks for representation learning, we employ a single hidden layer feed forward network with non-linear activation to extract the fea-ture vectors. Furthermore, people have different reactions to a particular task load. This inter-speaker difference in stress re-sponses presents a major challenge for stress level detection. We use a bootstrapped training process to learn the stress re-sponse of a particular speaker. We perform experiments using data sets from Cognitive Load with Speech and EGG (CLSE) provided for the Cognitive Load Sub-Challenge of the INTER-SPEECH 2014 Computational Paralinguistics Challenge. The results show that the system with our proposed strategies per-forms well on validation and test sets. Index Terms: cognitive load, GMM-supervector, neural net features

Speech is one of the most important signals that can be used to detect human emotions. When speech is modulated by different emotions, spectral distribution of speech is changed accordingly. A Gaussian Mixture Model(GMM) can model the changes in spectral distributions effectively. A GMM-supervector characterizes the spectral distribution of an emotion utterance by the GMM parameters such as the mean vectors and covariance matrices. In this paper, we propose to use the GMM-supervectors that characterize the emotional spectral dissimilarity measure for emotion classification. We employ the GMM-SVM kernel with Bhattacharyya based GMM distance to obtain dissimilarity measure. Beside the first-order statistics of mean, we consider dissimilarity measure using second-order statistics of covariance which describe the shape of the distribution. Experiments are conducted using SVM classifier to classify emotions of anger, happiness, neutral and sadness. We achieve average accuracy of 78.14% for speaker independent emotion classification.

This paper presents a design strategy for the speaker diarization system in the IIR submissions to the 2007 and 2009 NIST Rich Transcription Meeting Recognition Evaluations (RT07 and RT09) for the multiple distant microphone (MDM) condition. The system features two algorithms supporting two important steps in a diarization process. The first step is Initial Segmentation and Clustering (ISC), and the second one is cluster merging and purification. In the ISC step, we propose a histogram quantization and clustering technique based on time delay of arrival (TDOA) features by analyzing the correlation among the signals across multiple distant microphones. In the cluster merging and purification step, we further merge the speaker clusters using a Bayesian information criterion (BIC) to consolidate the clusters to arrive at one-cluster-per-speaker. The two steps work in tandem to form an integral process. We propose a novel Consensus Based Cluster Purification (CBCP) method that involves a technique to remove impure speaker segments in the speaker clusters before speaker modeling in the cluster purification process. The system reports a state-of-the-art performance of speaker diarization for RT07 and RT09 MDM condition with 7.47% and 8.77% Diarization error rates (DERs), respectively, for both overlapping and non-overlapping speech.

Speech is one of the most important signals that can be used to detect human emotions. Speech is modulated by different emotions by varying frequencyand energy-related acoustic parameters such as pitch, energy and formants. In this paper, we describe research on analyzing interand intra-subband energy variations to differentiate five emotions. The emotions considered are anger, fear, dislike, sadness, and neutral. We employ a Two-Layered Cascaded Subband (TLCS) filter to study the energy variations for extraction of acoustic features. Experiments were conducted on the Berlin Emotional Data Corpus (BEDC). We achieve average accuracy of 76.4% and 69.3% for speakerdependent and -independent emotion classifications, respectively.

In this paper, a voice conversion system that converts spoken vowels into singing vowels is proposed. Given the spoken vowels and their musical score, the system generates singing vowels. The system modifies the speech parameters of Fundamental frequency (F0), duration and spectral properties to produce singing voice. F0 contour is obtained using F0 fluctuation information from training singing voice and music score. Duration of each vowel of speech is stretched or shortened according to the length of the corresponding musical note. To transform speech spectrum to singing spectrum the following two approaches are employed. The first method employs spectral mean shifting and variance scaling method. And, the second approach uses weighted linear transformation method to transform speech to singing spectrum. The system is tested on the database including 75 speech and 30 singing voices sung using vowels. The results show that the proposed system is able to convert spoken vowels into singing vowels with a quality very close to the target singing voice.

This paper describes speaker diarization system on a NIST Rich Transcription 2007 (RT-07) meeting recognition evaluation data set for the task of multiple distant microphone (MDM). Our implementation includes three components: initial clustering, non-speech removal and cluster purification. Initial clusters are generated using directional of arrival (DOA) information and bootstrap clustering. Multiple GMM modeling for speech/non-speech classification is employed for non-speech removal component. In addition, a novel system fusion strategy using information from receiver operating curve (ROC) is proposed for non-speech removal component. Finally, consensus clustering approach together with iterative GMM clustering method is employed for speaker cluster purification. The system achieves the overall DER of 10.81%.

This paper describes a speaker diarization system in 2007 NIST Rich Transcription (RT07) Meeting Recognition Evaluation for the task of Multiple Distant Microphone (MDM) in meeting room scenarios. The system includes three major modules: data preparation, initial speaker clustering and cluster purification/merging. The data preparation consists of the raw data Wiener filtering and beamforming, Time Difference of Arrival estimate and speech activity detection. Based on the initial processed data, two-stage histogram quantization has been used to perform the initial speaker clustering. A modified purification strategy via high-order GMM clustering method is proposed. BIC criterion is applied for cluster merging. The system achieves a competitive overall DER of 8.31% for RT07 MDM speaker diarization task.

We propose a co-training algorithm to detect the singing voice segments from the pop songs. Co-training algorithm leverages compatible and partially uncorrelated information across different features to effectively boost the model from unlabeled data. We adopt this technique to take advantage of abundant unlabeled songs and explore the use of different acoustic features including vibrato, harmonic, attack-decay and MFCC (Mel Frequency Cepstral Coefficients). The proposed algorithm substantially reduces the amount of manual labeling work and computational cost. The experiments are conducted on the database of 94 pop solo songs. We achieve an average error rate of 17% in segment level singing voice detection.

Timbre is the quality of sound which allows the ear to distinguish between musical sounds. In this paper, we study timbre effects in identification of singing voice segments in popular songs. Firstly, we identify between singing voice and instrumental segments in a song. Then, singing voice segments are further categorized according to their singer identity. Timbre-motivated effects are formulated by fusion of systems that use the features from vibrato, harmonic information and other features extracted using Mel and Log frequency scale filter banks. Statistical methods to select singing voice segments with high confidence measure are proposed for better performance in singer identification process. The experiments conducted on a database of 214 popular songs show that the proposed approach is effective.

We present LyricAlly, a prototype that automatically aligns acoustic musical signals with their corresponding textual lyrics, in a manner similar to manually-aligned karaoke. We tackle this problem based on a multimodal approach, using an appropriate pairing of audio and text processing to create the resulting prototype. LyricAlly's acoustic signal processing uses standard audio features but constrained and informed by the musical nature of the signal. The resulting detected hierarchical rhythm structure is utilized in singing voice detection and chorus detection to produce results of higher accuracy and lower computational costs than their respective baselines. Text processing is employed to approximate the length of the sung passages from the lyrics. Results show an average error of less than one bar for per-line alignment of the lyrics on a test bed of 20 songs (sampled from CD audio and carefully selected for variety). We perform a comprehensive set of system-wide and per-component tests and discuss their results. We conclude by outlining steps for further development.

Perceptual features are motivated by human perception of sounds. In this paper, several perceptually-motivated features such as harmonic, vibrato and timbre are studied to detect singing voice segments in a song. In addition, singing formant and attack-decay envelope of the sound are also studied for acoustic feature formulation. The cepstral coefficients which reflect the timbre characteristics are formulated by combining information from harmonic content, vibrato, singing formant and attack-decay envelope of the sound. Bandpass filters that spread according to the octave frequency scale are used to extract vibrato and harmonic information. Several experiments are conducted using a database that includes 84 popular songs from commercially available CD recordings. The experiments show that the proposed feature formulation methods are effective.

Vibrato is a slightly tremulous effect imparted to vocal or instrumental tone for added warmth and expressiveness through slight variation in pitch. It corresponds to a periodic fluctuation of the fundamental frequency. It is common for a singer to develop a vibrato function to personalize his/her singing style. In this paper, we explore the acoustic features that reflect vibrato information in order to identify singers of popular music. We start with an enhanced vocal detection method that allows us to select vocal segments with high confidence. From the selected vocal segments, the cepstral coefficients which reflect the vibrato characteristics are computed. These coefficients are derived using bandpass filters, such as parabolic and cascaded bandpass filters, spread according to the octave frequency scale. The strategy of our classifier formulation is to utilize the high level musical knowledge of song structure in singer modeling. Singer identification is validated on a database containing 84 popular songs from commercially available CD recordings from 12 singers. We achieve an average error rate of 16.2% in segment level identification

It is commonthat a singer develops a vibrato to personalize his/her singing style. In this paper, we exxplore the acoustic features that reflect vibrato information, to identify singers of population music. We start with an enhanced vocal detection method that allows us to select vocal segments with high confidence. From the selected vocal segments, the cepstral coefficients which reflect the vibrato characteristics are computed. These coefficients are derived using cascaded bandpass filters spread according to the octave frequency scale. We empoly the high level musical knowledge of song structure in singer modeling. Singer identification is validated on a database containing 84 popular songs incommercially availbale CD records from 12 singers. We achieve an average error rate of 16.2% in segment level idintification.

Abstract This paper proposes an approach to automatically evaluate the prosody of Chinese Mandarin speech for language learning. In this approach, we grade the appropriateness of prosody of speech units according to a model speech corpus from a teacher’s voice. To this end, we build two models, which are the prosody model and the scoring model. The prosody model that is built from the teacher’s speech predicts the reference prosody for the learning text. The scoring model compares the student’s prosody with the reference prosody and gives a prosody rating score. Both the prosody model and the scoring model are built using regression tree. To make the two prosodies comparable, we transform the student’s prosody into the teacher’s prosody space. To build the scoring model, we derive from the corpus a reference data set, in which prosody rating is associated with prosody parameters. During speech evaluation, the student’s prosody is first transformed into the teacher’s prosody space and then evaluated by the scoring model. Experiments show that our model works well for speech of new speakers.

It is common for us to define audio types according to human perception instead of audio spectral properties. In this paper, we analyse the spectral properties of audio types and propose the acoustic features based on spectral properties and harmonic enhancement, to classify audio. By analyzing the spectral properties of sound types, a multi-model HMM is proposed to integrate the primitive spectral properties in statistical modeling. To validate the approach, we build a classifier to segment audio streams into speech, commercials, environmental sound, physical violence and silence in multiple steps. It is shown that the proposed approach outperforms conventional methods. Experimental evaluations on 20 audio tracks of the TRECVID broadcast news database have shown the effectiveness of the proposed approach.

We propose a novel technique for the automatic classification of vocal and non-vocal regions in an acoustic musical signal. Our technique uses a combination of harmonic content attenuation using higher level musical knowledge of key followed by sub-band energy processing to obtain features from the musical audio signal. We employ a Multi-Model Hidden Markov Model (MM-HMM) classifier for vocal and non-vocal classification that utilizes song structure information to create multiple models as opposed to conventional HMM training methods that employ only one model for each class. A statistical hypothesis testing approach followed by an automatic bootstrapping process is employed to further improve the accuracy of classification. An experimental evaluation on a database of 20 popular songs shows the validity of the proposed approach with an average classification accuracy of 86.7%.

We present a prototype that automatically aligns acoustic musical signals with their corresponding textual lyrics, in a manner similar to manually-aligned karaoke. We tackle this problem using a multimodal approach, where the appropriate pairing of audio and text processing helps create a more accurate system. Our audio processing technique uses a combination of top-down and bottom-up approaches, combining the strength of low-level audio features and high-level musical knowledge to determine the hierarchical rhythm structure, singing voice and chorus sections in the musical audio. Text processing is also employed to approximate the length of the sung passages using the textual lyrics. Results show an average error of less than one bar for per-line alignment of the lyrics on a test bed of 20 songs (sampled from CD audio and carefully selected for variety). We perform holistic and per-component testing and analysis and outline steps for further development.

Three systems for the classification of stress in speech are proposed. The first system makes use of linear short time log frequency power coefficients (LFPC), the second employs a Teager energy operator (TEO) based nonlinear frequency domain LFPC features (NFD-LFPC) and the third uses TEO based nonlinear time domain LFPC features (NTD-LFPC). The systems were tested using the SUSAS (speech under simulated and actual stress) database to categorize five stress conditions individually. Results show that the system using LFPC gives the highest accuracy, followed by the system using NFD-LFPC features, while the system using NTD-LFPC features gives the worst performance. For the system using linear LFPC features, average accuracy of 84% and best accuracy of 95% were obtained in classifying five stress categories.