Because a freshman-level introductory engineering course is typically one of few courses used in fulfilling a broad set of technical and non-technical program learning outcomes (i.e. for ABET accreditation purposes), a significant challenge involved in designing and teaching such courses is to cover a broad array of topics in a way that is engaging and beneficial to students. We revamped the freshman-level introductory engineering course at CSU-East Bay to better suit this purpose, and in this work, we focused on obtaining a deeper understanding of student learning across a set of six technical and non-technical concepts. We evaluated the relationship between student performance on each concept with the students' final course performance, and found that students who perform well on the technical concepts generally have better final course performance. We also investigated how performance on each concept affects performance on other concepts. We found that there is significant correlation between performances across a couple of concepts, and lack of significant correlation between most other concepts. This suggests that there are concept-specific tools and skillsets allowing students to attain proficiency in each concept, and learning the tools and skillsets for one concept may not benefit performance on other concepts. Students also performed significantly better on the non-technical concepts compared to the technical concepts. Deeper understanding of the difficulties facing the students in the technical concepts would benefit students in future iterations of the course.

This work presents improved audio-based user-verification analysis and results on Flickr videos, using a subset of the MediaEval 2011 [1] data set. User-verification is a new task, where the goal is to determine if two pieces of media are uploaded by the same user. Our best results, with a 19.7% Equal Error Rate, and a 53.9% Miss Rate at 1% False Positive, are obtained using an i-vector [2] system. A frequency-matching system that requires 96% less computation time than the other systems is also explored, and may be better suited for processing large datasets from Flickr and other social networks. The results have significant privacy implications as they present a framework for exploiting users' tendencies to assume that different accounts remain as separate realms.

Audio-based video event detection (VED) on user-generated content (UGC) aims to find videos that show an observable event such as a wedding ceremony or birthday party rather than a sound, such as music, clapping or singing. The difficulty of video content analysis on UGC lies in the acoustic variability and lack of structure of the data. The UGC task has been explored mainly by computer vision, but can be benefited by the used of audio. The i-vector system is state-of-the-art in Speaker Verification, and is outperforming a conventional Gaussian Mixture Model (GMM)-based approach. The system compensates for undesired acoustic variability and extracts information from the acoustic environment, making it a meaningful choice for detection on UGC. This paper employs the i-vector-based system for audio-based VED on UGC and expands the understanding of the system on the task. It also includes a performance comparison with the conventional GMM-based and state-of-the-art Random Forest (RF)-based systems. The i-vector system aids audio-based event detection by addressing UGC audio characteristics. It outperforms the GMM-based system, and is competitive with the RF-based system in terms of the Missed Detection (MD) rate at 4% and 2.8% False Alarm (FA) rates, and complements the RF-based system by demonstrating slightly improvement in combination over the standalone systems.

Over the recent years, the problem of video location estimation (i.e., estimating the longitude/latitude coordinates of a video without GPS information) has been approached with diverse methods and ideas in the research community and significant improvements have been made. So far, however, systems have only been compared against each other and no systematic study on human performance has been conducted. Based on a human-subject study with 11,900 experiments, this article presents a human baseline for location estimation for different combinations of modalities (audio, audio/video, audio/video/text). Furthermore, this article compares state-of-the-art location estimation systems with the human baseline. Although the overall performance of humans' multimodal video location estimation is better than current machine learning approaches, the difference is quite small: For 41% of the test set, the machine's accuracy was superior to the humans. We present case studies and discuss why machines did better for some videos and not for others. Our analysis suggests new directions and priorities for future work on the improvement of location inference algorithms.

We study how potential attackers can identify accounts on different social network sites that all belong to the same user, exploiting only innocuous activity that inherently comes with posted content. We examine three specific features on Yelp, Flickr, and Twitter: the geo-location attached to a user's posts, the timestamp of posts, and the user's writing style as captured by language models. We show that among these three features the location of posts is the most powerful feature to identify accounts that belong to the same user in different sites. When we combine all three features, the accuracy of identifying Twitter accounts that belong to a set of Flickr users is comparable to that of existing attacks that exploit usernames. Our attack can identify 37% more accounts than using usernames when we instead correlate Yelp and Twitter. Our results have significant privacy implications as they present a novel class of attacks that exploit users' tendency to assume that, if they maintain different personas with different names, the accounts cannot be linked together; whereas we show that the posts themselves can provide enough information to correlate the accounts.

We have performed city-verification of videos based on the videos' audio and metadata, using videos from the MediaEval Placing Task's video set, which contain consumer-produced videos “from-the-wild”. 18 cities were used as targets, for which acoustic and language models were trained, and against which test videos were scored. We have obtained the first known results for the city verification task, with an EER minimum of 21.8%, suggesting that ~80% of test videos, when tested against a correct target city, were identified as belonging to that city. This result is well above-chance, even as the videos contained very few city-specific audio and metadata features. We have also demonstrated the complementarity of audio and metadata for this task.

In this work, we have investigated the performance of 2D Gabor features (known as spectro-temporal features) for speaker recognition. Gabor features have been used mainly for automatic speech recognition (ASR), where they have yielded improvements. We explored different Gabor feature implementations, along with different speaker recognition approaches, on ROSSI [1] and NIST SRE08 databases. Using the noisy ROSSI database, the Gabor features performed as well as the MFCC features standalone, and score-level combination of Gabor and MFCC features resulted in an 8% relative EER improvement over MFCC features standalone. These results demonstrated the value of both spectral and temporal information for feature extraction, and the complementarity of Gabor features to MFCC features.

This paper presents a system for identifying the room in an audio or video recording through the analysis of acoustical properties. The room identification system was tested using a corpus of 13440 reverberant audio samples. With no common content between the training and testing data, an accuracy of 61% for musical signals and 85% for speech signals was achieved. This approach could be applied in a variety of scenarios where knowledge about the acoustical environment is desired, such as location estimation, music recommendation, or emergency response systems.

This article presents an attempt to link the uploaders of videos based on the audio track of the videos. Using a subset of the MediaEval Placing Task's Flickr video set, which is labeled with the uploader's name, we conducted an experiment with a similar setup as a typical NIST speaker recognition evaluation run. Based on the assumption that the audio might be matched in various ways (speaker, channel, environmental noise, etc.), we trained one of ICSI's simplified speaker recognition systems on the audio tracks of the Flickr videos. Note that since the selection of videos is essentially random, the audio track can contain any sounds. We obtain an equal error rate of 36.7% on 312 videos with 11,550 trials. The result has implications for audio research, security applications, and raises privacy concerns.

This paper investigates the usefulness of segmental phoneme-dynamics for classification of speaking styles. We modeled transition details based on the phoneme sequences emitted by a speech recognizer, using data obtained from a recording of 39 depressed patients with 7 different speaking styles- nor-mal, pressured, slurred, stuttered, flat, slow and fast speech. We designed and compared two set of phoneme models: a lan-guage model treating each phoneme as a word unit (one for each style) and a context-dependent phoneme duration model based on Gaussians for each speaking style considered. The experi-ments showed that language modeling at the phoneme level per-formed better than the duration model. We also found that better performance can be obtained by user normalization. To see the complementary effect of the phoneme-based models, the classi-fiers were combined at a decision level with a Hidden Markov Model (HMM) classifier built from spectral features. The im-provement was 5.7 % absolute (10.4 % relative), reaching 60.3% accuracy in 7-class and 71.0 % in 4-class classification. Index Terms: speaking styles, language model, phoneme, du-ration

We propose new data selection approaches based on speaker discriminability features, including kurtosis and a set of nasal-ity features which exploit spectral properties of nasal speech sounds. Data selected based on the speaker discriminability fea-tures are used to implement end-to-end speaker recognition sys-tems, which produce significant improvements when combined with the baseline system (which uses the speech-only data re-gions determined by a speech/non-speech detector), where the optimal combination of systems produces roughly a 24 % im-provement over the baseline. Results suggest that focusing the modeling power on data regions selected via the kurtosis and nasality speaker discriminability features, part of which are of-ten discarded in the speech/non-speech detection process, can improvement speaker recognition. Index Terms: speaker recognition, kurtosis, nasality features, data selection

We improve upon our measures relating feature vector distri-butions to speaker recognition (SR) performances for perfor-mance prediction and potential arbitrary data selection for SR, as described in [1]. In particular, we examine the means and variances of 11 features pertaining to nasality (each of which is denoted as a measure), computing them on feature vectors of phones to determine which measures give good SR performance prediction of phones. We’ve found that the combination of nasality measures give a 0.917 correlation with the Equal Error Rates (EERs) of phones on SRE08, exceeding the correlation of our previous best measure (mutual information) by 12.7%. When implemented in our data-selection scheme (which does not require a SR system to be run), the nasality measures allow us to select data with combined EER better than data selected via running a SR system in certain cases, at a fortieth of the computational costs. The nasality measures also require a tenth of the computational costs to compute compared to our previous best measure. Index Terms: Text-dependent speaker recognition, data selec-tion, nasality measures, relevance, redundancy

We’ve examined the speaker discriminative power of mel-, antimel- and linear-frequency cepstral coefficients (MFCCs, a-MFCCs and LFCCs) in the nasal, vowel, and non-nasal con-sonant speech regions. Our inspiration came from the work of Lu and Dang in 2007, who showed that filterbank energies at some frequencies mainly outside the telephone bandwidth pos-sess more speaker discriminative power due to physiological characteristics of speakers, and derived a set of cepstral coeffi-cients that outperformed MFCCs in non-telephone speech. Us-ing telephone speech, we’ve discovered that LFCCs gave 21.5% and 15.0 % relative EER improvements over MFCCs in nasal and non-nasal consonant regions, agreeing with our filterbank energy f-ratio analysis. We’ve also found that using only the vowel region with MFCCs gives a 9.1 % relative improvement over using all speech. Last, we’ve shown that a-MFCCs are valuable in combination, contributing to a system with 17.3% relative improvement over our baseline.

We extend the state-of-the-art by applying word-conditioning to constrain phone N-gram features used in speaker recognition. Feature-level combination of 52 word unigrams constraining phone N-grams of order 1, 2, and 3 proved to be the best approach. Our system achieves 18% and 27% improvements compared to a non word-conditioned phone N-grams system on SRE05 and SRE06, respectively. Furthermore, the system achieves 18% and 37% improvements compared to the non word-conditioned phone N-grams system when each system is combined with a GMM-based system on SRE05 and SRE06, suggesting that the word-conditioned features are more complementary. On both corpora, this approach achieves a 4.7% EER standalone, and a 3.3% EER in combination with the non word-conditioned phone N-grams and GMM-based systems. Note that the word-conditioning approach utilizes only 43% of SRE05 data.

We improve upon the current Hidden Markov Model (HMM) techniques for speaker recognition by using the means of Gaus-sian mixture components of keyword HMM states in a support vector machine (SVM) classifier. We achieve an 11 % improve-