In this work we investigate the impact of a large-scale self-supervised pretraining strategy for active speaker detection (ASD) on an unlabeled dataset consisting of over 125k hours of YouTube videos. When compared to a baseline trained from scratch on much smaller in-domain labeled datasets we show that with pretraining we not only have a more stable supervised training due to better audio-visual features used for initialization, but also improve the ASD mean average precision by 23% on a challenging dataset collected with Google Nest Hub Max devices capturing real user interactions.

The Berkeley Phonetics Machine is a Linux virtual machine imageproduced and used by the UC Berkeley Phonology Lab asa platform for phonetic research. It contains a full data analysisstack based on Python and R and also specialized tools for phoneticresearch. The machine is designed as a flexible and productiveplatform for established and novel research agendas thatcan be easily shared and reproduced. We list the software availablein the machine, which includes many command-line toolsfor acoustic analysis and media file manipulation, as well asspecialized Python libraries. We also discuss the use of this machinein the Phonology Lab and in phonetics courses. The overallexperience with the machine has been positive, as facultyand graduate students are able to share and execute scripts ina common working environment. Undergraduate students haveless opportunity to master the virtual machine environment butbenefit from simplified instructions and fewer installation andoperating problems. The primary difficulty that we have encounteredhas been with a few underpowered student computersthat cannot run the virutual machine or do not run it well.

VR applications provide an opportunity to study a variety of new applications. One of the focus areas of the media convergence, games and media integration (McGMI) program is to develop new media applications for the visually impaired population. We are particularly interested in developing applications which are at the same time interesting for the sighted population as well-hence the title-crossover applications. Bonnie Snyder, who has been working with the visually impaired population for more than twenty years, visited a group of students early in the Fall 2008 As many typical applications are geared toward sighted population, the cost of software and hardware systems tend to be a lot higher. In addition, several games, developed for primarily the sighted, provide minimal interaction for the blind. Although this issue remains a topic of discussion in both IEEE VR and ISMAR and related conferences, much more can be done. We used this as motivation and developed three applications for both the sighted and the visually impaired population (a) haptic chess program combines PHANTOM force feedback interaction with OpenAL audio; (b) Simple hand movement recognition on iPhone provides a hierarchical menu application; (c) Barnyard fun program uses interesting animal-sound feedback to facilitate spatial selection. In future, we expect to conduct testing of these applications in Denver Museum as possible.

Automatic Speech Attribute Transcription (ASAT), an ITR project sponsored under the NSF grant (IIS-04-27113), is a cross-institute effort involving Georgia Institute of Technology, The Ohio State University, University of California at Berkeley, and Rutgers University. This project approaches speech recognition from a more linguistic perspective: unlike traditional ASR systems, humans detect acoustic and auditory cues, weigh and combine them to form theories, and then process these cognitive hypotheses until linguistically and pragmatically consistent speech understanding is achieved. A major goal of the ASAT paradigm is to develop a detection-based approach to automatic speech recognition (ASR) based on attribute detection and knowledge integration. We report on progress of the ASAT project, present a sharable platform for community collaboration, and highlight areas of potential interdisciplinary ASR research. Index Terms: attributes, events, features, detection, speech recognition, speech attribute transcription, utterance verification

We present a preliminary analysis of transcriber consistency in labeling and segmentation of words and phones in the Buckeye corpus of spontaneous, informal speech. We find that pairwise inter-transcriber agreement on exact phone label match was 76%, and segmentation agreement within 20 % of phone pair length was 75%, though longer phones are more consistently segmented than shorter phones. Patterns of consistency variation in labeling are observed as a function of phonetic categories that are similar to patterns reported for read speech. More agreement is seen on consonants than on vowels, and on fricatives and labials than on other consonant classes. In general, we find that shorter, more reduced words and phones result in more transcriber disagreement. 1.