The communication of non-speech information (NSI) in closed captioning is essential in providing full access and increased enjoyment of video content, particularly for d/Deaf or Hard-of-Hearing (DHH) viewers. We identified the limitations and frustrations of current NSI captioning through needfinding interviews and then employed a critical design framework to develop a medium-fidelity audio-reactive animated overlay prototype to explore the opinions and values of DHH users regarding NSI communication through surveys and interviews. The results show that current NSI captioning strategies lack consistency across platforms, adequate temporal information, clarity in emotional conveyance, and lack customization options. The study suggests that novel sound communication technologies show promise in enhancing certain aspects NSI communication and that there is a strong desire to move beyond the current, inflexible format of single-track closed captions for NSI communication.

The inclusion of voice persona in synthesized voice can be significant in a broad range of human-computer-interaction (HCI) applications, including augmentative and assistive communication (AAC), artistic performance, and design of virtual agents. We propose a framework to imbue compelling and contextually-dependent expression within a synthesized voice by introducing the role of the vocal persona within a synthesis system. In this framework, the resultant ‘tone of voice’ is defined as a point existing within a continuous, contextually-dependent probability space that is traversable by the user of the voice. We also present initial findings of a thematic analysis of 10 interviews with vocal studies and performance experts to further understand the role of the vocal persona within a natural communication ecology. The themes identified are then used to inform the design of the aforementioned framework.

In this paper, we propose a method for removing linguistic information from speech for the purpose of isolating paralinguistic indicators of affect. The immediate utility of this method lies in clinical tests of sensitivity to vocal affect that are not confounded by language, which is impaired in a variety of clinical populations. The method is based on simultaneous recordings of speech audio and electroglotto-graphic (EGG) signals. The speech audio signal is used to estimate the average vocal tract filter response and amplitude envelop. The EGG signal supplies a direct correlate of voice source activity that is mostly independent of phonetic articulation. These signals are used to create a third signal designed to capture as much paralinguistic information from the vocal production system as possible-maximizing the retention of bioacoustic cues to affect-while eliminating phonetic cues to verbal meaning. To evaluate the success of this method, we studied the perception of corresponding speech audio and transformed EGG signals in an affect rating experiment with online listeners. The results show a high degree of similarity in the perceived affect of matched signals, indicating that our method is effective.

Through collaborative playlists (CPs), streaming platform users have co-curated music together for various purposes for over a decade. As the COVID-19 pandemic has transformed how people come together through technology and engage with music, CPs have also taken on new roles and value. To understand how CP usage and perception have evolved since the onset of COVID-19, we conducted a mixed-methods investigation of CPs in the United States. Survey results from primarily CP users (N=142) revealed that interest in and usage of CPs have mostly increased since the pandemic, and that the role of music in connecting with others is positively correlated with the perceived impact of COVID-19. Follow-up interviews (N=9) provided additional insights into changing perceptions and usage patterns of CPs during COVID-19; for instance, fewer collaborators per playlist reflects users’ greater focus on strengthening social connections and relationships. Taken together, findings and design implications on digitally mediated co-curation further elucidate the necessity for social and collaborative experiences with music supported by CPs during COVID-19.

The IPv6 protocol was designed with security in mind. One of the changes that IPv6 has introduced over IPv4 is a new 20-bit flow label field in its protocol header.We show that remote servers can use the flow label field in order to assign a unique ID to each device when communicating with machines running Windows 10 (versions 1703 and higher), and Linux and Android (kernel versions 4.3 and higher). The servers are then able to associate the respective device IDs with subsequent transmissions sent from those machines. This identification is done by exploiting the flow label field generation logic and works across all browsers regardless of network changes. Furthermore, a variant of this attack also works passively, namely without actively triggering traffic from those machines.To design the attack we reverse-engineered and cryptanalyzed the Windows flow label generation code and inspected the Linux kernel flow label generation code. We provide a practical technique to partially extract the key used by each of these algorithms, and observe that this key can identify individual devices across networks, VPNs, browsers and privacy settings. We deployed a demo (for both Windows and Linux/Android) showing that key extraction and machine fingerprinting works in the wild, and tested it from networks around the world.

We propose a probabilistic method for the joint segmentation and melody extraction for musical audio signals which arise from a monophonic score. The method operates on framewise short-time Fourier transform (STFT) peaks, enabling a computationally efficient inference of note onset, duration, and pitch attributes while retaining sufficient information for pitch determination and spectral change detection. The system explicitly models note events in terms of transient and steady-state regions as well as possible gaps between note events. In this way, the system readily distinguishes abrupt spectral changes associated with musical onsets from other abrupt change events. Additionally, the method may incorporate melodic context by modeling note-to-note dependences. The method is successfully applied to a variety of piano and violin recordings containing reverberation, effective polyphony due to legato playing style, expressive pitch variations, and background voices. While the method does not provide a sample-accurate segmentation, it facilitates the latter in subsequent processing by isolating musical onsets to frame neighborhoods and identifying possible pitch content before and after the true onset sample location

We describe a perceptual space for timbre, define an objective metric that takes into account perceptual orthogonality and measure the quality of timbre interpolation. We discuss two timbre representations and measure perceptual judgments. We determine that a timbre space based on Mel-frequency cepstral coefficients (MFCC) is a good model for perceptual timbre space.