This work investigates privacy-aware collaborative wake word detection (WWD) in acoustic sensor networks. To meet state-of-the-art privacy constraints, the proposed WWD scheme is based on privacy-aware unsupervised clustered federated learning that groups microphone nodes w.r.t. active sound sources and on a privacy-preserving high-level feature representation. Using the partition of microphone nodes into clusters, we apply intra- and inter-cluster feature enhancement strategies directly in the privacy-preserving feature domain and thus circumvent the need for communicating privacy-sensitive information between nodes. The approach is demonstrated for an acoustic sensor network deployed in a smart-home environment. We show that the proposed collaborative WWD system clearly outperforms independent decisions of individual microphone nodes. Index Terms: privacy, wake word detection, clustering, federated learning, unsupervised clustered federated learning

Computational models of the auditory periphery are important tools for understanding mechanisms of normal and impaired hearing and for developing advanced speech and audio processing algorithms. However, the simulation of accurate neural representations entails a high computational effort. This prevents the use of auditory models in applications with real-time requirements and the design of speech enhancement algorithms based on efficient bio-inspired optimization criteria. Hence, in this work we propose and evaluate DNN-based approximations of a state-of-the-art auditory model. The DNN models yield accurate neurogram predictions for previously unseen speech signals with processing times shorter than signal duration, thus indicating their potential to be applied in real-time.

In this paper we present a privacy-aware method for estimating source-dominated microphone clusters in the context of acoustic sensor networks (ASNs). The approach is based on clustered federated learning which we adapt to unsupervised scenarios by employing a light-weight autoencoder model. The model is further optimized for training on very scarce data. In order to best harness the benefits of clustered microphone nodes in ASN applications, a method for the computation of cluster membership values is introduced. We validate the performance of the proposed approach using clustering-based measures and a network-wide classification task.

Wake word detection and verification systems often involve a local, on-device wake word detector and a cloud-based verification node. In such systems, the audio representation sent to the cloud-based server may exhibit sensitive information that might be intercepted by an eavesdropper. To improve privacy of cloud-based wake word verification (WWV) systems, we propose to use a privacy-preserving feature representation that minimizes the automatic speech recognition (ASR) capability of a potential attacker. The proposed approach employs an adversarial training schedule that aims to minimize an attacker’s word error rate (WER) while maintaining a high WWV performance. To this end, we apply an adaptive weighting factor in the combined loss function to control the balance between minimizing the WWV loss and maximizing the ASR loss. We show that the proposed training method significantly reduces possible privacy risks while maintaining a strong WWV performance.

In this paper we investigate and tackle the privacy risks of deep-neural-network-based feature extraction for sound classification in acoustic sensor networks. To this end, we analyze a single-label domestic activity monitoring and a multi-label urban sound tagging scenario. We show that in both cases, the feature representations designed for sound classification also carry a significant amount of speaker-dependent data, thus posing serious privacy risks for speaker recognition attacks based on feature interception. We then propose to mitigate the aforementioned privacy risks by introducing a variational information feature extraction scheme that allows sound classification while, concurrently, minimizing the feature representation's level of information and hence, inhibiting speaker recognition attempts. We control and analyze the balance between the performance of the trusted and attacker tasks via the resulting model's composite loss function, its budget scaling factor, and latent space size. It is empirically demonstrated that the proposed privacy-preserving feature representation generalizes well to both single-label and multi-label scenarios with vast as well as reduced training-dataset resources. Furthermore, it exhibits robustness against x-vector-based, state-of-the-art speaker recognition attacks.

In this paper we propose a new scheme for vehicle engine noise classification as a more privacy-preserving alternative to classifying vehicles based on video recordings. We establish two scenarios: diesel vs. petrol and heavy goods vehicle vs. personal car classification. Our approach includes a novel modulation-spectrum-based feature representation that is used in conjunction with a siamese neural network classifier. Additionally, a database containing recordings from diverse urban acoustic scenarios is provided. The obtained results show the advantage of the proposed approach compared to conventional feature representations and classifiers. This is achieved by de-correlating background noise from target noise and by quantifying the degree of variation of noise characteristics.

This paper introduces a deep neural network based feature extraction scheme that aims to improve the trade-off between utility and privacy in speaker classification tasks. In the proposed scenario we develop a feature representation that helps to maximize the performance of a gender classifier while minimizing additional speaker identity information. Our approach is to use variational information feature extraction that allows for gender discrimination (utility) but minimizes the information level of the features, thus discouraging speaker identification adversarial attacks (privacy). We analyze the model's loss function and the budget scaling factor used to control the balance of utility vs. privacy. It is experimentally shown that the proposed method reduces privacy risks without significantly deprecating utility and that it also generalizes well to new speaker contexts.

In this paper we highlight the privacy risks entailed in deep neural network feature extraction for domestic activity monitoring. We employ the baseline system proposed in the Task 5 of the DCASE 2018 challenge and simulate a feature interception attack by an eavesdropper who wants to perform speaker identification. We then propose to reduce the aforementioned privacy risks by introducing a variational information feature extraction scheme that allows for good activity monitoring performance while at the same time minimizing the information of the feature representation, thus restricting speaker identification attempts. We analyze the resulting model’s composite loss function and the budget scaling factor used to control the balance between the performance of the trusted and attacker tasks. It is empirically demonstrated that the proposed method reduces speaker identification privacy risks without significantly deprecating the performance of domestic activity monitoring tasks.

In this paper we analyze the impact of temporal feature aggregation and feature dimensionality reduction on the performance of speaker identification tasks. We investigate these two processing steps in the context of communication layer constraints, such as limited bitrate, and privacy constraints at node level, of a wireless acoustic sensor network. To this end, we extract Modulation-MFCC features and state-of-the-art i-vectors for speaker identification, and investigate temporal aggregation and dimensionality reduction in the feature extraction process. In the evaluation, we use clean data as well as reverberant data to assess the feature sets for different application scenarios. It is found that temporal aggregation has a positive effect on increasing speaker identification performance while respecting the aforementioned privacy constraints and that Linear Discriminant Analysis can be successfully employed for dimensionality reduction.