In this paper, we introduce a novel framework for generating multi-speaker speech without relying on any audible inputs. Our approach leverages silent electromyography (EMG) signals to capture linguistic content, while facial images are used to match with the vocal identity of the target speaker. Notably, we present a pitch-disentangled content embedding that enhances the extraction of linguistic content from EMG signals. Extensive analysis demonstrates that our method can generate multi-speaker speech without any audible inputs and confirms the effectiveness of the proposed pitch-disentanglement approach.

Audio-language retrieval faces a fundamental challenge due to many-to-one mappings, where multiple audio instances correspond to the same or similar textual descriptions. This ambiguity leads to overlapping representations, making it difficult for models to learn discriminative features between samples. Queue-based contrastive learning helps mitigate this ambiguity by maintaining a large pool of negatives, improving retrieval performance. However, selecting appropriate hyperparameters, such as the queue size for optimal performance and the number of epochs to prevent overfitting, remains a challenge. In this study, we propose a LAtent Space Embedding Rank (LASER) analysis. Specifically, we analyze that a low-rank property of a queue indicates the presence of informative negative samples, and demonstrate that the best performance is achieved when the queue has the lowest rank. Furthermore, the convergence of rank variation can be interpreted as the model parameters reaching an optimal state during training. Thanks to the proposed rank-based analysis, the model hyperparameters can be selected analytically rather than through trial and error. Experimental results confirm that retrieval performance depends on queue rank, with lower ranks yielding better performance. These findings suggest that our LASER analysis can provide insight into how rank-based analysis can improve retrieval performance in audio-language tasks.

There has been growing interest in audio-language retrieval research, where the objective is to establish the correlation between audio and text modalities. However, most audio-text paired datasets often lack rich expression of the text data compared to the audio samples. One of the significant challenges facing audio-text datasets is the presence of similar or identical captions despite different audio samples. Therefore, under many-to-one mapping conditions, audio-text datasets lead to poor performance of retrieval tasks. In this paper, we propose a novel approach to tackle the data imbalance problem in audio-language retrieval task. To overcome the limitation, we introduce a method that employs a distance sampling-based paraphraser leveraging ChatGPT, utilizing distance function to generate a controllable distribution of manipulated text data. For a set of sentences with the same context, the distance is used to calculate a degree of manipulation for any two sentences, and ChatGPT's few-shot prompting is performed using a text cluster with a similar distance defined by the Jaccard similarity. Therefore, ChatGPT, when applied to few-shot prompting with text clusters, can adjust the diversity of the manipulated text based on the distance. The proposed approach is shown to significantly enhance performance in audio-text retrieval, outperforming conventional text augmentation techniques.

As the development of Internet and mobile device increases, there is a correspondingly increasing amount of data produced by users of such technology worldwide. It is thus essential to analyze such massive amounts of data reflective of the big data era. Recently, Apache Spark has become popular for analyzing big data, since it can process streaming data and support real-time in-memory computing. Also, it is known to execute applications faster than traditionally used Hadoop. Also cloud technology provides flexible resource utilization environment on-demand. When analyzing big data using Spark in existing environments, it is difficult to provision resources according to the system's changing environment and the influence of other users' executions. Using cloud technology however, it is possible to provision resources more effectively for the execution of jobs through dynamic resource provision methods. In this paper, we propose an auto-scaling framework with corresponding algorithms to manage resources dynamically in virtual environments, in order to meet user-specified SLA (Service Level Agreement) given a set of limited resources. Our experimental results on Spark in OpenStack demonstrate the effectiveness of scaling resources to satisfy user SLAs.