Creating a unified speech and music model requires expensive pre-training. Model merging can instead create a unified audio model with minimal computational expense. However, direct merging is challenging when the models are not aligned in the weight space. Motivated by Git Re-Basin, we introduce a correlation-permutation approach that aligns a music encoder’s internal layers with a speech encoder. We extend previous work to the case of merging transformer layers. The method computes a permutation matrix that maximizes the model’s features-wise cross-correlations layer by layer, enabling effective fusion of these otherwise disjoint models. The merged model retains speech capabilities through this method while significantly enhancing music performance, achieving an improvement of 14.83 points in average score compared to the linear interpolation model merging. This work allows the creation of unified audio models from independently trained encoders.

Evaluation of text-to-music systems is constrained by the cost and availability of collecting experts for assessment. AudioMOS 2025 Challenge track 1 is created to automatically predict music impression (MI) as well as text alignment (TA) between the prompt and the generated musical piece. This paper reports our winning system, which uses a dual-branch architecture with pre-trained MuQ and RoBERTa models as audio and text encoders. A cross-attention mechanism fuses the audio and text representations. For training, we reframe the MI and TA prediction as a classification task. To incorporate the ordinal nature of MOS scores, one-hot labels are converted to a soft distribution using a Gaussian kernel. On the official test set, a single model trained with this method achieves a system-level Spearman’s Rank Correlation Coefficient (SRCC) of 0.991 for MI and 0.952 for TA, corresponding to a relative improvement of $21.21 \%$ in MI SRCC and $31.47 \%$ in TA SRCC over the challenge baseline.

Accurate audio quality estimation is essential for developing and evaluating audio generation, retrieval, and enhancement systems. Existing non-intrusive assessment models predict a single Mean Opinion Score (MOS) for speech, merging diverse perceptual factors and failing to generalize beyond speech. We propose MMMOS, a no-reference, multidomain audio quality assessment system that estimates four orthogonal axes: Production Quality, Production Complexity, Content Enjoyment, and Content Usefulness across speech, music, and environmental sounds. MMMOS fuses frame-level embeddings from three pretrained encoders (WavLM, MuQ, and M2D) and evaluates three aggregation strategies with four loss functions. By ensembling the top eight models, MMMOS shows a $\mathbf{2 0 - 3 0 \%}$ reduction in mean squared error and a 4-5% increase in Kendall’s $\tau$ versus baseline, gains first place in six of eight Production Complexity metrics, and ranks among the top three on 17 of 32 challenge metrics.

Speech emotion recognition (SER) systems often exhibit gender bias. However, the effectiveness and robustness of existing debiasing methods in such multi-label scenarios remain underexplored. To address this gap, we present EMO-Debias—a large-scale comparison of 13 debiasing methods applied to multi-label SER. Our study encompasses techniques from pre-processing, regularization, adversarial learning, biased learners, and distributionally robust optimization. Experiments conducted on acted and naturalistic emotion datasets, using WavLM and XLSR representations, evaluate each method under conditions of gender imbalance. Our analysis quantifies the trade-offs between fairness and accuracy, identifying which approaches consistently reduce gender performance gaps without compromising overall model performance. The findings provide actionable insights for selecting effective debiasing strategies and highlight the impact of dataset distributions.

Modern speech quality prediction models are trained on audio data resampled to a specific sampling rate. When tested on audio with a higher sampling rate, these models can produce biased scores. We present HighRateMOS, the first non-intrusive mean opinion score (MOS) model that explicitly considers sampling rate. HighRateMOS ensembles three model variants that exploit the following information: (i) a learnable embedding of speech sampling rate, (ii) Wav2vec 2.0 selfsupervised embeddings, (iii) multi-scale CNN spectral features, and (iv) MFCC features. In AudioMOS 2025 Track 3, HighRateMOS ranked first in five of eight metrics. Our experiments confirm that modeling sampling rate leads to more robust and sampling-rate-agnostic speech quality predictions.

Bias in speech emotion recognition (SER) systems often stems from spurious correlations between speaker characteristics and emotional labels, leading to unfair predictions across demographic groups. Many existing debiasing methods require model-specific changes or demographic annotations, limiting their practical use. We present COVADA, a Confidence-Oriented Voice Augmentation Debiasing Approach that mitigates bias without modifying model architecture or relying on demographic information. CO-VADA identifies training samples that reflect bias patterns present in the training data and then applies voice conversion to alter irrelevant attributes and generate samples. These augmented samples introduce speaker variations that differ from dominant patterns in the data, guiding the model to focus more on emotion-relevant features. Our framework is compatible with various SER models and voice conversion tools, making it a scalable and practical solution for improving fairness in SER systems.

Real-world audio often mixes speech and music, yet models typically handle only one domain. This paper introduces a multi-teacher distillation framework that unifies speech and music models into a single one while significantly reducing model size. Our approach leverages the strengths of domain-specific teacher models, such as HuBERT for speech and MERT for music, and explores various strategies to balance both domains. Experiments across diverse tasks demonstrate that our model matches the performance of domain-specific models, showing the effectiveness of cross-domain distillation. Additionally, we conduct few-shot learning experiments, highlighting the need for general models in real-world scenarios where labeled data is limited. Our results show that our model not only performs on par with specialized models but also outperforms them in few-shot scenarios, proving that a cross-domain approach is essential and effective for diverse tasks with limited data. Code and models are released at https://github.com/johnwei0325/Multi-Distillation-from-Speech-and-Music-Representation-Models

Advances in speech synthesis intensify security threats, motivating real-time deepfake detection research. We investigate whether bidirectional Mamba can serve as a competitive alternative to Self-Attention in detecting synthetic speech. Our solution, Fake-Mamba, integrates an XLSR front-end with bidirectional Mamba to capture both local and global artifacts. Our core innovation introduces three efficient encoders: TransBiMamba, ConBiMamba, and PN-BiMamba. Leveraging XLSR’s rich linguistic representations, PN-BiMamba can effectively capture the subtle cues of synthetic speech. Evaluated on ASVspoof 21 LA, 21 DF, and In-The-Wild benchmarks, FakeMamba achieves 0.97 %, 1.74 %, and 5.85% EER, respectively, representing substantial relative gains over SOTA models XLSRConformer and XLSR-Mamba. The framework maintains realtime inference across utterance lengths, demonstrating strong generalization and practical viability. The code is available at https://github.com/xuanxixi/Fake-Mamba.

Yi-Cheng Lin, Kang-Chieh Chen, Zhe-Yan Li, Tzu-Heng Wu, Tzu-Hsuan Wu, Kuan-Yu Chen, Hung-yi Lee, Yun-Nung Chen. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

Speech Emotion Recognition (SER) is a crucial component in developing general-purpose AI agents capable of natural human-computer interaction. However, building robust multilingual SER systems remains challenging due to the scarcity of labeled data in languages other than English and Chinese. In this paper, we propose an approach to enhance SER performance in low SER resource languages by leveraging data from high-resource languages. Specifically, we employ expressive Speech-to-Speech translation (S2ST) combined with a novel bootstrapping data selection pipeline to generate labeled data in the target language. Extensive experiments demonstrate that our method is both effective and generalizable across different upstream models and languages. Our results suggest that this approach can facilitate the development of more scalable and robust multilingual SER systems. Our code is available at: https://github.com/hsi-che-lin/Improve-SER-via-S2ST

In multichannel speech enhancement, effectively capturing spatial and spectral information across different microphones is crucial for noise reduction. Traditional methods, such as CNN or LSTM, attempt to model the temporal dynamics of full-band and sub-band spectral and spatial features. However, these approaches face limitations in fully modeling complex temporal dependencies, especially in dynamic acoustic environments. To overcome these challenges, we modify the current advanced model McNet by introducing an improved version of Mamba, a state-space model, and further propose MCMamba. MCMamba has been completely reengineered to integrate full-band and narrow-band spatial information with sub-band and full-band spectral features, providing a more comprehensive approach to modeling spatial and spectral information. Our experimental results demonstrate that MCMamba significantly improves the modeling of spatial and spectral features in multichannel speech enhancement, outperforming McNet and achieving very promis- ing performance on the CHiME-3 dataset. Additionally, we find that Mamba performs exceptionally well in modeling spectral information.

Expressive performance in classical music plays a crucial role in shaping interpretations of musical pieces. However, existing datasets often provide limited attention to expressive markings in piano performances. This research addresses this gap by developing the Expressive Markings and Emotions 33 (EME33) dataset, which captures expressive piano performances in MIDI format, annotated with dynamic markings, expression markings, and additional emotional expressions. To validate the dataset’s applicability, we employ a Long Short-Term Memory (LSTM) model, which is suited to the size of our dataset, to render expressiveness in music. The results demonstrate the model’s effectiveness in capturing and reproducing expressive performance, highlighting the potential of the EME33 dataset for future research in music information retrieval.

Warning: This paper may contain texts with uncomfortable content.Large Language Models (LLMs) have achieved remarkable performance in various tasks, including those involving multimodal data like speech. However, these models often exhibit biases due to the nature of their training data. Recently, more Speech Large Language Models (SLLMs) have emerged, underscoring the urgent need to address these biases. This study introduces Spoken Stereoset, a dataset specifically designed to evaluate social biases in SLLMs. By examining how different models respond to speech from diverse demographic groups, we aim to identify these biases. Our experiments reveal significant insights into their performance and bias levels. The findings indicate that while most models show minimal bias, some still exhibit slightly stereotypical or anti-stereotypical tendencies.

Speech Integrated Large Language Models (SILLMs) combine large language models with speech perception to perform diverse tasks, such as emotion recognition to speaker verification, demonstrating universal audio understanding capability. However, these models may amplify biases present in training data, potentially leading to biased access to information for marginalized groups. This work introduces a curated spoken bias evaluation toolkit and corresponding dataset. We evaluate gender bias in SILLMs across four semantic-related tasks: speech-to-text translation (STT), spoken coreference resolution (SCR), spoken sentence continuation (SSC), and spoken question answering (SQA). Our analysis reveals that bias levels are language-dependent and vary with different evaluation methods. Our findings emphasize the necessity of employing multiple approaches to comprehensively assess biases in SILLMs, providing insights for developing fairer SILLM systems.

Despite their impressive success, training foundation models remains computationally costly. This paper investigates how to efficiently train speech foundation models with self-supervised learning (SSL) under a limited compute budget. We examine critical factors in SSL that impact the budget, including model architecture, model size, and data size. Our goal is to make analytical steps toward understanding the training dynamics of speech foundation models. We benchmark SSL objectives in an entirely comparable setting and find that other factors contribute more significantly to the success of SSL. Our results show that slimmer model architectures outperform common small architectures under the same compute and parameter budget. We demonstrate that the size of the pre-training data remains crucial, even with data augmentation during SSL training, as performance suffers when iterating over limited data. Finally, we identify a trade-off between model size and data size, highlighting an optimal model size for a given compute budget.

Neural audio codec models are becoming increasingly important as they serve as tokenizers for audio, enabling efficient transmission or facilitating speech language modeling. The ideal neural audio codec should maintain content, paralinguistics, speaker characteristics, and audio information even at low bitrates. Recently, numerous advanced neural codec models have been proposed. However, codec models are often tested under varying experimental conditions. As a result, we introduce the Codec-SUPERB challenge at SLT 20241, designed to facilitate fair and lightweight comparisons among existing codec models and inspire advancements in the field. This challenge brings together representative speech applications and objective metrics, and carefully selects license-free datasets, sampling them into small sets to reduce evaluation computation costs. This paper presents the challenge’s rules, datasets, participant systems, results, and findings.1https://codecsuperb.github.io/

Federated Learning (FL) allows Internet-of-Things (IoT) devices to train a global model collaboratively and circumvent the security issue. However, the current FL framework has three main drawbacks, the huge network overhead, single point of failure, and accuracy degradation in non-independent-and-identically-distributed (non-IID) data distribution. We propose a novel Deep Reinforcement Learning (DRL) based Decentralized Learning (DL) framework, DeepSelect, to 1) reduce the network overhead of conventional FL, 2) construct a good communication topology adaptively to mitigate the effect of non-IID data, and 3) accelerate the DL training by balancing the effects of hitting time (HT) and data bias. Moreover, DeepSelect with a subtly-designed DRL agent is reusable with different levels of non-IID data distributions. To the best of our knowledge, this paper is the first one to indicate that proper neighbor selection for exchanging parameters (not raw data) can counterbalance the data bias's effect and improve the DL convergence with non-IID data. The experiment results show that DeepSelect can reduce 18%-51% training rounds than the other heuristics on FashionMNIST and CIFAR-10 with non-IID data distributions.

A novel 4-times voltage multiplier rectifier with smart Time-Division-Multiplexing (TDM) control strategy for high step-up converters is proposed in this paper. Based on the proposed TDM control strategy, two full-wave voltage doubler rectifiers can be combined to realize the proposed 4-times voltage multiplier rectifier. The novel 4-times voltage multiplier rectifier and TDM control strategy can reduce transformer turn ratio and transformer size for high step-up converters and also reduce voltage stress for the output capacitors and rectifier diodes. Simulations and experiments are conducted in this paper to validate the proposed 4-times voltage multiplier rectifier with smart TDM control strategy for high step-up converters. The results show that the proposed smart TDM control strategy has great potential to be used in high step-up converters.

This paper proposes a maximum power point tracker (MPPT) which can accommodate widely output voltage range of solar panel under various environmental conditions. The controller in the MPPT employs the interval type-2 Takagi-Sugeno (IT2 TS) fuzzy technology. The main advantage of the proposed IT2 TS fuzzy controller is that it can handle the uncertainties in the modeling process. The experimental results are implemented to demonstrate the capability of IT2 TS fuzzy controller compared to type-1 T-S fuzzy controller.

A 4-port antenna is designed for evaluating the performance of an MIMO system in indoor environment. Diversity gain, correlation coefficient and channel capacity are calculated and discussed in this paper. The selection diversity becomes more unapparent when using more antennas in MIMO system. Correlation in both rich scattering and high K-factor environment are found below 0.2 and 0.5 respectively. Channel capacity is 82% of the ideal case with uncorrelated channel. The above-mentioned results suggest the effectiveness of the proposed antenna.

Reliability optimization has been a popular area of research, and received significant attention due to the critical importance of reliability in various kinds of systems. Most network reliability optimization problems are only focused on solving simple structured networks (e.g., series-parallel networks) of which the reliability function can be easily obtained in advance. However, modern networks are usually very complex, and it is impossible to calculate the exact network reliability function by using traditional analytical methods in limited time. Hence, a new particle swarm optimization (PSO) based on Monte Carlo simulation (MCS), named MCS-PSO, has been proposed to solve complex network reliability optimization problems. The proposed MCS-PSO can minimize cost under reliability constraints. To the best of our knowledge, this is the first attempt to use PSO combined with MCS to solve complex network reliability problems without requiring knowledge of the reliability function in advance. Compared with previous works to solve this problem, the proposed MCS-PSO can have better efficiency by providing a better solution to the complex network reliability optimization problem.

We present a software-based test methodology that utilizes an on-chip processor to perform test procedures for direct-mapped data cache. The cache system under test is divided into two major groups, namely the memory modules and the logic modules. For the memory modules which include the tag memory, the data memory, and the physical address tag memory, systematic procedures to transform a widely-used March algorithm into various executable instruction sequences are developed. For the logic modules, extensive analysis on the functions as well as the structures (architecture, RTL, and gate-level) of these modules is carried out and effective test instruction sequences based on the analysis are derived. A 100% fault coverage for six conventional RAM fault models and 99.13% test efficiency for single stuck-at fault model are obtained on a real 32-bit RISC processor. These results validate the viability and effectiveness of the proposed methodology for data-cache testing.

A coherent scattering model for tree canopies is employed in order to characterize the sensitivity of an interferometric SAR (INSAR) response to the physical parameters of forest stands. The concept of an equivalent scatterer for a collection of scatterers within a pixel, representing the vegetation particles of tree structures, is used for identifying the scattering phase center of the pixel whose height is measured by an INSAR. Combining the recently developed coherent scattering model for tree canopies and the INSAR /spl Delta/k-radar-equivalence algorithm, accurate statistics of the scattering phase-center location of forest stands are obtained numerically for the first time. The scattering model is based on a Monte Carlo simulation of scattering from fractal-generated tree structures, and therefore is capable of preserving the absolute phase of the backscatter. The model can also account for coherent effects due to the relative position of individual scatterers and the inhomogeneous extinction experienced by a coherent wave propagating through the random collection of vegetation particles. The location of the scattering phase center and the correlation coefficient are computed using the /spl Delta/k-radar equivalence simply by simulating the backscatter response at two slightly different frequencies. The model is successfully validated using the measured data acquired by JPL TOPSAR over a selected pine stand in Raco, MI. A sensitivity analysis is performed to characterize the response of coniferous and deciduous forest stands to a multifrequency and multipolarization INSAR in order to determine an optimum system configuration for remote sensing of forest parameters.

A coherent scattering model for tree canopies based on a Monte Carlo simulation of scattering from fractal-generated trees is developed and verified. In contrast to incoherent models, the present model calculates the coherent backscatter from forest canopies composed of realistic tree structures, where the relative phase information from individual scatterers is preserved. Computer generation of tree architectures faithful to the real stand is achieved by employing fractal concepts and Lindenmayer systems as well as incorporating the in situ measured data. The electromagnetic scattering problem is treated by considering the tree structure as a cluster of scatterers composed of cylinders (trunks and branches) and disks (leaves) above an arbitrary tilted plane (ground). Using the single scattering approximation, the total scattered field is obtained from the coherent addition of the individual scattering from each scatterer illuminated by a mean field. Foldy's approximation is invoked to calculate the mean field within the forest canopy that is modeled as a multilayer inhomogeneous medium. Backscatter statistics are acquired via a Monte Carlo simulation over a large number of realizations. The accuracy of the model is verified using the measured data acquired by a multifrequency and multipolarization synthetic aperture radar (SAR) [Spaceshuttle Imaging Radar-C (SIR-C)] from a maple stand at many incidence angles. A sensitivity analysis shows that the ground tilt angle and the tree structure may significantly affect the polarimetric radar response, especially at lower frequencies.

A procedure for retrieval of forest parameters is developed using the recently developed fractal-based coherent scattering model (FCSM) and a stochastic optimization algorithm. Since the fractal scattering model is computationally extensive, first a simplified empirical model with high fidelity for a desired forest stand is constructed using FCSM. Inputs to the empirical model are the influential structural and electrical parameters of the forest stand, such as the tree density, tree height, trunk diameter, branching angle, wood moisture, and soil moisture. Other finer structural features are embedded in the fractal model. The model outputs are the polarimetric and interferometric response of the forest as a function of the incidence angle. In this study, a genetic algorithm (GA) is employed as a global search routine to characterize the input parameters of a forest stand from a set of measured polarimetric/interferometric backscatter responses of the stand. The success of the inversion algorithm is demonstrated using a set of measured single-polarized interferometric synthetic aperture radar (SAR) data and several FCSM simulation results.

A three-step process is presented for estimation of forest biophysical properties from orbital polarimetric SAR data. Simple direct retrieval of total aboveground biomass is shown to be ill-posed unless the effects of forest structure are explicitly taken into account. The process first involves classification by (1) using SAR data to classify terrain on the basis of structural categories or (2) a priori classification of vegetation type on some other basis. Next, polarimetric SAR data at L- and C-bands are used to estimate basal area, height and dry crown biomass for forested areas. The estimation algorithms are empirically determined and are specific to each structural class. The last step uses a simple biophysical model to combine the estimates of basal area and height with ancillary information on trunk taper factor and wood density to estimate trunk biomass. Total biomass is estimated as the sum of crown and trunk biomass. The methodology is tested using SIR-C data obtained from the Raco Supersite in Northern Michigan on Apr. 15, 1994. This site is located at the ecotone between the boreal forest and northern temperate forests, and includes forest communities common to both. The results show that for the forest communities examined, biophysical attributes can be estimated with relatively small rms errors: (1) height (0-23 m) with rms error of 2.4 m, (2) basal area (0-72 m/sup 2//ha) with rms error of 3.5 m/sup 2//ha, (3) dry trunk biomass (0-19 kg/m/sup 2/) with rms error of 1.1 kg/m/sup 2/, (4) dry crown biomass (0-6 kg/m/sup 2/) with rms error of 0.5 kg/m/sup 2/, and (5) total aboveground biomass (0-25 kg/m/sup 2/) with rms error of 1.4 kg/m/sup 2/. The addition of X-SAR data to SIR-C was found to yield substantial further improvement in estimates of crown biomass in particular. However, due to a small sample size resulting from antenna misalignment between SIR-C and X-SAR, the statistical significance of this improvement cannot be reliably established until further data are analyzed. Finally, the results reported are for a small subset of the data acquired by SIR-C/X-SAR.>

An efficient and realistic electromagnetic scattering model for a tree trunk above a ground plane is presented. The trunk is modeled as a finite-length stratified dielectric cylinder with a corrugated bark layer. The ground is considered to be a smooth homogeneous dielectric with an arbitrary slope. The bistatic scattering response of the cylinder is obtained by invoking two approximations. In the microwave region, the height of the tree trunks are usually much larger than the wavelength. Therefore the interior fields in a finite length cylinder representing a tree trunk can be approximated with those of an infinite cylinder with the same physical and electrical radial characteristics. Also an approximate image theory is used to account for the presence of the dielectric ground plane which simply introduces an image excitation wave and an image scattered field. An asymptotic solution based on the physical optics approximation is derived which provides a fast algorithm with excellent accuracy when the radii of the tree trunks are large compared to the wavelength. The effect of a bark layer is also taken into account by simply replacing the bark layer with an anisotropic layer. It is shown that the corrugated layer acts as an impedance transformer which may significantly decrease the backscattering radar cross section depending on the corrugation parameters. It is also shown that for a tilted ground plane a significant cross-polarized backscattered signal is generated while the co-polarized backscattered signal is reduced.>