The Transformer model, particularly its cross-attention module, is widely used for feature fusion in target sound extraction which extracts the signal of interest based on given clues. Despite its effectiveness, this approach suffers from low computational efficiency. Recent advancements in state space models, notably the latest work Mamba, have shown comparable performance to Transformer-based methods while significantly reducing computational complexity in various tasks. However, Mamba’s applicability in target sound extraction is limited due to its inability to capture dependencies between different sequences as the cross-attention does. In this paper, we propose CrossMamba for target sound extraction, which leverages the hidden attention mechanism of Mamba to compute dependencies between the given clues and the audio mixture. The calculation of Mamba can be divided to the query, key and value. We utilize the clue to generate the query and the audio mixture to derive the key and value, adhering to the principle of the cross-attention mechanism in Transformers. Experimental results from two representative target sound extraction methods validate the efficacy of the proposed CrossMamba.

As artificial intelligence-generated content (AIGC) continues to evolve, video-to-audio (V2A) generation has emerged as a key area with promising applications in multimedia editing, augmented reality, and automated content creation. While Transformer and Diffusion models have advanced audio generation, a significant challenge persists in extracting precise semantic information from videos, as current models often lose sequential context by relying solely on frame-based features. To address this, we present TA-V2A, a method that integrates language, audio, and video features to improve semantic representation in latent space. By incorporating large language models for enhanced video comprehension, our approach leverages text guidance to enrich semantic expression. Our diffusion model-based system utilizes automated text modulation to enhance inference quality and efficiency, providing personalized control through text-guided interfaces. This integration enhances semantic expression while ensuring temporal alignment, leading to more accurate and coherent video-to-audio generation.

The traditional feedback Active Noise Control (ANC) algorithms are built upon linear filters, which leads to reduced performance when dealing with real-world noise. Deep learning-based feedback ANC algorithms have been proposed to overcome this problem. However, methods relying on pre-trained neural networks exhibit performance degradation when encountering noise from unseen scenes in the training dataset. This paper proposed a hybrid deep-online learning based spatial ANC system which combines online learning with pre-trained deep neural networks. The proposed method can keep the performance on noise from the trained scenes while improve the performance of cancelling noise from new scenes. Additionally, by incorporating wave domain decomposition, this paper achieves noise cancellation over a control spatial region. Simulation experiments validate the effectiveness of the combination of online learning and deep learning in handling previously unseen noise. Furthermore, the efficiency of wave domain decomposition in spatial noise cancellation is also verified.

In the current method for the sound field translation tasks based on spherical harmonic (SH) analysis, the solution based on the additive theorem usually faces the problem of singular values caused by large matrix condition numbers. The influence of different distances and frequencies of the spherical radial function on the stability of the translation matrix will affect the accuracy of the SH coefficients at the selected point. Due to the problems mentioned above, we propose a neural network scheme based on the dual-path transformer. More specifically, the dual-path network is constructed by the selfattention module along the two dimensions of the frequency and order axes. The transform-average-concatenate layer and upscaling layer are introduced in the network, which provides solutions for multiple sampling points and upscaling. Numerical simulation results indicate that both the working frequency range and the distance range of the translation are extended. More accurate higher-order SH coefficients are obtained with the proposed dual-path network.

In a reverberant environment, interferences such as reflections and background noise can degrade the perception of the sound source signal. Although the DNN-based methods have made a tremendous breakthrough in addressing this issue, the performance of these models is highly dependent on the completeness of the training dataset, which will limit its generalization under unknown environments. In this paper, we propose a physical model-based self-supervised learning (PMSSL) method to realize the DNN model optimization under unknown scenarios. This method incorporates a room reverberation physical model into the sound source enhancement model optimization process, realizing the self-learning of the DNN model under physical constraints. In this process, the time-frequency characteristics of the input signal and the spatial feature of the reverberation environment are utilized for parameter optimization, improving the adaptability of the DNN model under unknown scenarios. Experimental results based on simulated and measured data prove that the proposed method can obtain much more accurate source signal enhancement results compared with the pre-trained models, verifying its effectiveness and adaptability in new environments.

The performance of higherorder Ambisonics (HOA) signals obtained using spherical harmonics decomposition method is disturbed by two primary sources of errors, the noise pollution in low-frequency band and the spatial aliasing in high-frequency band. Inspired by the HOA signals upscale method, which is performed using the sparse character of the sound field, this paper propose a sound field decomposition model based on a sparse deep neural network that offers HOA signals with wider frequency bandwidth. We use the frequency domain multi-scale convolutional network to realize the spherical harmonics decomposition, as well as learning the spatial aliasing pattern, based on which the aliasing-free HOA signals can be derived. Besides, we apply a sparse encoding network to cpature the sparse feature of the sound field which will improve the model performance when the sparse condition is satisfied. The experiments results prove that the proposed model can obtain HOA signals with wider frequency range of operation under multiple sources (up to 10 sources) and low reverberant environments ($T_{60}\le$400 ms). When the sparsity feature cannot be satisfied ($T_{60} =$800 ms), the proposed network model still maintain the same performance as the traditional methods.

The Ambisonic technique has been widely used in fields such as sound-field reproduction, sound source localization, and acoustic noise control. In this paper, a recently proposed Ambisonic decoding method, the partially matching projection method (PMPD), was described in detail and extended for more practical use with near-field sources and reverberant playback environments. Several experiments were conducted to evaluate the performance of the method. In the experiments, we compared performances of PMPD method with three other Ambisonic decoding methods, “All-round Ambisonic Decoding” (AllRAD) method, mode matching decoding (MMD) method and regularized mode matching decoding (RMMD) method. Different playback conditions were considered to investigate the generalization of the method, including unevenly and evenly distributed loudspeakers, as well as free-field and reverberant playback environments. In terms of reproduction errors near sweep-spot and energy of loudspeaker gains, PMPD method worked well for different signals and playback environments. Whereas the other three methods encontered ill-posed problems or large reproduction errors in certain conditions. This means that PMPD method achieves a balence between precise reproductioin near sweet-spot and energy of loudspeaker gains with ill-posed problems. PMPD method has a more robust performance than the other three compared methods for different signals and playback environments. And we conclude that PMPD method is effective to be used as an Ambisonic decoding method, especially for different playback environments.

The lack of data is a major problem in individual HRTF modeling. There are many HRTF databases, but each database only has limited HRTFs with different characteristics, such as distance-dependent HRTFs or individual HRTFs. How to effectively model HRTFs through several different databases is an important task. In this paper, a method for predicting individual distance-dependent HRTFs using a few anthropometric parameters is proposed. By modeling the HRTFs in CIPIC database, which contains individual HRTFs in 1 meter, and the PKU&IOA database, which contains KEMAR HRTFs in eight distances, we predict the individual HRTFs in arbitrary directions and distances. The objective experiments show that the proposed model has less spectral distortions than distance variation function model. The subjective experiments show that the proposed model can predict the individual HRTFs in arbitrary directions and distances.

Model sound synthesis is a physically-based sound synthesis method used to generate audio content in games and virtual worlds. We present a novel learning-based impact sound synthesis algorithm called Deep-Modal. Our approach can handle sound synthesis for common arbitrary objects, especially dynamic generated objects, in real-time. We present a new compact strategy to represent the mode data, corresponding to frequency and amplitude, as fixed-length vectors. This is combined with a new network architecture that can convert shape features of 3D objects into mode data. Our network is based on an encoder-decoder architecture with the contact positions of objects and external forces embedded. Our method can synthesize interactive sounds related to objects of various shapes at any contact position, as well as objects of different materials and sizes. The synthesis process only takes ~0.01s on a GTX 1080 Ti GPU. We show the effectiveness of Deep-Modal through extensive evaluation using different metrics, including recall and precision of prediction, sound spectrogram, and a user study.

Head-related transfer function (HRTF) plays an important role in the construction of 3D auditory display. This article presents an individual HRTF modeling method using deep neural networks based on spatial principal component analysis. The HRTFs are represented by a small set of spatial principal components combined with frequency and individual-dependent weights. By estimating the spatial principal components using deep neural networks and mapping the corresponding weights to a quantity of anthropometric parameters, we predict individual HRTFs in arbitrary spatial directions. The objective and subjective experiments evaluate the HRTFs generated by the proposed method, the principal component analysis (PCA) method, and the generic method. The results show that the HRTFs generated by the proposed method and PCA method perform better than the generic method. For most frequencies the spectral distortion of the proposed method is significantly smaller than the PCA method in the high frequencies but significantly larger in the low frequencies. The evaluation of the localization model shows the PCA method is better than the proposed method. The subjective localization experiments show that the PCA and the proposed methods have similar performances in most conditions. Both the objective and subjective experiments show that the proposed method can predict HRTFs in arbitrary spatial directions.

The Ambisonic technique has been widely used for sound field recording and reproduction recently. However, the basic Ambisonic decoding method will break down when the playback loudspeakers distribute unevenly. Various methods have been proposed to solve this problem. This paper introduces several improvements to a recently proposed Ambisonic decoding method, the matching projection method, for uneven loudspeaker layouts. The first improvement is energy preserving; the second is introducing the "in-phase" weight, and the third is introducing partial projection coefficients. To evaluate the improved method, we compared it with the original one and the all-round Ambisonic decoding method with a 2-dimension unevenly arranged loudspeaker array. The result shows our method greatly improves the original method where the loudspeaker arranges very sparsely or densely.

In this paper, a method for modeling distance dependent head-related transfer functions is presented. The HRTFs are first decomposed by spatial principal component analysis. Using deep neural networks, we model the spatial principal component weights of different distances. Then we realize the prediction of HRTFs in arbitrary spatial distances. The objective and subjective experiments are conducted to evaluate the proposed distance model and the distance variation function model, and the results have shown that the proposed model has less spectral distortions than distance variation function model, and the virtual sound generated by the proposed model has better performance in terms of distance localization.

The basic Ambisonics decoding method will break down when the playback loudspeakers distribute unevenly. This paper proposes a modified Ambisonics method, the matching projection decoding method, for solving this problem. The matching projection decoding method is a kind of the greedy algorithm. It firstly calculates the projection value of the object Ambisonics signal over each Ambisonics signal of loudspeakers, then the maximum projection value is assigned to the corresponding loudspeaker. This process is repeated until all the loudspeakers have been assigned a gain value. The objective and subjective experiments were performed to evaluate the proposed system and the basic system. Objective evaluation results show that the accuracy of the sound field generated by the matching projection decoding method is better than that of the basic method; and the subjective evaluation results show a more correct directional perception of the matching projection decoding method than the basic one.

Basis pursuit algorithm is one of the most popular methods of sparse coding. The goal of the algorithm is to represent signal using as few coefficients as possible, which is suitable for acoustic signal compression. This paper presents a lossless coding/decoding method using the basis pursuit algorithm. In this method, wavelet packets bases were used to compose the dictionary because of their natural sparse property. Experimental results are obtained by comparing the proposed method with the four popular lossless coding/decoding methods using various types of acoustic signals. The results show that the proposed method is competitive with the well-known methods for lossless compression, in terms of compression ratio and computational efficiency.

A measurement of head-related transfer functions (HRTFs) with high spatial resolution was carried out in this study. HRTF measurement is difficult in the proximal region because of the lack of an appropriate acoustic point source. In this paper, a modified spark gap was used as the acoustic sound source. Our evaluation experiments showed that the spark gap was more like an acoustic point source than others previously used from the viewpoints of frequency response, directivity, power attenuation, and stability. Using this spark gap, high spatial resolution HRTFs were measured at 6344 spatial points, with distances from 20 to 160 cm, elevations from -40deg to 90deg, and azimuths from 0deg to 360deg. Based on these measurements, an HRTF database was obtained and its reliability was confirmed by both objective and subjective evaluations.