Stereo 3-D Sound Event Localization and Detection (SELD) faces a fundamental geometric conflict: the simultaneous regression of unitless angular direction and unbounded physical distance. This heterogeneity destabilizes training, as large-magnitude distance errors often overwhelm fine-grained directional gradients in the loss landscape. To resolve this, we propose WINTER (Wrapped Interval NormalizaTion for Elevation Representation), a novel embedding that projects the heterogeneous target space onto a unified, homogeneous manifold. By mapping scalar distance to the latent elevation axis of stereo signals, WINTER transforms the problem into a single-head regression on a pseudo 3-D unit sphere. This geometric unification restores the validity of angular metrics and enables the introduction of Stereo-DICE, a differentiable proxy for the polyphonic F1-score that optimizes joint localization and detection. Validated on the DCASE 2025 Task 3 benchmark, our framework achieves a relative reduction of up to 8.19% in overall SELD error compared to standard regression baselines, significantly improving far-field stability. Code is available at:https://github.com/itsjunwei/WINTER-SELD

Head-related transfer function (HRTF) is an essential component to create an immersive listening experience over headphones for virtual reality (VR) and augmented reality (AR) applications. Metaverse combines VR and AR to create immersive digital experiences, and users are very likely to interact with virtual objects in the near-field (NF). The HRTFs of such objects are highly individualized and dependent on directions and distances. Hence, a significant number of HRTF measurements at different distances in the NF would be needed. Using conventional static stop-and-go HRTF measurement methods to acquire these measurements would be time-consuming and tedious for human listeners. In this paper, we propose a continuous measurement system targeted for the NF, and efficiently capturing HRTFs in the horizontal plane within 45 secs. Comparative experiments are performed on head and torso similar (HATS) and human listeners to evaluate system consistency and robustness.

With rapid urbanization comes the increase of community, construction, and transportation noise in residential areas. The conventional approach of solely relying on sound pressure-level information to decide on the noise environment and to plan out noise control and mitigation strategies is inadequate. This article presents an end-to-end Internet-of-Things (IoT) system that extracts real-time urban sound metadata using edge devices, providing information on the sound type, location and duration, rate of occurrence, loudness, and azimuth of a dominant noise in nine residential areas. The collected metadata on environmental sound is transmitted to and aggregated in a cloud-based platform to produce detailed descriptive analytics and visualization. Our approach in integrating different building blocks, namely, hardware, software, cloud technologies, and signal processing algorithms to form our real-time IoT system is outlined. We demonstrate how some of the sound metadata extracted by our system are used to provide insights into the noise in residential areas. A scalable workflow to collect and prepare audio recordings from nine residential areas to construct our urban sound data set for training and evaluating a location-agnostic model is discussed. Some practical challenges of managing and maintaining a sensor network deployed at numerous locations are also addressed.

Parkinson's disease (PD) is an irreversible neurodegenerative disease that seriously affects patients' lives. To provide patients with accurate treatment in time and to reduce deterioration of the disease, it is critical to have an early diagnosis of PD and accurate clinical score predictions. Different from previous studies on PD, most of which only focus on feature selection methods, we propose a network combining joint learning from multiple modalities and relations (JLMMR) with sparse nonnegative autoencoder (SNAE) to further enhance the ability of feature expression. We first preprocess and extract features of the modal neuroimaging data with multiple time points. To extract discriminative and informative features from longitudinal data, we apply JLMMR method for feature selection to avoid over-fitting issues. We further exploit SNAE to learn longitudinal discriminative features for joint disease diagnosis and obtain clinical score predictions. Extensive experiments on the publicly available Parkinson's Progression Markers Initiative (PPMI) dataset show the proposed method produces promising classification and prediction performance, which outperforms state-of-the-art methods as well.

Several typical types of retinopathy are major causes of blindness. However, early detection of retinopathy is quite not easy since few symptoms are observable in the early stage, attributing to the development of non-mydriatic retinal cameras, these cameras produce high-resolution retinal fundus images that provide the possibility of Computer-Aided-Diagnosis (CAD) via deep learning to assist diagnosing retinopathy. Deep learning algorithms usually rely on a large number of labeled images that are expensive and time-consuming to obtain in the medical imaging area. Moreover, the random distribution of various lesions that often vary greatly in size also brings significant challenges to learn discriminative information from high-resolution fundus images. In this paper, we present generative adversarial networks simultaneously equipped with a “good” generator and a “bad” generator (GBGANs) to make up for the incomplete data distribution given limited fundus images. To improve the generative feasibility of the generator, we introduce a pre-trained feature extractor to acquire condensed features for each fundus image in advance. Experimental results on integrated three public iChallenge datasets show that the proposed GBGANs could fully utilize the available fundus images to identify retinopathy with little label cost.

Parkinson's disease (PD) is a neurodegenerative progressive disease that mainly affects the motor systems of patients. To slow this disease deterioration, early and accurate diagnosis of PD is an effective way, which alleviates mental and physical sufferings by clinical intervention. In this paper, we propose a joint regression and classification framework for PD diagnosis via magnetic resonance and diffusion tensor imaging data. Specifically, we devise a unified multitask feature selection model to explore multiple relationships among features, samples, and clinical scores. We regress four clinical variables of depression, sleep, olfaction, cognition scores, as well as perform the classification of PD disease from the multimodal data. The multitask model explores the relationships at the level of clinical scores, image features, and subjects, to select the most informative and diseased-related features for diagnosis. The proposed method is evaluated on the public Parkinson's progression markers initiative dataset. The extensive experimental results show that the multitask framework can effectively boost the performance of regression and classification and outperforms other state-of-the-art methods. The computerized predictions of clinical scores and label for PD diagnosis may offer quantitative reference for decision support as well.

Predicting the protein-protein interactions (PPIs) has played an important role in many applications. Hence, a novel computational method for PPIs prediction is highly desirable. PPIs endow with protein amino acid mutation rate and two physicochemical properties of protein (e.g., hydrophobicity and hydrophilicity). Deep polynomial network (DPN) is well-suited to integrate these modalities since it can represent any function on a finite sample dataset via the supervised deep learning algorithm. We propose a multimodal DPN (MDPN) algorithm to effectively integrate these modalities to enhance prediction performance. MDPN consists of a two-stage DPN, the first stage feeds multiple protein features into DPN encoding to obtain high-level feature representation while the second stage fuses and learns features by cascading three types of high-level features in the DPN encoding. We employ a regularized extreme learning machine to predict PPIs. The proposed method is tested on the public dataset of H. pylori, Human, and Yeast and achieves average accuracies of 97.87%, 99.90%, and 98.11%, respectively. The proposed method also achieves good accuracies on other datasets. Furthermore, we test our method on three kinds of PPI networks and obtain superior prediction results.

In [1], Baiying Lei was indicated as the corresponding author. Tianfu Wang and Baiying Lei should have been indicated as the corresponding authors.

Ultrasound imaging has become a prevalent examination method in prenatal diagnosis. Accurate acquisition of fetal facial standard plane (FFSP) is the most important precondition for subsequent diagnosis and measurement. In the past few years, considerable effort has been devoted to FFSP recognition using various hand-crafted features, but the recognition performance is still unsatisfactory due to the high intraclass variation of FFSPs and the high degree of visual similarity between FFSPs and other non-FFSPs. To improve the recognition performance, we propose a method to automatically recognize FFSP via a deep convolutional neural network (DCNN) architecture. The proposed DCNN consists of 16 convolutional layers with small 3 × 3 size kernels and three fully connected layers. A global average pooling is adopted in the last pooling layer to significantly reduce network parameters, which alleviates the overfitting problems and improves the performance under limited training data. Both the transfer learning strategy and a data augmentation technique tailored for FFSP are implemented to further boost the recognition performance. Extensive experiments demonstrate the advantage of our proposed method over traditional approaches and the effectiveness of DCNN to recognize FFSP for clinical diagnosis.

Accurate segmentation of cervical cells in Pap smear images is an important step in automatic pre-cancer identification in the uterine cervix. One of the major segmentation challenges is overlapping of cytoplasm, which has not been well-addressed in previous studies. To tackle the overlapping issue, this paper proposes a learning-based method with robust shape priors to segment individual cell in Pap smear images to support automatic monitoring of changes in cells, which is a vital prerequisite of early detection of cervical cancer. We define this splitting problem as a discrete labeling task for multiple cells with a suitable cost function. The labeling results are then fed into our dynamic multi-template deformation model for further boundary refinement. Multi-scale deep convolutional networks are adopted to learn the diverse cell appearance features. We also incorporated high-level shape information to guide segmentation where cell boundary might be weak or lost due to cell overlapping. An evaluation carried out using two different datasets demonstrates the superiority of our proposed method over the state-of-the-art methods in terms of segmentation accuracy.

Individualization of head-related transfer functions (HRTFs) can be realized using the person's anthropometry with a pretrained model. This model usually establishes a direct linear or non-linear mapping from anthropometry to HRTFs in the training database. Due to the complex relation between anthropometry and HRTFs, the accuracy of this model depends heavily on the correct selection of the anthropometric features. To alleviate this problem and improve the accuracy of HRTF individualization, an indirect HRTF individualization framework was proposed recently, where HRTFs are synthesized using a sparse representation trained from the anthropometric features. In this paper, we extend their study on this framework by investigating the effects of different preprocessing and postprocessing methods on HRTF individualization. Our experimental results showed that preprocessing and postprocessing methods are crucial for achieving accurate HRTF individualization.

Spatial audio reproduction addresses the growing commercial need to recreate an immersive listening experience of digital media content, such as movies and games. Primary-ambient extraction (PAE) is one of the key approaches to facilitate flexible and optimal rendering in spatial audio reproduction. Existing approaches, such as principal component analysis and time-frequency masking, often suffer from severe extraction error. This problem is more evident when the sound scene contains a relatively strong ambient component, which is frequently encountered in digital media. In this Letter, we propose a novel PAE approach by estimating the ambient phase with a sparsity constraint (APES). This approach exploits the equal magnitude of the uncorrelated ambient components in the two channels of a stereo signal and reformulates the PAE problem as an ambient phase estimation problem, which is then solved using the criterion that the primary component is sparse. Our experimental results demonstrate that the proposed approach significantly outperforms existing approaches, especially when the ambient component is relatively strong.

The diversity of today's playback systems requires a flexible, efficient, and immersive reproduction of sound scenes in digital media. Spatial audio reproduction based on primary-ambient extraction (PAE) fulfills this objective, where accurate extraction of primary and ambient components from sound mixtures in channel-based audio is crucial. Severe extraction error was found in existing PAE approaches when dealing with sound mixtures that contain a relatively strong ambient component, a commonly encountered case in the sound scenes of digital media. In this paper, we propose a novel ambient spectrum estimation (ASE) framework to improve the performance of PAE. The ASE framework exploits the equal magnitude of the uncorrelated ambient components in two channels of a stereo signal, and reformulates the PAE problem into the problem of estimating either ambient phase or magnitude. In particular, we take advantage of the sparse characteristic of the primary components to derive sparse solutions for ASE based PAE, together with an approximate solution that can significantly reduce the computational cost. Our objective and subjective experimental results demonstrate that the proposed ASE approaches significantly outperform existing approaches, especially when the ambient component is relatively strong.

One of the key issues in spatial audio analysis and reproduction is to decompose a signal into primary and ambient components based on their directional and diffuse spatial features, respectively. Existing approaches employed in primary-ambient extraction (PAE), such as principal component analysis (PCA), are mainly based on a basic stereo signal model. The performance of these PAE approaches has not been well studied for the input signals that do not satisfy all the assumptions of the stereo signal model. In practice, one such case commonly encountered is that the primary components of the stereo signal are partially correlated at zero lag, referred to as the primary-complex case. In this paper, we take PCA as a representative of existing PAE approaches and investigate the performance degradation of PAE with respect to the correlation of the primary components in the primary-complex case. A time-shifting technique is proposed in PAE to alleviate the performance degradation due to the low correlation of the primary components in such stereo signals. This technique involves time-shifting the input signal according to the estimated inter-channel time difference of the primary component prior to the signal decomposition using conventional PAE approaches. To avoid the switching artifacts caused by the varied time-shifting in successive time frames, overlapped output mapping is suggested. Based on the results from our experiments, PAE approaches with the proposed time-shifting technique are found to be superior to the conventional PAE approaches in terms of extraction accuracy and spatial accuracy.

Due to the physical size and frequency response constraints of miniaturized and flat panel loudspeakers, low frequency reproduction from these loudspeakers is generally limited and unsatisfactory. The virtual bass system (VBS) enhances the bass performance of such loudspeakers by tricking the human brain to perceive the fundamental frequency from its higher harmonics. Problematically, the additional harmonics from VBS can also introduce perceptual distortion and reduce the audio quality. Therefore, a reliable method to assess the quality of VBS-enhanced signals is necessary in the designing of VBS. Since subjective experiments are often time-consuming and may be inconsistent, it is desirable to develop an objective assessment method for VBS. Earlier studies only utilized some simple objective metrics, which generally do not consider the human auditory model and are unable to accurately predict the perceptual quality of VBS. In this paper, we introduce a perceptual quality-assessment method for VBS based on the model output variables (MOVs) of the ITU Recommendation ITU-R BS.1387. Suitable combinations of MOVs are selected to derive perceptual quality metrics that correlate closely to the subjective quality. A verification experiment is presented to justify the accuracy of the metrics.

Primary-ambient extraction (PAE) has been playing an important role in spatial audio analysis-synthesis. Based on the spatial features, PAE decomposes a signal into primary and ambient components, which are then rendered separately. PAE is performed in subband domain for complex input signals having multiple point-like sound sources. However, the performance of PAE approaches and their key influences for such signals have not been well-studied so far. In this paper, we conducted a study on frequency-domain PAE using principal component analysis (PCA) in the case of multiple sources. We found that the partitioning of the frequency bins is very critical in PAE. Simulation results reveal that the proposed top-down adaptive partitioning method achieves superior performance as compared to the conventional partitioning methods.

Audio signals for moving pictures and video games are often linear combinations of primary and ambient components. In spatial audio analysis-synthesis, these mixed signals are usually decomposed into primary and ambient components to facilitate flexible spatial rendering and enhancement. Existing approaches such as principal component analysis (PCA) and least squares (LS) are widely used to perform this decomposition from stereo signals. However, the performance of these approaches in primary-ambient extraction (PAE) has not been well studied and no comparative analysis among the existing approaches has been carried out so far. In this paper, we generalize the existing approaches into a linear estimation framework. Under this framework, we propose a series of performance measures to identify the components that contribute to the extraction error. Based on the generalized linear estimation framework and our proposed performance measures, a comparative study and experimental testing of the linear estimation based PAE approaches including existing PCA, LS, and three proposed variant LS approaches are presented.

In spatial audio analysis-synthesis, one of the key issues is to decompose a signal into cue and ambient components based on their spatial features. Principal component analysis (PCA) has been widely employed in cue extraction. However, the performance of PCA based cue extraction is highly dependent on the assumptions of the input signal model. One of these assumptions is the input signal contains highly correlated cue at zero lag. However, this assumption is often unmet. To overcome this problem, time shifted PCA is proposed in this paper, which involves time-shifting the input signal according to the estimated inter-channel time difference (ITD) of the input signal before cue extraction. From our simulation and listening tests results, the proposed method is found to be superior to the conventional PCA based cue extraction method.

Small and flat loudspeakers usually result in poor low-frequency (or bass) responses. Conventional gain equalization does not help significantly and may even result in overdriving and distortion. A psychoacoustic approach has been found to be suitable in tricking the human ear to perceive the fundamental frequency from its higher harmonics. Past research efforts have generally focused on weighting the harmonics based on the loudness matching method, but no work on timbre matching has been carried out so far. In this paper, we propose a new timbre matching technique, which can improve the sound quality of the psychoacoustically enhanced bass. This approach adjusts the amplitude of harmonics to produce similar timbre as the original audio content. Objective and subjective tests are carried out to compare the audio quality of the psychoacoustic bass enhanced signal using the equal-loudness weighting and the timbre matching methods.

In this paper, a robust audio watermarking scheme based on singular value decomposition (SVD) and differential evolution (DE) using dither modulation (DM) quantization algorithm is proposed. Two novel SVD-based algorithms, lifting wavelet transform (LWT)-discrete cosine transform (DCT)-SVD and discrete wavelet transform (DWT)-DCT-SVD, are developed for audio copyright protection. In our method, LWT\DWT is first applied to decompose the host signal and obtain the corresponding approximate coefficients followed by DCT to take advantage of “energy compaction” property. SVD is further performed to acquire the singular values and enhance the robustness of the scheme. The adaptive DM quantization is adopted to quantize the singular values and embed the watermark. To withstand desynchronization attacks, synchronization code is inserted using audio statistical characteristics. Furthermore, the conflicting problem of robustness and imperceptibility is effectively resolved by the DE optimization. Simulation results demonstrate that both the LWT-DCT-SVD and DWT-DCT-SVD methods not only have good imperceptibility performance, but also resist general signal processing, hybrid and desynchronization attacks. Compared with the previous DWT-DCT, support vector regression (SVR)-DWT-DCT and DWT-SVD methods, our method obtains more robustness against the selected attacks.

Psychoacoustic bass (low frequency) enhancement approach has received strong interests from consumer electronics manufacturers who demand good bass quality from small or flat panel speakers. Due to physical limitations and cost constraints, good bass performance is lacking from such speakers. A typical solution is based on the psychoacoustic phenomenon known as the “missing fundamental”, whereby human auditory system can perceive the fundamental frequency from its higher harmonics. This psychoacoustic bass enhancement system generally uses nonlinear devices (NLD) or phase vocoders (PV) to generate harmonics that enhance bass virtually. However, both approaches have their strengths and weaknesses. This paper presents a hybrid system, which combines these two approaches, to overcome their drawbacks and achieve good bass performance. The new approach first separates musical signals into transient and steady-state components, and applies NLD and PV on the separated signals. MUSHRA subjective test is used to evaluate the bass effect and audio quality of the proposed hybrid system against the NLD and PV methods.

Based on the Berktay's farfield solution, various preprocessing methods were proposed to reduce the distortion of the highly directional audible signal in the parametric loudspeaker. However, the Berktay's farfield solution is an approximated model of nonlinear acoustic propagation. To determine the effectiveness of these methods, we analyze various preprocessing methods theoretically for directional speech reproduction using the Khokhlov-Zabolotskaya-Kuznetsov (KZK) equation, which provides a more accurate model of nonlinear acoustic propagation. In order to reduce the distortion effectively in the parametric loudspeaker with these preprocessing methods, the initial sound pressure level of the carrier frequency is found to be less than 132 dB according to the KZK equation. Unlike the Berktay' farfield solution that results in a +12 dB/octave gain slope, different gain slopes are derived using the KZK equation and appropriate equalizers are proposed to improve the frequency response of the parametric loudspeaker. The optimal preprocessing method for directional speech reproduction is established based on the KZK equation, which has a relatively flat frequency response of the desired speech signal and the best total harmonic distortion performance.

A highly directional speech signal can be generated using parametric loudspeaker. The generation of highly directional sound beam is due to the nonlinear interaction of amplitude-modulated ultrasound waves in air. However, severe distortion is also generated during the reproduction of directional speech and several preprocessing techniques based on the Berktay's farfield model have been proposed by researchers to reduce the distortion. In this paper, we carried out a thorough investigation on the analytical performance of the recursive single-sideband amplitude modulation (RSSB-AM) technique, which has been found to perform well for directional speech reproduction. Several important characteristics of the performance of the RSSB-AM are observed and optimal parameters of the RSSB-AM are also presented.

Limited storage and bandwidth have long necessitated the use of images and videos in compressed formats. Resizing of images in compressed domains such as discrete cosine transform (DCT) are developed primarily to avoid the computation associated to decompression and compression for spatial techniques. An important application of this work is transcoding of image and video content to achieve reduction of spatial resolution. In this work, arbitrary resizing of images by factors of P/Q times R/S, where P, Q, R, and S are positive integers is considered. An efficient approach is derived by performing up-sampling and down-sampling simultaneously. All computations are carried out with N times N DCT blocks to simplify integration with block based coders such as JPEG and MPEG. Simulation results revealed that our proposed approach achieves significant reduction up to 50% of the computational cost and visually sharper images compared to existing methods.