This paper presents a causal speech enhancement (SE) model based on a complex two-branch nested U-Net architecture (CNUNet-TB) combined with a two-stage (TS) training method that leverages speech embeddings from a large self-supervised speech representation learning (SRL) model. The proposed architecture improves performance by simultaneously estimating complex masks and the speech spectrum, effectively handling complex speech data. The SE model is initially trained by fusing the self-supervised speech embeddings with the model’s latent vectors, which are then stored for the second stage of training. In the second stage, the SE model is trained to replicate these latent vectors without relying on the SRL model, ensuring causality and eliminating the need for the SRL model during inference. Experimental results demonstrate that the proposed CNUNet-TB-TS effectively replicates the stored latent vectors with strong speaker representation, achieving superior performance compared to recent causal SE models. The codes and demo samples are available at https://github.com/seorim0/SEusing-SRL-model.

This paper proposes a two-dimensional (2D) deep neural network (DNN) model for the electrocardiogram (ECG) abnormality classification, which effectively utilizes the inter and intra-lead information comprised in the 12-lead ECG. The proposed model is designed using a stack of residual U-shaped (ResU) blocks so that it can effectively capture ECG features in a multi-scale. The 2D features extracted by the ResU block are down-mixed to 1D features using a lead combiner block designed to merge features of the lead domain into both the time and channel domain. Through experiments, we confirm that our model outperforms other state-of-the-art models in various metrics. The code is made publicly available at https://github.com/seorim0/ResUNet-LC.

This study investigates the implication of utilizing the psychoacoustic model (PAM) within the neural audio coder (NAC), specifically focusing on the masking of quantization noise. We introduce a novel training strategy to incorporate the PAM into the NAC more accurately. This method involves a discriminator that directly or indirectly measures the PAM loss. For the indirect measurement, a multi-scale STFT discriminator (MS-STFTD) is incorporated to introduce an auxiliary loss term in addition to the existing PAM loss. Conversely, for the direct measurement, we have designed a multi-scale PAM discriminator (MS-PAMD) that quantifies PAM-specific parameters. Experimental results show that adding the discriminator masks the quantization noise better than the previous NAC, and it obtains audio quality comparable to the commercial AAC in both objective and subjective scores.

This paper proposes an end-to-end neural audio coder based on a mean-scale hyperprior model together with a perceptual optimization using a psychoacoustic model (PAM)-based loss function. The proposed coder estimates the mean and scale hyperpriors using a sub-network after assuming that the probability distribution of latent samples is Gaussian. The main network is an autoencoder based on Resnet-type gated linear units (ResGLUs), each comprising a generalized divisive normalization (GDN) layer. We train both networks to optimize perceptual attributes estimated using a multi-timescale scheme to obtain high perceptual quality. Experimental results show that the proposed model accurately predicts the mean and scale hyperpriors. Also, it obtains consistently higher audio quality than the commercial MP3 audio coder at all bitrates.

A new end-to-end audio coder based on a deep neural network (DNN) is proposed. To compensate for the perceptual distortion that occurred by quantization, the proposed coder is optimized to minimize distortions in both signal and perceptual domains. The distortion in the perceptual domain is measured using the psychoacoustic model (PAM), and a loss function is obtained through the two-stage compensation approach. Also, the scalar uniform quantization was approximated using a uniform stochastic noise, together with a compression-decompression scheme, which provides simpler but more stable learning without an additional penalty than the softmax quantizer. Test results showed that the proposed coder achieves more accurate noise-masking than the previous PAM-based method and better perceptual quality then the MP3 audio coder.

In this paper, we present a time-frequency (TF)-dependent a priori speech absence probability (SAP) estimator utilizing the magnitude square coherence (MSC) between two microphone signals. It is shown that the normalized SNR can be numerically computed from the MSC by solving a quadratic equation. Based on the fact that the normalized SNR is bounded between 0 and 1, we directly use it for the probability of speech absence in each TF-unit. Since this approach does not require prior statistical knowledge of noise and speech, it is not affected by the performance of the noise PSD estimator. Furthermore, unlike the conventional SNR-based estimator, additional mapping strategy is unnecessary. The algorithm was evaluated using the receiver operating characteristic (ROC) curve and it attained higher correct detection rate at a given false-alarm rate than the conventional algorithms.

In this letter, we propose a new auditory distance rendering (ADR) algorithm based on the interchannel phase difference (ICPD) control. In the conventional ICPD control, distance perception of the sound image is nonlinearly controlled, and directional localization of the sound image can be biased by changes of the interaural cues. These problems are caused by applying the frequency-independent ICPD without considering the acoustic transfer paths of the system setup. To solve these problems, first, the interaural cues of ear signals are analyzed by binaural auditory simulations. Then, stereophonic ADR filters are designed that produce ear signals with a linearly controlled interaural cross-correlation (IACC) and a consistent interaural level difference (IALD) for sophisticated distance perception under the given stereo setup. Subjective test results show that the proposed algorithm can provide better distance controllability than the conventional method with reduced lateralization blur of the sound image.

The conventional target power density spectrum (PSD) estimation methods based on the signal prediction inherently produce a biased target PSD because of irrelevant assumptions for the noisy environment. In this paper, an unbiased target PSD is obtained by removing the effect of diffuse noise on the prediction filter. In addition, by on-line estimation of both the noise PSD and target transfer function ratio (TFR) from the input signals, the proposed algorithm achieves robust noise suppression for an unknown target direction under a fast time-varying noisy environment. Computer simulations demonstrate the effectiveness and superiority of the proposed algorithm over the conventional methods.

This letter describes a numerical algorithm for synthesizing optimal low-sidelobe beampatterns. The pattern synthesis problem is formulated as a constrained optimization that minimizes the spatially weighted energy arriving at the array subject to unit gain in the look direction and a constant sidelobe level. The weighting takes on the form of a sensor correlation matrix, parameterized between uncorrelated sensor noise and correlated signal arrival terms according to a balancing factor. Setting the balancing factor to its extreme values of 0 and 1, produces, respectively, the optimal Riblet-Chebyshev and Delay-and-Sum type beamformers. In between, the method generates a low-sidelobe beamformer with varying beamwidths that provides a trade-off between White Noise Gain and Directivity Index. Simulation examples demonstrate optimal and intermediate designs for uniform and non-uniform sensor spacings.

In this paper, we present an unsupervised noise PSD estimation algorithm for binaural hearing aids in a time-varying diffuse noise field. It is shown that the noise PSD can be obtained from the eigenvalues of the input covariance matrix together with the noise coherence function effective at low frequencies. To reduce the estimation bias due to fast smoothing, pre- and post-compensation methods are proposed. The proposed algorithm is able to track non-stationary noise PSD without tracking delay or underestimation problems. Its performance is independent of the target speech direction and input SNR. Results of the objective parameter evaluation demonstrate the superiority of the proposed algorithm over conventional techniques.

The gradient adaptive lattice (GAL) algorithm is very attractive choice for active noise control of multiple sinusoidal interferences. In the GAL algorithm, a selection of step-size parameters trades off between convergence speed and steady-state performance. In this paper, we develop a variable step-size scheme for the filtered-x GAL (VSS-FxGAL) algorithm. This proposed algorithm achieves a good compromise between fast convergence speed and low steady-state mean-square error (MSE). In addition, comparing to the filtered-x affine projection (FxAP) algorithm, the proposed algorithm performs better when the filter input consists of multiple sinusoids.

This paper proposes an improved phase parameter analysis and synthesis method for parametric stereo audio coding. For stereo signal with low interchannel coherence (IC), conventional phase parameter analysis and synthesis methods show poor quality, since they overlook the effect of IC to the phase parameters defined. The proposed system defines new phase parameters reflecting IC information, so that we can alleviate the deterioration of audio quality due to the synthesis of excessive phase difference, especially when IC is low. Subjective listening test results are presented to show that the proposed method provides always better audio quality than the conventional methods.

This paper presents a mathematical analysis of the effects of interchannel amplitude and time differences in two channel (stereophonic) sound systems. The analysis is developed by computing the acoustic conditions at the listener's ears as a function of the stereophonic signal feature. We also present separate approximations of head-related transfer function (HRTF)-based panning according to predefined frequency bands. We attempt to create non-sophisticated models of the stereophonic listening mechanism with approximations in both the time and frequency domains. The models are based upon psychoacoustical theories that present the frequency-dependent relative importance of acoustical cues. Based on the model, we propose new panning methods that can enhance the localization accuracy of conventional panning methods, such as amplitude panning and HRTF-based panning. The localization performances of the new panning techniques are evaluated and compared by means of auditory model simulations and listening tests. Through simulations and listening test results, it is shown that the proposed panning method makes substantial improvements in the localization of virtual sources.

In this paper, we propose a method of restoring principal to ambient energy ratio (PAR) at the decoder in the principal component analysis (PCA)-based parametric audio coding. The conventional approach applies the post-scaling at the decoder using the energy information extracted from the input signal at the encoder. However, this approach has a problem that the relative energy of principal source in the reconstructed signal is smaller than the original signal and also affected by the panning angle of principal source. To restore the PAR at the decoder, the proposed method estimates the post-scaling factors using parametric information extracted from PCA-based formulation. The objective and subjective results verify the performance improvement of proposed method.

In this paper, we propose a new soft-decision adaptation mode controller (SD-AMC) for frequency domain generalized sidelobe canceller (GSC) as a speech enhancement system. Contrarily to conventional systems that update filter coefficients in a hard-decision manner using voice activity detection (VAD), the proposed method flexibly controls the step-sizes of adaptive filters depending on the probability of speech presence in each frequency bin. Therefore, it further improves the system performance for various environments without much consideration on noise type and signal to noise ratio (SNR) of input signal. It also improves the robustness of GSC system by avoiding the miss-classification error by the hard-decision logic. Experimental results with speech recognition systems verify that the SD-AMC shows higher performance than ideally designed hard-decision approaches.

This paper describes a method for jointly designing the crosstalk cancellation filters to facilitate binaural rendering of audio through loudspeakers. The minimax criterion is used to design the immersive audio rendering filters having finite impulse responses for a single listener using loudspeakers. The work presented is applied to the traditional Atal-Schroeder crosstalk canceler structure. The minimax approach provides improved low frequency performance and a better overall separation of the direct path and cross path transfer functions than the conventional least-squares designs.

This paper presents a method for jointly designing immersive audio rendering filters for a single listener using loudspeakers. The filters for crosstalk cancellation are assumed to have finite impulse responses and are designed using the minimax criterion. In addition to the traditional Atal-Schroeder crosstalk canceler structure, this paper explores an alternate topology that requires the approximation of a single filter. In general, the minimax approach provides improved low-frequency performance leading to a better overall separation of the direct-path and cross-path transfer functions than least-squares designs. The performance of the single-filter structure is better than that of the traditional crosstalk cancellation structure.

In this paper, we propose an audio transcoding algorithm for mobile multimedia applications. The algorithm is to provide high quality transcoded audio stream from a multimedia broadcasting media server system to mobile terminals. As a representative case, audio transcoder between T-DTV and T-DMB service is considered. While the Korean DTV audio standard adopted the Dolby AC-3, the Korean T-DMB service uses the MPEG-4 BSAC. The proposed algorithm reuses the bit allocation information of the AC-3 decoder in the process of BSAC encoding and simplifies the bit-allocation process with two independent loops instead of nested ones. We verified that our architecture could take the place of the nested loop with a significant simplification and no PAM. The proposed audio transcoding algorithm is suitable for the mobile multimedia applications due to its low complexity and minimum frame-by-frame complexity variations.

This paper presents a novel approach for implementing immersive audio rendering filters for a single listener using loudspeakers. We address the problem of crosstalk cancellation inherent in loudspeaker rendering and propose to implement the crosstalk cancellation filters using minimax finite impulse response (FIR) filters. The formulation is based on the Atal-Schroeder crosstalk canceller. The use of the optimal FIR filter design procedure ensures significant amount of separation between the direct path and the cross path. An alternative topology which requires the approximation of just one filter has also been explored using the same design principles. The minimax techniques provides superior solutions as compared to a least-squares design and the alternate structure is shown to be robust in its performance.

This letter presents a new fast search algorithm for the multitap adaptive codebook used in the G.723.1 standard speech coder. In contrast with the standard method that a closed-loop pitch lag and gains for a fifth-order pitch predictor are searched simultaneously, the proposed algorithm adopts a sequential and restricted approach to determine the parameters. In other words, the proposed scheme first determines a couple of pitch lag candidates using a first-order pitch predictor and then computes the pitch gains of the fifth-order predictor within a restricted search area. Experimental results confirm that the proposed algorithm reduces the total complexity by 30.69% in the encoding process and provides speech quality equivalent to the standard method.

The paper describes important issues on transcoding between different speech codecs by considering the paradigms of source and target coders. Conventional transcoding algorithms on LSP, pitch and adaptive/fixed codebook conversion are refined with regard to the structure of the coders. In addition, a new perceptual weighting filter, that plays a role in the post-filter and perceptual weighting filter together, is proposed to improve the performance further. The performance of the proposed algorithms is verified in a step-by-step manner with examples of transcoding between AMR, G.723.1 and G.729. By applying the proposed algorithm to the transcoders, the complexity is reduced by about 20-76.88% and quality is also improved compared to conventional approaches.

We propose a packet loss concealment algorithm for a code-excited linear prediction (CELP) speech coder. We perform a time-scale modification (TSM) using a waveform similarity overlap-add (WSOLA) technique to reconstruct the excitation signal of the lost or dropped frames. In addition, when a lost frame is classified as a voiced, an adaptive codebook gain and a fixed codebook gain are estimated by a modified gain parameter re-estimation (GRE) technique. By applying these techniques, we can reduce quality degradation of the decoded speech and error propagation effect through the adaptive codebook memory. We apply the proposed scheme to the ITU-T G.729 standard speech coder to evaluate the performance of the proposed method. The perceptual evaluation of speech quality (PESQ) and AB preference tests under various packet loss conditions verify that the proposed algorithm is superior to the concealment algorithm embedded in the G.729.

In this paper, a discrete multitone (DMT) modem for the client side G.LITE ADSL transceiver is presented. The DMT modem is one of the most computationally intensive parts in the transceiver. In the design, computational complexity and operational regularity were traded-off to decide suitable fast Fourier transform (FFT)/Inverse FFT and frequency-domain equalization (FEQ) algorithms. We used the radix-2 real FFT and the normalized LMS that guarantees an equal convergence per tone. Based on operational community between FFT and FEQ, a resource-shared design was implemented. Datapath of the DMT modem comprises one real multiplier and two real adders, which are connected in parallel and flexibly used under control. The designed modem has been coded in VHDL and synthesized with 0.35-/spl mu/m standard CMOS cell. Synthesis results exhibit that the proposed system can provide a cost-effective solution to asymmetric digital subscriber line devices owing to its small design area.

In this paper, we present a highly optimized one-chip digital audio decoder supporting both MP3 and AAC standards. The design has been focused on the hardware modularity for both MP3 and AAC. The hardware consists of a Huffman decoder module and 20-bit programmable DSP module. In addition, a hardware module for the prediction tool of AAC has been added to support AAC Main Profile. Utilizing the modularity of the designed hardware, MP3 and AAC decoding algorithms are distributed to each module in such a way that the system resource can be maximally utilized. The designed system can decode MPEG/Audio layer-3 bitstream using only 13.33 MIPS and MPEG-2 AAC Main Profile bitstream using only 16.9 MIPS while the high accuracy of the decoded output PCM can be obtained with it.

In this paper, we present a new bandwidth-scalable coder for wide band speech and audio signals. The proposed coder splits 8 kHz signal bandwidth into two narrow bands, and different coding schemes are applied to each band. The lower-band speech is coded with ITU-T G.729 Annex E, and the higher-band signal is compressed using a new algorithm based on the gammatone filter band with an invertible auditory model. Due to the split-band architecture and completely independent coding schemes for each band, the output speech of the decoder can be selected to be a narrowband or wideband coding according to the channel conditions. Subjective tests showed that, for wideband speech and audio signals, the proposed coder at 18 kbit/s produces superior quality to ITU-T 24 kbit/s G.722.1 with the shorter algorithmic delay.

In OFDM systems, the symbol timing error over multipath fading channels should stay inside the ISI free range of the guard interval. Although correct symbol timing of DAB system can be obtained using a 2048-point IFFT of the phase-demodulated pilot symbol, this approach demands very high computational works. In this paper, a new computationally efficient symbol synchronization algorithm is presented. The presented algorithm employs a 64-point IFFT of the decimated pilot symbol, but it still can provide accurate symbol timing that is comparable to the method associated with the 2048-point IFFT. The proposed method can save more than 80% of the computation works while the performance is not deteriorated.

A system architecture optimized for the 2-channel main-profile MPEG-2 AAC decoder is presented. In order to enable an efficient job scheduling and allocation, the presented system comprises three hardware modules: Huffman decoder module, predictor module, and processing core module which is programmable using an assembly language of its own. The Huffman decoder is designed to finish the requested job in only 1 clock cycle time and the predictor forms parallel processing with other modules, so that utilization of the system resource is maximized. The developed system has been coded in VHDL and the MPEG-2 AAC decoding algorithm is programmed using the assembly language of the processing core. For the verification of the decoding algorithm, the 16-bit PCM output of the system was compared with the result of the floating-point simulation, and the result showed the maximum of 2-bit difference. Functional simulation verified that the developed system can decode standard MPEG-2 AAC main-profile bitstreams in real-time with high accuracy.

Dual-rate G.723.1 speech coder has been widely applied to real-time video and teleconferencing applications where reduced bandwidth and good voice quality is required. This paper presents an efficient implementation of G.723.1 speech coder. To simplify the excitation quantization procedure which is the most computationally demanding, we propose fast algorithms for adaptive codebook and fixed codebook search. In the fast adaptive codebook search, pitch delay and pitch gains are computed sequentially. In the fast fixed codebook search, the codebook structure is redesigned based on the interleaved single-pulse permutation (ISPP) design at high rate mode and the depth-first tree search is applied instead of nested-loop search at low rate mode. A real-time implementation is achieved using a 16-bit fixed-point TMS320C62x DSP. The implemented G.723.1 speech coder requires 8.70 and 10.29 MHz clock cycles at low and high rate, respectively, 57.8 kByte of program memory and 55 kByte of data memory. Thus, more than 16 channels of G.723.1 coder can be operated in real-time using a single TMS320C62x DSP.

Interoperability is one the most important factors for a successful integration of the speech network. To operate speech networks employing different speech coders, but integrated as one, bitstreams generated by one coder should be translated seamlessly to those of the other coders. Connecting two coders in tandem may be the simplest way to accomplish this. However, coders in tandem connection often produce problems such as poor speech quality, high computational load, and additional transmission delay. We propose an efficient transcoding algorithm that can provide interoperability to the networks employing ITU-T G.723.1 and TIA IS-127 EVRC speech coders. Subjective and objective quality evaluation have confirmed that the speech quality produced by the proposed transcoding algorithm was equivalent to, or better than the tandem coding, while it had shorter processing delay and less computational complexity.

In this paper, we present an efficient noise control algorithm based on the delayless subband adaptive filter. The algorithm reduces the computational complexity of the delayless subband filter by decomposing the secondary path response into a set of subband functions. In this new algorithm, the filtered reference signal is generated for each subband by using a short impulse response filter that models the secondary path transfer function in a subband-decomposed form. Computational efficiency of the presented method originates from the fact that the filtering process occurs only in one subband for each reference input sample. Furthermore, computational efficiency is enhanced when the presented algorithm is combined with an online identification of the secondary path transfer function. We also propose a frequency-domain implementation of the presented algorithm. In this implementation, it is shown that the computational complexity is further reduced by employing the block-processing approach.

Stereophonic sound becomes more important in a growing number of hands-free applications where spatial realism is demanded. Such hands-free systems need stereophonic acoustic echo cancellers to reduce echoes that result from coupling between loudspeakers and microphones. In this paper, we propose a new adaptive algorithm for an stereophonic acoustic echo canceller based on human auditory properties and affine projection (AP) algorithm. The proposed algorithm employs a pre-processor generating speech-like noise to decorrelate the input signals without degrading perceptual speech quality. The decorrelation is based on the masking patterns of the human auditory system. By showing that the AP algorithm can be represented by a vector update as a combination of the Gram-Schmidt (GS) orthogonalization followed by the normalized LMS (NLMS) algorithm, the proposed adaptive algorithm integrates the pre-processor into the adaptive algorithm. Subjective listening test and computer simulation show the effectiveness of the proposed algorithm.

This paper presents an optimum space-time moving target indication (MTI) processor for the airborne radar. The optimization is based on a stochastic target model, rather than deterministic target models adopted in most space-time MTI processor designs. The optimum solution that maximizes the improvement factor yielded by the processor is shown to be the generalized eigenvector corresponding to the smallest generalized eigenvalue of the signal and clutter covariance matrices. A suboptimal, but computationally simpler solution to this problem is also derived. This approach requires the solution of a linearly constrained minimum variance (LCMV) problem. Unlike typical LCMV problems, our solution also calculates the response vector specifying the frequency response along the look direction. Experimental results demonstrating the usefulness of our methods are included in the paper. The results indicate that the suboptimal solution does not suffer from significant performance loss.

In this paper, a new active noise control algorithm based on a delayless subband adaptive filter architecture is presented. Also, an on-line system identification method implemented in the subband structure is suggested. To implement the filtered-x LMS algorithm in the subband structure, the secondary path transfer function is decomposed into sets of subband functions. The two filter on-line modeling algorithm is then applied to each subband to estimate the secondary-path transfer function in a decomposed form. In this manner, the computational load for the on-line system identification is reduced by a factor 3 compared with the wideband approach. Simulation results are presented to show the efficiency of the new ANC algorithm and the performance of the on-line system identification scheme.