This paper proposes an algorithm to estimate the melody pitch line (the most dominant pitch sequence) of a given polyphonic audio based on melody range estimation and pitch candidate extraction using a harmonic structure model similar to that proposed by Goto. This paper defines melody pitch candidate as a list of pitch candidates that produces the best-fit harmonic models to the polyphonic audio. In many melody extraction algorithms proposed in the past, multiple-pitch extractor (MPE) is often performed for extracting melody pitch candidates; however, the MPE serves the purpose of estimating all pitches within a frame of a polyphonic audio and does not necessarily provide melody pitch candidates. The estimated weights of the harmonic structure model which must be obtained for extracting the pitch candidates are liable to octave error and strong low frequency interference, and therefore, certain refinement after the estimation must be performed. As a refinement, the algorithm measures the degree of harmonic fitness of each candidate. Furthermore, a melody pitch range is estimated to reduce false-positive pitch candidates. The melody pitch range is estimated based on the distribution of the best pitch candidates with long duration. Experimental results show that the proposed extraction algorithm performed better than many of the algorithms proposed in the past.

This paper considers a psychoacoustically constrained and distortion minimized speech enhancement algorithm. Noise reduction, in general, leads to speech distortion, and a balanced tradeoff between noise reduction and speech distortion must be attained. A constrained optimization problem is set to reduce noise so that speech distortion is minimized while the sum of speech distortion and residual noise is kept below the masking threshold of the clean speech. Obtaining a solution to the optimization problem may be infeasible under certain conditions, and a slack variable is introduced to allow certain deviation from the constraint conditions. To estimate the power spectral density and also the masking threshold of clean speech, a speech model that assumes coexisting deterministic and stochastic components in speech is used. Experimental results show that the considered algorithm outperforms some of the more popular algorithms in terms of improvement in segmental signal-to-noise ratio (SegSNR), spectral distance (SD), modified Bark spectral distortion (MBSD), and mean opinion score (MOS).

A psychoacoustically constrained and distortion minimized speech enhancement algorithm is considered. In general, noise reduction leads to speech distortion, and thus, the goal of an enhancement algorithm should reduce noise and speech distortion so that both are inaudible. In this paper, a constrained optimization problem is formulated so that speech distortion is minimized while distortion that includes residual noise and speech distortion is kept below the masking threshold of the clean speech. Experimental results show that the algorithm considered in this paper outperforms some of the more popular algorithms in terms of improvement in segmental signal-to-noise ratio (SegSNR) and spectral distance (SD).

A simple free viewpoint video system which is able not only to display user-specified views at arbitrary angle but also to efficiently stream the necessary video over a network is described. Virtual view synthesis based on projection theory was implemented to obtain intermediate views between multiple cameras. The depth information, which is required for the virtual view synthesis, was obtained using the segmentation-based stereo matching algorithm. For realtime rendering, our system was optimized using Single Instructions, Multiple Data (SIMD) technology. For efficient streaming, a novel method of combining the video with the depth information is proposed.

A novel speech enhancement algorithm based on both a decomposition of speech into coexisting deterministic and stochastic components and a psychoacoustic model is proposed. Noisy speech is first decomposed into deterministic and stochastic components, and then each component is enhanced preserving its individual characteristics. A psychoacoustic model is taken into account when enhancing the stochastic component which usually has much lower energy than the deterministic component. Simulation results show that the proposed algorithm performs better than some of the more popular algorithms in terms of segmental signal-to-noise ratio (SNR) and speech recognition rate.