Multichannel active noise control (ANC) systems have been widely investigated for low-frequency noise attenuation over a spatial region. Using a conventional centralized control strategy based on the multichannel filtered-xleast mean square (FxLMS) algorithm has been demonstrated to be effective for multichannel ANC systems, but the high computational burden restricts its practical applications. Meanwhile, a decentralized control strategy suffers from stability problems although it has been successful in reducing the computational load. Recently, distributed control strategies, such as the multitask diffusion adaptation scheme, have been introduced to ANC systems and shown to mitigate the stability problems in decentralized systems. However, distributed ANC systems using the diffusion FxLMS algorithm require strong symmetry of acoustic paths because of the dependence on node-based adaptation and neighborhood-based combination. To overcome this limitation, this paper proposes an Augmented Diffusion FxLMS algorithm with neighborhood-based adaptation and node-based combination. A theoretical formulation and convergence analysis are presented and simulations are performed to compare the proposed algorithm with existing ones under different system configurations for tonal, multi-tonal, narrowband and broadband signals. Simulation results demonstrate that the proposed algorithm has the same noise reduction performance as centralized method even if the acoustic paths are strongly asymmetrical, which is superior over existing distributed multitask diffusion strategy.

Personal audio systems with multiple sound zones for listeners to enjoy different music/audio contents privately in a shared physical space have attracted great research interest in the past two decades. Acoustic Contrast Control (ACC) is one of the most popular methods for generating multiple personal sound zones because it produces the minimum inter-zone interference. However, the ACC method has been found to be inferior to the pressure matching method in terms of sound quality due to an uneven frequency response and nonuniform spatial sound field distribution in the bright zone. This paper proposes a spatial uniformity constraint on time-domain broadband ACC in addition to the frequency response trend estimation constraint with the aim of ensuring a uniform sound field distribution in the bright zone. Simulation results with measured room impulse responses demonstrate that the proposed algorithm reduces the magnitude variations in the bright zone to be less than 1 dB higher than the just noticeable level difference at a cost of a perceptually negligible degradation in acoustic contrast.

Remote error sensing is required in active noise control systems when they are used to create a quiet zone in a noisy environment with the constraint that the error microphones cannot be inside the zone. The challenge in remote error sensing is to estimate the sound pressure in the target zone with a small number of physical microphones outside it. The spatial harmonic decomposition method uses wave domain sound field parameterisation to reduce the required number of the error microphones but can only provide accurate estimation below a certain frequency. This paper presents an improved approach to increase the effective frequency range based on the acoustic modelling. The simulation results demonstrate the proposed method can provide more than 20 dB noise reduction up to 1650 Hz for a quiet zone with a radius of 0.1 m by using only three microphones under the studied situations.

In an active noise control (ANC) system, computational complexity is one major concern when designing practical control algorithms. For an ANC system with multiple secondary sources and error microphones, one approach to reducing computational complexity is to apply a decentralized control scheme rather than centralized approaches. A decentralized scheme attempts to control a number of small-size ANC subsystems independently. In this paper, we consider the decentralized control of a two-channel ANC system tackling a noise disturbance in the frequency domain, where each channel consists of one secondary source and one error microphone. We propose a decentralized control method that is able to achieve the same noise reduction performance as the centralized controller with guaranteed convergence. The key step in designing the control method is to properly shape the eigenvalues of a matrix that models the two-channel secondary paths for each frequency index.

Personal audio generates sound zones in a shared space to provide private and personalized listening experiences with minimized interference between consumers. Regularization has been commonly used to increase the robustness of such systems against potential perturbations in the sound reproduction. However, the performance is limited by the system geometry such as the number and location of the loudspeakers and controlled zones. This paper proposes a geometry optimization method to find the most geometrically robust approach for personal audio amongst all available candidate system placements. The proposed method aims to approach the most “natural” sound reproduction so that the solo control of the listening zone coincidently accompanies the preferred quiet zone. Being formulated in the SVD-based modal domain, the method is demonstrated by applications in three typical personal audio optimizations, i.e., the acoustic contrast control, the pressure matching, and the planarity control. Simulation results show that the proposed method can obtain the system geometry with better avoidance of “occlusion,” improved robustness to regularization, and improved broadband equalization.

Room geometry estimation from corresponding Room Impulse Responses (RIRs) has attracted much attention in recent years, and a key challenge is to find the first order image source locations from the RIRs under different environments. Unlike the existing approaches which require a priori knowledge of the room or require some ideal conditions, this paper proposes an intuitive geometrical method based on the acoustical image source model. The proposed approach does not need any a priori knowledge of the room, only the RIRs from one arbitrary source location to five arbitrary receiving locations. The first order image sources of the walls in a room are identified first, then the room geometry is estimated based on the wall locations using a geometrical approach. Simulations with 2D and 3D convex polyhedral rooms demonstrate the feasibility and the precision of the proposed approach is discussed.

Emotional factors usually affect users' preferences for and evaluations of images. Although affective image analysis attracts increasing attention, there are still three major challenges remaining: 1) it is difficult to classify an image into a single emotion type since different regions within an image can represent different emotions; 2) there is a gap between low-level features and high-level emotions and 3) it is difficult to collect a training set of reliable emotional image content. To address these three issues, we propose an emotion classification method based on multi-scale blocks using Multiple Instance Learning (MIL). We firstly extract blocks of an image at multiple scales using different image segmentation methods pyramid segmentation and simple linear iterative clustering (SLIC) and represent each block using the bag-of-visual-words (BoVW) method. Then, to bridge the “affective gap”, probabilistic latent semantic analysis (pLSA) is employed to estimate the latent topic distribution as a mid-level representation of each block. Finally, MIL, which reduces the need for exact labelling, is employed to classify the dominant emotion type of the image. Experiments carried out on three widely used datasets demonstrate that our proposed method with S-LIC effectively improves the state-of-the-art results of image emotion classification 5.1% on average.

This paper presents an approach for multizone wideband sound field generation using an efficient harmonic nested (EHN) dictionary for sparse loudspeakers' placement and weight. Effectively, the nested arrays provide a priori knowledge of prospective loudspeaker locations based on the frequency bands of interest. The nested arrays are then further optimized in the Lasso stage to form an efficient loudspeakers' location dictionary. The final loudspeaker locations and weightings are estimated by a two-stage Lasso-LS pressure matching optimization. In the first stage Lasso algorithm, the center band frequencies of octave bands from 1 kHz to 8 kHz were used to select active loudspeakers. A second stage then optimizes reproduction using all selected loudspeakers on the basis of a regularized LS algorithm. The results demonstrate that the proposed approach provides a solution for the multizone sound system with the mean squared error (MSE) under -30 dB across the targeted frequency range (500 Hz- 16 kHz) using a linear array e.g. 13 loudspeakers. While, the single-stage LS approach generates the MSE peaks of -10 dB and -9 dB at 9 kHz within the active and silent zones respectively using an identical number of loudspeakers and array length.

Saliency maps that integrate individual feature maps into a global measure of visual attention are widely used to estimate human gaze density. Most of the existing methods consider low-level visual features and locations of objects, and/or emphasize the spatial position with center prior. Recent psychology research suggests that emotions strongly influence human visual attention. In this paper, we explore the influence of emotional content on visual attention. On top of the traditional bottom-up saliency map generation, our saliency map is generated in cooperation with three emotion factors, i.e., general emotional content, facial expression intensity, and emotional object locations. Experiments, carried out on National University of Singapore Eye Fixation (a public eye tracking data set), demonstrate that incorporating emotion does improve the quality of visual saliency maps computed by bottom-up approaches for the gaze density estimation. Our method increases about 0.1 on an average of area under the curve of receiver operation characteristic curve, compared with the four baseline bottom-up approaches (Itti's, attention based on information maximization, saliency using natural, and graph-based vision saliency).

In this paper a one-shot method to determine the shape of an object from overlapping cosine fringes projected from multiple projectors is presented. This overcomes the limitation with single projector systems that do not allow imaging the entire object with a single shot. The proposed method projects orthogonal fringe patterns from different projectors and uses Fourier domain filtering to isolate the fringes, which are demodulated using an unscented particle filter. Sources of error are discussed and their effects on the resulting parameter estimation are shown, as well as methods to reduce their impact. The proposed method is tested on simulations and real world objects and it is shown to be effective to isolate interfering fringes and determine the shape of an object.

This paper proposes the use of the Iterated Extended Kalman Filter (IEKF) in a real-time 3D mapping framework applied to Microsoft Kinect RGB-D data. Standard EKF techniques typically used for 3D mapping are susceptible to errors introduced during the state prediction linearization and measurement prediction. When models are highly nonlinear due to measurement errors e.g., outliers, occlusions and feature initialization errors, the errors propagate and directly result in divergence and estimation inconsistencies. To prevent linearized error propagation, this paper proposes repetitive linearization of the nonlinear measurement model to provide a running estimate of camera motion. The effects of iterated-EKF are experimentally simulated with synthetic map and landmark data on a range and bearing camera model. It was shown that the IEKF measurement update outperforms the EKF update when the state causes nonlinearities in the measurement function. In the real indoor environment 3D mapping experiment, more robust convergence behavior for the IEKF was demonstrated, whilst the EKF updates failed to converge.

Standard surround systems for generation of isolated wideband soundfields employ uniformly-spaced array of speakers in the horizontal plane. For these systems, the evaluation of sound reproduction with height is important due to listener's variable heights. Previous work demonstrated that controlling both the speakers' location and their complex weights using two-stage Lasso-LS pressure matching optimization allows isolated sound reproduction with limited number of speakers within the speakers' plane. This work demonstrates that deployment of this technique can also give up to 24dB in suppression of sound at heights between zero and one meter from speakers' plane over single-stage LS using e.g. 90 speakers in a semicircular array.

This paper introduces a dereverberation algorithm based on Linear Prediction (LP) applied to the outputs of an Acoustic Vector Sensor (AVS). The approach applies adaptive beamforming to take advantage of the directional outputs of the AVS array to obtain a more accurate LP spectrum than can be obtained with a single channel or Uniform Linear Array (ULA) with a comparable number of channels. This is then used within a modified version of the Spatiotemporal Averaging Method for Enhancement of Reverberant Speech (SMERSH) algorithm derived for the AVS to enhance the LP residual signal. In a highly reverberant environment, the approach demonstrates a significant improvement compared to a ULA as measured by both the Signal to Reverberant Ratio (SRR) and Speech to Reverberation Modulation Energy Ratio (SRMR) for sources ranging from at 1m to 5m from the array.

An understanding of the dynamics underlying emotional interactions between speakers is essential to the design of effective conversational strategies for interviews, mental health therapies, teaching and counseling, as well as the design of naturalistic human-machine communication systems. The present study introduces a new approach to the modeling of emotional influences during parent-adolescent conversations. The proposed dynamic influence model (DIM) estimates the joint conditional probabilities of speaker's states as a linear combination of simpler inter- and intra-speaker conditional probabilities. Contrary to the previously existing influence models (IMs), the DIM's coefficients are given not as static, constant values but as dynamically changing functions of the time delay between the current and the previous state. The speaker's states were annotated using four labels (speech with positive emotion, speech with negative emotion, emotionally neutral speech and silence with undefined emotion). Experimental results based on the audio recordings of 63 different naturalistic (not acted) parent-adolescent conversations showed that the proposed method leads to psychologically plausible observations. It was also demonstrated that the proposed DIM can achieve up to 20 percent higher accuracy of discriminating between emotional influence patterns of parents and adolescents when compared to the previously used static IM.

This paper presents a technique for low bit rate compression of three-dimensional (3D) audio produced by multiple loudspeaker channels. The approach is based on the time-frequency analysis of the localization of spatial sound sources within the 3D space as rendered by a multi-channel audio signal (in this case 16 channels). This analysis results in the derivation of a stereo downmix signal representing the original 16 channels. Alternatively, a mono-downmix signal with side information representing the location of sound sources within the 3D spatial scene can also be derived. The resulting downmix signals are then compressed with a traditional audio coder, resulting in a representation of the 3D soundfield at bit rates comparable with existing stereo audio coders while maintaining the perceptual quality produced from separate encoding of each channel.

The prohibitive number of speakers required for the reproduction of isolated soundfields is the major limitation preventing solution deployment. This paper addresses the provision of personal soundfields (zones) to multiple listeners using a limited number of speakers with an underlying assumption of fixed virtual sources. For such multizone systems, optimization of speaker positions and weightings is important to reduce the number of active speakers. Typically, single stage optimization is performed, but in this paper a new two-stage pressure matching optimization is proposed for wideband sound sources. In the first stage, the least-absolute shrinkage and selection operator (Lasso) is used to select the speakers' positions for all sources and frequency bands. A second stage then optimizes reproduction using all selected speakers on the basis of a regularized least-squares (LS) algorithm. The performance of the new, two-stage approach is investigated for different reproduction angles, frequency range and variable total speaker weight powers. The results demonstrate that using two-stage Lasso-LS optimization can give up to 69 dB improvement in the mean squared error (MSE) over a single-stage LS in the reproduction of two isolated audio signals within control zones using e.g. 84 speakers.

The recent ubiquity of mobile telephony has posed the challenge of forensic speech analysis on compressed speech content. Whilst existing research studies have investigated the effect of mobile speech compression on speaker and speech parameters, this paper addresses the effect of speech compression on parameters when an interfering background speaker is present in clean and noisy conditions. Preliminary evaluations presented in this paper study the effect of the Adaptive Multi-Rate (AMR) and Adaptive Multi-Rate Wideband (AMR-WB) speech coders on the Linear Prediction (LP) speech spectrum, Line Spectral Frequencies (LSFs), and Mel Frequency Cepstral Coefficients (MFCCs). Results indicate that due caution should be employed for the forensic analysis of mobile telephony speech: speech coder parameters are significantly degraded when an interfering speaker or noise is present, compared to parameters obtained from the main speaker alone. Moreover, at high SNR the speech parameters exhibit values that gradually transition from those ideally and independently obtained from the main speaker to those of the background speaker as the amplitude of the background interfering speaker increases.

While higher order ambisonic approaches can be used to generate multiple zone soundfields, this paper adopts a Least Squares matching approach which provides a more flexible formulation. The base approach, adopted from computes speaker weights which allow for the placement of single sources in the soundfield. In this paper the approach is extended firstly to two multi-frequency sources and then to narrowband speech signals. The results for multi-frequency sources explore the zonal soundfield errors resulting from varied source positions. For speech signals, the approach provides a potential solution for multiple conversation reproduction in a multi user environment. The paper results indicate that the approach is feasible for zones which do not suffer occlusion effects from other zones. However, for more versatile multizone soundfield reproduction a 3D approach is recommended.

This paper investigates the performance of a new technique for speech enhancement which combines Linear Predictive (LP) spectrum-based perceptual filtering to the recordings obtained from an Acoustic Vector Sensor (AVS). The technique takes advantage of the directional polar responses of the AVS to obtain a significantly more accurate representation of the LP spectrum of a target speech signal in the presence of noise when compared to single channel, omni-directional recordings. Comparisons between the speech quality obtained from the proposed technique and existing beamforming-based speech enhancement techniques for the AVS are made through Perceptual Evaluation of Speech Quality (PESQ) tests and Mean Opinion Score (MOS) listening tests. Results show significant improvements in PESQ and MOS scores of 0.2 and 1.6, respectively, for the proposed enhancement technique. Being based on a miniature microphone array, the approach is particular suitable for hands free communication applications in mobile telephony.

This paper considers a self-calibration approach to the estimation of motion parameters for an unknown camera used for video-based augmented reality. Whilst existing systems derive four SVD solutions of the essential matrix, which encodes the epipolar geometry between two camera views, this paper presents eight possible solutions derived from mathematical computation and geometrical analysis. The eight solutions not only reflect the position and orientation of the camera in static displacement but also the dynamic, relative orientation between the camera and an object in continuous motion. This paper details a novel algorithm that introduces three geometric constraints to determine the rotation and translation matrix from the eight possible essential matrix solutions. An OpenGL camera motion simulator is used to demonstrate and evaluate the reliability of the proposed algorithms; this directly visualizes the abstract computer vision parameters into real 3D.

This paper proposes a flexible, markerless registration method that addresses the problem of realistic virtual object placement at any position in a video sequence. The registration consists of two steps: four points are specified by the user to build the world coordinate system, where the virtual object is rendered. A self-calibration camera tracking algorithm is then proposed to recover the camera viewpoint frame-by-frame, such that the virtual object can be dynamically and correctly rendered according to camera movement. The proposed registration method needs no reference fiducials, knowledge of camera parameters or the user environment, where the virtual object can be placed in any environment even without any distinct features. Experimental evaluations demonstrate low errors for several camera motion rotations around the X and Y axes for the self-calibration algorithm. Finally, virtual object rendering applications in different user environments are evaluated.

This paper investigates how the directional characteristics of an Acoustic Vector Sensor (AVS) can be used to separate speech sources. The technique described in this work takes advantage of the frequency domain direction of arrival estimates to identify the location, relative to the AVS array, of each individual speaker in a group of speakers and separate them accordingly into individual speech signals. Results presented in this work show that the technique can be used for real-time separation of speech sources using a single 20ms frame of speech, furthermore the results presented show that there is an average improvement in the Signal to Interference Ratio (SIR) for the proposed algorithm over the unprocessed recording of 15.1 dB and an average improvement of 5.4 dB in terms of Signal to Distortion Ratio (SDR) over the unprocessed recordings. In addition to the SIR and SDR results, Perceptual Evaluation of Speech Quality (PESQ) and listening tests both show an improvement in perceptual quality of 1 Mean Opinion Score (MOS) over unprocessed recordings.

This paper investigates multichannel speech enhancement for colocated microphone recordings based on Independent Component Analysis (ICA). Comparisons are made between co-located microphone arrays that contain microphones with mixed polar responses with traditional uniform linear arrays formed from omni-directional microphones. It is shown that polar responses of the microphones are a key factor in the performance of ICA applied to co-located microphones. Results from PESQ testing show a significant improvement in speech quality of ICA separated sources as a result of using an AVS rather than other types of microphone arrays.

With new social media technologies arising daily, this paper reports on a pilot user survey that studies how tertiary educated users are engaging with social media. The results indicate sporadic use of social media by the tertiary educated users studied; they are generally aware of the key social media sites and facilities, but are not actively utilizing these services. The reasons for, and the implications of a lack of tertiary educated users in the egalitarian and participatory environments intrinsic to social media are discussed. Further, the paper suggests potential technological barriers that might be at the root of such a lack of engagement amongst tertiary educated users.

Spatially squeezed surround audio coding (S3AC) has been previously shown to provide efficient coding with perceptually accurate soundfield reconstruction when applied to ITU 5.1 multichannel audio. This paper investigates the application of S3AC to the coding of Ambisonic audio recordings. Traditional ambisonics achieve compression and backward compatibility through the use of the UHJ matrixing approach to obtain a stereo signal. In this paper the relationship to Ambisonic B-format signals is described and alternative approaches that derive a stereo or mono-downmix signal based on S3AC are presented and evaluated. The mono-downmix approach utilizes side information consisting of spatial cues that are quantized based on novel source localization listening experiments. Objective and subjective tests demonstrate significant improvements in the localization of sound sources resulting from decoding the compressed B-format signals to a 5.1 speaker playback.

Automatic generation of metadata is an important component of multimedia search-by-content systems as it both avoids the need for manual annotation as well as minimising subjective descriptions and human errors. This paper explores the automatic attachment of basic descriptions (or dasiaTagspsila) to human motion held in a motion-capture database on the basis of a dynamic time warping (DTW) approach. The captured motion is held in the Acclaim ASF/AMC format commonly used in game and movie motion capture work and the approach allows for the comparison and classification of motion from different subjects. The work analyses the bone rotations important to a small set of movements and results indicate that only a small set of examples is required to perform reliable motion classification.

Recent research in speech localization and dereverberation introduced processing of the multichannel linear prediction (LP) residual of speech recorded with multiple microphones. This paper investigates the novel use of intra- and inter-channel speech prediction by proposing the use of a multichannel LP model derived from multivariate autoregression (MVAR), where current LP approaches are based on univariate autoregression (AR). Experiments were conducted on simulated anechoic and reverberant synthetic speech vowels and real speech sentences; results show that, especially at low reverberation times, the MVAR model exhibits greater prediction gains from the residual signal, compared to residuals obtained from univariate AR models for individually or jointly modelled speech channels. In addition, the MVAR model more accurately models the speech signal when compared to univariate LP of a similar prediction order and when a smaller number of microphones are deployed.

Several recent standards address virtual containers for rich multimedia content: collections of media with metadata describing the relationships between them and providing an immersive user experience. While these standards - which include MPEG-21 and TVAnytime - provide numerous tools for interacting with rich media objects, they do not provide a framework for streaming or delivery of such content. This paper presents the bitstream binding language (BBL), a format-independent tool that describes how multimedia content and metadata may be bound into delivery formats. Using a BBL description, a generic processor can map rich content (an MPEG-21 digital item, for example) into a streaming or static delivery format. BBL provides a universal syntax for fragmentation and packetization of both XML and binary data, and allows new content and metadata formats to be delivered without requiring the addition of new software to the delivery infrastructure. Following its development by the authors, BBL was adopted by MPEG as Part 18 of the MPEG-21 Multimedia Framework.

This paper presents a novel solution to multichannel spatial audio coding: spatial squeezing surround audio coding (S3AC). The S3AC scheme analyses a multichannel audio signal and downmixes it into a stereo signal pair containing both the monophonic properties of audio sources and their localization information; this avoids the need for side information. The approach uses time-frequency analysis of a spatial audio scene and exploits virtual sources and amplitude panning techniques to 'squeeze' 360deg of a horizontal soundfield to a 60deg stereo signal pair. In comparison with other spatial audio coding techniques, S3AC significantly advances in-band encoding of the localization information in the original sound scene and achieves accurate recoverability of dynamic localized sources.

MQF is a new XML based format designed to facilitate communication between disparate systems for applications involving multimedia query by content. Currently no standardized protocol exists which are able to provide flexibility in formulation of a query, such as the combination of any multimedia format (image, video, sound) to serve as query terms, combined with very complicated query conditions that can utilize a hierarchy of meta-search engines. In this work, we propose MQF as a flexible solution to serve as a communication format between a client and server for use in content based multimedia searching.

A dynamic P2P architecture was proposed in our previous work to support resource adaptation/personalization according to the surrounding usage environment and user preferences. In this paper, we propose an adaptive resource replication strategy based on the proposed P2P architecture. It uses resource request rate as the metric to trigger the resource replication process, and proportionally replicate multimedia resources into various configuration states according to the properties of peers (i.e., terminal capabilities and user preferences) and the size of peer clusters. Also, the strategy uses peer related information stored on super peers to determine which peers should be selected to perform adaptive replications and where the resulting replicas should be stored. Simulation results show that the proposed strategy reduces network delays while increasing resource hit rate in comparison to FreeNet and random replication strategies.

XML is a popular approach to interoperable exchange of multimedia metadata between a wide range of devices. This paper explores extending the use of the remote XML exchange protocol (previously proposed by the authors) as a mechanism to provide efficient interaction with complex multimedia XML documents and their associated schemas. This is particularly applicable to users with limited application complexity devices and/or limited bandwidth connections. Many XML documents do not fully utilize all the information present in a given schema; thus, users download substantial redundant information for the current application. This paper introduces the use of RXEP for the transmission of small, relevant schema sections. The paper investigates the advantages of schema retrieval using RXEP in terms of the bandwidth saved

The Bitstream Binding Language (BBL) is a new technology developed by the authors and being standardized by MPEG, which describes how multimedia content and metadata can be mapped onto streaming formats. This paper describes a particular application of BBL - format-independent multimedia streaming. This means that streaming servers no longer require additional software modules in order to support new content formats as they are introduced. Instead, the server requires only a BBL description of the mapping between the content format and the stream, and any content in the new format may then be delivered by the streaming server. This approach is validated using the H.264/AVC format as an example, and performance data are provided.

This paper addresses efficiency issues identified in the Bitstream Syntax Description Language used by the MPEG-21 generic multimedia adaptation framework. In particular, when used to adapt modern content formats such as H.264/AVC, the time required for processing increases exponentially relative to the duration of the bitstream. In response, the paper proposes several additional features for the Bitstream Syntax Description Language which reduce the complexity of adaptation using BSDL to a linear function of bitstream duration. These features are implemented and validated using bitstreams of real-world length.

A blind signal separation algorithm (SCAtemp) that exploits both the sparse time-frequency representation and temporal structure of speech is proposed. SCAtemp compares each speech signal's adherence to the sparsity and temporal criteria, before switching to the most appropriate criteria to estimate each signal. This algorithm is shown to improve the real time separation performance of conventional BSS algorithms exclusively exploiting either the temporal structure, sparsity or statistical independence of signals. The improvement of SCAtemp over conventional BSS algorithms can be attributed to the use of additional a priori knowledge of speech in the temporal short term

This paper focuses on metadata-based, multimedia resource representation and retrieval in a P2P environment as a means of universal multimedia access (UMA). The primary focus of the work is a P2P architecture, which uses MPEG-21 as standards based technique to dynamically adapt resources to various usage environments. In the architecture, a super peer based approach is used to cluster peers, store peer information, perform searches and instruct peers to adapt/send resources. Also a two-stages based adaptation method is introduced to adapt search results and resources in an intelligent manner based on the usage environment attributes. The concept is demonstrated using a test-bed built based on the JXTA peer-to-peer framework. In addition, simulation results show that the proposed architecture reduces download time while increasing resource availabilities and download speed in the network when compared to traditional P2P systems.

This work investigates the validity and accuracy of using spatial cues with time-delay estimation (TDE) as a method of segmenting multichannel recorded speech by speaker location. In environments such as meetings where speakers do not significantly alter position, segmentation by speaker location essentially leads to segmentation by speaker 'turn'. The proposed system calculates location information using TDEs and spatial cues extracted from multichannel meeting audio recordings. This location information is then input into a simple segmentation algorithm. Experiments have been performed on both theoretical and real meeting recordings with non-overlapping speakers, and theoretical recordings with overlapping speakers. Segmentation results reveal the most robust cue to be a combination of spatial information and TDEs. This cue combination leads to greater segmentation accuracy for classifying individual speakers and detecting overlapping sections than using spatial cues or time-delay information alone.

XML is becoming increasingly popular as the ubiquitous standard for metadata; consequently, it is being incorporated into many multimedia applications, such as those based on MPEG-7 and MPEG-21. However, XML is often verbose and transmitting the large files can be wasteful in bandwidth and power-limited mobile applications. This paper introduces an XML access mechanism, RXEP, which combines XML compression with a fragment access protocol. RXEP ensures that essential information is exchanged efficiently while minimizing superfluous XML content transmission. This makes XML containers an attractive technique for multimedia content delivery

Bitstream binding language (BBL) provides an abstraction layer between XML multimedia containers and the way their resources and metadata are published in a bitstream. It allows multiple bindings from a single source document to facilitate interoperability and applicability of the multimedia content to a wide range of terminals and users. BBL introduces a number of features not found in other XML fragmentation techniques-such as support for random-access-and provides a highly general framework for combining metadata with multimedia content in a transport format. This paper presents an overview of BBL and provides details of a Java implementation using the MPEG-21 multimedia framework as a sample application

The MPEG-21 Multimedia Framework aims to realize interoperable access to content across heterogeneous networks and devices. Within the Framework, the concept of Digital Items is introduced as a structured digital representation for multimedia. To demonstrate the applicability of MPEG-21 to seamless multimedia interactions on limited platforms, the authors have produced an implementation of MPEG-21 for a mobile device, in Java 2 Micro Edition (J2ME). This paper examines the design and implementation of the Mobile MPEG-21 Peer, including a specialized architecture and processing mechanisms specific to the J2ME platform.

A sequential approach to sparse component analysis (SeqTIF) is proposed in this paper. Although SeqTIF employs the estimation process of the simultaneous TIFROM algorithm, a source cancellation and deflation technique are also incorporated to sequentially estimate speech signals in the mixture. Results indicate that SeqTIF's separation performance shows a clear improvement upon the simultaneous TIFROM approach, due to the less restrictive assumptions it places upon the signals in the mixture. In particular, the analysis indicates SeqTIF's data efficiency is high, enabling the sequential approach to track a time-varying mixture with much greater accuracy than the simultaneous algorithm. Furthermore, SeqTIF is a more flexible approach, free from the constraints that a simultaneous approach places upon the mixing system

We propose a new blind signal separation (BSS) technique, developed specifically for speech, that exploits a priori knowledge of speech production mechanisms. In our approach, the autoregressive (AR) structure and fundamental frequency (F0) production mechanisms of speech are jointly modeled. We compare the separation performance of our joint AR-F0 algorithm to existing BSS algorithms that model either speech's AR structure or F0 individually. Experimental results indicate that the joint algorithm demonstrates superior separation performance to both the individual AR algorithm (up to 77% improvement) and F0 (up to 50% improvement) algorithms. This suggests that speech separation performance is improved by employing a BSS model with a more realistic description of the speech production process.

At the core of the MPEG-21 Multimedia Framework is the concept of the Digital Item, a virtual container for a hierarchical structure of metadata and resources. This paper considers the Digital Item Declaration Language (DIDL), gives examples of its usage, and discusses how it is used to integrate other parts of MPEG-21. The paper then discusses how Digital Item Identification integrates with the DIDL to allow MPEG-21 to utilize standard identifiers from many application spaces. Finally, an alternative, compressed form of the XML Digital Item Declaration is described. This uses schema-based compression to significantly reduce the size of these XML documents.

The paper discusses media streaming using dynamic resource adaptation and update as a means of facilitating universal multimedia access (UMA): the concept of accessing multimedia content through a variety of possible schemes (Bormans, J. et al., IEEE Sig. Process. Magazine, 2003). As background, the paper summarizes the most common content negotiation approaches and addresses their facets and problems. MPEG-21, the multimedia framework and its relationship to UMA are then explained. Our primary focus is an end-to-end approach to content adaptation which takes advantage of MPEG-21 to facilitate the UMA concept in a media streaming environment. The concept is validated using a media streaming test-bed which provides for wide adaptation according to broad usage descriptions.

The paper proposes a technique for scalable to lossless audio compression. The scheme presented is perceptually scalable and also provides for lossless compression. It produces smooth objective scalability, in terms of SegSNR, from lossy to lossless compression. The proposal is built around the introduced perceptual SPIHT algorithm, which is a modification of the SPIHT algorithm. Both objective and subjective results are reported and demonstrate both perceptual and objective measure scalability. The subjective results indicate that the proposed method performs comparably with the MPEG-4 AAC coder at 16, 32 and 64 kbps, yet also achieves a scalable-to-lossless architecture.

This paper investigates waveform interpolation (WI) applied low bit rate wideband speech coding. An analysis of the evolutionary behaviour of wideband characteristic waveforms (CW) shows that direct application of the classical WI algorithm may not be appropriate for wideband speech. We propose a modification whereby CW quantisation is performed using classical WI decomposition for the low frequency region and noise modelling for the high frequency region. Wideband WI coders incorporating this modification and operating at 4 kbit/s and 6 kbit/s are described. Subjective testing of these coders shows that WI is a promising approach to low bit rate wideband speech compression.

This paper proposes a technique for scalable and also provides for lossless compression. It reduces smooth objective scalability, in terms of SegSNR, from lossy to lossless compression. The proposal is built around the perceptual SPIHT algorithm, which is a modification of the SPIHT algorithm and is introduced in this paper. Both objective and subjective results are reported and demonstrate that the proposed method performs comparably with the MPEG-4 AAC coder at 16, 32 and 64 kbps, yet also achieves a scalable-to-lossless architecture.

This paper investigates waveform interpolation (WI) applied low bit rate wideband speech coding. An analysis of the evolutionary behaviour of wideband characteristic waveforms (CWs) shows that direct application of the classical WI algorithm may not be appropriate for wideband speech. We propose a modification whereby CW quantisation is performed using classical WI decomposition for the low frequency region and noise modelling for the high frequency region. Wideband WI coders incorporating this modification and operating at 4 kbps and 6 kbps are described. Subjective testing of these coders shows that WI is a promising approach to low bit rate wideband speech compression.

This paper extends a scalable to lossless compression scheme to allow scalability in terms of sampling rate as well as quantization resolution. The scheme presented is an extension of a perceptually scalable scheme that scales to lossless compression, producing smooth objective scalability, in terms of SNR, until lossless compression is achieved. The scheme is built around the Perceptual SPIHT algorithm, which is a modification of the SPIHT algorithm. An analysis of the expected limitations of scaling across sampling rates is given as well as lossless compression results showing the competitive performance of the presented technique. 1.

Audio stream segregation is a task performed constantly by the human auditory system, yet is difficult to reproduce with a computer. The research detailed in this paper looks at performing just one method of stream segregation - the temporal coherence boundary - using a fuzzy clustering system. The main focus of the paper is on examining the effectiveness of several time-frequency distributions as the feature vectors for the system. Three time-frequency distributions are examined and their effectiveness evaluated in terms of correct separation and computational complexity. The main evaluation compares the popular gamma-tone filter bank with the MPEG-7 audio spectrum envelope. The results are promising, indicating that the less computationally expensive MPEG-7 descriptor performs well, implying that stream segregation may be possible using the MPEG-7 audio low-level description scheme.

This paper considers low-rate waveform interpolation (WI) coding. It compares the existing, common slowly evolving waveform (SEW) quantisation scheme with two new schemes for representing and quantising the SEW. The first scheme uses a minimum phase estimate to reconstruct the SEW whilst the second scheme uses a pulse model whose parameters are implicitly transmitted in the quantised rapidly evolving waveform (REW). These new schemes maintain or reduce the bit rate required for transmission of the SEW. Results indicate that, for low rate WI coding, necessarily coarse SEW magnitude spectrum quantisation limits the contribution of the SEW to perceptual quality. Perceptual tests indicate that avoiding coarse spectral shape quantisation and using a fixed shape model that lends itself to smooth interpolation, maintains the perceptual quality of the synthesized speech. The proposed fixed shape model requires no bits for transmission, allowing a 12 percent reduction in the overall coder bit rate.

Linear prediction (LP) is the cornerstone of most modern speech compression algorithms. Previously it has been shown that incorporating a weighting function based on the simultaneous masking property of the ear into the calculation of the LP coefficients (SMWLPC) allows the filter to better model the unmasked sections of the input spectrum. This paper conducts a detailed analysis of the implementation of SMWLPC in low rate speech codecs. The analysis allows the cause of inconsistencies in the technique to be identified and solutions formulated. Experimental results show that when combined with the proposed changes, the SMWLPC technique is suitable for implementation in any low rate LP based speech codec and the net result is an improvement in the perceptual quality of synthesised speech for all speakers.

In this paper, we present new results on Temporal Decomposition (TD) applied to the Line Spectral Frequencies (LSFs) derived for wideband speech. The paper shows that by incorporating a dynamic programming search algorithm into TD, near transparent quantisation of wideband LSFs can be obtained at approximately 1 kbps. We also show that TD performs significantly better than Split Vector Quantisation at low bit rates. We propose that TD is a promising approach to low rate wideband speech coding for applications such as unicast streaming.

Reducing the bit rate of waveform interpolation speech coders while maintaining the perceptual quality has been the focus of a great deal of research. This letter proposes a new method of slowly evolving waveform (SEW) quantization specifically targeted at low rate coding. The proposed method uses a pulse model whose parameters are implicitly contained in the quantized rapidly evolving waveform (REW) parameters, thus requiring no bits for transmission. Results indicate no degradation in perceptual speech quality when compared to that of the existing SEW quantization method. This retention of perceptual quality is in spite of a 12% reduction in the overall coder bit rate.

Whilst linear prediction is the cornerstone of most modern speech coders, few of these coders incorporate the perceptual characteristics of hearing into the calculation of the linear predictor coefficients (LPCs). This paper proposes a method of incorporating simultaneous masking into the calculation of the LPCs. This modification requires only a modest increase in computational complexity and results in the linear predictor removing more perceptually important information from the input speech signal. This results in a filter that better models the formants of the input speech spectrum. The net effect is that an improvement in quality is achieved for a given bit rate or alternately a bit rate reduction can be achieved while maintaining perceived quality. These results have been confirmed through subjective listening tests.

This correspondence uses a pitch synchronous wavelet transform (PSWT) as an alternative characteristic waveform decomposition method for the waveform interpolation (WI) paradigm. The proposed method has the benefit of providing additional scalability in quantization than the existing WI decomposition to meet desired quality requirements. The PSWT is implemented as a quadrature mirror filter bank and decomposes the characteristic waveform surface into a series of reduced time resolution surfaces. Efficient quantization of these surfaces is achieved by exploiting their perceptual importance and inherent transmission rate requirements. The multiresolution representation has the additional benefit of more flexible parameter quantization, allowing a more accurate description of perceptually important scales, especially at higher coding rates. The proposed PSWT-WI coder is very well suited to high quality speech storage applications.

For efficient coding of speech, it is desirable to separate the slowly and rapidly evolving spectral components to take advantage of their different perceptual qualities. We present a multi-level wavelet decomposition mechanism, using low delay FIR filters, applied to waveform interpolation (WI) coding. The technique overcomes the substantial delay problems encountered by Chong, Burnett, Chicharo and Thomson (see Proc. Int Conf. Acoustics, Speech, Sig. Processing, Seattle, USA, vol. 1., p.513-16, 1998) and identifies a preferred technique for the quantisation of the decomposed surfaces. The phase is shown to be particularly sensitive to the compounding of quantisation errors within the tree-structured transform. The proposed solution involves the use of variable dimension vector quantisation (VDVQ) on separately decomposed magnitude/phase surfaces. This approach provides for coarse or no phase quantisation while maintaining high speech quality. The techniques discussed may also be applied to other transforms and to the quantisation of surfaces in the standard waveform interpolation coder.

In this paper we investigate the spectral density of line spectral frequency (LSF) content in languages. The results show that the phonetic variation of languages is reflected in the LSF space. This leads to an alternative approach to the design of LSF quantisers. A trained LSF codebook, like the phonetic inventory of a language, is a static description of spectral behaviour of speech. As clear relationships exist between phonetic segments and LSFs, the structure of an LSF codebook can be analysed in terms of the phonetic segments. The new approach incorporates phonetic information into the structure of LSF codebooks through combining individual phonetic codebooks. The investigation leads to the conclusion that phonetic information can be usefully employed in codebook training in terms of perceptual performance and bit-rate reductions.

A new characteristic waveform decomposition method based on wavelets is proposed for the waveform interpolation (WI) paradigm. In WI, pitch-cycle waveforms are filtered in the evolution domain to decompose the signal into two waveform surfaces, one characterising voiced speech and a second representing unvoiced speech. The slow roll-off of FIR filters leads, however, to a significant inter-relationship between the decomposed surfaces. We present the pitch synchronous wavelet transform (PSWT) as an alternative decomposition mechanism. Filtering is again performed in the evolutionary waveform domain, producing characteristic surfaces at several resolutions. This multi-scale characterisation leads to more flexible quantisation of parameters, especially at higher rates than WI's 2.4 kb/s. FIR filters are replaced in the wavelet filter bank by causal, stable FIR filters which achieve significant delay reductions over their IIR counterparts. Furthermore, FIR filters track the dynamic aspects of the evolutionary surfaces faster, overcoming problems existing in the current WI decomposition.

In this paper, we present an alternative characteristic waveform (CW) decomposition mechanism for the Waveform Interpolation (WI) paradigm based on the Pitch Synchronous Wavelet Transform (PSWT). In this technique, IIR filters replace the conventional FIR filters of the PSWT, offering computational and spectral magnitude performance advantages, in addition to significant delay reductions. Previously, the PSWT has only incorporated filter banks with slowly reacting FIR wavelet filters. While these filters possess the desirable properties of linear phase, and design simplicity, a large delay is incurred which increases exponentially with increasing resolution. The progression to IIR filter banks gives rise to a multi-resolution decomposition mechanism, beneficial for real-time applications, such as speech coding, where delay is an important issue.

In this paper we investigate an alternative approach to the design of low-bit rate (LBR) quantisation. This approach incorporates phonetic information into the structure of Line Spectral Frequency (LSF) codebooks. In prior work vector quantisation (VQ) has been used to quantise stochastic processes. Speech signals can, however, be described in terms of phonetic segments and linguistic rules. A trained LSF codebook, like the phonetic inventory of a language, is a static description of spectral behaviour of speech. As clear relationships exist between phonetic segments and LSFs the structure of an LSF codebook can be analysed in terms of the phonetic segments. The investigation leads to the conclusion that phonetic information can be usefully employed in codebook training in terms of perceptual performance and bit-rate reductions.

A new mechanism for using analysis-by-synthesis techniques in low rate waveform interpolation based coders is introduced. The algorithm, implemented as part of a multi-prototype waveform coder, exploits the high quality speech produced by interpolating unquantised speech-domain prototype waveforms. In the new scheme, a frame of prototype waveforms is quantised using two sets of codebook searches, one representing the slowly evolving prototype shape and the other the rapid, noisy components. The scheme offers performance advantages over the previous open-loop multi-prototype waveform coder, particularly when perceptual weighting is incorporated in the search. Reductions in search complexity and the use of the scheme for quantisation at higher rates are also considered. This results in a generalised analysis-by-synthesis waveform interpolation architecture with closed-loop optimisation of all prototype waveform properties.

This paper considers the issue of the effect of languages and linguistic perception on low-rate speech coding. Current algorithms exploit the redundancies of speech but these redundancies are not common across all languages. Similarly speech coder evaluation techniques do not take into account the nuances of linguistic perception across languages. This paper illustrates some of the linguistic sensitivities experienced by low-rate coders and explores approaches to low-rate coder design. This is achieved through an evaluation of cross-language spectral distortion measures which account for specific linguistic peculiarities influencing linguistic perception.

Telecommunications networks are exposed to a plethora of accents and languages.Fundamental to current and future systems are low rate speech coders.This paper examines the problems associated with speech coding of different languages and accents.Our investigations show that most low-rate (8kb/s and below) speech coders show bias towards non-accented English.When the coders are used for heavily accented English or other languages, significant performance degradation is noted.This paper examines the reasons for such variations and some approaches for improving coder performance.

The paper describes new techniques for prototype waveform (PW) coding at coding rates as low as 2.84 kb/s. The algorithm produces good communications quality speech with significant improvements over previously reported PW coding schemes. In multi-prototype waveform (MPW) coding, prototypes are extracted at 2.5 ms intervals. No distinction is necessary between voiced and unvoiced speech since the normalised LP residual prototypes are coded as a combination of a noise vector and smoothly evolving pitch pulse. At low bit rates it is unnecessary to explicitly code the underlying pulse shape-the relative magnitude of the rapidly evolving noise vectors is sufficient to describe the level of periodicity in each prototype. Prototypes are quantised using either an open or closed-loop scheme with similar results. Quantised prototypes are interpolated by continuous phase interpolation of Fourier coefficients in the discrete frequency domain.