Multi-instrument automatic music transcription (AMT) is a critical but less investigated problem in the field of music information retrieval (MIR). With all the difficulties faced by traditional AMT research, multi-instrument AMT needs further investigation on high-level music semantic modeling, efficient training methods for multiple attributes, and a clear problem scenario for system performance evaluation. In this article, we propose a multi-instrument AMT method, with signal processing techniques specifying pitch saliency, novel deep learning techniques, and concepts partly inspired by multi-object recognition, instance segmentation, and image-to-image translation in computer vision. The proposed method is flexible for all the sub-tasks in multi-instrument AMT, including multi-instrument note tracking, a task that has rarely been investigated before. State-of-the-art performance is also reported in the sub-task of multi-pitch streaming.

The multi-instrument transcription task refers to joint recognition of instrument and pitch of every event in polyphonic music signals generated by one or more classes of music instruments. In this paper, we leverage multi-object semantic segmentation techniques to solve this problem. We design a time-frequency representation, which has multiple channels to jointly represent the harmonic structure and pitch saliency of a pitch activation. The transcription task therefore becomes a pixel-wise multi-task classification problem including pitch activity detection and instrument recognition. Experiments on both single- and multi-instrument data verify the competitiveness of the proposed method.

Spectral features are limited in modeling musical signals with multiple concurrent pitches due to the challenge to suppress the interference of the harmonic peaks from one pitch to another. In this paper, we show that using multiple features represented in both the frequency and time domains with deep learning modeling can reduce such interference. These features are derived systematically from conventional pitch detection functions that relate to one another through the discrete Fourier transform and a nonlinear scaling function. Neural networks modeled with these features outperform state-of-the-art methods while using less training data.

Fast independent component analysis (FastICA) algorithm separates the independent sources from their mixtures by measuring non-Gaussian. FastICA is a common offline method to identify artifact and interference from their mixtures such as electroencephalogram (EEG), magnetoencephalography (MEG), and electrocardiogram (ECG). Therefore, it is valuable to implement FastICA for real-time signal processing. In this paper, the FastICA algorithm is implemented in a field-programmable gate array (FPGA), with the ability of real-time sequential mixed signals processing by the proposed pipelined FastICA architecture. Moreover, in order to increase the numbers precision, the hardware floating-point (FP) arithmetic units had been carried out in the hardware FastICA. In addition, the proposed pipeline FastICA provides the high sampling rate (192 kHz) capability by hand coding the hardware FastICA in hardware description language (HDL). To verify the features of the proposed hardware FastICA, simulations are first performed, then real-time signal processing experimental results are presented using the fabricated platform. Experimental results demonstrate the effectiveness of the presented hardware FastICA as expected.

The human brain /spl sim/20 Hz rhythm measured by electroencephalography (EEG) and magnetoencephalography (MEG) has been used as a clinical examination index of motor function which originates in the anterior bank of the central sulcus in the human brain. In human voluntary movement, it is composed of three phases, planning, execution and recovery which has been suggested that localized event-related alpha desynchronization (ERD) upon movement can be viewed as an EEG/MEG correlate of an activated cortical motor network, servicing planning and execution, while beta event-related synchronization (ERS)may reflect deactivation/inhibition during the recovery phase in the underlying cortical network. The single-trial detection of /spl sim/20 Hz rhythm is challenged because of its low signal amplitude and its signal-to-noise ration (SNR) in EEG/MEG measured neural activities. This present study proposes a method based on independent component analysis (ICA) for extraction of the sensorimotor rhythm from magnetoencephalography (MEG) measurements of right finger lifting task in a single trail. ICA decomposes a single trial recording into a set of temporal independent components (IC) and corresponding spatial maps in which the task-related components are selected by visual inspection. Pertinent ICs are then selected by visual inspection to reconstruct task-related components beta oscillatory activity which is then subjected to beta rebound quantification and source estimation in further analyses. Since the event-related oscillatory activity of human brain is related to subject's-related oscillatory activity of human brain is related to subject's performance and state, the ICA-based single trial method enables the possibility of studying a single-trial, which in turn may shed light on the intricate dynamics of the brain.

Dynamic-susceptibility-contrast MR imaging is a widely used perfusion imaging technique that records signal changes on images caused by the passage of contrast-agent particles in the human brain after a bolus injection of contrast agent. The signal changes over time on different brain tissues represent distinct blood supply patterns and are crucial for studying cerebral hemodynamics. By assuming the spatial independence among these patterns, independent component analysis (ICA) was applied to classify different tissues, i.e., artery, gray matter, white matter, vein and sinus and choroid plexus, so that the spatio-temporal hemodynamics of these tissues were decomposed and analyzed. An arterial input function was modeled using the concentration-time curve of the arterial area for the deconvolution calculation of relative cerebral blood flow. The cerebral blood volume (CBV), relative cerebral blood flow (CBF), and relative mean transit time (MTT), were computed and their averaged ratios between gray matter and white matter were in good agreement with those in the literature.

We propose a novel method to estimate and track the 3D line of sight of a person based on 3D computer vision techniques. Most of the existing nonintrusive gaze tracking methods share a common drawback that users have to perform certain experiments in calibrating the user-dependent parameters before using the gaze tracking systems. These parameters are functions of the radius of the cornea, the position of the pupil, the position of user's head, etc. Our approach, in contrast, employs multiple cameras and multiple point light sources to estimate the light of sight without using any of the user-dependent parameters. As a consequence, the users can avoid the inconvenient calibration process which may produce possible calibration errors. Computer simulations were performed to confirm the proposed method.

A motion estimation algorithm using wavelet approximation as an optical flow model has been developed to estimate accurate dense optical flow from an image sequence. This wavelet motion model is particularly useful in estimating optical flows with large displacement. Traditional pyramid methods which use the coarse-to-fine image pyramid by image burring in estimating optical flow often produce incorrect results when the coarse-level estimates contain large errors that cannot be corrected at the subsequent finer levels. This happens when regions of low texture become flat or certain patterns result in spatial aliasing due to image blurring. Our method, in contrast, uses large-to-small full-resolution regions without blurring images, and simultaneously optimizes the coarser and finer parts of optical flow so that the large and small motion can be estimated correctly. We compare results obtained by using our method with those obtained by using one of the leading optical flow methods, the Szeliski pyramid spline-based method. The experiments include cases of small displacement (less than 4 pixels under 128/spl times/128 image size or equivalent displacement under other image sizes), and those of large displacement (10 pixels). While both methods produce comparable results when the displacements are small, our method outperforms pyramid spline-based method when the displacements are large.

Nonlinear system identification using a doubly random input function which is a Poisson train of events with random amplitudes as a system input is investigated. These doubly random input functions are useful for identifying systems that naturally require amplitude modulated point process inputs as stimuli such as the hippocampal formation in the central nervous system. This is an extension of earlier work in which a Poisson train of events with only constant amplitude was used as the input for system identification. Analogous to the Wiener theory, we have developed both continuous and discrete functionals up to second-order for this doubly random input function. Closed form solutions for the diagonal terms of the second-order kernels in both cases have been obtained and convergence properties are demonstrated. Two hypothetical discrete second-order nonlinear systems are illustrated and one of them was simulated to test the theory presented. Discrete kernels computed from the simulated data agree with the theoretical prediction.