We propose a probabilistic method for the joint segmentation and melody extraction for musical audio signals which arise from a monophonic score. The method operates on framewise short-time Fourier transform (STFT) peaks, enabling a computationally efficient inference of note onset, duration, and pitch attributes while retaining sufficient information for pitch determination and spectral change detection. The system explicitly models note events in terms of transient and steady-state regions as well as possible gaps between note events. In this way, the system readily distinguishes abrupt spectral changes associated with musical onsets from other abrupt change events. Additionally, the method may incorporate melodic context by modeling note-to-note dependences. The method is successfully applied to a variety of piano and violin recordings containing reverberation, effective polyphony due to legato playing style, expressive pitch variations, and background voices. While the method does not provide a sample-accurate segmentation, it facilitates the latter in subsequent processing by isolating musical onsets to frame neighborhoods and identifying possible pitch content before and after the true onset sample location

Many events of interest in human activity-based multimedia applications exhibit a high degree of temporal structure. This structure generates expectancies regarding the occurrence and location of subsequent events. In the context of switching state-space models, we develop a general Bayesian framework for representing temporal expectancies and fusing them with raw sense-data to improve both event segmentation and temporal structure identification. Furthermore, we develop a new cognitive model for event anticipation which adapts to incoming sense-data in real time. Comparative advantages of the proposed framework are realized in controlled experiments involving partially-observed, quasi-periodic event streams