The ball state tracking and detection technology plays a significant role in volleyball game analysis for volleyball team supporting and tactics development. This paper proposes a ball event detection method to achieve high detection rate by solving challenges including: the great variety of event length, the large intra-class difference of one event and the influence caused by ball trajectories. Proposed state vector covers both the event type and the event period length so that the system model can transits various lengths of event period and predicts event types by volleyball game rules. The curve segmental observation model avoids the tracking error influence to evaluate the event period likelihood by referring neighbouring trajectories of the ball. And according to the standard of the ball event, the feature of the distance between the ball and specific court line are extracted to evaluate the ball event type in observation. At last a two-layer estimation method estimates the posterior state which is a joint probability distribution. Experiments of the proposed method implemented on 3D trajectories tracked from multi-view volleyball game videos shows the detection rate reaches 90.43%.

Automatically extraction of accurate volleyball game data from game videos plays an important role in making contribution to game data analysis, TV broadcasting and performance evaluations. In this paper, a particle filter based physical and conceptual ball state estimation method is proposed to track the 3D ball trajectory and ball event simultaneously with high accuracy. The physical ball state includes 3D ball position and velocity. Besides the ball event, the conceptual state also includes flag of the external force on the ball. The system model is adaptive to this external force predicted through proposed spatial hitting points dense distribution. Observation of the external force is evaluated by hitting point likelihood, which uses not only the past tracked trajectory but also the image noise feature so that image noise is transferred into useful feature and unidirectional dependency on trajectory is avoided. Experimental results based on multi-view HDTV video sequences show the tracking success rate of ball state achieves 92.43%.

The 3D position of the ball plays a crucial role in professional sport analysis. In ball sports, tracking of ball's precise position accurately is highly required, whose performance is affected by inaccurate 3D coordinates and occlusion problem. In this paper, we propose anti-occlusion observation model and automatic recovery by 3D ball detection based on multiview videos to track the ball in 3D space. The anti-occlusion observation model evaluates each camera's image and eliminates the influence of the cameras in which the ball is occluded. The automatic recovery method detects the ball's 3D position by homography relation of the multi-video and generates a new distribution to initiate the tracker when tracking failure is detected. Experimental results based on the HDTV video sequences, which were captured by four cameras located at the corners of the court, show that the success rate of the 3D ball tracking achieves 99.14%.

We have developed a new, three dimensional vocal tract mechanical model for an anthropomorphic talking robot WT-7R (Waseda Talker No. 7 Refined), to improve the resonance of the vocal tract. The Waseda Talker robot series aims to reproduce the human speech mechanism with three-dimensional accuracy. The tongue of the previous model, WT-7 (Waseda Talker No. 7), was made of rigid links and covered with a thermoplastic rubber (Septon). This mechanism could deform the tongue shape and work as a part of the vocal tract, however, the cover thickness was not sufficient enough to prevent sound leakage, and did not have sufficient resonance. As a result, the produced sounds were unclear. To resolve this problem, the inner area of the tongue was filled with liquid. We experimented to select the filling liquid which is minimizes damage to the Septon cover and provides adequate resonance characteristics. The ethylene glycol was selected because it does little damage to the Septon and is relatively non-flammable. An oil seal and liquid gasket prevent leakage into the robot. WT-7 also has problems with its tongue link deformation range and open lip control range-these problems were also addressed. The improvements made the vowel production of WT-7R clearer than that of the previous robot, and the bandwidth of the formant peak in spectral analysis became sharper.

We have developed a new three-dimensional talking robot Waseda Talker No. 6 (WT-6), which generates speech sounds by mechanically simulating articulatory motions as well as aero-acoustic phenomena in the vocal tract. WT-6 has 17-DOF vocal mechanism. It has three-dimensional lips, tongue, jaw and velum which form the 3D vocal tract structure. It also has an independent jaw opening/closing mechanism, which controls the relative tongue position in the vocal tract as well as the oral opening. The previous robot in the series had a 2D tongue and was not able to realize precise closure to produce human-like consonants such as /s/ or /r/. The new tongue, which can be controlled to form 3D shapes, is able to produce more realistic vocal tract shapes. The vocal cord model was also improved by adding a new pitch control mechanism that pushes from the side of the vocal cords. The pitch range is broader than that of the previous robot; it is sufficiently broad so as to be able to reproduce normal human speech. Preliminary experimental results showed improved synthesized speech quality for the vowels /a/, /u/ and /o/.

We developed a sensor feedback mechanism for an anthropomorphic talking robot WT-5 (Waseda Talker No. 5). In human speech, sensory feedback is more important when we producing obstacle consonant sounds, such as /t/ and /d/, compared to the auditory feedback mechanism. We reproduce this mechanism by placing tactile sensors and a pressure sensor on the palate of a talking robot and reducing the error between the pressure of the human voice and the robot consonant production. In addition, we developed more efficient optimization methods than those of WT-4, using speech recognition and the pre-optimized memory of vowels. Using these mechanisms, we realized continuous mimic speaking that includes consonant sounds

We propose a sinusoidal synthesis method based on instantaneous frequency (IF) attractors, which correspond to harmonic frequency trajectories. The algorithm is novel in extracting accurate sinusoidal components. Since the continuity of IF attractors is well-defined and simple to detect, the IF attractors can be extracted from audio signals without any explicit constraint or complicated algorithm; therefore, they can be directly applied to sinusoidal synthesis. Accuracy of IF attractor analysis can be further improved by time-warping analysis, which improves resolution for harmonic components whose frequencies that change rapidly. We describe the procedure for sinusoidal synthesis and precise phase estimation along with examples and also evaluate the effect of time-warping on analysis and synthesis of speech

This paper describes development of a new human-like talking robot WT-4 (Waseda Talker No. 4). WT-4 was developed to overcome the problems related to the lack of variation in the formant frequency of the sounds generated by the previous talking robot WT-3. These problems arose particularly on the first formant (F1) where frequencies of less than 500 [Hz] could not be reached. And the differences among WT-3’s vowels were not clear; which may have been caused by the sound source of the vocal cords and the lips. Therefore, the connection between the vocal cords and the vocal tract was improved, as well as the lips mechanism. As the result of these improvements, WT-4’s sounds were quite similar to a human’s, and WT-4 was able to reproduce suitable sounds using auditory feedback.

We have developed a new talking robot, WT-5 (Waseda Talker No. 5), having novel vocal cords and lips based on human biological structures. The vocal cords are made from the thermoplastic rubber "Septon", available from Kuraray Co. Ltd, which has a similar elasticity to human tissue. The vocal cord model was constructed with a similar structure to the biological structure of the human vocal cords. The vocal cords vibrated like those of a human. The new lips were able to attain a large deformation in a similar manner to human lips, without the leakage of air. This allowed clear pronunciation of vowels. With these new mechanisms, the robot could reproduce human speech in a more biological manner and could produce voices closer to those of a human.

This paper describes the mechanisms and the speech production of a new advanced talking robot WT-3 (Waseda Talker-No.3) that improves on WT-2 (Waseda Talker-No.2) and is based on human acoustic theory for the reproduction of human speech. WT-3 consists of 1-DOF lungs and 3-DOF vocal cords and articulators (the 7-DOF tongue, 5-DOF lips, 1-DOF teeth, nasal cavity and 1-DOF soft palate), and can reproduce human-like articulatory motion; the total DOF is 18. The oral cavity is designed based on the MRI images of the human sagittal plane, although the cross section of the vocal tract is rectangular in shape except for the mouth. The width of the vocal tract is 30 [mm]. The average length of the vocal tract is approximately 175 [mm] and the same as that of a human's. Compared to the previous robots, WT-3 can produce vowels more clearly, and produce stops, fricatives and nasal sounds with the new flexible mechanisms that function as the human vocal tract area and the other mechanisms. WT-3 can mechanically reproduce human speech.

This paper describes an autonomous control method of an anthropomorphic talking robot WT-4 (Waseda Talker No.4) to mimic continuous human speech sounds by auditory feedback. WT-4 consisted of 1-DOF lungs, 4-DOF vocal cords and articulators (the 7-DOF tongue, 5-DOF lips, 1-DOF teeth, nasal cavity and 1-DOF soft palate), and could reproduce human-like articulatory motion; the total DOF was 19. In this method, the trajectory of each robot parameter was controlled so that the acoustic parameters (pitch, sound power, formant frequencies that are resonant frequencies of the vocal tract and have the peak of the output spectrum, and the timing of the switch between voiced and voiceless sounds) generated from the robot were close to those of human speech sounds. The trajectory of each robot parameter was optimized by inputting the acoustic parameters. This method would help to clarify the human speech mechanism and to create a new speech production system.

We present a method that determines articulatory movements from speech acoustics using a Hidden Markov Model (HMM)-based speech production model. The model statistically generates speech spectrum and articulatory parameters from a given phonemic string. It consists of HMMs of articulatory parameters for each phoneme and an articulatory-to-acoustic mapping for each HMM state. For a given speech spectrum, maximum a posteriori estimation of the articulatory parameters of the statistical model is presented. The performance on sentences was evaluated by comparing the estimated articulatory parameters with the observed parameters. The average RMS errors of the estimated articulatory parameters were 1.50 mm from the speech acoustics and the phonemic information in an utterance and 1.73 mm from the speech acoustics only.

We propose a sinusoidal synthesis method based on the instantaneous frequency (IF) attractors which well correspond to harmonic frequency trajectories. We demonstrate that IF attractors can be directly applied to sinusoidal synthesis. Since the continuity of IF attractors is well-defined and simple to detect, extraction of IF attractors can be done without any explicit constraint and needs no complicated algorithm such as peak-tracking of short-time spectra. We also describe the procedure for sinusoidal synthesis and precise phase estimation along with examples.

This paper describes the development of a new talking robot WT-2 that improved on WT-1R and has advanced vocal tract mechanisms for the reproduction of human vocal movement. WT-2 consists lungs, vocal cords, a vocal cavity and nasal cavity; the total DOF is 15. The length of the vocal tract is about 175 mm and is almost the same as that of an adult male. Compared to WT-1 or WT-1R developed previously, WT-2 could utter Japanese vowels more clearly, and produce stops, fricatives and nasal sounds by a new flexible tongue mechanism as well as decrease sound leaks from the tongue except those from the lips or nostrils.

This paper describes the modeling and analysis of an elastic tongue mechanism of a talking robot WT-2 and the development of an acoustic simulator for the construction of the computational model of human speech production. WT-2 consists of lungs, vocal cords and articulators. It is essential to accurately estimate and analyze the shapes of these organs to develop an acoustic simulator. However, it was difficult to estimate and analyze the mechanisms, because they were composed of an elastic rubber to make various shapes and to seal the air and sound. Therefore, we constructed a modeling method to simulate the shape of the elastic mechanisms. This method is applicable to all the elastic body. We developed an acoustic simulator using this modeling method in the process of constructing the speech production model.

We present a method that determines articulatory movements from speech acoustics using an HMM (Hidden Markov Model)-based speech production model‥ The model statistically generates speech acoustics and articulatory movements from a given phonemic string. It consists of HMMs of articulately movements for each phoneme and an articulatory-to-acoustic mapping. For a given speech acoustics, maximum likelihood estimation of the articulatory parameters in the statistical model is presented. An experiment was conducted to evaluate the method on sentences, and estimated articulatory parameters were compared with the observed parameters. The rms error of the estimated articulatory parameters is about 2.12 mm on average.

Diagram manipulation in conventional CAD systems requires frequent mode switching and explicit placement of the pivot for rotation and scaling. In order to simplify this process, we propose an interaction technique called integrated manipulation, where the user can move, rotate, and scale without mode switching. In addition, the pivot for rotation and scaling automatically snaps to a contact point during moving operation. We performed a user study is performed using our prototype system and a commercial CAD system. The results showed that users could perform a diagram manipulation task much more rapidly using our technique.

A method used to determine the vocal-tract cross-sectional area function from acoustical measurements at the lips is analyzed. Under the framework described by Sondhi and Gopinath (1971) and implemented by Sondhi and Resnick (1983), a sensitivity analysis of the vocal-tract area function, derived from the impedance or reflectance at the lips is performed. It indicates that, in the ideal case, the area function is not heavily affected by random distortions of the impulse response at the lips. Simulations and real measurements show that the method works relatively well, except for regions behind narrow constrictions. In this case, an excitation pulse with high energy, as well as a fine sampling, proved to be important. The excitation used is a time stretched pulse. It produces an excitation with high energy without the necessity of a high power sound generator device.

A speech analysis-synthesis method based on linear predictive coding (LPC) residual phase equalization is presented for low-bit- rate natural sounding speech coding. In this method, the phase-equalized LPC residual signal is represented as the output of the quasi-periodic impulsive excitation to an all-zero filter, which captures harmonic deviation in the residual spectrum which has not been compensated for by LPC inverse filtering of the input speech signal. The impulsive excitation is specified by the instantaneous pitch excitation instant and amplitude. A procedure is presented to determine the excitation parameters and the all-zero filter using a waveform matching criterion. The coder performance is studied objectively and subjectively with a focus on the potential of the excitation model to produce high-quality speech. A speech coder designed for bit rates lower than 4.8 kb/s is discussed.>

A speaker adaptation method is presented based on a piecewise linear mapping of spectral code vector space into the spectral vector space of an unknown speaker. Adaptation is performed by modifying the code vectors to give a better fit to input the spectral vectors, while maintaining the local continuity of distribution of the original code vectors. Two adaptation algorithms are presented: a minimum-distortion method which provides a monotonic nonincreasing distortion for the training data and a simplified method. These algorithms are investigated in terms of distortion performance and adaptation speed. Experimental results show that the method provides much higher adaptation speed than the LBG (Linde-Buzo-Gray) algorithm and that the distortion performance approaches that of the LBG algorithm as the size of the training data increases.>

A medium-band speech coder is proposed that uses a weighted vector quantization scheme in the transformed domain. The linear prediction residue is transformed and vector-quantized. In order to control the quantization errors in the transformed domain, adaptively weighted matching is used instead of conventional adaptive bit allocation. Therefore, the residual signal can be reconstructed by the decoder, even if the spectral envelope parameters are destroyed due to transmission errors. This coder is also capable of maintaining higher SNR (signal-to-noise ratio) performance than time-domain vector quantization coders for a wide range of computation complexities and bit rates. Coded speech is natural and unaffected by background noise. The mean opinion score for this coder at 7.2 kb/s is comparable to that of 5.5-bit log PCM coded speech sampled at 6.4 kHz.>

A new transform coding of speech is proposed which employs a weighted vector quantization scheme. First, the characteristics of the weighted vector quantization are checked. Then based on these considerations, a frequency domain coder is designed for medium-band (4.8-9.6 kbps) speech coding. In this coding scheme, the linear prediction residue is transformed and vector quantized. In order to control the quantization errors in the frequency domain, adaptively weighted matching is employed instead of the conventional adaptive bit allocation. Therefore, the residual signal can be reconstructed by the decoder, even if the spectral envelope parameters are destroyed due to transmission errors. The coded speech is natural and unaffected by background noise and its mean opinion score at 7.2 kbps is comparable to that of 5.5-bit log PCM coded speech.

A new speech processing and coding method is proposed which makes use of perceptual redundancy for slowly varying short-time phase characteristics. The method employs waveform conversion through a phase-equalizing filter, which is based on the time domain matched filter for the residue of Linear Predictive Coding (LPC). Phase-equalized speech is found to be almost perceptually equivalent and to be efficiently encoded by a two-stage quantization. In the first stage, vector quantization is performed for the pulse pattern in the time domain. In the second stage, vector-scalar quantization is applied to the spectral components using adaptive bit allocation. The proposed coder is proven to be superior to other coders both in terms of the SNR and the subjective quality. The averaged subjective quality at 9.6 kbps is comparable to that of a 6 bit log PCM.

An adaptive predictive coding with adaptive bit allocation (APC-AB) is presented for speech encoding at low to medium bit rates (6.4kb/s-24kb/s). In this system, a split-band predictive coding scheme and a bit allocation scheme are employed in order to remove the redundancies due to a periodic concentration of the prediction residual energy as well as nonuniform nature of the speech spectrum. Quantization bits are dynamically allocated both over the sub-bands(frequency domain) and over the subintervals(time domain) in accordance with the distribution of the residual energies in the time-frequency domain. Optimum bit allocation is derived based on the mean square error criterion on the speech waveform, and the SNR gain is presented in relation to the prediction gain of the full-band signal. This system is evaluated in terms of the segmental SNR and speech quality. The result shows that the APC-AB system has advantage over the conventional full-band APC system in the segmental SNR and the stability of the prediction loop. It was also shown that this system can provide speech quality subjectively equivalent to 7 bit Log-PCM at 16 kb/s, and to 6 bit Log-PCM at 9.6 kb/s.

This paper will describes a digital speech signal processor(DSSP) designed for a narrow band and a medium band speech coding system, both of them are recently developed at the E.C.L., N.T.T.. The narrow band system is LSP(Line Spectrum Pair) vocoder, which is based on the frequency domain representation of LPC parameters. The LSP parameters are superior to conventional LPC parameters in view of its quantization and interpolation properties. The medium band system is a split band adaptive predictive coding with adaptive bit-allocation, APC-AB. This system can achieve a toll quality coding at 16 Kbit/s. We have made an extensive software simulation of LSP and APC-AB, and decided the specification of the DSSP. The DSSP can handle arithmetic and shifting operations in both 12 and 24 bit mode. Multiplications are done by a 12- by-12 parallel multiplier. Multiplier, ALU and shifters can operate concurrently in a pipeline mode at the minimum interval of 0.5 µs per multiplication-accumulation.

This paper describes objective quality measures to evaluate speech quality for various kinds of voiceband CODECs in common. The voiceband CODECs studied were PCM, ADM, ADPCM, ATC (Adaptive Transform Coding) and APC-AB (Adaptive Predictive Coding with Adaptive Bit Allocation). First, several objective quality measures in time and frequency domain were defined. They were SNR, Segmental SNR, Spectral Distortion, LPC Cepstrum Distance, COSH, Likelihood Ratio and Weighted Likelihood Ratio. Second, speech quality for voiceband CODECs were evaluated by subjective and objective quality measures. The subjective measures used were based on opinion test and articulation test. Finally, the relationship between objective measures and subjectively evaluated values was studied. It was concluded that the LPC Cepstrum Distance measure had best correspondence to Mean Opinion Score, among the objective measures studied. It was also concluded that the Wighted Likelihood Ratio measure had best correspondence to Articulation Score.