This paper presents a phonetic segment classification method based on joint embeddings learned from processing Ultrasound Tongue Imaging (UTI) and audio data. For constructing the embeddings, we compiled an ultrasound image dataset synchronized with audio that encompasses common speech scenarios. The embeddings are obtained from artificial neural network models trained on this dataset. During testing, our model processes only audio data, making it practical for speech therapy as no ultrasound imaging is required. Experiments show that our method yields similar performance compared to methods that simultaneously use both audio and UTI data. However, it outperforms the methods utilizing solely audio or UTI data in real-time classification.

Prostate cancer is one of the most frequent cancers among men. Abdominal ultrasound imaging is a very practical alternative to more precise but inconvenient transrectal ultrasound imaging for the diagnosis and treatment of prostate cancer. However, detection of the prostate region alone is very difficult for the abdominal ultrasound images. This paper presents a new prostate detection method that models the abdominal images as the classes of neighboring anatomical regions of the prostate. The proposed method has two levels: Pixel level detection assigns class scores to each pixel in the image. Model level detection uses these scores to determine the final positions of the anatomical regions in the image. The proposed approach is very effective for the specific problems of the abdominal ultrasound scans. Extensive experiments performed on real patient data with and without pathologies produce very promising results.

This paper proposes a new focus measure operator for Shape From Focus to recover a dense depth map of a scene. The method can handle depth discontinuities effectively by using adaptively shaped and weighted support windows. The support window shapes and weights are determined from the image characteristics of the all-focused image of the scene. Similar and closer pixels in the support window get higher weights, which inhibits the problems caused by the depth discontinuities. The size of the support window can be increased conveniently for a more robust depth estimation without introducing any window size related Shape From Focus problems. The large support window sizes also addresses the edge bleeding problem. The experiments on the real and synthetically refocused images show that the introduced ideas work effectively and efficiently in real world applications. 1

This paper presents a novel level set method with shape priors. The method keeps the level set deformations and the integration of the prior information as separate processes and hence it can be used with any level set formulation without complicating the level set functional. The method does not need any explicit training phase and by the addition of an appropriate deformable contour matching method, it can be used for any specific application. The system is tested and verified by the task of extraction of the inner and outer heart walls (endocardium and epicardium) from the echocardiographic images of the left ventricle. 1

This paper presents a novel system that uses two synchronous optimization processes to recover 3D structure from recti ed stereo image pairs. The synchronization of the processes are done by energy terms that inform the optimization processes about the recovered positions of each other. This information is used to direct the optimizations towards a better direction. The system is initialization insensitive and it is very robust against local minima. We performed experiments on real and synthetic images with ground truth that showed the effectiveness and the robustness of our system. We also compared our system to other systems for further validation. 1

This paper introduces a novel coarse-to-fine deformable contour optimization framework, which is composed of two main components. The first component uses scale-space and information theories to produce a coarser representation of the input image to be used in a coarse-to-fine optimization scheme. The employment of information theory ensures that maximal image information is propagated to the coarse images and employment of scale spaces provides a mechanism to change the image coarseness locally based on the deformable contour model definition. The second component of this framework uses a novel combination of dynamic programming and gradient descent methods to optimize the contour energy on coarser representations and then use the obtained coarse contour positions in finer optimizations. The motivation in using a combination of dynamic programming and gradient descent method is to take advantage of each method's efficiency and avoid their drawbacks. In order to verify the performance of this framework, we constructed a deformable contour model for the spatiotemporal tracking of closed contours and optimized the model energy under this framework. Experiments on this system performed using synthetic images and real world echocardiographic sequences demonstrated the effectiveness and practicality of this framework.

Application of dynamic programming to the deformable contours has many advantages, such as guaranteed optimality and numerical stability. However, long execution times of these methods almost always force researchers to use dynamic programming in combination with multiresolution methods. Multiresolution methods shorten the execution time by subsampling the original images after an application of a smoothing filter. However, this speedup comes at the expense of contour optimality due to the loss of details in the decreased resolution. In this paper, we present a new multi-level framework for deformable contour optimization, which can achieve faster optimization times and performs better than current multiresolution methods. To form the new levels, this method uses a very efficient algorithm to segment the original images with respect to the deformable contour external energy instead of subsampling. An exhaustive search on these segments is carried out by dynamic programming. A novel gradient descent algorithm is employed to find optimal internal energy for large image segments, where the external energy remains constant due to segmentation. We also introduce a new algorithm to pass the contour information more precisely between the levels. We present an analysis of time and performance comparisons with the current multiresolution methods by the experiments done on variety of medical images, which confirmed efficiency and accuracy of our framework.

We propose a novel method for continuous 3D depth recovery and tracking using calibrated stereo. The method integrates stereo correspondence, surface reconstruction and tracking by using a new single deformable dual mesh optimization, resulting in simplicity, robustness and efficiency. In order to combine stereo correspondence and structure recovery, the method introduces an external energy function defined for a 3D volume based on cross-correlation between the stereo pairs. The internal energy functional of the deformable dual mesh imposes smoothness on the surfaces and it serves as a communication tool between the two meshes. Under the forces produced by the energy terms, the dual mesh deforms to recover and track the 3D surface. The newly introduced dual-mesh model, which is one of the main contributions of this paper, makes the system robust against local minima and yet it is efficient. A coarse-to-fine minimization approach makes the system even more efficient. Tracking is achieved by using the recovered surface as an initial position for the next time frame. Although the system can effectively utilize initial surface positions and disparity data, they are not needed for a successful operation, which makes this system applicable to a wide range of areas. We present the results of a number of experiments on stereo human face and cloud images, which proves that our new method is very effective.

Computerized analysis of the tongue surface movement can provide valuable information to speech and swallowing research. Ultrasound technology is currently the most attractive modality for the tongue imaging mainly because of its high video frame rate. However, problems with ultrasound imaging, such as noise and echo artifacts, refractions, and unrelated reflections pose significant challenges for computer analysis of the tongue images and hence specific methods must be developed. This paper presents a system that is developed for automatic extraction and tracking of the tongue surface movements from ultrasound image sequences. The ultrasound images are supplied by the head and transducer support system (HATS), which was developed in order to fix the head and support the transducer under the chin in a known position without disturbing speech. In this work, we propose a novel scheme for the analysis of the tongue images using deformable contours. We incorporate novel mechanisms to 1) impose speech related constraints on the deformations; 2) perform spatiotemporal smoothing using a contour postprocessing stage; 3) utilize optical flow techniques to speed up the search process; and 4) propagate user supplied information to the analysis of all image frames. We tested the system's performance qualitatively and quantitatively in consultation with speech scientists. Our system produced contours that are within the range of manual measurement variations. The results of our system are extremely encouraging and the system can be used in practical speech and swallowing research in the field of otolaryngology.

This paper presents a system for automatic extraction and tracking of 2D contours of the tongue surfaces from digital ultrasound image sequences. The input to the system is provided by a Head and Transducer Support System (HATS), which is developed for use in ultrasound imaging of the tongue movement. We developed a novel active contour (snakes) model that uses several temporally adjacent images during the extraction of the tongue surface contour for an image frame. The user supplies an initial contour model for a single image frame in the whole sequence. Using optical flow and multi-resolution methods, this initial contour is then used to find the candidate contour points in the temporally immediate adjacent images. Subsequently, the new snake mechanism is applied to estimate optimal contours for each image frame using these candidate points. In turn, the extracted contours are used as models for the extraction process of new adjacent frames. Finally, the system uses a novel postprocessing technique to refine the positions of the contours. We tested the system on 11 different speech sequences, each containing about 25 images. Visual inspection of the detected contours by the speech experts shows that the results are very promising and this system can be effectively employed in speech and swallowing research.

We present a system that extracts and tracks 2D mid-sagital tongue surfaces from ultrasound image sequences produced by a head and transducer support system (HATS). Such a system can be a valuable and practical tool for speech and swallowing research. We extend our previous work by introducing a deformable contour model to impose restrictions of spatiotemporal coherency by assuming that the 2D mid-sagital tongue surface contours sweep a coherent 3D structure in spatiotemporal 3D space. Another novel contribution of this paper is a dynamic programming minimization method to optimize the new energy functional, which can be employed in general snake minimization tasks. We tested this system on a number of speech and swallowing sequences each containing 24 to 30 frames. We verified with speech experts and found that our system produces promising results, especially for the swallowing sequences, which are more problematic.