We introduce a mobile system that is able to detect arousal in realtime based on electrocardiogram and electrodermal activity. The system is using an Android smartphone and wearable sensors, which include a smart watch and a heart rate belt that gather skin conductance and heart rate data, respectively. Algorithms for processing the skin conductance and heart rate data, as well as an automated method for labeling the collected `arousal' and `non-arousal' experimental data are developed. Small-scale user tests show 84% 10-fold, 83% between-subject, and 68% new-subject arousal detection accuracy.

We introduce a method for the automatic generation of magnetic resonance imaging brain atlases from a set of brains without any prior on their structure and number. This is a bottom-up, data-driven approach whose goal is to generate brain atlases that mirror as closely as possible the properties of arbitrary brains. This is important, for example, in the segmentation of brain structures of highly deformed brains as that of advanced Alzheimer's disease patients. The method is as follows. First, we register all brain pairs and obtain similarity weights. Second, we construct a graph whose nodes represent volumes and the links the similarity weights. Third, we use Dijkstra's method to select atlas candidates. Fourth, we iteratively merge clusters of atlas candidates until a good subset of them remains. We applied our method to a set of 61 ADNI brain volumes and found that a good selection could be made.

Distinguishing neurodegenerative diseased patients (e.g., suffering from Alzheimer's Disease (AD)) from healthy individuals with the aid of MRI images is one of the challenges that need to be addressed in the field of Computational Anatomy (CA). A crucial feature in the analysis is the rate of atrophy of brain structures like the hippocampus or the ventricles. Until recently, analysis of atrophy rate has been restricted mainly to `localized atrophy', i.e. atrophy within one brain structure. Distinguishing correlations of local atrophy rates between different brain structures could possibly provide additional information about the disease process. Therefore, in this paper, we introduce four correlation parameters to measure and analyze correlations of atrophy rate between hippocampus and ventricles. We combine these parameters with three local atrophy rate parameters into a seven-dimensional vector, and use various vector classification methods to see if the methods can distinguish AD patients from normal (NL) subjects in 31 longitudinal MRI baseline images and their follow-ups from the ADNI database. We obtain a good agreement between our classification results and the ground truth data. The analysis is facilitated with the aid of a specially designed graphical user interface.

In this paper we propose a new method for quantification of the rate of brain tissue deformation of patients with neurode-generative diseases. Our method allows for generating intermediary (virtual) MR brain volumes from real scans. In addition, we introduce two new descriptor spaces: (i) joint brain tissue deformation displacement vector magnitude and Jacobian and (ii) joint polar angles of displacement vector. In order to conduct our analysis, we built a visualization environment that illustrates images and flows, and it quantifies parameters. Through the calculation of the parameters, of the intermediary MR volumes, we create N-fold information simulating the deformation process. While our method is suitable for every neurodegenerative disease, we empirically tested our scheme on patients with diagnosed Alzheimer's disease, mild cognitive impairment and normal controls from the ADNI database. We focus on regions containing either ventricles or hippocampus. A distinction between different subjects could clearly be made.

The aging population and the growing amount of medical data have increased the need for automated tools in the neurology departments. Although the researchers have been developing computerized methods to help the medical expert, these efforts have primarily emphasized to improve the effectiveness in single patient data, such as computing a brain lesion size. However, patient-to-patient comparison that should help improve diagnosis and therapy has not received much attention. To this effect, this paper introduces a fast and robust region-of-interest retrieval method for brain MR images. We make the following various contributions to the domains of brain MR image analysis, and search and retrieval system: 1) we show the potential and robustness of local structure information in the search and retrieval of brain MR images; 2) we provide analysis of two complementary features, local binary patterns (LBPs) and Kanade-Lucas-Tomasi feature points, and their comparison with a baseline method; 3) we show that incorporating spatial context in the features substantially improves accuracy; and 4) we automatically extract dominant LBPs and demonstrate their effectiveness relative to the conventional LBP approach. Comprehensive experiments on real and simulated datasets revealed that dominant LBPs with spatial context is robust to geometric deformations and intensity variations, and have high accuracy and speed even in pathological cases. The proposed method can not only aid the medical expert in disease diagnosis, or be used in scout (localizer) scans for optimization of acquisition parameters, but also supports low-power handheld devices.

In the medical domain, experts usually look at specific anatomical structures to identify the cause of a pathology, and therefore they can largely benefit from automated tools that retrieve relevant slice(s) from a patient's image volume in diagnosis. Accordingly, this paper introduces a novel search and retrieval work for finding relevant slices in brain MR (magnetic resonance) volumes. As intensity is non-standard in MR we explore performance of two complementary intensity invariant features, local binary patterns and Kanade-Lucas-Tomasi feature points, their extended versions with spatial context, and a simple edge descriptor with spatial context. Experiments on real and simulated data showed that the local binary patterns with spatial context is fast, highly accurate, and robust to geometric deformations and intensity variations.

Automatic indexing of large collections of multimedia data is important for enabling retrieval functions. Current approaches mostly draw on a single or dual modality of video content analysis. Here we describe a framework for the integration of multimedia content and context information, which generalizes and systematizes current methods. Content information in the visual, audio, and text domains, is described at different levels of granularity and abstraction. Context describes the underlying structural information that can be used to constrain the possible number of interpretations. We introduce a probabilistic framework that combines (a) Bayesian networks that describe both content and context and (b) hierarchical priors that describe the integration of content and context. We present an application that uses this framework to segment and index TV programs. We discuss experimental results on segment classification on six and a half hours of broadcast video. In our experiments we used audio context information. Classification results for financial segments yield 83% and for celebrity segments 89%.

Currently available personal video recorders find and store whole TV programs. Our system, Video Scouting, not only finds and stores programs; it automatically segments and indexes story segments from the programs according to viewers' profiles. The extracted descriptions serve the viewers' content information requests for program segment selection, e.g., play the three minute interview with Hillary Clinton. To achieve this, the system combines information from the audio, visual, and transcript domains in a probabilistic framework based on Bayesian networks. We describe the overall architecture, a system implementation, and discuss some experimental results.

We describe integrated multimedia processing for Video Scout, a system that segments and indexes TV programs according to their audio, visual, and transcript information. Video Scout represents a future direction for personal video recorders. In addition to using electronic program guide metadata and a user profile, Scout allows the users to request specific topics within a program. For example, users can request the video clip of the USA president speaking from a half-hour news program. Video Scout has three modules: (i) video pre-processing, (ii) segmentation and indexing, and (iii) storage and user interface. Segmentation and indexing, the core of the system, incorporates a Bayesian framework that integrates information from the audio, visual, and transcript (closed captions) domains. This framework uses three layers to process low, mid, and high-level multimedia information. The high-level layer generates semantic information about TV program topics. This paper describes the elements of the system and presents results from running Video Scout on real TV programs.

Current personal Vido recorders make it very easy for consumers to record whole TV programs. Our research however, focuses on personalizing TV at a sub-program level. We use a traditional Content-Based Information Retrieval system architecture consisting of archiving and retrieval modules. The archiving module employs a three-layered, multimodal integration framework to segment, analyze, characterize, and classify segments. The retrieval module relies on users personal preferences to deliver both full programs and video segments of interest. We tested retrieval concepts with real users and discovered that they see more value in segmenting non-narrative programs (e.g. news) than narrative programs (e.g. movies). We benchmarked individual algorithms and segment classification for celebrity and financial segments as instances of non-narrative content. For celebrity segments we obtained a total precision of 94.1% and recall of 85.7%, and for financial segments a total precision of 81.1% and a recall of 86.9%.

In this paper, we describe a robust method for the extraction of the apparent 3-D camera velocity and 3-D scene structure information. Our method performs the extraction of the apparent 3-D camera velocity in a fully automated way without any knowledge about 3-D scene content information as used in current methods. This has the advantage that it can be used to fully automate the generation of natural-looking virtual/augmented environments, as well as in video-database browsing. First, we describe our method for the robust extraction of 3-D parameters. This method is a combination of the eight-point method in structure-from-motion with a statistical technique to automatically select feature points in the image, irrespective of 3-D content information. Second, we discuss two applications which use the results of the 3-D parameter extraction. The first application is the generation of sprite layers using 3-D camera velocity information to represent an eight-parameter perspective image-to-sprite mapping plus 3-D scene depth information for the sprite layering. The second application is the use of 3-D camera velocity for the indexing of large video databases according to a set of seven independent types of camera motion. In this application, we discuss a formal video description structure-the camera-motion descriptor-which was successfully included in the working draft of the MPEG-7 video standard.

Nonlinear video editors manipulate video sequences by contents irrespective of frame order. These are computer based tools that contrast with analogue linear tape editing technologies. The latter are extremely taxing of videographers time and resources. Current computerized editing methods represent video in terms of individual images. This poses a formidable task to the manipulation task due to the large data volumes associated with them. We discuss a framework-Generative Video, which deals with this problem in an efficient way. Generative Video represents video sequences in terms of constructs-compact models. These are world images and generative operators. World images are augmented images, which contain the non-redundant information in the video sequence, and they describe video contents information. For each independently moving object we have a different world image. World images are stratified in layers according to occlusion information. The generative operators access video contents information, such as the shape and motion of objects moving in the sequence. Nonlinear video editing is realized by applying generative operators to world images. This approach to nonlinear editing facilitates the access, storage, and manipulation of video contents information. We describe the main properties of Generative Video and demonstrate nonlinear editing on a real video sequence.

Current video compression standards compress video sequences at NTSC quality with factors in the range of 10-100, like in MPEG-1 and MPEC-2. To operate beyond this range, that is, MPEG-4, radically new techniques are needed. We discuss one such technique called generative video (GV). Video compression is realized in GV in two steps. First, the video sequence is reduced to constructs. Constructs are world images, corresponding to augmented images containing the non-redundant information on the sequence, and window, figure, motion, and signal processing operators, representing video sequence properties. Second, the world images are spatially compressed. The video sequence is reconstructed by applying the various operators to the decompressed world images. We apply GV to a 10 sec video sequence of a real 3-D scene and obtain compression ratios of about 2260 and 4520 for two experiments done with different quantization codebooks. The reconstructed video sequence exhibits very good perceptual quality.

The compact representation of video sequences is important for many applications, including very low bit-rate video compression and digital image libraries. We discuss here a novel approach, called generative video, by which video sequences are compactly represented in terms of their contents. This is achieved by reducing the video sequence to constructs. Constructs encode video sequence contents, such as, the shape and the velocity of independently moving objects, and the camera motion. Constructs are of two types: world images and generative operators. World images are augmented images incrementally generated. Generative operators, access video sequence contents and reconstruct the sequence from the world images. The reduction of a video sequence to constructs proceeds in steps. First, the shape of independently moving regions in the image is tessellated into rectangles. Second, world images are generated using the tessellated shape representation. This is described with an experiment using a real video sequence.

Describes how, in the process of extracting the optical flow through space-time filtering, one has to consider the constraints associated with the motion uncertainty, as well as the spatial and temporal sampling rates of the sequence of images. The motion uncertainty satisfies the Cramer-Rao (CR) inequality, which is shown to be a function of the filter parameters. On the other hand, the spatial and temporal sampling rates have lower bounds, which depend on the motion uncertainty, the maximum support in the frequency domain, and the optical flow. These lower bounds on the sampling rates and on the motion uncertainty are constraints that constitute an intrinsic part of the computational structure of space-time filtering. The author shows that if he uses these constraints simultaneously, the filter parameters cannot be arbitrarily determined but instead have to satisfy consistency constraints. By using explicit representations of uncertainties in extracting visual attributes, one can constrain the range of values assumed by the filter parameters.>

Proposes a statistical model for the perception of motion transparency and coherence which is given by a two stage process for the extraction of the optical flow and the velocity histogram. In the first stage, a sequence of images is divided into regions; in each region, an optical flow or a normal component of the optical flow is computed. In the second stage, in each of the regions mentioned above, a local velocity histogram is computed and the local histogram is summarised into a global histogram. As for experiment, the authors describe the perception of motion coherence and transparency for synthetic images, such as line patterns and filled regions. They then extend the model for real scenes and apply it to the analysis of the motion of pedestrians.>