Display advertising is a multi-billion dollar industry where advertisers promote their products to users by having publishers display their advertisements on popular Web pages. An important problem in online advertising is how to forecast the number of user visits for a Web page during a particular period of time. Prior research addressed the problem by using traditional time-series forecasting techniques on historical data of user visits; (e.g., via a single regression model built for forecasting based on historical data for all Web pages) and did not fully explore the fact that different types of Web pages have different patterns of user visits.

We propose a method for forecasting high-dimensional data (hundreds of attributes, trillions of attribute combinations) for a duration of several months. Our motivating application is guaranteed display advertising, a multi-billion dollar industry, whereby advertisers can buy targeted (high-dimensional) user visits from publishers many months or even years in advance. Forecasting high-dimensional data is challenging because of the many possible attribute combinations that need to be forecast. To address this issue, we propose a method whereby only a sub-set of attribute combinations are explicitly forecast and stored, while the other combinations are dynamically forecast on-the-fly using high-dimensional attribute correlation models. We evaluate various attribute correlation models, from simple models that assume the independence of attributes to more sophisticated sample-based models that fully capture the correlations in a high-dimensional space. Our evaluation using real-world display advertising data sets shows that fully capturing high-dimensional correlations leads to significant forecast accuracy gains. A variant of the proposed method has been implemented in the context of Yahoo!'s guaranteed display advertising system.

In this paper, we propose an approach for learning appearance models of moving objects directly from compressed video. The appearance of a moving object changes dynamically in video due to varying object poses, lighting conditions, and partial occlusions. Efficiently mining the appearance models of objects is a crucial and challenging technology to support content-based video coding, clustering, indexing, and retrieval at the object level. The proposed approach learns the appearance models of moving objects in the spatial-temporal dimension of video data by taking advantage of the MPEG video compression format. It detects a moving object and recovers the trajectory of each macroblock covered by the object using the motion vector present in the compressed stream. The appearances are then reconstructed in the DCT domain along the object's trajectory, and modeled as a mixture of Gaussians (MoG) using DCT coefficients. We prove that, under certain assumptions, the MoG model learned from the DCT domain can achieve pixel-level accuracy when transformed back to the spatial domain, and has a better band-selectivity compared to the MoG model learned in the spatial domain. We finally cluster the MoG models to merge the appearance models of the same object together for object-level content analysis.

Image segmentation is conventionally formulated as a pixel-labeling problem, in which "hard" decisions have to be made to partition pixels into regions. As image segmentation is usually used as a preprocessing step in many image analysis applications, the segmentation errors introduced by the "hard" decisions bring difficulties to higher-level image analysis. In this paper, we propose a "soft" image segmentation method to model the object appearance and spatial layouts in an image with an incremental mixture of probabilistic models. The proposed approach extracts "soft" regions incrementally using adaptive apertures without making any hard decisions. We show that "soft" regions not only bring more robustness than conventional "hard" regions but also enable a higher-level region-based analysis.

This paper presents a robust object tracking method via a spatial bias appearance model learned dynamically in video. Motivated by the attention shifting among local regions of a human vision system during object tracking, we propose to partition an object into regions with different confidences and track the object using a dynamic spatial bias appearance model (DSBAM) estimated from region confidences. The confidence of a region is estimated to re ect the discriminative power of the region in a feature space, and the probability of occlusion. We propose a novel hierarchical Monte Carlo (HAMC) algorithm to learn region confidences dynamically in every frame. The algorithm consists of two levels of Monte Carlo processes implemented using two particle filtering procedures at each level and can efficiently extract high confidence regions through video frames by exploiting the temporal consistency of region confidences. A dynamic spatial bias map is then generated from the high confidence regions, and is employed to adapt the appearance model of the object and to guide a tracking algorithm in searching for correspondences in adjacent frames of video images. We demonstrate feasibility of the proposed method in video surveillance applications. The proposed method can be combined with many other existing tracking systems to enhance the robustness of these systems.

Social interaction plays an important role in our daily lives. It is one of the most important indicators of physical or mental changes in aging patients. In this article, we investigate the problem of detecting social interaction patterns of patients in a skilled nursing facility using audio/visual records. Our studies consist of both a “Wizard of Oz” style study and an experimental study of various sensors and detection models for detecting and summarizing social interactions among aging patients and caregivers. We first simulate plausible sensors using human labeling on top of audio and visual data collected from a skilled nursing facility. The most useful sensors and robust detection models are determined using the simulated sensors. We then present the implementation of some real sensors based on video and audio analysis techniques and evaluate the performance of these implementations in detecting interactions. We conclude the article with discussions and future work.

We propose a novel unsupervised learning algorithm to extract the layout of an image by learning latent object-related aspects. Unlike traditional image segmentation algorithms that segment an image using feature similarity, our method is able to learn high-level object characteristics (aspects) from a large number of unlabeled images containing similar objects to facilitate image segmentation. Our method does not require human to annotate the training set and works without supervision. We use a graphical model to address the learning of aspects and layout extraction together. In particular, aspect-feature dependency from multiple images is learned via the expectation-maximization algorithm. We demonstrate that, by associating latent aspects to spatial structure, the proposed method achieves much better layout extraction results than using probabilistic latent semantic analysis.

People identification is an essential task for video content analysis in a surveillance system. To construct a good classifier requires a large amount of training data, which may not be obtained in some scenario. In this paper, we propose an approach to augment insufficient training data by labeling identical video images that have removed people's identities by masking faces. We show user study results that human subjects can perform reasonably well in labeling pairwise constraints from face obscured images. We also present a new discriminative learning algorithm WPKLR to handle uncertainties in pairwise constraints. The effectiveness of the proposed approach is demonstrated using video captured in a nursing home environment. The experiments show that the WPKLR approach can obtain a high accuracy of people identification using limited labeled data and noisy pairwise constraints, and meanwhile minimize the risk of exposing people's identities

Patient monitoring via video and physiological data recording can now be performed outside hospitals. This procedure, usually performed in a prolonged manner, generates a considerable amount of data, which calls for efficient ways for archiving and transmission. In this work, we present a specialized system to code the video and the physiological data recorded from a patient, aiming at a reduced bandwidth requirement compared to the conventional methods. We've developed an object-based approach to coding the monitoring video. By applying two change detection methods, we decompose a video frame into three video object planes (VOPs) representing the background, the stationary foreground and the moving foreground. These VOPs are coded at different frame rates, leading to a reduced overall bit rate. For coding the physiological data (using electroencephalogram, i.e. EEG, as an example), we present an effective solution by using a combination of the lifting scheme and the SPIHT algorithm. This approach is featured with a wavelet-quantization algorithm that enables a scalable transmission. The feasibility of this proposed system is demonstrated by our experimental results

Multiparty collaborative applications in which groups of people act in concert to achieve some real-world goal abound. In these situations, it is useful for a central planning agent to receive online audio-visual information from all participants. However, as the size of the group grows, it becomes difficult to process all the sensory streams; cognitive overload prevents direct analysis of sensory streams for situational awareness. To avoid this situation, an automatic method is needed to assign value to each stream and direct the attention of the planning agent to those streams which are most valuable. We present an audio-based blind value assignment (BVA) method to address this problem, and experiments demonstrating the method's efficacy. We demonstrate that use of audio BVA techniques results in automatic value judgments which are broadly similar to human value judgments and superior to automatic judgments based on video information

Context-aware computer systems are characterized by the ability to consider user state information in their decision logic. One example application of context-aware computing is the smart mobile telephone. Ideally, a smart mobile telephone should be able to consider both social factors (i.e., known relationships between contactor and contactee) and environmental factors (i.e., the contactee's current locale and activity) when deciding how to handle an incoming request for communication.Toward providing this kind of user state information and improving the ability of the mobile phone to handle calls intelligently, we present work on inferring environmental factors from sensory data and using this information to predict user interruptibility. Specifically, we learn the structure and parameters of a user state model from continuous ambient audio and visual information from periodic still images, and attempt to associate the learned states with user-reported interruptibility levels. We report experimental results using this technique on real data, and show how such an approach can allow for adaptation to specific user preferences.

Training a good text detector requires a large amount of labeled data, which can be very expensive to obtain. Co-training has been shown to be a powerful semi-supervised learning tool for solving many problems using a large amount of unlabeled data. However, augmented data from a co-training process could potentially degrade the performance of classifiers due to added noises from unlabeled data. This paper makes two contributions by proposing a modified co-training scheme for text detection. First, to get cleaner augmented data, the new algorithm integrates some authority knowledge of unlabeled data into co-training. Text recognition output of each selected unlabeled image patch is used as the authority that is combined with classifier prediction to decide if the sample will be added to the augmented set. Second, instead of evenly combining predictions of two co-training classifiers, a weighted combination is learned and used to produce the final prediction. Contributions of the new algorithm have been evaluated on a standard text detection dataset.

This paper presents a probabilistic algorithm for segmenting and recognizing text embedded in video sequences based on adaptive thresholding using a Bayes filtering method. The algorithm approximates the posterior distribution of segmentation thresholds of video text by a set of weighted samples. The set of samples is initialized by applying a classical segmentation algorithm on the first video frame and further refined by random sampling under a temporal Bayesian framework. This framework allows us to evaluate a text image segmentor on the basis of recognition result instead of visual segmentation result, which is directly relevant to our character recognition task. Results on a database of 6944 images demonstrate the validity of the algorithm.

In this paper, we propose a multimodal system for detecting human activity and interaction patterns in a nursing home. Activities of groups of people are firstly treated as interaction patterns between any pair of partners and are then further broken into individual activities and behavior events using a multi-level context hierarchy graph. The graph is implemented using a dynamic Bayesian network to statistically model the multi-level concepts. We have developed a coarse-to-fine prototype system to illustrate the proposed concept. Experimental results have demonstrated the feasibility of the proposed approaches. The objective of this research is to automatically create concise and comprehensive reports of activities and behaviors of patients to support physicians and caregivers in a nursing facility.

This paper presents a probabilistic algorithm for segmenting and recognizing text embedded in video sequences. The algorithm approximates the posterior distribution of segmentation thresholds of video text by a set of weighted samples. After initialization the set of samples is recursively refined by random sampling under a temporal Bayesian framework. The proposed methodology allows us to estimate the optimal text segmentation parameters directly in function of the string recognition results instead of segmentation quality. Results on a database of 6944 images demonstrate the validity of the algorithm.

A method for segmenting and recognizing text embedded in video and images is proposed. Multiple segmentation of the same text region is performed, thus producing multiple hypotheses of binary text images. The segmentation algorithm is stated as a statistical labeling and is based on a Markov random field (MRF) model of the label map. Background regions in each hypothesis are then removed by performing a connected component analysis and by enforcing a more stringent constraint (called GCC - grayscale consistency constraint) on the text characters' grayscale values using a robust 1D-median operator. Each text image hypothesis is then processed by optical character recognition (OCR) software. The final result is then selected from the set of output strings. Results show that both the use of multiple hypotheses and the GCC significantly improve the results.

The paper presents a fast and robust algorithm to identify text in image or video frames with complex backgrounds and compression effects. The algorithm first extracts the candidate text line on the basis of edge analysis, baseline location and heuristic constraints. Support Vector Machine (SVM) is then used to identify text line from the candidates in edge-based distance map feature space. Experiments based on a large amount of images and video frames from different sources showed the advantages of this algorithm compared to conventional methods in both identification quality and computation time.

Awareness support system are based on formal and specific context information such as location, or on video-mediated general context information such asa view into a remote office. We propose a new approach based on fusion of these different kinds of context information. In this approach we distinguish white box context, used by the awareness system for reasoning, and black box context, which can only be interpreted by humans. Our approach uses a variety of perception techniques to obtain white box context from audio and video streams. White box context is then used for further processing of context information, for instance to derive additional context. It is further used to generate a storyboard-like multimedia representation of collected and extracted context information. This storyboard provides a condensed view of recent activity to collaboration partners.

Summary form only given. A sprite which is an image composed of pixels belonging to a video object visible throughout a video segment is a very important concept proposed by MPEG4. Because of the search region limitation in the global motion estimation, the performance of traditional sprite coding technology is not satisfactory in the case of fast camera motion. Only enlarging the search region is difficult to ensure the right motion estimation. An improved algorithm is proposed with enlarging the search region, predicting the motion of the current VOP (video object plane) and shortening the iterative time. Three main techniques are adopted in the new algorithm. They are: (1) enlarging search region; (2) weighting sum of absolute difference (SAD); and (3) shortening iteration. Two group experiments present a comparison between the original algorithm and the improved algorithm. The first group experiment shows the improved algorithm gets the same performance as the original algorithm in coding the general motion sequences. In the second group experiment, the Stefan background sequences which frame rates of 15 Hz, 10 Hz, 7.5 Hz, and 6 Hz are encoded in various transformations. The results given in a table show the coding performances of the algorithm are significantly improved.

A robust numerical model for the charge balance condition (CBC) on floating gates (FG) is presented, in which the quasi-Fermi level, instead of the potential (as in conventional models), is assumed to be a constant in the FG. By solving Poisson's equation in the FG region with an extra equation for the CBC, the accurate distribution of both potential and charge can be obtained. A quasi-stationary scheme is developed to simulate the writing or erasing procedure of EEPROM cells, which may serve as a useful tool for accurate simulation of devices such as EEPROMs or EPROMs. Simulation results reveal a significant degradation of tunneling efficiency when N/sub FG/>