A huge fraction of cameras used nowadays is based on CMOS sensors with a rolling shutter that exposes the image line by line. For dynamic scenes/cameras this introduces undesired effects like stretch, shear and wobble. It has been shown earlier that rotational shake induced rolling shutter effects in hand-held cell phone capture can be compensated based on an estimate of the camera rotation. In contrast, we analyse the case of significant camera motion, e.g.\ where a bypassing street level capture vehicle uses a rolling shutter camera in a 3D reconstruction framework. The introduced error is depth dependent and cannot be compensated based on camera motion/rotation alone, invalidating also rectification for stereo camera systems. On top, significant lens distortion as often present in wide angle cameras intertwines with rolling shutter effects as it changes the time at which a certain 3D point is seen. We show that naive 3D reconstructions (assuming global shutter) will deliver biased geometry already for very mild assumptions on vehicle speed and resolution. We then develop rolling shutter dense multiview stereo algorithms that solve for time of exposure and depth at the same time, even in the presence of lens distortion and perform an evaluation on ground truth laser scan models as well as on real street-level data.

We present a novel algorithm that takes as input an uncalibrated unordered set of spherical panoramic images and outputs their relative pose up to a global scale. The panoramas contain both indoor and outdoor shots and each set was taken in a particular indoor location e.g. a bakery or a restaurant. The estimated pose is used to build a map of the location, and allow easy visual navigation and exploration in the spirit of Google's Street View. We also present a dataset of 9 sets of panoramas, together with an annotation tool and ground truth point correspondences. The manual annotations were used to obtain ground truth relative pose, and to quantitatively evaluate the different parameters of our algorithm, and can be used to benchmark different approaches. We show excellent results on the dataset and point out future work.

While image clustering has many important applications ranging from personal to Web image management, its use is often limited by the difficulty of extracting reliable semantics from low level image features. The image clusters can be improved by using features extracted from image regions rather than the whole image. Region segmentation can be improved in turn, by considering all images within the same cluster rather than segmenting each image independently. This observation leads to the unified Bayesian framework for image clustering and segmentation presented in this paper. The experimental results, reported using several types of visual feature extractors on a database of Web documents containing over 6000 images, illustrates a significant improvement over existing techniques.

With the proliferation of digital cameras, the size of personal media data such as digital photos, videos, etc. has grown extremely large. The personal nature of the data has heightened the demands for a media management system on personal desktops. Existing solutions for media management target mostly server-based Web databases and rely on extensive metadata (i.e., labels) generation to aid retrieval. Personal media databases, on the other hand, have very limited labels generated by the end users themselves. This paper introduces a method for learning concept templates from web images to query personal image databases. The proposed method has the advantage of leveraging Web resources to ease personal photo retrieval in order to avoid costly annotation of personal image databases.

With the advent and proliferation of digital cameras and computers, the number of digital photos created and stored by consumers has grown extremely large. This created increasing demand for image retrieval systems to ease interaction between consumers and personal media content. Active learning is a widely used user interaction model for retrieval systems, which learns the query concept by asking users to label a number of images at each iteration. In this paper, we study sampling strategies for active learning in personal photo retrieval. In order to reduce human annotation efforts in a content-based image retrieval setting, we propose using multiple sampling criteria for active learning: informativeness, diversity and representativeness. Our experimental results show that by combining multiple sampling criteria in active learning, the performance of personal photo retrieval system can be significantly improved

Most programs are repetitive, where similar behavior can be seen at different execution times. Algorithms have been proposed that automatically group similar portions of a program's execution into phases, where samples of execution in the same phase have homogeneous behavior and similar resource requirements. In this paper, we examine applying these phase analysis algorithms and how to adapt them to parallel applications running on shared memory processors. Our approach relies on a separate representation of each thread's activity. We first focus on showing its ability to identify similar intervals of execution across threads for a single run. We then show that it is effective at identifying similar behavior of a program when the number of threads is varied between runs. This can be used by developers to examine how different phases scale across different number of threads. Finally, we examine using the phase analysis to pick simulation points to guide multithreaded simulation.

In many novel application scenarios such as smart rooms or sensing rooms visual sensors (such as cameras) need to know which visual actuators (such as displays) are visible to them. Often only parts of a display are visible from a camera. Therefore, a novel algorithm for precise visibility determination is presented. The algorithm makes the assumption that the displays are active, i.e., they can be controlled by the application. Under these conditions the algorithm determines precisely where which parts of a display are imaged by a camera

Recent studies have shown that most SPEC CPU2K benchmarks exhibit strong phase behavior, and the Cycles per Instruction (CPI) performance metric can be accurately predicted based on program's control-flow behavior, by simply observing the sequencing of the program counters, or extended instruction pointers (EIPs). One motivation of this paper is to see if server workloads also exhibit such phase behavior. In particular, can EIPs effectively predict CPI in server workloads? We propose using regression trees to measure the theoretical upper bound on the accuracy of predicting the CPI using EIPs, where accuracy is measure by the explained variance of CPI with EIPs. Our results show that for most server workloads and, surprisingly, even for CPU2K benchmarks, the accuracy of predicting CPI from EIPs varies widely. We classify the benchmarks into four quadrants based on their CPI variance and predictability of CPI using EIPs. Our results indicate that no single sampling technique can be broadly applied to a large class of applications. We propose a new methodology that selects the best-suited sampling technique to accurately capture the program behavior.

This paper describes a system for learning a face model that is used for 3D tracking of the human face. The face is modeled as a linear combination of shape basis vectors and action vectors. Shape space models the difference in face shape of different people while action space models the facial expressions. First, real stereo tracking data is used to learn the space of these shape and action vectors using Principal Component Analysis. Then this low-complexity model is used to simultaneously track shape, pose and expression from a monocular image sequence. The main contribution of this paper is in learning shape and action deformation models simultaneously from real data. Results of monocular model-based tracking for subjects not included in the training set show that the model derived from data is robust and generalizes well.

A light field parameterized on the surface offers a natural and intuitive description of the view-dependent appearance of scenes with complex reflectance properties. To enable the use of surface light fields in real-time rendering we develop a compact representation suitable for an accelerated graphics pipeline. We propose to approximate the light field data by partitioning it over elementary surface primitives and factorizing each part into a small set of lower-dimensional functions. We show that our representation can be further compressed using standard image compression techniques leading to extremely compact data sets that are up to four orders of magnitude smaller than the input data. Finally, we develop an image-based rendering method, light field mapping, that can visualize surface light fields directly from this compact representation at interactive frame rates on a personal computer. We also implement a new method of approximating the light field data that produces positive only factors allowing for faster rendering using simpler graphics hardware than earlier methods. We demonstrate the results for a variety of non-trivial synthetic scenes and physical objects scanned through 3D photography.

This paper describes a two-stage system for 3D tracking of pose and deformation of the human face in monocular image sequences without the use of special markers. The first stage of the system learns the space of all possible facial deformations by applying principal component analysis on real stereo tracking data. The resulting model approximates any generic shape as a linear combination of shape basis vectors. The second stage of the system uses this low-complexity deformable model for simultaneous tracking of pose and deformation of the face from a single image sequence. This stage is known as model-based monocular tracking. There are three main contributions of this paper. First we demonstrate that a data-driven approach for model construction is suitable for tracking non rigid objects and offers an elegant and practical alternative to the task of manual construction of models using 3D scanners or CAD modelers. Second, we show that such a method exhibits good tracking accuracy (errors less than 5 mm) and robustness characteristics. Third, we demonstrate that our system exhibits very promising generalization properties in enabling tracking of multiple persons with the same 3D model.

We present a multi-PC/camera system that can perform 3D reconstruction and ellipsoid fitting of moving humans in real time. The system consists of five cameras. Each camera is connected to a PC which locally extracts the silhouettes of the moving person in the image captured by the camera. The five silhouette images are then sent, via local network, to a host computer to perform 3D voxel-based reconstruction by an algorithm called SPOT. Ellipsoids are then used to fit the reconstructed data. By using a simple and user-friendly interface, the user can display and observe, in real time and from any view-point, the 3D models of the moving human body. With a rate of higher than 15 frames per second, the system is able to capture non-intrusively, a sequence of human motions.

This paper describes a stereo-based approach for gesture recognition that works well under extreme lighting conditions and tolerates a large range of hand poses. The approach proposes a hybrid gesture representation that models the user's arm as a 3D line and uses images to represent the hand gestures. The algorithm finds the arm orientation and the hand location from the disparity data and uses this information to initialize a color based segmentation algorithm that cleanly separates the hand from the background. Finally, our approach uses the arm orientation to compute a frontal view of the hand through perspective unwarping producing easily recognizable hand gesture templates. The classification algorithm uses statistical moments of the binarized gesture templates to find the match. We achieve 96% recognition rates under varying lighting and 3D poses for a set of 6 gestures.

A method for reconstructing 3D scene geometry from a set of projected shadows is presented. It is composed of two stages. First, the scene geometry is retrieved up to three scalar unknowns using only the information contained in the observed shadow edges on the image plane. Then, the three remaining unknowns are computed making use of the known depths at three points. This technique improves upon previous results in that it does not require the presence of a reference plane in the background. A mathematical analysis is presented using dual-space geometry, a formalism that provides adequate tools to carry out all the derivations in a compact and intuitive manner. A linear algorithm based on singular value decomposition (SVD) is presented leading to a closed form solution for reconstruction.

A simple and inexpensive approach for extracting the three-dimensional shape of objects is presented. It is based on 'weak structured lighting'; it differs from other conventional structured lighting approaches in that it requires very little hardware besides the camera: a desk-lamp, a pencil and a checker-board. The camera faces the object, which is illuminated by the desk-lamp. The user moves a pencil in front of the light source casting a moving shadow on the object. The 3D shape of the object is extracted from the spatial and temporal location of the observed shadow. Experimental results are presented on three different scenes demonstrating that the error in reconstructing the surface is less than 1%.

We assess the usefulness of monocular recursive motion estimation techniques for vehicle navigation in the absence of a model for the environment. For this purpose we extend a recently proposed recursive motion estimator, the Essential filter, to handle scale estimation. We examine experimentally the accuracy with which the motion and position of the vehicle may be computed on an 8000 frame indoors sequence. The issues of sampling time frequency and number of necessary features in the environment are addressed systematically.>