When people look at pictures, they fixate on specific areas. The sequences of such fixations are so characteristic for certain pictures that metrics can be derived that allow successful grouping of similar pieces of visual art. However, determining enough fixation sequences by eye tracking is not practically feasible for large groups of people and pictures. In order to get around this limitation, we present a novel algorithm that simulates eye movements by calculating scan paths for images and time frames in real time. The basis of our algorithm is an attention model that combines and optimizes rectangle features with Adaboost. The model is adapted to the characteristics of the retina, and its input is dependent on a few earlier fixations. This method results in significant improvements compared to previous approaches. Our simulation process delivers the same data structures as an eye tracker, thus can be analyzed by standard eye-tracking software. A comparison with recorded data from eye tracking experiments shows that our algorithm for simulating fixations has a very good prediction quality for the stimulus areas on which many subjects focus. We also compare the results with those from earlier works. Finally, we demonstrate how the presented algorithm can be used to calculate the similarity of pictures in terms of human perception.

We present a scalable technique to explore surveillance videos by scatter/gather browsing of trajectories of moving objects. Trajectories are clustered according to a variety of properties, such as location, orientation, and velocity that can be selected by the users. These properties allow for faceted video exploration and refinement of previous browsing steps. The proposed approach facilitates interactive clustering of trajectories by an effective way of cluster visualization that we term schematic summaries. This novel visualization illustrates cluster summaries in a schematic, nonphotorealistic style. To reduce visual clutter, we introduce the trajectory bundling technique. Further, schematic summaries include a timeline view and a showcase view to represent the facets present in a cluster. The fusion of schematic summaries, a variety of facets, and user interaction lead to efficient hierarchical exploration of video data. Examples of different browsing scenarios and initial user feedback demonstrate the potentials of our method.

Abstract In recent years, a collection of new techniques which deal with video as input data, emerged in computer graphics and visualization. In this survey, we report the state of the art in video‐based graphics and video visualization. We provide a review of techniques for making photo‐realistic or artistic computer‐generated imagery from videos, as well as methods for creating summary and/or abstract visual representations to reveal important features and events in videos. We provide a new taxonomy to categorize the concepts and techniques in this newly emerged body of knowledge. To support this review, we also give a concise overview of the major advances in automated video analysis, as some techniques in this field (e.g. feature extraction, detection, tracking and so on) have been featured in video‐based modelling and rendering pipelines for graphics and visualization.

We evaluate and compare video visualization techniques based on fast-forward. A controlled laboratory user study (n = 24) was conducted to determine the trade-off between support of object identification and motion perception, two properties that have to be considered when choosing a particular fast-forward visualization. We compare four different visualizations: two representing the state-of-the-art and two new variants of visualization introduced in this paper. The two state-of-the-art methods we consider are frame-skipping and temporal blending of successive frames. Our object trail visualization leverages a combination of frame-skipping and temporal blending, whereas predictive trajectory visualization supports motion perception by augmenting the video frames with an arrow that indicates the future object trajectory. Our hypothesis was that each of the state-of-the-art methods satisfies just one of the goals: support of object identification or motion perception. Thus, they represent both ends of the visualization design. The key findings of the evaluation are that object trail visualization supports object identification, whereas predictive trajectory visualization is most useful for motion perception. However, frame-skipping surprisingly exhibits reasonable performance for both tasks. Furthermore, we evaluate the subjective performance of three different playback speed visualizations for adaptive fast-forward, a subdomain of video fast-forward.

Background subtraction is one of the key techniques for automatic video analysis, especially in the domain of video surveillance. Although its importance, evaluations of recent background subtraction methods with respect to the challenges of video surveillance suffer from various shortcomings. To address this issue, we first identify the main challenges of background subtraction in the field of video surveillance. We then compare the performance of nine background subtraction methods with post-processing according to their ability to meet those challenges. Therefore, we introduce a new evaluation data set with accurate ground truth annotations and shadow masks. This enables us to provide precise in-depth evaluation of the strengths and drawbacks of background subtraction methods.

We present a new and scalable technique to explore surveillance videos by scatter/gather browsing of trajectories of moving objects. The proposed approach facilitates interactive clustering of trajectories by an effective way of cluster visualization that we term schematic summaries. This novel visualization illustrates cluster summaries in a schematic, non-photorealistic style. To reduce visual clutter, we introduce the trajectory bundling technique. The fusion of schematic summaries and user interaction leads to efficient hierarchical exploration of video data. Examples of different browsing scenarios demonstrate the effectiveness of the proposed method.