Video slide shows constitute a very frequent video type. Typically, in order to produce such a video, a series of still images are used and processed. We present a method to solve the video slide show inversion problem where we are given a video slide show and we wish to extract the sequence of still images that have been used to produce this video. Our approach relies on a fast and efficient key-frame extraction method that partitions the video into content homogeneous segments and extracts a representative key-frame for each video segment. An important characteristic of this method is that the number of key-frames is determined automatically. Next, the set of key-frames is further processed to solve various problems such as mixing of images, fade-in/fade-out and zooming effects that produce keyframes not belonging to the original image sequence, thus they must be detected and discarded. We provide illustrative examples from the application of the method on several video slide shows with different characteristics.

Most of the existing approaches for camera motion detection are based on optical flow analysis and the use of the affine motion model. However, these methods are computationally expensive due to the cost of optical flow estimation and may be inefficient in the presence of moving objects whose motion is independent of the camera motion. We present an effective approach to detect camera motions by considering four trapezoidal regions in each frame and computing the horizontal and vertical translations of those regions. Then, simple decision rules based on the translations of the regions are employed in order to decide for the existence and the type of camera motion in each frame. In this way, three signals are constructed (pan, tilt, zoom) which are subsequently filtered to improve the robustness of the method. Comparative experiments on a variety of videos indicate that our method efficiently detects any type of camera motion (pan, tilt, zoom), even in the case where moving objects exist in the video sequence.

An efficient shot summarization method is presented based on agglomerative clustering of the shot frames. Unlike other agglomerative methods, our approach relies on a cluster merging criterion that computes the content homogeneity of a merged cluster. An important feature of the proposed approach is the automatic estimation of the number of a shot's most representative frames, called keyframes. The method starts by splitting each video sequence into small, equal sized clusters (segments). Then, agglomerative clustering is performed, where from the current set of clusters, a pair of clusters is selected and merged to form a larger unimodal (homogeneous) cluster. The algorithm proceeds until no further cluster merging is possible. At the end, the medoid of each of the final clusters is selected as keyframe and the set of keyframes constitutes the summary of the shot. Numerical experiments demonstrate that our method reasonable estimates the number of ground-truth keyframes, while extracting non-repetitive keyframes that efficiently summarize the content of each shot.

Reliable video summarization is one of the most important problems in digital video processing and analysis. The most common approach used for shot representation is the extraction of a set of key-frames sufficiently representing the total content of the shot. In such way, the whole video content can be represented using only a few, cautiously picked, non redundant key-frames maintaining at the same time a great percentage of information. A typical approach is to extract key frames using clustering. However, using a single image descriptor to extract key-frames is not sufficient due to large variations in the visual content of videos. In our approach, a weighted multi-view clustering algorithm is employed to combine two different image descriptors into a single similarity matrix, that serves as an input to a spectral clustering algorithm. Each image descriptor (view) does not contribute equally to the similarity matrix, but the weighted multi-view clustering algorithm associates a weight with each view and learns these weights automatically. Numerical experiments using a variety of videos demonstrate that our method is capable of efficiently summarizing video shots regardless of the characteristics of the visual content of the video.

Video indexing requires the efficient segmentation of video into scenes. The video is first segmented into shots and a set of key-frames is extracted for each shot. Typical scene detection algorithms incorporate time distance in a shot similarity metric. In the method we propose, to overcome the difficulty of having prior knowledge of the scene duration, the shots are clustered into groups based only on their visual similarity and a label is assigned to each shot according to the group that it belongs to. Then, a sequence alignment algorithm is applied to detect when the pattern of shot labels changes, providing the final scene segmentation result. In this way shot similarity is computed based only on visual features, while ordering of shots is taken into account during sequence alignment. To cluster the shots into groups we propose an improved spectral clustering method that both estimates the number of clusters and employs the fast global k-means algorithm in the clustering stage after the eigenvector computation of the similarity matrix. The same spectral clustering method is applied to extract the key-frames of each shot and numerical experiments indicate that the content of each shot is efficiently summarized using the method we propose herein. Experiments on TV-series and movies also indicate that the proposed scene detection method accurately detects most of the scene boundaries while preserving a good tradeoff between recall and precision.

Video indexing requires the efficient segmentation of the video into scenes. In the method we propose, the video is first segmented into shots and key-frames are extracted using the global k-means clustering algorithm that represent each shot. Then an improved spectral clustering method is applied to cluster the shots into groups based on visual similarity and a label is assigned to each shot according to the group that it belongs to. Next, a method for segmenting the sequence of shot labels is applied, providing the final scene segmentation result. Numerical experiments indicate that the method we propose correctly detects most of the scene boundaries while preserving a good trade off between recall and precision.