This survey introduces the current state of the art in image and video retargeting and describes important ideas and technologies that have influenced the recent work. Retargeting is the process of adapting an image or video from one screen resolution to another to fit different displays, for example, when watching a wide screen movie on a normal television screen or a mobile device. As there has been considerable work done in this field already, this survey provides an overview of the techniques. It is meant to be a starting point for new research in the field. We include explanations of basic terms and operators, as well as the basic workflow of the different methods.

With the popularity of social networks, users can communicate with each other in a more convenient way. However, the increasing amount of data poses new challenges for the analysis of the social activities of the users. In this paper, we propose to visualize the heterogeneous information of user interactions in a social network in a three-dimensional way using the concept of solar systems. The target user represents the center of the solar system. We determine physical variables from interaction frequencies between the user and their friends to be visualized. This gives the user a better insight about their relationships on Facebook. To show the interactions between the user and their friends, we choose four variables related to the solar system: the size of each planet, its angular velocity, and the semi-major and the semi-minor axis of its elliptical orbit. Our system measures the interaction frequencies between a user and their friends based on a linear model. Its coefficients are derived from an online survey we performed. The experimental results indicate that the accuracy of our estimated interaction frequency is better than the accuracies for each individual interaction feature. The average accuracy improvement is 15.93%.

Social media platforms provide large amounts of user-generated text which can be utilized for text-based Sentiment Analysis in order to obtain insights about opinions on many aspects of life. Current approaches that are based on supervised learning require manually annotated data sets that are time-consuming to create and are specific to a single language. In this work, we present an approach to generate ground truth sentiment values for a data set from Twitter. We use a sentiment emoji lexicon and distribute known polarities of hashtags over neighbors in a graph we build on them. This approach is language-independent. Native speakers of five different languages evaluate the accuracy of the sentiment values assigned by our method on a corpus of Tweets. Our experiments show that the quality of our automatically assigned sentiment values is sufficiently high to be used for training of machine learning-based sentiment analysis.

We estimate the velocity field of the blood flow in a human face from videos. Our approach first performs spatial preprocessing to improve the signal-to-noise ratio (SNR) and the computational efficiency. The discrete Fourier transform (DFT) and a temporal band-pass filter are then applied to extract the frequency corresponding to the subject's heart rate. We propose two techniques for reducing the noise from the resulting phase and amplitude maps. The 2D blood flow field is then estimated from the relative phase shift between the pixels. We evaluate our approach on real and synthetic face videos using two different metrics. Our method produces a velocity field with an angular error of 20° and an error in magnitude of 29% on the average.

The re-capturing of video content poses significant challenges to algorithms in the fields of video forensics, watermarking, and near-duplicate detection. Using a camera to record a video from a display introduces a variety of artifacts, such as geometric distortions, luminance transformations, and temporal aliasing. A deep understanding of the causes and effects of such phenomena is required for their simulation, and for making the affected algorithms more robust. In this paper, we provide a detailed model of the temporal effects in re-captured video. Such effects typically result in the re-captured frames being a blend of the original video's source frames, where the specific blend ratios are difficult to predict. Our proposed parametric model captures the temporal artifacts introduced by interactions between the video renderer, display device, and camera. The validity of our model is demonstrated through experiments with real re-captured videos containing specially marked frames.

In this paper, a novel warping-based retargeting approach for stereoscopic video is proposed. It considers the three conflicting goals: preserving salient image content, avoiding flickering and maintaining consistency between the two views. The first step of the approach focuses on content-aware image resizing and considers image saliency, motion saliency, and depth saliency. In the second step, temporal coherence is preserved by tracking and optimizing deformed pathlines. The proposed algorithm maps the mesh from the left to the right view to guarantee consistency between the deformation of objects between the views. Users rated the quality of the adapted videos as good. In particular, distortions in the perceived depth are not noticeable, and the temporal stability is significantly higher compared to seam carving approaches.

In a remote surveillance system, a high resolution surveillance camera streams its video to a user's handheld device. Such devices are unable to make use of the high resolution video due to their limited display size and bandwidth. In this paper, we propose a method to assist the mobile operator of the surveillance camera in focusing on sensitive regions of the video. Our system automatically identifies relevant regions. We introduce a pan and zoom strategy to ensure that the operator is able to see fine details in these areas while maintaining contextual knowledge. Regions of interest are identified using foreground detection as well as face and body detection. The efficacy of the proposed method is demonstrated through a user study. Our proposed method was reported to be more useful than two comparable approaches for getting an understanding of the activities in a surveillance scene while maintaining context.

In this paper, we present a fast parallel algorithm for the retargeting of videos. It combines seam carving and cropping and is aimed for real-time adaptation of video streams. The basic idea is to first find an optimal cropping path over the whole sequence with the target size. Then, the borders are slightly extended to be reduced again by seam carving on a frame-by-frame basis. This allows the algorithm to get more important content into the cropping window as it is also able to remove pixels from within the window. In contrast to the previous SeamCrop algorithm, the presented technique is optimized for parallel processes and a CUDA GPU implementation. In comparison, the computation time of our GPU algorithm is 10.5 times faster (on a 960 x 540 video with a retarget factor of 25%) than the already efficient CPU implementation.

We present a novel system to assist users in choosing suitable exposure values for photography on smartphones. The user specifies the desired shape of the histogram of the captured image by adjusting the parameters of a function. The system then uses the smartphone's camera and varying exposure values to calculate a high dynamic range (HDR) histogram of the scene. It contains the entire brightness range from the darkest to the brightest area of the scene. For any exposure value, the HDR histogram can be used to synthesize a conventional histogram of a low dynamic range (LDR) image as if captured by this exposure value. This process is much faster than capturing the image itself. By comparing a number of synthesized LDR histograms to the user's desired histogram shape, the system determines an exposure value that best meets the user's preference.

We present a novel system for automatically detecting salient image regions in stereoscopic videos. Our proposed algorithm considers information based on three dimensions: salient colors in individual frames, salient information derived from camera and object motion, and depth saliency. These three components are dynamically combined into one final saliency map based on the reliability of the individual saliency detectors. Such a combination allows using more efficient algorithms even if the quality of one detector degrades. For example, we use a computationally efficient stereo correspondence algorithm that might cause noisy disparity maps for certain scenarios. In this case, however, a more reliable saliency detection algorithm such as the image saliency is preferred. To evaluate the quality of the saliency detection, we created modified versions of stereoscopic videos with the non-salient regions blurred. Having users rate the quality of these videos, the results show that most users do not detect the blurred regions and that the automatic saliency detection is very reliable.

We present a system that is capable of capturing, processing and displaying High Dynamic Range (HDR) videos at 23 frames per second. For each frame in the video, a sequence of regular images is captured in quick succession at optimally chosen shutter speeds. Intermediate camera motion is compensated and the images are combined into an HDR frame which is tone mapped for display. Our system makes use of novel algorithms that are fast enough for processing the frames in real-time. The most time-consuming operations are parallelized and executed on a graphics card for a speed-up of 15 to 1 over the sequential version.

The quality achieved by simply scaling a sports video to the limited display resolution of a mobile device is often insufficient. As a consequence, small details like the ball or lines on the playing field become unrecognizable. In this paper, we present a novel approach to analyzing court-based ball sports videos. We have developed new techniques to distinguish actual playing frames, to detect players, and to track the ball. This information is used for advanced video retargeting, which emphasizes essential content in the adapted videos. We evaluate the precision and recall achieved in the analysis and measure the computational time taken. In addition, we compare our new approach to other video retargeting techniques based on scaling, cropping, and seam carving.

We introduce a novel approach for image registration for high dynamic range (HDR) videos. We estimate a translation vector between two low dynamic range (LDR) frames captured at different exposure settings. By using row and column histograms, counting the number of dark and bright pixels in a row or column, and maximizing the correlation between the histograms of two consecutive frames, we reduce the two-dimensional problem to two one-dimensional searches. This saves computation time, which is critical in recording HDR videos in real-time. The robustness of our estimation is increased through application of a Kalman filter. A novel certainty criterium controls whether the estimated translation is used directly or discarded and extrapolated from previous frames. Our experiments show that our proposed approach performs registration more robustly on videos and is 1.4 to 3 times faster than comparable algorithms.

We present an approach for integrating voice communication into massively multiplayer online games (MMOG). The currently available third-party software solutions involve additional fees and are not connected to the game logic. Other approaches use the game servers directly for mixing and transmitting speech. Doing so allows for a tight coupling of speech and in-game events but consumes large amounts of bandwidth and computational power on the servers. Our novel idea is a mix between server-based signaling and peer-to-peer data transmission. While the entire game context resides on the server, audio streams are transmitted in a peer-to-peer fashion and processed at the peer nodes only. This technique splits up the workload among all machines while still allowing to relate speech processing to game events. We further improve the processing performance on the clients by using SOUND, a novel technique for distance based resampling.

In this paper we present an optimized approach to capture high dynamic range (HDR) images. It is based on existing methods of creating HDR images by fusing a set of differently exposed low dynamic range (LDR) images. We optimize the capturing process of LDR images towards improved capture speed by using partial re-exposures. That is, we make use of the idea that it is not always necessary to capture full size images when only small portions of the scene require HDR. By analyzing captured images for badly exposed regions and re-exposing selectively, we save overall capture time and increase the frame rate when image sequences are recorded.

In this demo proposal we present Planet Π4, a Massively Multiplayer Online Game (MMOG) developed to evaluate and compare peer-to-peer-based MMOG systems with scalability in mind. The game requires low network latency and creates frequent game state updates. It has a modular architecture that can be adapted and extended with new functionality. Using this modular design we have developed different peer-to-peer infrastructures. Workshop participants will be able to play the game and compare the versions with each other.