In recent years, digital endoscopy has established as key technology for medical screenings and minimally invasive surgery. Since then, various research communities with manifold backgrounds have picked up on the idea of processing and automatically analyzing the inherently available video signal that is produced by the endoscopic camera. Proposed works mainly include image processing techniques, pattern recognition, machine learning methods and Computer Vision algorithms. While most contributions deal with real-time assistance at procedure time, the post-procedural processing of recorded videos is still in its infancy. Many post-processing problems are based on typical Multimedia methods like indexing, retrieval, summarization and video interaction, but have only been sparsely addressed so far for this domain. The goals of this survey are (1) to introduce this research field to a broader audience in the Multimedia community to stimulate further research, (2) to describe domain-specific characteristics of endoscopic videos that need to be addressed in a pre-processing step, and (3) to systematically bring together the very diverse research results for the first time to provide a broader overview of related research that is currently not perceived as belonging together.

The rapidly increasing volume of videos recorded in the course of endoscopic screenings and surgeries poses demanding challenges to video retrieval and browsing systems. Surgeons typically have to use standard video players to retrospectively review their procedures, which is an extremely cumbersome and time-consuming process. We present an HTML5-based video explorer that is specially tailored to this purpose and enables a time-efficient post-operative review of procedures. It incorporates various interactive browsing mechanisms as well as domain-specific content-based features based on previous research results. Preliminary interviews with surgeons indicate that this tool can considerably improve retrieval and browsing efficiency for users in the medical domain and allows surgeons to more easily and quickly revisit specific moments in recordings of their endoscopic surgeries.

Common lossy video compression methods have already reached a very high performance level and are about to reach their natural limit. However, their compression efficiency is still insufficient for certain domains where a plethora of video data should be archived. One example is the domain of medical endoscopy where entire surgeries are recorded for documentation with high visual quality because it is not possible to anticipate if and when exceptional situations may happen. Eventually, only a fraction of the video footage is of actual relevance for post-operative review. In this paper, we introduce the concept of domain-specific video compression by the example of this very specific domain. We identify several characteristics of endoscopic surgery videos that can be exploited by content-based analysis algorithms to achieve a significantly higher compression efficiency than in current practice. The extensive evaluation shows that the proposed methods massively reduce the data volume without any semantic information loss and thus allow for long-term video archiving of endoscopic surgery videos. Moreover, we expect to stimulate future research in this challenging new field.

In recent years it has become common practice to archive video recordings of laparoscopic surgeries for documentation purposes and for retrospective review. Typically, the videos are captured in High Definition (HD) format but encoded with legacy coding standards like MPEG-2 requiring an enormous storage capacity. In this paper we present the results of a subjective quality assessment study with 37 medical experts. We identify appropriate encoding configurations for the H.264 AVC coding standard to guarantee a visually loss less quality with a significant bitrate reduction. Further, we show that it is not necessary to capture the highest possible quality for documentation and retrospective analysis. A lower technical quality with a substantially lower bitrate still provides sufficient semantic quality. We finally present basic recommendations for an efficient encoding strategy with an appropriate tradeoff between visual quality and bitrate.

Parallel recordings made at the same event with different devices, e.g. by visitors of a concert, contain semantically the same content but do not run at the same speed when played back in parallel on a computer, which makes their synchronization difficult. This effect, time drift, concerns all current consumer multimedia recording devices and results from their internal clocks not running at the same speed, leading to deviations from their nominal sampling rates. We present a mobile application capable of conducting instant measurements of this time drift, thus helping in determining devices that go well together, or correcting the speed differences in post processing.

No abstract available.

The actual content of endoscopic videos is typically limited to a circular area in the image center. This area has a dynamic position and size and is surrounded by a dark, but noisy border. In this paper we present a novel algorithm that (1) classifies which frames of an endoscopic video feature the circular content area and (2) determines its exact position and size, if present. This information is very useful for improving the performance of subsequent analysis techniques. It can also be used for more efficient video encoding and economic printing of still images in findings and reports. The evaluation shows that the proposed method is very accurate, robust and efficient in terms of runtime.

In recent years, it became common to record video footage of laparoscopic surgeries. This leads to large video archives that are very hard to manage. They often contain a considerable portion of completely irrelevant scenes which waste storage capacity and hamper an efficient retrieval of relevant scenes. In this paper we (1) define three classes of irrelevant segments, (2) propose visual feature extraction methods to obtain irrelevance indicators for each class and (3) present an extensible framework to detect irrelevant segments in laparoscopic videos. The framework includes a training component that learns a prediction model using nonlinear regression with a generalized logistic function and a segment composition algorithm that derives segment boundaries from the fuzzy frame classifications. The experimental results show that our method performs very good both for the classification of individual frames and the detection of segment boundaries in videos and enables considerable storage space savings.

As mobile devices become more and more pervasive, they are increasingly used to record and watch personal and professional videos. Common video browsers on smart phones and tablets, however, fail to provide users an efficient and engaging experience to browse through video content. In this demo paper, we present an early prototype for browsing videos on tablets: browsing video with a 3D filmstrip. A video is split up into equidistant, uniformly sampled time segments. The segments are represented by key-frames shown on the surface of a filmstrip, allowing an immediate overview of great parts of the video. Further, a user can scroll through the filmstrip, start playback for any segment and refine the preview of segments by simple touch gestures.

In this work we propose an early prototype of a video browser for mobile devices with touchscreens. We concentrate on utilizing the thumbs because of the natural posture used with the devices when watching videos in landscape mode. The controls are only displayed when the user touches the screen and automatically rearrange themselves depending on the position of the thumbs. A combination of a radial menu and an extended seeker control with hierarchical browsing and bookmarking features enables the user to navigate quickly through videos.

Dynamic, large social events with concurrent activities, in which participants view, but also provide high-quality content to other visitors, pose one of many challenges to multimedia delivery. Distributing content in such a situation is difficult, and requires strong support from the underlying network, in order to quickly respond to changes in supply and demand. A Flock is a flexible overlay built in a self-organized, decentralized manner, relying on a simple interest-property concept. In this paper, we use Flocks to build large overlays that are both content- and Quality-of-Service (QoS) aware, and show how they adapt their structure dynamically, depending on interests in available QoS. We evaluate their performance under churn and large scale failures through extensive simulations, and practical experiments under Internet conditions on the Planet Lab, and present a selection of our results.

This paper presents a novel method for video-based traffic state detection on motorways performed on smart cameras. Camera calibration parameters are obtained from the known length of lane markings. Mean traffic speed is estimated from Kanade-Lucas-Tomasi (KLT) optical flow method using a robust outlier detection. Traffic density is estimated using a robust statistical counting method. Our method has been implemented on an embedded smart camera and evaluated under different road and illumination conditions. It achieves a detection rate of more than 95% for stationary traffic.

In VoIP applications, packet loss, delay and delay jitter are inevitable and have a large impact on the perceived speech quality. Jitter buffers are commonly deployed to compensate for jitter in order to play out the received packets continuously. For mobile devices, due to limited battery power, computational complexity has to be kept to a minimum. In this paper, we propose a jitter buffer management which takes complexity into consideration. The algorithm adaptively adjusts the play-out delay under certain complexity constraints. Simulation results show that our proposed algorithm can keep complexity under specified constraints while still optimizing jitter induced packet losses and average delay.

Interactive image and video search tools typically use a grid-like arrangement of thumbnails for preview purpose. Such a display, which is commonly known as storyboard, provides limited flexibility at interactive search and it does not optimally exploit the available screen estate. In this paper we design and evaluate alternatives to the common two-dimensional storyboard. We take advantage of 3D graphics in order to present image thumbnails in cylindrical arrangements. Through a user study we evaluate the performance of these interfaces in terms of visual search time and subjective performance.

This paper presents a new method for estimating the level of service (LOS) on motorways in the compressed video domain. The method performs statistical computations on motion vectors of MPEG4 encoded video streams within a predefined region of interest to determine a set of four motion features describing the speed and density of the traffic stream. These features are fed into a Gaussian radial basis function network to classify the corresponding LOS. To improve the classification results, vectors of moving objects are clustered and outliers are eliminated. The proposed method is designed to be executed on a server system, where a large number of camera live streams can be analyzed in parallel in real-time. Evaluations with a comprehensive set of real-world training and test data from an Austrian motorway have shown an average accuracy of 86.7% on the test data set for classifying all four LOS levels. With a mean execution time of 48 microseconds per frame on a common server, hundreds of video streams can be analyzed in real-time.

Manual synchronization of audio and video recordings is a very annoying and time consuming task, especially if the tracks are very long and/or of large quantity. If the tracks aren't just short clips (of a few seconds or minutes) and recorded from heterogeneous sources, an additional problem comes into play - time drift - which arises if different recording devices aren't synchronized. This demo paper presents the experimental software Audio Align, which aims to simplify the manual synchronization process with the ultimate goal to automate it altogether. It gives a short introduction to the topic, discusses the approach, method, implementation and preliminary results and gives an outlook at possible improvements.

Summary form only given. Automated code analysis is technology aimed at locating, describing and repairing areas of weakness in code. Code weaknesses range from security vulnerabilities, logic errors, concurrency violations, to improper resource usage, violations of architectures or coding guidelines. Common to all code analysis techniques is that they build abstractions of code and then check those abstractions for properties of interest. For instance a type checker computes how types are used, abstract interpreters and symbolic evaluators check how values flow, model checkers analyze how state evolves. Building modern program analysis tools thus requires a multi-pronged approach to find a variety of weaknesses. In this talk I will discuss and compare several program analysis tools, which MSR build during the last ten years. They include theorem provers, program verifiers, bug finders, malware scanners, and test case generators. I will describe the need for their development, their innovation, and application. Many of these tools had considerable impact on Microsoft's development practices, as well as on the research community. Some of them are being shipped in products such as the Static Driver Verifier or as part of Visual Studio. Performing program analysis as part of quality assurance is meanwhile standard practice in many software development companies. However several challenges have not yet been resolved. Thus, I will conclude with a set of open challenges in program analysis which hopefully triggers new aspiring directions in our joint quest of delivering predictable software that is free from defect and vulnerabilities.

We demonstrate an interactive 3D storyboard that take advantage of 3D graphics in order to overcome certain limitations of conventional 2D storyboards when used for the task of image and video browsing.

This paper introduces a scalable approach for efficient, low-cost multi-level Quality of Service (QoS) estimation in large overlay networks (ON). We modify an existing distributed partitioning algorithm, and use it to create "QoS maps". QoS maps empower applications to quickly predict several QoS metrics for any given route, and to obtain multiple alternative routes to any target node in the ON. We show that our modifications of the partitioning algorithm permit the aggregation of large hubs, but still preserve the sub-linear runtime of the original heuristic. Simulations with large ONs are performed to evaluate the proposed approach and demonstrate its scalability. Finally, we outline our estimation algorithm that we use to predict QoS and perform QoS aware routing in any given ON.

We propose a video browsing tool supporting new efficient navigation means and content-based search within a single video, allowing for interactive exploration and playback of video content. The user interface provides flexible navigation indices by visualizing low-level features and frame surrogates along one or more timelines, called interactive navigation summaries. By applying simple and fast content analysis, navigation summary computation becomes feasible during browsing, enabling addition, removal, and update of navigation summaries at runtime. Semantically similar video segments will be visualized by similar patterns in certain navigation summaries, which enables users to quickly recognize and navigate to potential similar segments. Moreover, free-shape regions of video frames or video segments within navigation summaries can be selected by the user to launch a fast content-based search to find frames with similar regions or segments with similar navigation summaries.

We present a video browsing tool that uses a novel and powerful visualization technique of video motion. The tool provides an interactive navigation index that allows users to quickly and easily recognize content semantics like scenes with fast/slow motion (in general or according to a specific direction), scenes showing still/moving objects in front of a still/moving background, camera pans, or camera zooms. Moreover, the visualization facilitates identification of similar segments in a video. A first user study has shown encouraging results.

We present a new approach for video browsing using visualization of motion direction and motion intensity statistics by color and brightness variations. Statistics are collected from motion vectors of H.264/AVC encoded video streams, so full video decoding is not required. By interpreting visualized motion patterns of video segments, users are able to quickly identify scenes similar to a prototype scene or identify potential scenes of interest. We give some examples of motion patterns with different semantic value, including camera zooms, hill jumps of ski-jumpers, and the repeated appearance of a news speaker. In a user study we show that certain scenes of interest can be found significantly faster using our video browsing tool than using a video player with VCR-like controls.

We present a video content browsing tool which is supposed to be used for two purposes: (1) efficiently searching a certain scene within a long video sequence and (2) quickly identifying videos out of interest. Instead of a complex user interface with many features we rather use a simple but flexible player-like interface which can, however, support the user on taking advantage of his/her knowledge about the content semantics. The content analysis is based on some simple features which can be extracted from compressed data of H264/AVC and, thus, allow very fast analysis. An early user study has shown encouraging results.

Streaming video data over best effort networks is a challenging task concerning the quality of the received content. The quality decreases with the number of frames that are corrupted, lost or received after the playback time. The main reasons for lost, delayed or corrupted frames are overloaded streaming servers and crowded network paths.

We investigate the dynamics of a distributed video proxy-cache system that is able to adapt the number of running nodes depending on conditions like client request patterns, network load etc. Since we've already examined the split operation used to expand the system by adding new nodes, we explore in detail two operations (hibernate and shut down) used to reduce the number of active nodes. Also, several scenarios for object (video data) movement and replication between participants are taken into consideration. Further more, we study the system's efficiency by measuring the byte hit rate under different test scenarios

Universal plug and play (UPnP) is a widely accepted standard for automatically detecting devices and services in a local area network as well as for describing and controlling them. In order to deal with multimedia devices and especially content, in 2002 the UPnP-AV standard definition was released. It defines device and service descriptions for media servers and renderers. Thereby, the media server's content directory service allows an easy management and the exchange of metadata about the provided media data. Media content became browsable by semantic meta information about it. There are still two major drawbacks of UPnP-AV, which make its usage in real world multimedia communication scenarios very difficult. First, searching for similar content on distributed media servers with a huge number of media files is not economically possible. Second, the media content must be consumed by Renderers as provided by the Servers, independently of their terminal capabilities and network connections. In order to deal with these two drawbacks, this work proposes a novel approach of metadata integration and media transcoding in UPnP networks. First, the Media Server is extended by a control point which offers discovery of other media servers and fetches metadata from their content directories. Furthermore, it integrates the gathered information in its own content directory. Control Points are then able to query this Integrating Media Server for a desired content, and get a network-complete search result. Second, terminal and network capabilities of the Renderers are taken into account in order to transcode and transmit the content in a suitable way for the consuming device. These two approaches of metadata integration and media data adaptation enable searchable logical views on tailored multimedia content in UPnP-AV networks.

Due to the increasing availability and use of digital video data on the Web, video caching will be an important performance factor in the future WWW. We propose an architecture of a video proxy cache that integrates modern multimedia and communication stan-dards. Especially we describe features of the MPEG-4 and MPEG-7 multimedia standards that can be helpful for a video proxy cache. QBIX1 supports real-time adaptation in the compressed and in the decompressed domain. It uses adaptation to improve the cache replacement strategies in the proxy, but also to realize media gate-way functionality driven by the clients ’ terminal capabilities.

Video caching will be an important performance factor in future networked multimedia systems. A major component of caches is the replacement strategy. This paper presents replacement strategies for video caches that incorporate quality reduction and the use of metadata given by the content provider. The strategies are evaluated by simulation.

No abstract available.