The ACM Lifelog Search Challenge (LSC) is an annual comparative benchmarking exercise that brings together researchers in the field of multimedia retrieval to evaluate interactive search systems using a large-scale multimodal lifelog dataset. This paper presents an overview of the ninth edition of the challenge (LSC’26), held as a workshop during the ACM International Conference on Multimedia Retrieval (ICMR ’26) in Amsterdam. To broaden its scope, the workshop now features three submission tracks: the traditional Challenge Track for real-time search performance, a new General Lifelog Research Track for theoretical and architectural advancements, and an additional Open Source Track aimed at enhancing reproducibility and reducing barriers to entry for new participants.

We present a novel 360° video player with dynamic thumbnails to address the challenge of navigating 360° content on 2D displays. A user study compared this player to a traditional 360° player, focusing on the reduction of fear of missing out (FOMO) and timeline navigation efficiency without compromising usability. Results showed dynamic thumbnails significantly reduced FOMO, as users reported less anxiety and performed fewer camera rotations. Task completion times and gaze patterns were comparable across both players. The proposed player received a System Usability Scale (SUS) score of 84.7, categorized as “Good.” While users appreciated the improved situational awareness, minor usability issues were identified. These results highlight the potential of dynamic thumbnails for enhancing 360° video navigation, warranting further optimization and research.

For the sixth time since 2018, the Lifelog Search Challenge (LSC) was organized as a comparative benchmarking exercise for various interactive lifelog search systems. The goal of this international competition is to test system capabilities to access large multimodal lifelogs. LSC’23 attracted twelve participanting teams, each of whom had developed a competitive interactive lifelog retrieval system. The benchmark was organized in front of live audience at the LSC workshop at ACM ICMR’23. As in previous editions, this introductory paper presents the LSC workshop and introduces the participating lifelog search systems.

Different brain diseases are associated with distinct patterns of affected brain regions. Neuroscientists aim to understand the variability of these patterns across different brain diseases. We used a user-centered design approach to develop support for neuroscientists in exploring and comparing which regions are affected by which diseases based on neuroscience literature. Our study involved six neuroscientists and nine visualization experts who evaluated the usability and explainability of our visualization tool for comparing disease patterns in brain regions. Our results show that our tool can help neuroscientists explore and visualize relationships between brain regions and diseases, and compare patterns of affected regions in an immersive AR environment.

360° videos provide an immersive experience, especially when watched in virtual reality (VR). Yet, even though the video data is inherently three-dimensional, interfaces to access datasets of such videos in VR almost always use two-dimensional thumbnails shown in a grid on a flat or curved plane. We claim that using spherical and cube-shaped 3D thumbnails may provide a better user experience and be more effective at conveying the high-level subject matter of a video or when searching for a specific item in it. A comparative study against the most used existing representation, that is, 2D equirectangular projections, showed that the spherical 3D thumbnails did indeed provide the best user experience, whereas traditional 2D equirectangular projections still performed better for high-level classification tasks. Yet, they were outperformed by spherical thumbnails when participants had to search for details within the videos. Our results thus confirm a potential benefit of 3D thumbnail representations for 360-degree videos in VR, especially with respect to user experience and detailed content search and suggest a mixed interface design providing both options to the users. Supplemental materials about the user study and used data are available at https://osf.io/5vk49/.

For the fifth time since 2018, the Lifelog Search Challenge (LSC) facilitated a benchmarking exercise to compare interactive search systems designed for multimodal lifelogs. LSC'22 attracted nine participating research groups who developed interactive lifelog retrieval systems enabling fast and effective access to lifelogs. The systems competed in front of a hybrid audience at the LSC workshop at ACM ICMR'22. This paper presents an introduction to the LSC workshop, the new (larger) dataset used in the competition, and introduces the participating lifelog search systems.

To support scientists in maintaining an overview of disciplinary concepts and their interrelations, we investigate whether Augmented Reality can serve as a platform to make automated methods more accessible and integrated into current literature exploration practices. Building on insights from text and immersive analytics, we identify information and design requirements. We embody these in DatAR, a system design and implementation focussed on analysis of co-occurrences in neuroscientific text collections. We conducted a scenario-based video survey with a sample of neuroscientists and other domain experts, focusing on participants’ willingness to adopt such an AR system in their regular literature review practices. The AR-tailored epistemic and representational designs of our system were generally perceived as suitable for performing complex analytics. We also discuss several fundamental issues with our chosen 3D visualisations, making steps towards understanding in which ways AR is a suitable medium for high-level conceptual literature exploration.

The Lifelog Search Challenge (LSC) is an annual benchmarking challenge for comparing approaches to interactive retrieval from multi-modal lifelogs. LSC'21, the fourth challenge, attracted sixteen participants, each of which had developed interactive retrieval systems for large multimodal lifelogs. These interactive retrieval systems participated in a comparative evaluation in front of an online live-audience at the LSC workshop at ACM ICMR'21. This overview presents the motivation for LSC'21, the lifelog dataset used in the competition, and the participating systems.

The Lifelog Search Challenge (LSC) is an annual comparative benchmarking activity for comparing approaches to interactive retrieval from multi-modal lifelogs. LSC'20, the third such challenge, attracts fourteen participants with their interactive lifelog retrieval systems. These systems are comparatively evaluated in front of a live-audience at the LSC workshop at ACM ICMR'20 in Dublin, Ireland. This overview motivates the challenge, presents the dataset and system configuration used in the challenge, and briefly presents the participating teams.

Presence and the sense of embodiment are essential concepts for the experience of our self and virtual bodies, but there is little quantitative evidence for a relation between these, and this relation becomes more complicated when there are real and virtual bodies in augmented reality (AR). We investigate the experience of body ownership, agency, self-location and self-presence in AR where users can see their real body and a virtual body from behind. Active arm movement congruency and virtual anthropomorphism are varied. We found significant effects of movement congruency but not anthropomorphism, a strong correlation between self-presence and body ownership, and a moderate correlation between self-presence and agency and self-location.

The classic rubber hand illusion (RHI) experiment studies the sense of embodiment over a fake limb. Distinguished subcomponents of embodiment are ownership (sense of self-attribution of a body), agency (sense of having motor control), and self-location (the spatial experience of being inside a body), and are typically evoked in either reality or virtual reality. In augmented reality (AR), however, visually present real limbs can be augmented with (multiple) fake virtual limbs, which results in a variation of the RHI, the augmented reality supernumerary hand illusion (ARSHI). Such conditions occur, for example, in first-person AR games and in AR-interfaces for tele-robotics. In this article, we examined to what extent humans can experience the sense of embodiment over a supernumerary virtual arm in addition to one or two real arms. We also examine how embodiment is affected by the perceptual visual-tactile synchronicity of the virtual and real limbs, and by the synchronicity of active movement of the virtual and real hand. Embodiment was measured subjectively by questionnaire and objectively by skin conductance responses (SCRs). Questionnaire responses show that ownership, agency, and self-location can be evoked over the virtual arm in the presence of a real arm, and that they are significantly stronger for synchronous conditions than for asynchronous conditions. The perceptual and motorical synchronous condition with three visible hands led to an experience of owning the virtual hand. These responses further show that agency was also strongly experienced over the supernumerary virtual arm, and responses regarding self-location suggest a shift in sensed location when one real arm was in view and an additional location when both real arms where in view. SCRs show no significant effect of condition, but do show a significant habituation effect as a function of the number of conditions performed by participants. When analyzing the relations at the individual participant level between the questionnaire data and skin conductance, we found two clusters of participants: (1) participants with low questionnaire responses and low-medium SCRs and (2) participants with high questionnaire responses and low-high SCRs. Finally, we discuss how virtual hand appearance/realism and willingness to accept virtual limbs could play an important role in the ARSHI, and provide insights on intricacies involved with measuring and evaluating RHIs.

Loitering is a suspicious behavior that often leads to criminal actions, such as pickpocketing and illegal entry. Tracking methods can determine suspicious behavior based on trajectory, but require continuous appearance and are difficult to scale up to multi-camera systems. Using the duration of appearance of features works on multiple cameras, but does not consider major aspects of loitering behavior, such as repeated appearance and trajectory of candidates. We introduce an entropy model that maps the location of a person's features on a heatmap. It can be used as an abstraction of trajectory tracking across multiple surveillance cameras. We evaluate our method over several datasets and compare it to other loitering detection methods. The results show that our approach has similar results to state of the art, but can provide additional interesting candidates.

It is very difficult to fully automate the detection of loitering behavior in video surveillance, therefore humans are often required for monitoring. Alternatively, we could provide a list of potential loiterer candidates for a final yes/no judgment of a human operator. Our system, VisLoiter+, realizes this idea with a unique, user-friendly interface and by employing an entropy model for improved loitering analysis. Rather than using only frequency of appearance, we expand the loiter analysis with new methods measuring the amount of person movements across multiple camera views. The interface gives an overview of loiterer candidates to show their behavior at a glance, complemented by a lightweight video playback for further details about why a candidate was selected. We demonstrate that our system outperforms state-of-the-art solutions using real-life data sets.

Drawing is an important part of elementary school education, especially since it contributes to the development of spatial skills. Virtual reality enables us to draw not just on a flat 2D surface, but in 3D space. Our research aims at showing if and how this form of 3D drawing can be beneficial for art education. This paper presents first insights into potential benefits and obstacles when introducing 3D drawing at elementary schools. In an experiment with 18 children, we studied practical aspects, proficiency, and spatial ability development. Our results show improvement in the children's 3D drawing skills but not in their spatial abilities. Their drawing skills also do seem to be correlated with their mental rotation ability, although further research is needed to conclusively confirm this.

Interactive video retrieval tools developed over the past few years are emerging as powerful alternatives to automatic retrieval approaches by giving the user more control as well as more responsibilities. Current research tries to identify the best combinations of image, audio and text features that combined with innovative UI design maximize the tools performance. We present the last installment of the Video Browser Showdown 2015 which was held in conjunction with the International Conference on MultiMedia Modeling 2015 (MMM 2015) and has the stated aim of pushing for a better integration of the user into the search process. The setup of the competition including the used dataset and the presented tasks as well as the participating tools will be introduced . The performance of those tools will be thoroughly presented and analyzed. Interesting highlights will be marked and some predictions regarding the research focus within the field for the near future will be made.

Virtual and augmented reality offer various ways to enhance art exhibitions and create virtual museum experiences. In this work, we investigate the impact of digital additions to paintings in a VR museum setting. In particular, we evaluate if adding visuals in the same style of a painting or small 3D animations related to its content can have a positive impact on the experience and on how the art is perceived. Using Parsons' theory of how we understand art, our results demonstrate an effect on the experience and thus confirm the potential of such digital additions for VR museums.

The digitization of paintings enables us to do various modifications with them and also to add digital content that complements such artworks. In our research we investigate how such digital extensions can be used in a VR museum setup. This Creative Showcase demo presents a first implementation of such an "enhanced VR museum" based on initial results and related user feedback from earlier experiments. By using techniques such as "inpainting", "stylization" of objects or other visuals in the style of a painting, and by adding of content-related 3D animations we aim at creating VR museum techniques that provide a better experience by, for example, being more immersive, creating a higher emotional connection, or having an educational value.

Virtual reality (VR) and augmented reality (AR) are expected by many to become the next wave of computing with significant impacts on our daily lives. Motivated by this, we organized a workshop on “Multimodal Virtual and Augmented Reality (MVAR)” at the 18th ACM International Conference on Multimodal Interaction (ICMI 2016). While current VR and AR installations mostly focus on the visual domain, we expect multimodality to play a crucial role in future, next generation VR/AR systems. The submissions for this workshop reflect the potential of multimodality for VR and AR, illustrate interesting new directions, and pinpoint important issues. This paper gives a short motivation for the workshop and its aim, and summarizes the aforementioned trends and challenges identified from the submissions.

Mobile or handheld augmented reality uses a smartphone's live video stream and enriches it with superimposed graphics. In such scenarios, tracking one's fingers in front of the camera and interpreting these traces as gestures offers interesting perspectives for interaction. Yet, the lack of haptic feedback provides challenges that need to be overcome. We present a pilot study where three types of feedback (audio, visual, haptic) and combinations thereof are used to support basic finger-based gestures (grab, release). A comparative study with 26 subjects shows an advantage in providing combined, multimodal feedback. In addition, it suggests high potential of haptic feedback via phone vibration, which is surprising given the fact that it is held with the other, non-interacting hand.

Augmented reality applications using stereo head-mounted displays are not capable of perfectly blending real and virtual objects. For example, depth in the real world is perceived through cues such as accommodation and vergence. However, in stereo head-mounted displays these cues are disconnected since the virtual is generally projected at a static distance, while vergence changes with depth. This conflict can result in biased depth estimation of virtual objects in a real environment. In this research, we examined whether redundant tactile feedback can reduce the bias in perceived depth in a reaching task. In particular, our experiments proved that a tactile mapping of distance to vibration intensity or vibration position on the skin can be used to determine a virtual object's depth. Depth estimation when using only tactile feedback was more accurate than when using only visual feedback, and when using visuotactile feedback it was more precise and occurred faster than when using unimodal feedback. Our work demonstrates the value of multimodal feedback in augmented reality applications that require correct depth perception, and provides insights on various possible visuotactile implementations.

We present a new approach for collaborative video search and video browsing relying on a combination of traditional, index-based video retrieval complemented with large-scale human-based visual inspection. In particular, a traditional PC interface is used for query-based search using advanced indexing and querying methods (e.g., concept search), whereas a visualization of the database on a tablet is used for pure human-based browsing. Both parts are coupled to compensate for mutual disadvantages; human visual inspection allows for a better, more detailed analysis of the data - also bridging the semantic gap - but comes at the price of an unfiltered database - a disadvantage that is resolved by using the results from the query engine to change visualization order.

This demo illustrates several use cases for utilizing vibrotactile feedback, i.e., tactile sensations created by small, body-worn vibration motors, in an augmented reality setting with head-mounted displays. In particular, we address how such feedback can be used to create different experiences and also how it can be applied to provide additional information to a user that might be beneficial when performing certain tasks, such as interactions requiring a high accuracy that is hard to achieve with pure visual feedback (e.g., correct placement of virtual objects at a specific location in the real world).

Haptic feedback, such as the sensation of 'being touched', is an essential part of how we experience our environment. Yet, it is often disregarded in current virtual reality (VR) systems. In addition to the technical challenge of creating such tactile experiences there are also human aspects that are not fully understood, especially with respect to how humans integrate multimodal stimuli. In this research, we proved that the visual stimuli in a VR setting can influence how vibrotactile stimuli are perceived. In particular, we identified how visual cues that are associated with the characteristic of weight influence tactile perception, whereas a similar effect could not be achieved for a temperature-related visual cue. Our results have technical implications - for example, suggesting that a rather simple vibration motor may be sufficient to create a complex tactile experience such as perceiving weight - and relevance for practical implementations - for example, indicating that vibration intensities need to be 'exaggerated' to achieve certain effects.

The Video Browser Showdown (VBS) is an annual event where researchers evaluate their video search systems in a competitive setting. Searching in videos is often a two-step process: first some sort of pre-filtering is done, where, for example, users query an indexed archive of files, followed by a human-based browsing, where users skim the returned result set in search for the relevant file or portion of it. The VBS aims at this whole search process, focusing in particular on its interactive aspects. Encouraged by previous years' results, we created a system that purely addresses the latter issue, i.e., interface and interaction design. By eliminating all kind of video indexing and query processing, we were aiming to demonstrate the importance of good interface design for video search and that its relevance is often underestimated by today's systems. This claim is clearly proven by the results our system achieved in the VBS 2015 competition, where our approach was on a par with the top performing ones. In this paper, we will describe our system along with related design decisions, present our results from the VBS event, and discuss them in further detail.

Unlike desktop screens, mobile devices can be moved around freely. This allows us to create different experiences when exploring 3D data and virtual environments on such handhelds. Yet, this additional degree of freedom not only introduces exciting new possibilities, but also potential issues when used in actual applications. We present a demo environment that enables users to explore different kinds of 3D visualizations on smartphones and tablets, experiment with various characteristics and implementation options, and experience the advantages and disadvantages of these different realizations.

In this work, we evaluate the feasibility of tracking-based interaction using a mobile phone's or tablet's camera in order to create and edit 3D objects in augmented reality applications. We present a feasibility study investigating if and how gestures made with your finger can be used to create such objects. A revised interface design is evaluated in a user study with 24 subjects that reveals a high usability and entertainment value, but also identifies issues such as ergonomic discomfort and imprecise input for complex tasks. Hence, our results suggest huge potential for this type of interaction in the entertainment, edutainment, and leisure domain, but limited usefulness for serious applications.

The goal of this research is to explore new interaction metaphors for augmented reality on mobile phones, i.e. applications where users look at the live image of the device’s video camera and 3D virtual objects enrich the scene that they see. Common interaction concepts for such applications are often limited to pure 2D pointing and clicking on the device’s touch screen. Such an interaction with virtual objects is not only restrictive but also difficult, for example, due to the small form factor. In this article, we investigate the potential of finger tracking for gesture-based interaction. We present two experiments evaluating canonical operations such as translation, rotation, and scaling of virtual objects with respect to performance (time and accuracy) and engagement (subjective user feedback). Our results indicate a high entertainment value, but low accuracy if objects are manipulated in midair, suggesting great possibilities for leisure applications but limited usage for serious tasks.

In this paper we present our findings from an experiment designed to test the effectiveness of complex, thumbnail based layouts and the interaction methods required, in video retrieval scenarios for handheld devices such as smartphones. Our evaluation explores the relationship between the number of thumbnails (Quantity) visible on screen and their discernible detail (Quality) with regards to the related necessary amount of interaction. The results indicate that such layouts achieve a very high rate of accuracy and speed but that the nature of the interaction is of critical importance for the success of the system.

This paper presents an interactive thumbnail-based video browser for mobile devices such as smartphones featuring a touch screen. Developed as part of on-going research and supported by user studies, it introduces HiStory, a hierarchical storyboard design offering an interface metaphor that is familiar and intuitive yet supports fast and effective completion of Known-Item-Search tasks by rapidly providing an overview of a video's content with varying degrees of granularity.

Motivated by the increasing popularity of video on handheld devices and the resulting importance for effective video retrieval, this paper revisits the relevance of thumbnails in a mobile video retrieval setting. Our study indicates that users are quite able to handle and assess small thumbnails on a mobile's screen - especially with moving images - suggesting promising avenues for future research in design of mobile video retrieval interfaces.

Virtual reality panoramic images are becoming increasingly popular on handheld devices, but navigating them remains a challenge due to small screen sizes. In this paper, we present a formal evaluation and usability studies comparing two interaction concepts. In the first one, the device is seen as a static peephole and the data is moved behind it via touch screen-based scrolling. In the second one, a mobile phone's sensors are used to create a dynamic peephole that can be moved over static content. In the results of our formal analysis sensor-based dynamic peephole navigation performed twice as good in an orientation task, 75% better in an object size discrimination task, and was preferred by 80% of the users. Despite these advantages, additional usability studies indicate that if they are sitting, a majority of users resort to touch screen-based static peephole navigation when interacting. Our results therefore demonstrate benefits of dynamic peephole navigation for virtual reality panoramas but also illustrate its limitations depending on the current context of the user.

Given small screen sizes, navigation in large data on mobile devices often becomes a problem. In this paper, we present different interfaces for pen-based navigation in VR (Virtual Reality) panoramic images on cell phones with touch-sensitive displays. Four different solutions have been implemented and evaluated in an initial user study. Our designs offer users advanced navigation functionality thus increasing usability and making VR panoramas on mobile devices more useful.

In this paper, we describe two interface designs for mobile video browsing on pen-based handheld devices. Both feature different interaction types, such as speed- and position-based navigation which enable users to interactively skim a video's content along the timeline at different granularity levels. The feasibility and usefulness of the proposed designs is demonstrated in a comparative user study.

The field of multimedia is composed of a variety of research areas. This diversity makes multimedia such a special and interesting research field. However, the different vocabularies, methods, and cultures of the involved communities also introduce barriers that make it difficult to teach the field as a unified subject. The panel invites experts to discuss if and how we can teach the diversity in multimedia as a single subject and how we as researchers and educators can help to foster this goal. The following text provides background information to the topic and introduces the organizer's hypotheses to be discussed at the panel.

Browsing video on mobile devices such as smartphones and PDAs requires new interface designs and interaction concepts because of their small screen sizes. In this paper, we present four different interfaces which enable users to skim video at different replay speed levels: An interface supporting flicking similar to text browsing on an iPhone™, an elastic slider, and two variations which also allow for interactive speed manipulation. Based on a heuristic evaluation with all four designs, revised versions of the two most promising ones have been implemented. A comparative user study proved the usefulness of the proposed designs. Both interfaces showed the same performance (measured in time needed to solve typical browsing tasks) but achieved different results in subjective user assessments.

Small screen sizes and limited interaction possibilities appear to be the main challenges when developing video browsing interfaces for mobile devices, such as smartphones and PDAs. In this demo, we present several pen-based designs for interactive video browsing on a PDA. Each implementation enables users to skim a video along the timeline at different speeds and granularity levels. Using the whole touch-sensitive screen as input area, different pen movements are mapped to different browsing behaviors, offering more functionality and creating a better user experience.

We present an interface for interactive video browsing on pen-based handheld devices. Our solution enables users to navigate through a video along the timeline at different granularity levels. In addition, one can skim a file's content by continuous modification of replay speed. Both interaction concepts are smoothly integrated into a single interface that takes full advantage of the limited screen space being available.

In this demonstration, we present a system that enables users to easily generate quizzes for mobile devices. With our program, they can create multiple choice tests and general question-answer pairs by just entering (or copy-pasting) text into an input mask. The text is automatically processed and formatted and corresponding images or text notes are created for representation on an Apple iPod.

Today, videos can be replayed on modern handheld devices, such as multimedia cellphones and personal digital assistants (PDAs), due to significant improvements in their processing power. However, screen size remains a limiting resource making it hard, if not impossible to adapt common approaches for video browsing to such mobile devices. In this paper we propose a new interface for the pen-based navigation of videos on PDAs and multimedia cellphones. Our solution - called the MobileZoomSlider - enables users to intuitively skim a video along the timeline on different granularity levels. In addition, it allows for continuous manipulation of replay speed for browsing purposes. Both interaction concepts are seamlessly integrated into the overall interface, thus taking optimum advantage of the limited screen space. Our claims are verified with a first evaluation which proves the suitability of the overall concept.

Compared to desktop PCs, mobile devices have several constraints such as restricted screen size and reduced interactivity. Hence, it is not clear if it is helpful to provide educational multimedia material for mobile media players. In this paper, we present a study of algorithm animations on mobile devices, i.e. an Apple iPod with video functionality. The results of our experiment indicate that despite limitations there is an actual learning effect and therefore mobile media players offer a potential to increase learning that deserves further investigation.

We present the AV-ZoomSlider interface for video browsing. It complements existing approaches, such as storyboards and video skims by enabling users to interactively navigate along the time line of a video file. Our solution smoothly integrates position- and speed-based navigation concepts and provides synchronized audio-visual feedback during scrolling when applicable.

In this demo, we present new interfaces for interactive navigation in continuous, time-based multimedia files, such as video recordings. In contrast to common techniques for multimedia skimming, our approaches enable users to interactively navigate audio files along the timeline and support time-synchronized browsing in both audio as well as visual data streams.

We present the ZoomSlider, a new interface for skimming and browsing video content in a flexible and interactive way. It circumvents common problems of existing video browsing approaches, such as their lack of scalability to large document sizes and unpleasant because jerky visual feedback. With the ZoomSlider a user can visually skim through video files at different granularity levels thus providing the power and flexibility needed for highly interactive tasks. First usability tests are presented in order to prove the feasibility of the overall concept.

Literature on speech skimming reports on techniques for playing speech backwards in a way that is still intelligible to the user. However, so far there is no empirical evidence for reasonable parameter settings of the respective algorithms and few examinations have been conducted to verify the usefulness of this feature for actual tasks. We present a user study testing different ways of backward skimming in relation to topic classification. Our evaluation shows a high classification performance and suggests implications for the design of the user interface.

In This work we introduce new approaches for visual information seeking in video documents. We argue that a discrete visualization of continuous media streams via static keyframes is not sufficient to fulfill all information needs and therefore additional, more interactive techniques are required. We extend the concept of elastic interfaces, which has been used successfully with discrete documents, by applying it to continuous data streams. In addition, we propose a different integration of the elastic browsing functionality called "elastic panning". A first usability test is presented in order to prove its feasibility and to highlight its main advantages, namely the ability to access any position within a file independent of its length, and the smoother visual feedback during navigation compared to a standard slider interface.

In this demonstration we present three interface designs which enable users to visually browse video data by moving a slider thumb along the timeline. In such a case, scrolling granularity is usually limited because of the fixed length of the corresponding slider. In contrast, our interaction designs enable users to skim the data at different granularity levels by providing the possibility to continuously change the slider's scale, by using a nonlinear scale, and by enabling interactive manipulation of the scrolling speed, respectively.

Today's interfaces for time-scaled audio replay have limitations especially regarding highly interactive tasks such as skimming and searching, which require quick temporary speed changes. Motivated by this shortcoming, we introduce a new interaction technique for speech skimming based on the so called rubber-band metaphor. We propose an "elastic" audio slider which is especially useful for temporary manipulation of replay speed and which integrates seamlessly into standard interface designs. The feasibility of this concept is proven by an initial user study.

This demonstration illustrates different ways to support users dealing with recorded live presentations in order to improve the usability of the corresponding documents. It highlights different problems in this context and presents solutions and alternative approaches for both, multimedia indexing and query processing as well as user interface issues in order to support users who are skimming or browsing such documents in search for information.

The tremendous amount of data resulting from the regular usage of tools for automatic presentation recording demand for elaborate search functionality. A detailed analysis of the according multimedia documents is required to allow search at a very detailed level. Unfortunately, the produced data differs significantly from traditional documents. In this demo, we discuss the problems appearing in the presentation retrieval scenario and introduce aofSE, a search engine to study and illustrate these problems as well as to develop and present according solutions and new approaches for this task.

In presentation recording special effort is usually put into the automation of the production process, that is in automatically creating high quality data files without much or any need for manual recording and post-editing [5]. With the advent of such systems and their usage in classroom teaching, at conferences, etc., there is an increasing need for techniques and abilities which enable users to search in those documents and to localize some specific information. In this paper we describe how we integrated information retrieval techniques into the Authoring on the Fly (AOF) system, an approach for automatic presentation recording. We have chosen the AOF system for two reasons. On the one hand, it is a well-established way for presentation recording, used by various universities and institutions1. On the other hand it is general enough to illustrate typical problems and challenges a developer is facing when designing a system for information retrieval from multimedia data streams which occur in the presentation recording scenario.

In order to improve the acceptance of recorded presentations, we introduce a new open document type covering a wide range of different media classes typically appearing in this scenario. Instances of this document type can be replayed using our time-based synchronization model. Random access in combination with the realized stream/media-layered synchronization mechanism results in essential features such as Random Visible Scrolling and Unrestricted Cross-Referencing. The proposed synchronization model has been implemented and is routinely used by a variety of institutions. Furthermore we show how the properties of the considered documents and the functionality of the proposed synchronization model affect cross-modal retrieval techniques.

We present work on improving the performance of automated speech recognizers by using additional visual information: (lip-/speechreading); achieving error reduction of up to 50%. This paper focuses on different methods of combining the visual and acoustic data to improve the recognition performance. We show this on an extension of an existing state-of-the-art speech recognition system, a modular MS-TDNN. We have developed adaptive combination methods at several levels of the recognition network. Additional information such as estimated signal-to-noise ratio (SNR) is used in some cases. The results of the different combination methods are shown for clean speech and data with artificial noise (white, music, motor). The new combination methods adapt automatically to varying noise conditions making hand-tuned parameters unnecessary.