This paper presents Movie Stream, an exhibition-ready framework that combines natural-language video retrieval with real-time generative AI reinterpretation. The demo indexes an audiovisual archive using CLIP-based multimodal embeddings and enables users to retrieve precise video segments via text prompts. Retrieved sequences are then re-rendered live through a diffusion-based style-transfer pipeline (StreamDiffusion) and presented through a tactile web interface integrated with TouchDesigner. Implemented and evaluated as an installation in museum contexts, Movie Stream makes the mediation of GenAI models tangible: users can explore how genres and directorial styles are operationalized and discuss related questions of authorship and copyright in the film domain. The modular architecture—offline indexing, online search, playback control, and UI orchestration—supports robust standalone deployment and provides a basis for future extensions such as relevance feedback, multi-result retrieval, and expanded generative controls.

The performative art installation is a rose uses plants as natural interfaces in order to establish a direct relation between natural and technological systems. The installation is used to visualize digital-physical interaction that is not necessarily explicit – triggered by touch or air movement, by direct and non-direct manipulation, depicting the sum of all interactions during the course of the day. The technical realization consists of detection of the movement of plants caused by the movements in their immediate vicinity and the subsequent deformation of a computer-generated sphere. The paper is explaining several different layers of meanings the artist was motivated by when developing the artwork.

The segmentation of point clouds is conducted with the help of deep reinforcement learning (DRL) in this contribution. We want to create interactive virtual reality (VR) environments from point cloud scans as fast as possible. These VR environments are used for secure and immersive trainings of serious real life applications such as the extinguishing of a fire. It is necessary to segment the point cloud scans to create interactions in the VR. Existing geometric and semantic point cloud segmentation approaches are not powerful enough to automatically segment point cloud scenes that consist of diverse unknown objects. Hence, we tackle this problem by considering point cloud segmentation as markov decision process and applying DRL. More specifically, a deep neural network (DNN) sees a point cloud as state, estimates the parameters of a region growing algorithm and earns a reward value. The point cloud scenes originate from virtual mesh scenes that were transformed to point clouds. Thus, a point to segment relationship exists that is used in the reward function. Moreover, the reward function is developed for our case where the true segments do not correspond to the assigned segments. This case results from, but is not limited to, the usage of the region growing algorithm. Several experiments with different point cloud DNN architectures such as PointNet [13] are conducted. We show promising results for the future directions of the segmentation of point clouds with DRL.

Photo Sprayer is a virtual painting tool for creation of digital art in virtual reality (VR) by using images of the real-world surrounding as its painting palette. The users can capture images of local sights, objects, faces or random details and turn them into colors and patterns for their digital artwork on 2D surfaces and 3D objects in VR. The application is conceptualized as a playful approach to the predominance of contemporary digital imagery, symbolically empowering the user to emancipate from the image dominance. It was developed for simple use by anyone interested in digital art, enabling a wide array of people to assume the role of an artist. The projection of the real-time painting process allows the observers to continuously follow the progress, while they become part of the artwork though their presence captured in the images.

The performance and security in outdoor climbing sports can depend on anxieties. These appear in frightening situations where, e.g. a deep fall is risked. Deep falls can lead to serious injuries or even to fatal accidents. Such situations can be trained in order to be mentally resistant to them and thereby to make climbing safer. However, drawbacks have to be taken into account. The trainee has to bring himself in a possible hazardous situation and nature is not directly reachable for every person. Thus, we present a system, where a user can climb at low heights in reality and simultaneously on a high cliff in VR. In this contribution, we describe the system architecture and future possibilities to safely train stressful outdoor climbing situations indoors.

Given the ongoing demographic changes in the world, there is not only a growing senior population, but also more people with dementia. Unfortunately, pharmaceutical interventions are limited. However, psychosocial interventions, e.g. reminiscence, are promising in improving the mood of people with dementia as well as the communication between them and their caregivers. At the same time, new possibilities to support reminiscence and communication of people with dementia are arising with advances in multimedia, sensor technologies and tangible computing. To date, the use of technology is not widespread in care practice and more research is needed to advance this. This research examines how tangible multimedia can be of added value in supporting reminiscence and communication for people with dementia and their caregivers and how to open up a dialog with caregivers about technical probes. Qualitative research using six experiential probes was conducted to explore different aspects of tangible interfaces and multimedia. The research participants were care professionals, IT experts and people with dementia. We conducted a number of interviews, field studies, focus groups and an expert workshop. We used qualitative content analysis and thematic analysis to identify themes from the data for each probe, which were consolidated into four design considerations for technologies supporting reminiscence of people with dementia. In addition, we reflect on the probe itself as a way to engage stakeholders in the design of new care technologies.

Our perception of the environment becomes more limited through, (1) an acceleration of everyday life through technology and, (2) restricted access to information resulting from online filtering. As a result, exploring unknown environments and making surprising discoveries (physical or digital) have become rare. However, it is exactly these experiences that extend our perspectives. In this paper we describe a mobile service to create and experience music channels motivating users to explore their urban surroundings. The SoundtrackOfYourLife app allows a user to place songs in urban spaces, linked to GPS coordinates, in order to create a digital sound channel. Other users can find these channels and follow them to explore the music placed by the first user and possibly walk a new path in the city. We elaborate on the design and implementation of the app as well as results from a first exploratory user study.

This paper summarizes the results of the design and implementation of an audio-visual installation, which focuses on the interaction of two casual players with complementary designed musical sphere interfaces. While one user interacts with a small dome touchscreen and controls the harmonic sound space, the other user stands inside a larger dome creating melodies by continuous gestures in mid-air. Similar to a theremin notes are played without touch using an invisible two-dimensional melodic table of notes with a dedicated layout that allows creating harmonic expressions by continuous gestures. The generation of musical expressions is visualized on both instruments with several visualization techniques. For prototyping purposes and to illustrate the idea of the installation without the need to set up the complex installation we also provide an immersion user experience by simulating the system using an Oculus Rift head-mounted display. The goal of this project is to provide and explore new methods of musical interaction and to exploit the complementary character of the installation to create a harmonic sound environment based on cooperative play.

We present a system that combines a real piano with computer generated algorithmic musical expressions, which are directly controlled by a conductor's gestures and trigger a real-time visualization.

This paper presents an interactive media installation that aims at providing users with the experience to sing like an opera singer from the 19th century. We designed a set of tangible and body-related interaction and feedback techniques and developed a singing voice synthesizer system that is controlled by the user's mouth shapes and gestures. This musical interface allows users to perform an aria without real singing. We adapted techniques from 3D body tracking, face recognition, singing voice synthesis, 3D rendering and tangible interaction to integrate them into an interactive musical interface.

In this work we present "Datenreise" (data journey), a media installation that consists of an extendible set of "data transmission" components that transform digital bits to tangible bits and vice versa using different modalities.

In this paper we describe the design of a virtual reality simulator for traditional intuitive archery. Traditional archers aim without a target figure. Good shooting results require an excellent body-eye coordination that allows the user to perform identical movements when drawing the bow. Our simulator provides a virtual archery experience and supports the user to learn and to practice the motion sequence of traditional archery in a virtual environment. We use a low-cost tracking system to capture the user's motion in order to correct his movement. To provide a realistic haptic feedback a real bow is used as interaction device. Our system should provide a believable user experience and support the user to learn how to shoot in the traditional way. Following a user-centered iterative design approach we developed a number of prototypes and evaluated them for refinement in sequent iteration cycles.

Mixed reality techniques have high potential to support the development of complex systems that operate in a real world environment, especially mechatronic systems. In our paper we present the Mixed Reality in the Loop design process that enables a seamless progression from an initial virtual prototype to the final system along the mixed reality continuum.

We describe the implementation of a playful multi-user virtual riding lesson for beginners. Our work focused on the integration of a wooden horse, equipped with real snaffle and saddle, which is used as input interface to navigate in a 3D environment. With the help of the wooden horse, the user is able to interact with a virtual 3D horse via reins and legs, similar to a real riding scenario.

Actors in virtual studio productions are faced with the challenge that they have to interact with invisible virtual objects because these elements are rendered separately and combined with the real image later in the production process. Virtual sets typically use static virtual elements or animated objects with predefined behavior so that actors can practice their performance and errors can be corrected in the post production. With the demand for inexpensive live recording and interactive TV productions, virtual objects will be dynamically rendered at arbitrary positions that cannot be predicted by the actor. Perceptive aids have to be employed to support a natural interaction with these objects.In our work we study the effect of haptic feedback for a simple form of interaction. Actors are equipped with a custom built haptic belt and get vibro-tactile feedback during a small navigational task (path following). We present a prototype of a wireless vibro-tactile feedback device and a small framework for evaluating haptic feedback in a virtual set environment. Results from an initial pilot study indicate that vibro-tactile feedback is a suitable non-visual aid for interaction that is at least comparable to audio-visual alternatives used in virtual set productions.

This paper describes a pragmatic approach for the design of hybrid user interfaces based on a number of extensions of an existing 3D authoring system. We present the design and realization of a visual framework dedicated to the prototyping of hybrid user interfaces. The rapid development environment was applied in a teaching context during lectures on advanced user interface design. The results showed that our framework provides a suitable tool to quickly design and test hybrid user interfaces.

A 3D information space is a three-dimensional visualization that contains 2D visualizations and puts them in a semantic context. In this paper, we sketch how mixed reality (MR) can be exploited as a technology in order to realize better interaction in a 3D information space and, as a result, to develop new interactive visualization techniques. Here, the real space is used as a metaphor for interacting with the 3D information space.

We describe our approach of developing AR applications with expressive visual representations and realistic object behavior. We combine concepts from high-level scene graphs, physical simulation libraries and a Java binding for ARToolkit with the ideas of iterative prototyping and show that rapid development of expressive AR applications using Java is possible.

This paper applies the idea of a continuously testable design representation to authoring of augmented realities for mobile devices.

We describe a framework that allows to easily design and implement virtual and augmented reality worlds. Based on a structured design approach for interactive animated 3D conten we want to supply designers and content experts of complex virtual environments with a component based to olset for the structured design of the visual and abstract components of 3D applications.