Plant breeders use field trials across locations and years to identify superior plant varieties with traits such as disease resistance and higher yield. However, comparing breeding candidates across locations and years is challenging and resource demanding. To address this, we developed an integrated system that combines data acquisition through a field robot with an immersive virtual reality (VR) interface for remote assessments. The robot autonomously collects images, spectral data and 3D scans of canola breeding trials. Our VR application, developed through a user-centered approach, offers photo-realistic 3D visualizations, enabling breeders to compare candidates across locations and growth stages—capabilities unavailable in field assessments. In a user study, five breeders conducted visual trait scoring in VR to evaluate how well the system supported typical field trial tasks. The results demonstrated consistent scoring patterns among raters. Feedback from breeders indicated that the ability to compare candidates across locations and growth stages enhanced their decision making in trait assessment. This work highlights the potential of combining robotics and VR to transform data-intensive processes in agriculture.

The field of Sports Human-Computer Interaction (SportsHCI) investigates interaction design to support a physically active human being. Despite growing interest and dissemination of SportsHCI literature over the past years, many publications still focus on solving specific problems in a given sport. We believe in the benefit of generating fundamental knowledge for SportsHCI more broadly to advance the field as a whole. To achieve this, we aim to identify the grand challenges in SportsHCI, which can help researchers and practitioners in developing a future research agenda. Hence, this paper presents a set of grand challenges identified in a five-day workshop with 22 experts who have previously researched, designed, and deployed SportsHCI systems. Addressing these challenges will drive transformative advancements in SportsHCI, fostering better athlete performance, athlete-coach relationships, spectator engagement, but also immersive experiences for recreational sports or exercise motivation, and ultimately, improve human well-being.

Physical fitness presents a significant challenge in ensuring proper exercise posture. Individuals who work out need help maintaining correct exercise posture during their workouts. Maintaining correct form is critical for ensuring the safety and effectiveness of fitness routines. Yet, it is often challenging for individuals to keep proper form without professional guidance, which usually comes at expensive costs. The paper presents a novel method that utilizes the capabilities of YOLOv7 and a primary web camera to offer immediate feedback and correction on body posture during gym activities. Such a method empowers individuals to correct themselves and promotes motivation even without the presence of a professional trainer. This system has been developed to provide immediate, personalized feedback for various fitness exercises. It efficiently counts repetitions and provides textual guidance for improvement, tailored to the specific requirements of fitness enthusiasts. To determine the efficacy of our technology, we carried out a user study in a controlled laboratory setting simulating a gym environment. The study compares our interactive system with the traditional training method, involving participants of varied fitness levels. It showed significant improvements in exercise technique with real-time feedback. These findings are crucial for AI-supported training systems in strength training, underscoring the need for adaptive technologies for different user experiences. The research contributes to human-computer interaction and fitness technology discussions, highlighting interactive models’ potential to augment and sometimes replicate personal training benefits in exercise form and posture improvement.

One challenge in highly automated driving is the safe transfer of control (ToC). A safe ToC requires estimating the take-over time depending on the driver’s state in different environmental conditions, to adapt the timing and design of the ToC request. We introduce environmental complexity as one factor that affects the ToC time. In a driving simulator experiment (N=12), the participants drove in five scenes having different environmental complexities (i.e. density and height of the background objects) with and without a secondary task. The results revealed that the ToC time is proportional to the environmental complexity. Thus, in the same driving task and the same traffic, an increasing environmental complexity yields higher ToC times in both conditions, with and without a secondary task. Our model of environmental complexity is a first step towards measuring the complexity of the real world, for a better prediction of ToC times in highly automated driving.

Over recent decades, with the fast growth of urbanization, new health-related issues have resulted from the changed lifestyles. People have become more sedentary. They suffer from fully-packed calendars, which does not give them enough time to care for their physical and psychological well-being. While this is a growing challenge, advancements made in internet of things (IoT) devices in the private context offer the chance to develop applications that tackle these issues. While many such applications have been envisioned, the user’s views and requirements for them remain unclear. To better understand this, we conducted a literature review in which we identified the four most common health-related scenarios in which such technologies are used. Next, we conducted an online survey (N=80), in which participants envisioned how smart home devices could realize these scenarios. Based on the results, we derive design implications that show that users value a high degree of customization and the ability to select a specific automation level through an all-in-one, easy-to-use platform.

Abstract Personalizing gameful applications is essential to account for interpersonal differences in the perception of gameful design elements. Considering that an increasing number of people lead sedentary lifestyles, using personalized gameful applications to encourage physical activity is a particularly relevant domain. In this article, we investigate behavior change intentions and Hexad user types as factors to personalize gameful fitness applications. We first explored the potential of these two factors by analyzing differences in the perceived persuasiveness of gameful design elements using a storyboards-based online study ( $$N=178$$ N=178 ). Our results show several significant effects regarding both factors and thus support the usefulness of them in explaining perceptual differences. Based on these findings, we implemented “Endless Universe,” a personalized gameful application encouraging physical activity on a treadmill. We used the system in a laboratory study ( $$N=20$$ N=20 ) to study actual effects of personalization on the users’ performance, enjoyment and affective experiences. While we did not find effects on the immediate performance of users, positive effects on user experience-related measures were found. The results of this study support the relevance of behavior change intentions and Hexad user types for personalizing gameful fitness systems further.

Nowadays, many wearable devices such as smartwatches exist that can be used to track and analyze sports activities. Generally, these devices are equipped with high-resolution screens, but most applications provide only textual status information as real-time visual feedback during the respective activities. This limited amount of information is particularly the case for running, which is among the most frequently tracked sports activities with wearable devices. So far, only a few products and prototypes provide assistance and feedback related to running technique and efficiency, but also predominantly by means of textual data representations.

Manipulating the appearance of a Virtual Environment to enable natural walking has so far focused on modifications that are intended to be unnoticed by users. In our research, we took a radically different approach by embracing the overt nature of the change. To explore this method, we designed the Space Bender, a natural walking technique for room-scale VR. It builds on the idea of overtly manipulating the Virtual Environment by "bending" the geometry whenever the user comes in proximity of a physical boundary. Our aim was to evaluate the feasibility of this approach in terms of performance and subjective feedback. We compared the Space Bender to two other similarly situated techniques: Stop and Reset and Teleportation, in a task requiring participants to traverse a 100 m path. Results show that the Space Bender was significantly faster than Stop and Reset, and preferred to the Teleportation technique, highlighting the potential of overt manipulation to facilitate natural walking.

In contrast to conventional retail stores, online shopping comes with many advantages, like unrestricted opening hours and is more focused on functionality. However, these pros often come at a cost of complex search and limited product visualization. Virtual Reality (VR) has the potential to create novel shopping experiences that combine the advantages of e-commerce sites and conventional stores. In this work, we propose a VR shop concept where product placement is not organized in shelves but through spatial placement in appropriate locations in an apartment environment. We thus investigated how the spatial arrangement of products in a non-retail environment affects the user, and how the actual shopping task can be supported in VR. In order to answer these questions, we designed two product selection and manipulation techniques (grabbing and pointing) and two VR shopping cart concepts (a realistic basket and an abstract one) and evaluated them in a user study. The results indicate that product interaction using pointing in combination with the abstract cart concept performs best with regard to error rate, user experience and workload. Overall, the proposed apartment metaphor provides excellent customer satisfaction, as well as a particularly high level of immersion and user experience, and it opens up new possibilities for VR shopping experiences that go far beyond mimicking real shop environments in VR.

Gaze estimation error can severely hamper usability and performance of mobile gaze-based interfaces given that the error varies constantly for different interaction positions. In this work, we explore error-aware gaze-based interfaces that estimate and adapt to gaze estimation error on-the-fly. We implement a sample error-aware user interface for gaze-based selection and different error compensation methods: a naïve approach that increases component size directly proportional to the absolute error, a recent model by Feit et al. that is based on the two-dimensional error distribution, and a novel predictive model that shifts gaze by a directional error estimate. We evaluate these models in a 12-participant user study and show that our predictive model significantly outperforms the others in terms of selection rate, particularly for small gaze targets. These results underline both the feasibility and potential of next generation error-aware gaze-based user interfaces.

In this paper we present Slackliner, an interactive slackline training assistant which features a life-size projection, skeleton tracking and real-time feedback. As in other sports, proper training leads to a faster buildup of skill and lessens the risk for injuries. We chose a set of exercises from slackline literature and implemented an interactive trainer which guides the user through the exercises and gives feedback if the exercise was executed correctly.

Wearable sports technology such as fitness trackers, smart watches or heart rate sensors has become ubiquitous in our everyday lives. This technology enables even recreational athletes to keep track of their workouts in a comprehensive manner. Besides the general assumption that this technology improves motivation to exercise more often, it also enables the athlete to get a better understanding of her current fitness level. However, current technology is mainly focussed on (quantifiable) performance indicators such as mileage, pace, cadence, watts, heart rate, etc.

Today, ambitioned amateur athletes often do not have access to professional coaching but still invest great effort in becoming faster runners. Apart from a pure increase in the quantitative training load, a change of the running technique, e.g. transitioning from heel striking to fore- or midfoot running, can be highly effective and usually prevents knee-related injuries.

Today, ambitious amateur athletes often do not have access to professional coaching but still invest great effort in becoming faster runners. Apart from a pure increase in the quantitative training load, a change of the running technique, e.g. transitioning from heel striking to mid-/forefoot running, can be highly effective and usually prevents knee-related injuries.

Rock climbing involves complex movements and therefore requires guidance when acquiring a new technique. The classic approach is mimicking the movements of a more experienced climber. However, the trainee has to remember every nuance of the climb, since the sequence of movements cannot be performed in parallel to the experienced climber. As a solution to this problem, we present a video recording and replay system for climbing. The replay component allows for different in-situ video feedback methods. We investigated the video feedback component of the system by studying two example visualization techniques, i.e. a life-sized in-place projection and a real-time third-person view of the climber, augmented by a video showing a successful ascent. The latter is presented to the user on both Google Glass and a projected display. The results indicate that a life-sized projection was perceived as easiest to follow, while most of the climbers had problems with the context switches between the augmented video and the climbing wall. These findings can aid in the design of assistance systems that teach complex movements.

Wearable sports devices like GPS watches and heart rate monitors are ubiquitous in sports like running or road cycling and enable the users to receive real-time performance feedback. Although rock climbing is a trending sport, there are little to no consumer electronics available to support rock climbing training during exercise. In this paper, we investigated the acceptance and appropriateness of wearables in climbing on different body parts. Based on an online survey with 54 climbers, we designed a wearable device and conducted a perception study with 12 participants in a climbing gym. Using vibro-tactile, audible, and visual cues while climbing an easy route and a hard route, requiring high physical and cognitive load, we found that the most suited notification channel is sound, directly followed by vibro-tactile output. Light has been found to be inappropriate for the use in the sport of climbing.

Gaze estimation error is inherent in head-mounted eye trackers and seriously impacts performance, usability, and user experience of gaze-based interfaces. Particularly in mobile settings, this error varies constantly as users move in front and look at different parts of a display. We envision a new class of gaze-based interfaces that are aware of the gaze estimation error and adapt to it in real time. As a first step towards this vision we introduce an error model that is able to predict the gaze estimation error. Our method covers major building blocks of mobile gaze estimation, specifically mapping of pupil positions to scene camera coordinates, marker-based display detection, and mapping of gaze from scene camera to on-screen coordinates. We develop our model through a series of principled measurements of a state-of-the-art head-mounted eye tracker.

Today, sports and activity trackers are ubiquitous. Especially runners and cyclists have a variety of possibilities to record and analyze their workouts. In contrast, climbing did not find much attention in consumer electronics and human-computer interaction. If quantified data similar to cycling or running data were available for climbing, several applications would be possible, ranging from simple training diaries to virtual coaches or usage analytics for gym operators. This paper introduces a system that automatically recognizes climbed routes using wrist-worn inertia measurement units (IMUs). This is achieved by extracting features of a recorded ascent and use them as training data for the recognition system. To verify the recognition system, cross-validation methods were applied to a set of ascent recordings that were assessed during a user study with eight climbers in a local climbing gym. The evaluation resulted in a high recognition rate, thus proving that our approach is possible and operational.

Some recent mobile devices have autostereoscopic displays that enable users to perceive stereoscopic 3D without lenses or filters. This might be used to improve depth discrimination of objects overlaid to a camera viewfinder in augmented reality (AR). However, it is not known if autostereoscopy is useful in the viewing conditions typical to mobile AR. This paper investigates the use of autostereoscopic displays in an psychophysical experiment with twelve participants using a state-of-the-art commercial device. The main finding is that stereoscopy has a negligible if any effect on a small screen, even in favorable viewing conditions. Instead, the traditional depth cues, in particular object size, drive depth discrimination.

With the increasing distribution of multi-touch capable devices multi-touch interaction becomes more and more ubiquitous. Especially the interaction with complex data (e.g. medical or geographical data), which until today mostly rely on mice and keyboard input or intense instrumentation, can benefit from this development. Multi-touch interaction offers new ways to deal with 3D data allowing a high degree of freedom (DOF) without instrumenting the user. This paper evaluates indirect multi-touch 3D selection techniques that can be used to interact with stereoscopic data. In this paper two gestural multi-touch selection techniques are presented and investigated with respect to positions on a stereoscopic multi-touch display and special consideration of objects displayed with different parallaxes. In an experiment it was shown that position and parallax have a significant impact on the interaction.

During sales conversations, gestures and mimics are of high importance to communicate information about a product. One prominent example for such sales gestures is the meat and cheese counter, which is one of the remaining spots in supermarkets where sales persons interact with customers. Interactions at such counters in supermarkets normally follow a simple protocol. The customer points at an item of choice. The employee takes out the item and, in most of the cases the product needs to be cut to fit the amount the customer wants to buy. Often it is ambiguous about what specific product the customer and the employees are talking about. Up to now, there are just a few efforts in HCI research to enrich communication at the point of sale. In this paper we report and analyze one scenario in which an intelligent natural user interface can support communication between customer and employee in a sales conversation. Furthermore, we report on our prototype that is able to track pointing gestures by using a depth camera and to display information about items pointed at.

Multi-touch interaction offers opportunities to interact with complex data. Especially the exploration of geographical data, which until today mostly relies on mice and keyboard input, could benefit from this interaction paradigm. However, the gestures that are required to interact with complex systems like Geographic Information Systems (GIS) increase in difficulty with every additional functionality. This paper describes a novel interaction approach that allows non-expert users to easily explore geographic data using a combination of multi-touch gestures and handpostures. The use of the additional input modality -- handpose -- is supposed to avoid more complex multi-touch gestures. Furthermore the screen of a wearable device serves as another output modality that on one hand avoids occlusion and on the other hand serves as a magic lens.

In the last years 3D is getting more and more popular. Besides the increasing number of movies for 3D stereoscopic cinemas and television, serious steps have also been undertaken in the field of 3D gaming. Games with stereoscopic 3D output are now available not only for gamers with high-end PCs but also on handheld devices equipped with 3D autostereoscopic displays.