A Diaphragm Wall Hydraulic Grab (DWHG), used in civil engineering for bulk excavation, is featured with a cable-suspended end-effector (also named attachment tool). The end-effector begins to oscillate during DWHG operation. Essential insights into the DWHG’s performance or productivity in operation could be derived from end-effector oscillations. However, due to limited robustness, the end-effector oscillation curves from a DWHG in operation can not be tracked directly by motion sensors attached to the end-effector. This article presents and evaluates a novel approach for DWHGs to infer cable-suspended end-effector oscillations from their hydraulic actuators’ responses. Hydraulic actuators’ responses depict the changes of hydraulic parameters in hydraulic actuators due to end-effector oscillations. The main contribution of this article is an investigation of how far oscillations of a cable-suspended end-effector in a DWHG can be inferred from their hydraulic actuators’ responses. For this purpose, a workflow is conducted: firstly, end-effector oscillations and their hydraulic actuators’ responses are collected for a typical DWHG steering sequence. Secondly, the collected end-effector oscillation and hydraulic actuators’ response curves are analyzed and processed. Thirdly, a model to infer end-effector oscillations from hydraulic actuators’ responses is built. Fourthly, an evaluation shows that the inferred oscillation curves (model output) are similar to the true end-effector oscillationsNote to Practitioners—This article is motivated by the demand to determine the oscillation curves of the cable-suspended end-effector (also named attachment tool) at a Diaphragm Wall Hydraulic Grab (DWHG) used in civil engineering for bulk excavation. Existing approaches for construction machines commonly install motion sensors at the end-effector to track its oscillations. However, mounting any sensors at the end-effector in DWHGs is not practicable because the end-effector is exposed to mechanical shocks and moisture. Thus, mounted sensors would be damaged or demolished. This article suggests and investigates a novel approach for DWHGs to infer end-effector oscillations from pressure data in hydraulic actuators that actuate the end-effector. Therefore, the pressure curves at the hydraulic actuators are tracked while the end-effector oscillates and are utilized afterward to infer these oscillations. A practical guide for a DWHG is presented and evaluated to collect, analyze, process, and utilize pressure data from hydraulic actuators to infer end-effector oscillations.

In Virtual Reality (VR), visual and auditory sensations are effectively leveraged to create immersive experiences. However, touch is significantly underutilized in immersive media. We enhance the VR image viewing experience by integrating haptic and auditory feedback into 3D environments constructed from immersive media. We address the challenges of utilizing depth maps from various image formats to create intractable environments. The VR experience is enhanced using vibrohaptic feedback and audio cues triggered by controller collisions with haptic materials.

Real-time ray tracing has been a goal and a challenge in the graphics field for many decades.With recent advances in the hardware and software domains, this is becoming a reality today.In this work, we describe how we got to this point by taking a look back at one of the first fully ray traced games: "Quake 3: Ray Traced".We provide insight into the development steps of the project with unreleased internal details and images.From a historical perspective, we look at the challenges pioneering in this area in the year 2004 and highlight the learnings in implementing the system, many of which are relevant today.We start by going from a blank screen to the full ray traced gaming experience with dynamic animations, lighting, rendered special effects and a simplistic implementation of the gameplay with basic AI enemies.We describe the challenges encountered with aliasing and the methods used to alleviate it.Lastly, we describe for the first time the unofficial continuation of the project, code named "Quake 3: Team Arena Ray Traced", and provide an overview of the changes over the past 15 years that made it possible to generate fully ray-traced interactive gaming experiences with mass market hardware and an open software stack.

Using stereo-based depth cameras outdoors on drones can lead to challenging situations for stereo algorithms calculating a depth map.A false depth value indicating an object close to the drone can confuse obstacle avoidance algorithms and lead to erratic behavior during the drone flight.We analyze the encountered issues from real-world tests together with practical solutions including a post-processing method to modify depth maps against outliers with wrong depth values.

An important criterion for virtual reality experiences is that they are very immersive. The person inside the head-mounted display feels like really being in the virtual environment. While this can be a very pleasant experience, the opposite can happen as well. The concepts of personal spaces and people or unfriendly avatars entering them, can lead to the same discomfort as if it would happen in real life. In this work, we propose to define multi-level artificial barriers for other avatars and objects, respecting the personal spaces as defined by users. We apply this to both interactive rendered environments and as much as possible also to 360 degree photo and video content.

To enable high immersion for virtual reality head-mounted displays (HMDs), a wide field of view of the display is required. Today's consumer solutions are mostly around 90 to 110 degrees field of view. The full human field of view for both eyes together has been measured to be between 200 and 220 degrees. Prototypes of HMDs with such properties have been shown. As the rendering workload increases with more pixels to fill the field of view, we propose a novel rendering method optimized for HMDs that cover the full human field of view. We target lower end HMDs where the cost of eye tracking would increase the price too much. Our method works without eye tracking, making use of certain human vision properties that appear once the full human field of view is covered. We achieve almost twice the rendering performance using our method.

Interacting with real-time rendered 3D content from powerful machines on smaller devices is becoming ubiquitous through commercial products that enable in-home streaming within the same local network.However, support for high resolution, low latency in-home streaming at high image quality is still a challenging problem.To enable this, we enhance an existing open source framework for in-home streaming.We add highly optimized DXT1 (DirectX Texture Compression) support for thin desktop and notebook clients.For rendered light fields, we improve the encoding algorithms for higher image quality.Within a 10 Gigabit Ethernet (10 GbE) network, we achieve streaming up to 5K resolution at 55 frames per second.Through new low-level algorithmic improvements, we increase the compression speed of ETC1 (Ericsson Texture Compression) by a factor of 5. We are the first to bring ETC2 compression to real-time speed, which increases the streamed image quality.Last, we reduce the required data rate by more than a factor of 2 through foveated compression with real-time eye tracking.

Virtual Reality is reaching into the consumer space. Mobile virtual reality solutions are nowadays widely available and affordable for many smartphones, by adding a case with attached lenses around the phone to create a head-mounted display. While using these in public places or at social gatherings where the head-mounted display is given around to others, it can lead to problems regarding social acceptance, as the surrounding people are not aware of what the virtual reality user is seeing and doing. We address this problem by adding a second, front-facing screen to the head-mounted display. We build and evaluate two prototypes for this usage.

Virtual Reality has hit the consumer market with affordable head-mounted displays. When using these, it quickly becomes apparent that the resolution of the built-in display panels still needs to be highly increased. To overcome the resulting higher performance demands, eye tracking can be used for foveated rendering. However, as there are lens distortions in HMDs, there are more possibilities to increase the performance with smarter rendering approaches. We present a new system using optimizations for rendering considering lens astigmatism and combining this with foveated rendering through eye tracking. Depending on the current eye gaze, this delivers a rendering speed-up of up to 20%.

With the adaption of virtual reality in the consumer space, spherical panorama photos are gaining popularity. Through wide-angle head-mounted displays, they can be experienced in a natural way and offer the user an immersive view of the captured scene. While being used in virtual reality, the alignment of the saved image does not matter much. However, when displaying the panorama on a 2D screen, the alignment can make a difference on how pleasant the image looks. We propose an automatic method to do lossless shifting of the image to make it look better on 2D screens.

With increasing spatial and temporal resolution in head-mounted displays (HMDs), using eye trackers to adapt rendering to the user is getting important to handle the rendering workload. Besides using methods like foveated rendering, we propose to use the current visual field for rendering, depending on the eye gaze. We use two effects for performance optimizations. First, we noticed a lens defect in HMDs, where depending on the distance of the eye gaze to the center, certain parts of the screen towards the edges are not visible anymore. Second, if the user looks up, he cannot see the lower parts of the screen anymore. For the invisible areas, we propose to skip rendering and to reuse the pixels colors from the previous frame. We provide a calibration routine to measure these two effects. We apply the current visual field to a renderer and get up to 2x speed-ups.

Lenses in modern consumer HMDs introduce distortions like astigmatism: only the center area of the displayed content can be perceived sharp while with increasing distance from the center the image gets out of focus. We show with three new approaches that this undesired side effect can be used in a positive way to save calculations in blurry areas. For example, using sampling maps to lower the detail in areas where the image is blurred through astigmatism, increases performance by a factor of 2 to 3. Further, we introduce a new calibration of user-specific viewing parameters that increase the performance by about 20-75%.

Today, mobile devices like smartphones and tablets are becoming more powerful and exhibit enhanced 3D graphics performance.However, the overall computing power of these devices is still limited regarding usage scenarios like photorealistic gaming, enabling an immersive virtual reality experience or real-time processing and visualization of big data.To overcome these limitations application streaming solutions are constantly gaining focus.The idea is to transfer the graphics output of an application running on a server or even a cluster to a mobile device, conveying the impression that the application is running locally.User inputs on the mobile client side are processed and sent back to the server.The main criteria for successful application streaming are low latency, since users want to interact with the scene in near real-time, as well as high image quality.Here, we present a novel application framework suitable for streaming applications from high-end machines to mobile devices.Using real-time ETC1 compression in combination with a distributed rendering architecture we fully leverage recent progress in wireless computer networking standards (IEEE 802.11ac) for mobile devices, achieving much higher image quality at half the latency compared to other inhome streaming solutions.

High-quality head mounted displays are becoming available in the consumer space. These displays provide an immersive gaming experience by filling the wearer's field of view. To achieve immersion with low cost, a commodity display panel is placed a short distance in front of each eye, and wide-angle optics are used to bring the image into focus. However, these optics introduce spatial and chromatic distortion into the image seen by the viewer. As a result, the images to be displayed must be pre-warped to cancel this distortion. This correction can be performed by warping the image in a post-processing step, by warping the scene geometry before rendering, or by modeling corrective optics in the virtual camera.