This paper contains a succinct survey on the recent major advances in contactless ultrasonic tactile feedback. It is a haptic technology which enables easy mid-air interactions with rich multisensory feedback and creates effects which are otherwise impossible. It can become a disruptive technology for multimedia interfaces, applications, and mobile computing. We summarize and discuss the advantages, problems and applications of the technology. This survey provides an introduction to the topic for anyone interested in applying or researching it.

We present a live gigapixel virtual reality system employing a 360° camera, a superzoom camera with a pan-tilt robotic head, and a head-mounted display (HMD). The system is capable of showing on-demand gigapixel-level subregions of 360° videos. Similar systems could be used to have live feed for foveated rendering HMDs.

Additional illumination improves the capture of omnidirectional 360° video and images, especially for dark or high-contrast environments. There is no "behind" for 360° cameras, so the placement of lights is a problem. We explore ways to position lights on some 360° cameras, and propose two good locations.

Immersive 360° video needs new ways of interaction. We compared three different interaction methods to find out which one of them is the most applicable for controlling 360° video playback. The compared methods were: remote control, pointing with head orientation, and hand gestures. A WebVR-based 360° video player was built for the experiment.

We present a novel signaling method for head-mounted displays, which surpasses eye (pupil) and delivers guiding light signals directly to retina through tissue near the eyes. This method preserves full visual acuity on the display and does not block view to the scene, while also delivering additional visual signals.

We present a new optical design for head-mounted displays (HMD) which has an exceptionally wide field of view (FOV). It can cover even the full human FOV. It is based on seamless lenses and screens curved around the eyes. The proof-of-concept prototypes are promising, and one of them far exceeds the human FOV, although the effective FOV is limited by the anatomy of the human head. The presented optical design has advantages such as compactness, light weight, low cost and super-wide FOV with high resolution. Even though this is still work-in-progress and display functionality is not yet implemented, it suggests a feasible way to significantly expand the FOV of HMDs.

Virtual and physical worlds are merging. Currently users of head-mounted displays cannot have unobtrusive tactile feedback while touching virtual objects. We present a mid-air tactile feedback system for head-mounted displays. Our prototype uses the focus of a modulated ultrasonic phased array for unobtrusive mid-air tactile feedback generation. The array and the hand position sensor are mounted on the front surface of a head-mounted virtual reality display. The presented system can enhance 3D user interfaces and virtual reality in a new way.

The Fog Screen can create high-quality walk-through images onto a translucent fog plane with rear-projection, so that they appear to float in mid-air. Projector's hotspot may distract viewers, but here we present a method based on fluorescence and front-projection to apparently remove the projector's light altogether. The fluorescent Fog Screen can create even more magical images than the standard Fog Screen.

In the entertainment industry, technology advances are continuously improving and evolving the portfolio of services and distinguished features that can be experienced by the audience, whether it is attending the event on site or remotely. In this context, the SkyMedia system aims at exploring, designing, and demonstrating a novel multimedia end-to-end architecture that can provide unique immersive media experiences to audience during live events. UAV and WSN technologies play a central role as enabling technologies. UAVs capture HD and 3D videos and images from the sky, while wireless sensors form an augmentation network acquiring side-information to be fused with the main audiovisual contents. All technologies come together to form a very advanced Multimedia Service Platform able to manage in real-time HD/3D processing, and an immersive media experience demonstration is going to be performed during a public live event in order to prove the effectiveness of the system.

Displays are the most visible part of most computer applications. Novel display technologies strongly influence and inspire new forms of computer use and interaction. We are particularly interested in the interplay of novel displays and interaction for ubiquitous computing or ambient media environments, as emerging display technologies may become game-changers in how we define and use computers, possibly changing the context of computing fundamentally. We present some of our experiments and lessons learnt with a new category of displays, the “immaterial” FogScreen. It can be described as a novel media platform, exhibiting some fundamental differences to and advantages over other displays. It also enables novel kinds of user interfaces and experiences. In this paper we give insights about the special properties and strengths of the FogScreen by looking at a set of successfully demonstrated interfaces and applications. We also discuss its future potential for user interface design.

There has been wide interest and even enthusiasm towards 3D and mid-air displays for well over a century. Recently many kinds of "holographic" displays have been seen in sci-fi movies like Star WarsTMetc. Media or 3D display manufacturers nowadays often claim that a specific novel display is like the ones seen in Star Wars. We survey and assess if how and to what extent the Star Wars mid-air displays are realizable with contemporary display technology, what are the strengths and weaknesses of various approaches, and when we can expect the mid-air displays to take over the display market.

The display is the last component in a chain of activity from image acquisition, compression, coding transmission and reproduction of 3-D images through to the display itself. There are various schemes for 3-D display taxonomy; the basic categories adopted for this paper are: holography where the image is produced by wavefront reconstruction, volumetric where the image is produced within a volume of space and multiple image displays where two or more images are seen across the viewing field. In an ideal world a stereoscopic display would produce images in real time that exhibit all the characteristics of the original scene. This would require the wavefront to be reproduced accurately, but currently this can only be achieved using holographic techniques. Volumetric displays provide both vertical and horizontal parallax so that several viewers can see 3-D images that exhibit no accommodation/convergence rivalry. Multiple image displays fall within three fundamental types: holoform in which a large number of views give smooth motion parallax and hence a hologram-like appearance, multiview where a series of discrete views are presented across viewing field and binocular where only two views are presented in regions that may occupy fixed positions or follow viewers' eye positions by employing head tracking. Holography enables 3-D scenes to be encoded into an interference pattern, however, this places constraints on the display resolution necessary to reconstruct a scene. Although holography may ultimately offer the solution for 3DTV, the problem of capturing naturally lit scenes will first have to be solved and holography is unlikely to provide a short-term solution due to limitations in current enabling technologies. Liquid crystal, digital micromirror, optically addressed liquid crystal and acoustooptic spatial light modulators (SLMs) have been employed as suitable spatial light modulation devices in holography. Liquid crystal SLMs are generally favored owing to the commercial availability of high fill factor, high resolution addressable devices. Volumetric displays provide both vertical and horizontal parallax and several viewers are able to see a 3-D image that exhibits no accommodation/convergence rivalry. However, the principal disadvantages of these displays are: the images are generally transparent, the hardware tends to be complex and non-Lambertian intensity distribution cannot be displayed. Multiple image displays take many forms and it is likely that one or more of these will provide the solution(s) for the first generation of 3DTV displays.

Future 3D television critically relies on mechanisms for automatically acquiring and visualizing high quality 3D content of both indoor and outdoor scenes. The envisioned goal is that a photo-realistic 3D real-time rendering from the actual and potentially arbitrary viewpoint of the beholder who is watching 3DTV becomes possible. Such scenes include movie sets in studios, e.g., for talk shows, TV series and blockbuster movies, but also outdoor scenes, e.g., buildings in a neighborhood for a car chase or cultural heritage sites for a documentary. The goal of 3D model acquisition is to provide the 3D background models where potential 3D actors can be embedded. We present both the 3D acquisition and semi-immersive 3D visualization to give an impression how a future 3D television system could be like.

Smart walls, interactive displays, and immersive projection dis are increasingly used in visualization and multimedia system. Tracking the user is one key element to implement inter, or e.g., to render new views according to the viewer's position to creat virtual reality-type displays. Persons viewing traditional screens or virtual reality rooms (CAVE) can be track with elec, optical, or many other tracking methods. If optica or computer vision track is used, then line of sight between the cameras and the tracked object is required. Opaque projection screens severely restrict the possibl camera setup, especially in CAVE-like environments.We present a novel, patented method to track users in three di through a projection screen. Our novel idea is to emplo a screen material that is opaque in the visible spectrum, and trans in the near-infrared (IR) spec, and then to use com vision -based or other optical tracking method to track the viewer in infrared spectrum. Non-solid screens like the FogScree seem to be one screen material that can deliver our objective, but also solid screens may be possible.

Various performing arts increasingly employ projection screens and other information displays as essential elements of the show. We present some possibilities and lessons learned for the interactiv, "immaterial" walk-through FogScreen in performing arts, analyze its suit for stage setups, and give some guidelines for show desig. We also describe our experiments on composing inter 3D graph for a FogScreen-based performance.

We present an interactive wall-sized immaterial display that introduces a number of interesting possibilities for advanced interface design. The immaterial nature of a thin sheet of fog allows users to penetrate and even walk through the screen, while its dual-sided nature allows for new possibilities in multi-user faceto- face collaboration and pseudo-3D visualization.

FogScreen is a free space 2D projection screen, which enables to touch and walk through an immaterial image. FogScreen consists of flowing air with a little visible humidity in the center of flow, and enables high-quality projected images in thin air, and many new applications.The FogScreen can be made interactive. Robust tracking of the user's pointing is a key element of the interactive system. In this short paper we present robust tracking employing laser scanning.

A system is presented to visualize information about a city with a three-dimensional Web user interface. We present a novel way to connect a VRML world to an information database. As the data, code and models are kept separate, updates and maintenance of the system is easy. The information visualization of a city becomes easier and more intuitive in 3D. The location of any service, shop etc. is shown both in a 3D world and on a 2D map, which are interconnected. Our implementation works on standard Web browsers. In addition, we built an optional steering wheel navigation, which is an intuitive way of moving around in a city for home Internet users. An experimental immersive VR interface was also made.