Recent developments of video and sensing technology have led to large volumes of digital media data. Current media production relies on videos from the principal camera together with a wide variety of heterogeneous source of supporting data [photos, light detection and ranging point clouds, witness video camera, high dynamic range imaging, and depth imagery]. Registration of visual data acquired from various 2D and 3D sensing modalities is challenging because current matching and registration methods are not appropriate due to differences in structure, format, and noise characteristics for multimodal data. A combined 2D/3D visualization of this registered data allows an integrated overview of the entire data set. For such a visualization, a Web-based context presents several advantages. In this paper, we propose a unified framework for registration and visualization of this type of visual media data. A new feature description and matching method is proposed, adaptively considering local geometry, semiglobal geometry, and color information in the scene for more robust registration. The resulting registered 2D/3D multimodal visual data are too large to be downloaded and viewed directly via the Web browser, while maintaining an acceptable user experience. Thus, we employ hierarchical techniques for compression and restructuring to enable efficient transmission and visualization over the Web, leading to interactive visualization as registered point clouds, 2D images, and videos in the browser, improving on the current state-of-the-art techniques for Web-based visualization of big media data. This is the first unified 3D Web-based visualization of multimodal visual media production data sets. The proposed pipeline is tested on big multimodal data set typical of film and broadcast production, which are made publicly available. The proposed feature description method shows two times higher precision of feature matching and more stable registration performance than existing 3D feature descriptors.

A typical high-end film production generates several terabytes of data per day, either as footage from multiple cameras or as background information regarding the set (laser scans, spherical captures, etc). This paper presents solutions to improve the integration of the multiple data sources, and understand their quality and content, which are useful both to support creative decisions on-set (or near it) and enhance the postproduction process. The main cinema specific contributions, tested on a multisource production dataset made publicly available for research purposes, are the monitoring and quality assurance of multicamera set-ups, multisource registration and acceleration of 3-D reconstruction, anthropocentric visual analysis techniques for semantic content annotation, and integrated 2-D–3-D web visualization tools. We discuss as well improvements carried out in basic techniques for acceleration, clustering and visualization, which were necessary to deal with the very large multisource data, and can be applied to other big data problems in diverse application fields.

In this paper, we present a web-based application allowing the analytic visualisation of on-set media data and metadata, which combines research from several fields of image processing and 3D graphics. Film and television production is currently undergoing a paradigm shift, away from the concept of capturing a `shot' with a single principal camera view, to recording everything possible about the set, the environment, the objects and the actors within it. In parallel, the rise of cloud-based storage and processing is pushing towards applications being controlled via web or browser based interfaces. The application presented in this paper has been created to meet both of these needs, visualising Big Media Data via a web-based interface, and supporting the integration of multiple data formats and analysis. We discuss the different modalities and processing included in the current tool, stress the integration into the web and the solutions to the challenges posed, and show how the tool has been initially tested by professionals, with positive outcomes.

Consumer-grade head-mounted displays (HMD) such as the Oculus Rift have become increasingly available for Virtual Reality recently. Their high degree of immersion and presence provokes usually amazement when first used. Nevertheless, HMDs also have been reported to cause adverse reactions such as simulator sickness. As their impact is growing, it is important to understand such side effects. This paper presents the results of a relatively large scale user experiment which compares using a conventional game controller versus positioning in the virtual world based upon the signal of the internal Inertial Measurement Unit (IMU) using Oculus Rift DK1. We show that simulator sickness is significantly reduced when using a position estimation system rather than using the more traditional game controller for navigation. However the sense of presence was not enhanced by the possibility of 'real walking'. We also show the impact of other factors, such as prior experience or motion history, and discuss the results.

In this paper we will present The Listening Room, an interactive audio installation that holds more or less meaningful conversations with up to three people at any one time. Conceived as an artwork that explores the boundaries between virtual and 'real world' experience, The Listening Room incorporates a number of speech technologies. This paper will give an account of the conceptual framework for The Listening Room and will describe the technologies employed in its realisation. To give context to the ideas behind The Listening Room we will describe the work with reference to two previous interactive works by the authors - Face Value(2000) and Alter Ego (2005).

In this paper we present Conversation Piece, an interactive audio installation that can hold conversations with up to three people at any one time. Conceived as an artwork that explores the boundaries between virtual and 'real world' experience, Conversation Piece incorporates a number of speech technologies. This paper describes the installation and gives an account of the technologies employed in its realization.

Automatic segmentation of the liver has the potential to assist in the diagnosis of disease, preparation for organ transplantation, and possibly assist in treatment planning. This paper presents initial results from work that extends on previous two-dimensional (2D) segmentation methods by implementing full three-dimensional (3D) liver segmentation, using a self-reparameterising discrete deformable model. This method overcomes many of the weaknesses inherent in 2D segmentation techniques, such as the inability to automatically segment separate lobes of the liver in each image slice, and sensitivity to individual-slice noise. Results are presented showing volumetric and overlap comparison of twelve automatically segmented livers with their corresponding manually segmented livers, which were treated as the gold standard for this study

We introduce a novel form of shape representation based on recording the distribution of pairwise geometric relationships between local shape features. It is shown that the geometric histograms used to record these distributions can be easily and robustly acquired from image data and can support recognition even when the shape extracted from the image is badly degraded by fragmentation noise and occlusion. Moreover, the processing involved in establishing correspondences between model and image features is both simple and parallel and has many advantages over previous search based methods. 1

Representation of arbitrary shape for the purposes of visual object recognition is an unsolved problem. This paper outlines some theoretical properties of shape representations based on stored geometric constraints that make them suitable for input to a 3D object recognition system. The construction of such representations is detailed together with an evaluation of their efficacy based on experimental findings.>