Image compositing is a method used to generate realistic yet fake imagery by inserting contents from one image to another. Previous work in compositing has focused on improving appearance compatibility of a user selected foreground segment and a background image (i.e. color and illumination consistency). In this work, we instead develop a fully automated compositing model that additionally learns to select and transform compatible foreground segments from a large collection given only an input image background. To simplify the task, we restrict our problem by focusing on human instance composition, because human segments exhibit strong correlations with their background and because of the availability of large annotated data. We develop a novel branching Convolutional Neural Network (CNN) that jointly predicts candidate person locations given a background image. We then use pre-trained deep feature representations to retrieve person instances from a large segment database. Experimental results show that our model can generate composite images that look visually convincing. We also develop a user interface to demonstrate the potential application of our method.

This paper introduces our ongoing work on the Foot Controller Device as an interface for navigation in Immersive Virtual Environments. The Foot Controller is a special, arduino-based control mat equipped with an array of pressure sensors and able to function as a touch surface for feet. What makes it special is that sensor cells can be accessed individually, allowing for a sophisticate control algorithms based on pressure distribution. The low-level software provides a set of recognition features which can be used to implement intuitive navigation metaphors. In this paper we introduce the device and its operating modes, present its use as a navigation interface, and discuss the results of a preliminary pilot user study aimed to evaluate its usability.

In this paper we investigate user performance and user behavior, related to the issue of who controls the point of view in a remote assistance scenario. We describe an experiment that examined users completing two different tasks with the aid of a remote gesturing device under two conditions: when control of the camera and gesturing point of view was in the hands of the remote helper, and when it was in the hands of the worker. Results indicate that in general, when most of the knowledge is with the helper, it is preferable to leave control in the hands of the helper. However, these results may depend on the situation and task at hand.

In the BEAMING project we have been extending the scope of collaborative mixed reality to include the representation of users in multiple modalities, including augmented reality, situated displays and robots. A single user (a visitor) uses a high-end virtual reality system (the transporter) to be virtually teleported to a real remote location (the destination). The visitor may be tracked in several ways including emotion and motion capture. We reconstruct the destination and the people within it (the locals). In achieving this scenario, BEAMING has integrated many heterogeneous systems. In this paper, we describe the design and key implementation choices in the Beaming Scene Service (BSS), which allows the various processes to coordinate their behaviour. The core of the system is a light-weight shared object repository that allows loose coupling between processes with very different requirements (e.g. embedded control systems through to mobile apps). The system was also extended to support the notion of presence awareness. We demonstrate two complex applications built with the BSS.

TeleAdvisor is a novel solution designed to support remote assistance tasks in many real-world scenarios. It consists of a video camera and a small projector mounted at the end of a tele-operated robotic arm. This enables a remote helper to view and interact with the workers' workspace, while controlling the point of view. It also provides the worker with a hands-free transportable device to be placed anywhere in his or her environment. Active tracking of the projection space is used in order to reliably correlate between the camera's view and the projector space.

Intelligent Environments most commonly take a physical form such as homes, offices, hotels, restaurants, shops, that are equipped with advanced networked computer based systems, which enable better or new lifestyles for people. However, Intelligent Environments can also take the form of virtual online spaces such as SecondLife, which can both mimic the real world and provide functionalities which could not be provided in reality, such as advanced simulations and movement. There is the growing trend for people to spend more time in such virtual environments and, to these ends, this work in progress paper reports on a new project, +Spaces which is developing a range of virtual world tools for e-government applications, and presents some of the concepts and technical challenges involved in creating these intelligent virtual spaces for e-government.