Abstract As the range of handheld, mobile and desktop devices expands and worldwide demand for collaborative application tools increases, there is a growing need for higher speed impromptu cross-device application sharing to keep up with workplace requirements for on-site or remote collaborations. To address this, we have developed CamCutter, a cross-device interaction technique enabling a user to quickly select and share an application running on another screen using the camera of a handheld device. This technique can accurately identify the targeted application on a display using our adapted computer vision algorithm, system architecture and software implementation, allowing impromptu real-time and synchronized application sharing between devices. For desktop and meeting room set-ups, we performed a technical evaluation, measuring accuracy and speed of migration. For a single-user reading task and a collaborative composition task, we carried out a user study comparing our technique with commercial screen sharing applications. The results of this study showed both higher performance and preference for our system. Finally, we discuss CamCutter’s limitations and present insights for future vision-based cross-device application sharing.

We propose Coupled-clay, a bi-directional 3D collaborative interactive environment that supports the 3D modeling work between groups of users at remote locations. Coupled-clay consists of two network-connected workspaces, the Physical Interaction Space and the Virtual Interaction Space. The physical interaction space allows a user to directly manipulate a physical object whose shape and position are precisely tracked. This tracked 3D information is transferred to the virtual interaction space in real time. The virtual interaction space is made of an interactive multi-user stereoscopic 3D tabletop, or other 3D displays with adequate interaction device. The users at the virtual interaction space observe the virtual 3D object which corresponds to the physical object and manipulate its geometrical attributes (e.g., translation, rotation and scaling). Additionally, they can control the graphical attributes of the virtual object such as color and texture. Information about changes in geometrical and graphical attributes are sent back to the physical interaction space in real time and reflected to the object in the physical interaction space by a robotic arm and a top-mounted projector. Coupled-clay can be used to remotely collaborate on 3D modeling tasks such as between a skilled designer and novice learners. This paper details our Coupled-clay implementation and presents its interaction capabilities.