This paper introduces dynamic composite physicalizations, a new class of physical visualizations that use collections of self-propelled objects to represent data. Dynamic composite physicalizations can be used both to give physical form to well-known interactive visualization techniques, and to explore new visualizations and interaction paradigms. We first propose a design space characterizing composite physicalizations based on previous work in the fields of Information Visualization and Human Computer Interaction. We illustrate dynamic composite physicalizations in two scenarios demonstrating potential benefits for collaboration and decision making, as well as new opportunities for physical interaction. We then describe our implementation using wheeled micro-robots capable of locating themselves and sensing user input, before discussing limitations and opportunities for future work.

We present a first prototype of an open-ended interactive physical game aiming at developing children's understanding of dynamic systems in a playful and embodied way. We use a swarm user interface, Zooids, developed by Le Goc et al., made of independent self-propelled elements that move collectively and react to user input. Papert promoted an active way of developing a computational literacy, through programming a turtle with LOGO, from which Resnick proposed StarLogo, a "multi-turtles" language to simulate complex systems behaviors. Our interface is positioned in between these two perspectives: it allows to physically interact with multiple "turtles", each having its own dynamic. Each Zooid can be assigned an action that will affect the system behavior. Based on this principle, our first prototype invites children to resolve situations by changing individual actions in a dynamic system.

Passive haptic proxy objects allow for rich tangible interaction, and this is especially true in VR applications. However, this requires users to have many physical objects at hand. Our paper proposes robotic assembly at run-time of low-resolution haptic proxies for tangible interaction and virtual reality. These assembled physical proxy objects are composed of magnetically attached blocks which are assembled by a small multi robot system, specifically Zooids. We explore the design of the basic building blocks and illustrate two approaches to assembling physical proxies: using multirobot systems to (1) self-assemble into structures and (2) assemble 2.5D structure with passive blocks of various heights. The success rate and completion time are evaluated for both approaches. Finally, we demonstrate the potential of assembled proxy objects for tangible interaction and virtual reality through a set of demonstrations.

We introduce a novel method based on physical proxies for investigating fundamental differences between touch and tangible interfaces. This method uses physical chips to emulate the flat, non-graspable objects that make up touch interfaces, in a way that supports direct comparison with tangible interfaces. We ran an experiment to test the effect of object thickness on participants' behavior, performance and subjective experience in spatial rearrangement tasks. We found that for the tasks tested, thick objects are faster but less accurate to operate, and that their graspability is only used occasionally. We also found that coarse manipulation of multiple thin objects is error-prone, an issue that only thick objects may allow to alleviate.

This paper introduces swarm user interfaces, a new class of human-computer interfaces comprised of many autonomous robots that handle both display and interaction. We describe the design of Zooids, an open-source open-hardware platform for developing tabletop swarm interfaces. The platform consists of a collection of custom-designed wheeled micro robots each 2.6 cm in diameter, a radio base-station, a high-speed DLP structured light projector for optical tracking, and a software framework for application development and control. We illustrate the potential of tabletop swarm user interfaces through a set of application scenarios developed with Zooids, and discuss general design considerations unique to swarm user interfaces.

SmartTokens are small-sized tangible tokens that can sense multiple types of motion, multiple types of touch/grip, and send input events wirelessly as state-machine transitions. By providing an open platform for embedding basic sensing capabilities within small form-factors, SmartTokens extend the design space of tangible user interfaces. We describe the design and implementation of SmartTokens and illustrate how they can be used in practice by introducing a novel TUI design for event notification and personal task management.

We contribute a thin, transparent, and low-cost design for electric field sensing, allowing for 3D finger and hand tracking and gestures on mobile devices. Our approach requires no direct instrumentation of the hand or body, and is non-optical, allowing for a compact form-factor that is resilient to ambient illumination. Our simple driver electronics are based on an off-the-shelf chip that removes the need for building custom analog electronics. We describe the design of our transparent electrode array, and present a machine learning algorithm for mapping from signal measurements at the receivers to 3D positions. We demonstrate non-contact motion gestures, and precise 3D hand and finger localization. We conclude by discussing limitations and future work.