The paper describes the design story of the ColorTable, a tangible user interface in support of urban planners and diverse stakeholders collaboratively envisioning urban change, which was developed in an iterative process of design-evaluation-feedback-redesign in a series of workshops with users in the context of real urban planning projects. It seeks to clarify a number of more general design issues related to tangible user interfaces -- how to make use of material and spatial properties in designing both, physical interface and multiple and simultaneous interactions; how to handle the complexity of urban projects while keeping interfaces and interactions simple and transparent.

This paper discusses technological interventions in support of planners, citizens and other stakeholders in envisioning and nego-tiating an urban project. A set of prototypal tools, including a tangible user interface, has been developed that allow users to create and manipulate visual/auditory scenes and mesh these scenes with the real environment of an urban site. The paper dis-cusses how to support users – different types of stakeholders- in the collaborative creation of mixed reality configurations as an integral part of expressing their ideas about an urban project, dis-tinguishing between different types and levels of cooperation. It also looks into how to use mixed reality tools for enhancing an already highly developed representational culture.

The ability to physically change properties of real objects used in augmented reality (AR) applications is limited. Geometrical properties (shape, size) and appearance (color, texture) of a real object remain unchanged during a single application run. However, an AR system can be used to provide a virtual texture for the real object. The texture can be changed dynamically based on user interactions. The developed AR system includes two components, the and the Painter. The 3D Table is a table where real objects are placed. The tabletop is used as a projection surface, making it possible to add a context to the real object. The Texture Painter makes it possible to paint on the real object, using a real brush and virtual ink (texture). ARToolkit markers are placed on the 3D Table tabletop to augment the environment with the virtual objects. Markers are either physical (printouts on the tabletop) or virtual (projections). The scene is recorded with a camera and the composed video is projected in real time. The projection shows a virtual environment, real objects painted with virtual ink, and virtual objects positioned where real or virtual ARToolkit markers are placed. The developed system is used in architectural design applications where, due to the different qualities of real architectural models and rendered architectural models, real models are still used. The system was tested at the Academy of Fine Arts in Vienna where it is used as a support tool for architecture students.

AR puppet is a hierarchical animation framework for augmented reality agents, which is a research area combining augmented reality (AR), sentient computing and autonomous animated agents into a single coherent human-computer interface paradigm. While sentient computing systems use the physical environment as an input channel, AR outputs virtual information superimposed on real world objects. To enhance man-machine communication with more natural and efficient information presentation, this framework adds animated agents to AR applications that make autonomous decisions based on their perception of the real environment. These agents are able to turn physical objects into interactive, responsive entities collaborating with both anthropomorphic and non-anthropomorphic virtual characters, extending AR with a previously unexplored output modality. AR puppet explores the requirements for context-aware animated agents concerning visualization, appearance, behavior, in addition to associated technologies and application areas. A demo application with a virtual repairman collaborating with an augmented LEGO/spl reg/ robot illustrates our concepts.