Abstract In this paper, we revisit the issues surrounding the lack of gender diversity in computing and build a theory on the roles and effects of genders in computing. Our intention is to transform human experiences with computing technologies to more equitably reflect and represent diversity. To support gender diversity in design, we work to create an integrated trans-feminist theory. In doing so we draw from diverse fields, including English, psychology, philosophy, cultural theory, law, medicine, and feminist, queer, disability, indigenous, post-colonial, Black, and Chicana studies. We show how and why marginalized people need to develop our own languages and voices as a step in empowering our identities. We assemble quantitative data showing the paucity of people with historically marginalized genders in computer science education and in our best papers. We use the participation gap in computing, combined with the mental health impact, to argue that computing, as a field, needs to critically examine our cis/heteronormative tendencies, which perpetuate a vicious cycle of erasure, and instead frame scholarship in terms of gender identities and presentations.

Multiscale design is the widely practiced use of space and scale to visually explore and articulate relationships. Free-form web curation (FFWC) is an approach to supporting multiscale design, involving creative strategies of collecting content, assembling it to juxtapose and organize, sketching, writing, shifting perspective to navigate, and exhibiting to share and collaborate. Our long term goal is to support design students with automatic, on demand feedback.

We investigate new media to improve how teams of students create and organize artifacts as they perform design. Some design artifacts are readymade-e.g., prior work, reference images, code framework repositories-while others are self-made-e.g., storyboards, mock ups, prototypes, and user study reports. We studied how computer science students use the medium of free-form web curation to collect, assemble, and report on their team-based design projects. From our mixed qualitative methods analysis, we found that the use of space and scale was central to their engagement in creative processes of communication and contextualization.

Our observations of landscape architecture students revealed a new phenomenon-interstices. Their bimanual interactions with a pen and touch surface involved various sustained hand gestures, interleaved between their regular commands. Positioning of the non-preferred hand indicates anticipated actions, including: sustained hovering near the surface; pulled back but still floating above the surface; and resting in their laps. We ran a second study with 14 landscape architect students which confirmed our observations, and uncovered a new interstice i.e. stabilizing the preferred hand while handwriting. We conclude with directions for future research and challenges for designers and researchers.

We derive a theory of multiscale design through a study of a landscape architecture studio classroom. We find that processes of designing, to meet a site's situated needs, involve creating and connecting representations across levels, such as overview and detail. We introduce multiscale design theory, which works to understand how designers explore, juxtapose, and synthesize relationships across levels of scale. We identify three design strategies landscape architecture students use to work with scale: multiply, map, and shift perspective. We combine these strategies with prior literature, across fields, to initiate a theory of multiscale design.

LGBT+ people adjust the presentation of their gender and sexual identities in response to social pressures, but their level of visibility differs between social media. We interviewed seventeen LGBT+ students at a socially-conservative university to investigate: (1) how do social media affect LGBT+ user experience of managing self presentation; and (2) how do social media affect participation in LGBT+ communities? We develop implications for design to support queering social media. (1) Give people abilities to present themselves with selective visibility, enabling choices about privacy and sharing, in contrast with the HCI design principle of indiscriminate 'making visible'. That is, enable participants to define their social media identities in their own ways. (2) Conduct studies, with a methodology likewise ensures that participants can define their gender and sexual identities in their own ways, rather than according to a predetermined vocabulary.

In recent years, online education's reach and scale have increased through new platforms for large and small online courses. However, these platforms often rely on impoverished modalities, which provide limited support for participation in social learning experiences. We present Collaborative Live Media Curation (CLMC), a new medium for sharing context and participation in online learning. CLMC involves collaborative, synchronous collection, creation, and assemblage of web media, including images, text, video, and sketch. CLMC integrates live media including streaming video, screenshares, audio, and text chat. We deploy and study LiveMâché, a CLMC technology probe, in four situated online learning contexts. We discovered student and instructor strategies for sharing context and participating including creating curations in advance, sketching to illustrate and gesture, real-time transformations, sharing perspective, and assembling live streams. We develop implications through live experience patterns, which describe how spatial and computing structures support social activities.

Social media platforms such as Twitter and Facebook have become a critical tools for the coordination of activist movements and demonstrations. Additionally, live streaming platforms like Periscope have started to emerge as modalities for reporting on and remotely experiencing activist activities. In this design fiction, we present LiveDissent, a fictional media platform that leverages both established and emerging technologies to support remote participation in activist demonstrations. These technologies include mobile live streaming, telepresence robots, unmanned aerial vehicles, and digitally augmented picket signs. LiveDissent brings these technologies together into an aggregated platform for remotely watching and participating in demonstrations. We discuss a planned in the wild study at an upcoming demonstration, through which we will explore the design and implications of social media and telepresence for participation in activism. Finally, our authors' statement connects to related prior work and explores potential research and ethics questions.

On Reddit, users from tens of thousands of communities create and promote internet content, including pictures, videos, news, memes, and creative writing. However, like most social media feeds, subscribing to a very small subset of available content creates filter bubbles. These bubbles, while created unintentionally, skew perceptions of reality. This phenomena provides an impetus for researchers to design techniques breaking out of filter bubbles. Virtual reality provides opportunities for new environments that contextualize social media among multiple perspectives. We present one solution to the filter bubble problem: Blue Link City, which enables contextualized exploration of Reddit.

We synthesize principles across art, design, writing, and psychology to articulate strategies of free-form web curation, for stimulating students' creative engagement with prior work, in performance of ideation tasks. We manifest the strategies in an interactive system and pedagogy. Free-form web curation is a form of new media--designed to enable users to conceptualize and create new contexts--in which they discover, interpret, and represent relationships, by composing readymade and self-made content elements. The strategies are Collect, Assemble, Shift Perspective, Sketch, Write, and Exhibit. We conducted a field study involving 1247 students over 4 semesters. We connect visual, quantitative, and experiential data to show how students engage with prior work through the free-form web curation strategies, system, and pedagogy. We develop implications for design of web curation systems and education. We coalesce a theory of free-form web curation, based on the strategies, emphasizing the roles of space and context.

We identify emerging phenomena of distributed liveness, involving new relationships among performers, audiences, and technology. Liveness is a recent, technology-based construct, which refers to experiencing an event in real-time with the possibility for shared social realities. Distributed liveness entails multiple forms of physical, spatial, and social co-presence between performers and audiences across physical and virtual spaces. We interviewed expert performers about how they experience liveness in physically co-present and distributed settings. Findings show that distributed performances and technology need to support flexible social co-presence and new methods for sensing subtle audience responses and conveying engagement abstractly.

Web curation involves choosing, organizing, and commenting on content. Popular web curation apps-- e.g., Facebook, Twitter, and Pinterest-- provide linear feeds that show people the latest content, but provide little support for articulating relationships among content elements. The new medium of free-form web curation enables multimedia elements to be spontaneously gathered from the web, written about, sketched amidst, manipulated, and visually assembled in a continuous space. Through free-form web curation, content is collected, interpreted, and arranged, creating context. We conducted a field study of 1581 students in 6 courses, spanning diverse fields. We derive patterns of free-form curation through a visual grounded theory analysis of the resulting dataset of 4426 curations. From the observed range of invocations of the patterns in the performance of ideation tasks, we conclude that free-form is valuable as a new medium of web curation in how it supports creative visual thinking.

We introduce, LayerFish, a bimanual interaction technique for layering overlapping content in large 2D design spaces. Designers compose visual elements, often producing overlap and requiring layering in the z-dimension. A common approach in design tools is to provide a scene index, as an ordered list of layered elements. Scene indexes become difficult, in terms of physical effort and human cognition, to deal with when working with hundreds of elements. LayerFish renders an in-place scene index as a fisheye to reduce demands on effort and attention. Bimanual gestures support selection, scrolling, and manipulation. We evaluated layering task performance with LayerFish in comparison to a traditional scene index. Findings indicate that the fisheye reduces time to find an element and selection reduces layering time when elements do not heavily overlap.

We extend the Twitter interface to stimulate exploratory browsing of social media and develop a creative cognition method to establish its efficacy. Exploratory browsing is a creative process in which users seek and traverse diverse and novel information as they investigate a conceptual space. The TweetBubble browser extension extends Twitter to enable expansion of social media associations@usernames and #hashtags-in-context, without overwriting initial content. We build on a prior metadata type system, developing new presentation semantics, which enable an integrated look and feel consistent with Twitter.

We investigate how presentation in a free-form medium stimulates free-form thinking and discussion in the classroom. Most classroom presentations utilize slideware (e.g. PowerPoint). Yet, slides add intrusive segregations that obstruct the flow of information. In contrast, in a free-form medium of presentation, content is not separated into rigid slide compartments. Instead, it is visually arranged and transformed in a continuous space.

Previously, video streaming sites were at the fringes of online social media. In the past two years, live streams of video games, on sites such as Twitch.tv, have become very popular. Live streams serve as meeting grounds for player communities. The Twitch streaming medium combines broadcast video with open IRC chat channels. In conjunction with gameplay, viewer participation and community building gain emphasis. Twitch streams range in size and nature, from intimate communities with fifty viewers, to massive broadcasts with tens of thousands. In this paper, we present an ethnographic investigation of the live streaming of video games on Twitch.

We develop new understanding of how people engage in digital curation. We interview twenty users of Pinterest, a social curation platform. We find that through collecting, organizing, and sharing image bookmarks, users engage in processes of everyday ideation. That is, they use digital found objects as creative resources to develop ideas for shaping their lives. Curators assemble information into new contexts, forming and sharing ideas with practical and emotional value. We investigate cognitive and social aspects of creativity that affect the digital curation practices of everyday ideation. We derive implications for the design of curation environments that support information-based ideation.

Evaluating creativity support environments is challenging. Some approaches address people’s experiences of creativity. The present method measures creativity, across conditions, in the products that people make. This research introducesinformation-based ideation(IBI), a paradigm for investigating open-ended tasks and activities in which users develop new ideas. IBI tasks span imagining, planning, and reflecting on a weekend, vacation, outfit, makeover, paper, internship, thesis, design, campaign, crisis response, career, or invention. What products do people create through engagement in IBI?Curationof digital media incorporates conceptualization, finding and choosing information objects, annotation, and synthesis. Through engagement in IBI tasks, people create curation products. This article formulates a quantitative methodology for evaluating IBI support tools, building on prior creative cognition research in engineering design to derive a battery ofideation metrics of curation. Elemental ideation metricsevaluate creativity within curated found objects.Holistic ideation metricsevaluate how a curation puts elements together. IBI support environments are characterized by their underlying medium of curation. Curation media include lists, such as listicles, and grids, such as the boards of Pinterest. An in-depth case study investigatesinformation composition, an art-based medium representing a curation as a freeform visual semantic connected whole. We raise two creative cognition challenges for IBI. One challenge is overcoming fixation—for instance, when a person gets stuck in a counterproductive mental set. The other challenge is to bridge information visualization’ssynthesis gap, by providing support for connecting findings. To address the challenges, we developmixed-initiative information composition(MI2C), integrating human curation of information composition with automated agents of information retrieval and visualization. We hypothesize that MI2C generatesprovocative stimulithat help users overcome fixation to become more creative on IBI tasks. We hypothesize that MI2C’s integration of curation and visualization bridges the synthesis gap to help users become more creative. To investigate these hypotheses, we apply ideation metrics of curation to interpret results from experiments with 44 and 49 participants.

Reflection, interpretation, and curation play key roles in learning, creativity, and problem solving. Reflection means looking back and forward among building blocks constituting a space of ideas, contextualizing with processes including tasks, activities, and one's internal thinking and meditating, and deriving new understandings, known as interpretations. Curation, in the digital age, means searching, gathering, collecting, organizing, designing, reflecting on, and interpreting information.

Research in multi-touch interaction has typically been focused on direct spatial manipulation; techniques have been create to result in the most intuitive mapping between the movement of the hand and the resultant change in the virtual object. However, as we attempt to design for more complex operations, the expectation of spatial manipulation becomes infeasible.

ZeroTouch (ZT) is a unique optical sensing technique and architecture that allows precision sensing of hands, fingers, and other objects within a constrained 2-dimensional plane. ZeroTouch provides tracking at 80 Hz, and up to 30 concurrent touch points. Integration with LCDs is trivial. While designed for multi-touch sensing, ZT enables other new modalities, such as pen+touch and free-air interaction. In this paper, we contextualize ZT innovations with a review of other flat-panel sensing technologies. We present the modular sensing architecture behind ZT, and examine early diverse uses of ZT sensing.

We present culturally based design (CBD), a new paradigm for designing embodied natural user interaction (NUI) with digital information by drawing on customary ways that people use physical objects. CBD coalesces experiences, practices, and embodied mental models of pre-digital activities as a basis for the design of interactive systems. We apply CBD to address trans-surface interaction, the manipulation of information artifacts from one device to another. We develop Trans-Surface Rummy, because the game involves highly dynamic combinations of turn taking and non-linear out of turn play, while transferring information artifacts to and from private and social surfaces. Through the CBD process, we create the trans-surface wormhole, an embodied interface technique. We investigate the trans-surface wormhole's efficacy and other aspects of culturally based design with young students, and with elderly members of our local bridge club. We derive implications for the design of trans-surface interaction, and more broadly, from the process of CBD. We initiate a research agenda for trans-surface interaction.

We used the situated context of real-time strategy (RTS) games to address the design and evaluation of new pen + touch interaction techniques. RTS play is a popular genre of Electronic Sports (eSports), games played and spectated at an extremely high level. Interaction techniques are critical for eSports players, because they so directly impact performance.

Fire emergency responders rely on team coordination to survive and succeed in high-stress environments, but traditional education does not directly teach these essential skills. Prior simulations seek the highest possible fidelity, employing resources to capture concrete characteristics of operating environments. We take a different tack, hypothesizing that a zero-fidelity approach, focusing on human-centered aspects of work practice, will improve team coordination learning. Such an approach promotes simulation focus by developing an alternative environment that stimulates participants to engage in distributed cognition. The costs of simulation development are reduced.

A use case involving integrating results from search engines illustrates how the meta-metadata language facilitates interoperable metadata semantics. Formal semantics can be hard to obtain directly. For example, search engines may only present results through web pages; even if they do provide web services, they don't provide them according to a mutually interoperable standard.

The adjustment of multidimensional abstract parameter spaces, used in human-in-the-loop systems such as simulations and visualizations, plays an important role for multi-touch interaction. We investigate new natural forms of interaction to manipulate such parameter spaces. We develop separable multi-touch interaction techniques for abstract parameter space manipulation. We investigate using the index and thumb to perform the often-repeated sub-task of switching between parameters. A user study compares these multi-touch techniques with mouse-based interaction, for the task of color selection, measuring performance and efficiency. Our findings indicate that multi-touch interaction techniques are faster than mouse based interaction.

Crisis response engenders a high-stress environment in which teams gather, transform, and mutually share information. Prior educational approaches have not successfully addressed these critical skills. The assumption has been that the highest fidelity simulations result in the best learning. Deploying high-fidelity simulations is expensive and dangerous; they do not address team coordination. Low-fidelity approaches are ineffective because they are not stressful. Zero-fidelity simulation develops and invokes the principle of abstraction, focusing on human-information and human-human transfers of meaning, to derive design from work practice. Our principal hypothesis is that crisis responders will experience zero-fidelity simulation as effective simulation of team coordination. We synthesize the sustained iterative design and evaluation of the Team Coordination Game. We develop and apply new experimental methods to show that participants learn to cooperate and communicate, applying what they learn in practice. Design implications address how to employ the abstraction principle to develop zero-fidelity simulations.

Collecting, organizing, and thinking about diverse information resources is the keystone of meaningful digital information experiences, from research to education to leisure. Metadata semantics are crucial for organizing collections, yet their structural diversity exacerbates problems of obtaining and manipulating them, strewing end users and application developers amidst the shadows of a proverbial tower of Babel. We introduce meta-metadata, a language and software architecture addressing a metadata semantics lifecycle: (1) data structures for representation of metadata in programs; (2) metadata extraction from information resources; (3) semantic actions that connect metadata to collection representation applications; and (4) rules for presentation to users. The language enables power users to author metadata semantics wrappers that generalize template-based information sources. The architecture supports development of independent collection representation applications that reuse wrappers. The initial meta-metadata repository of information source wrappers includes Google, Flickr, Yahoo, IMDb, Wikipedia, and the ACM Portal. Case studies validate the approach.

In this paper we outline the aims and organization of the ACM MM 10 workshop on 'Interactive Multimedia Computing for Creativity and Expression'.

We describe a new method of multi-touch sensing which can be unobtrusively added to existing displays. By coupling individually controlled optoelectronics to the edge of a planar waveguide, our scanning approach overcomes prior disadvantages of optoelectronic multi-touch sensing. Our approach allows for a completely transparent touch surface and easy integration with existing LCD displays.

The representation of information collections needs to be optimized for human cognition. While documents often include rich visual components, collections, including personal collections and those generated by search engines, are typically represented by lists of text-only surrogates. By concurrently invoking complementary components of human cognition, combined image-text surrogates will help people to more effectively see, understand, think about, and remember an information collection. This research develops algorithmic methods that use the structural context of images in HTML documents to associate meaningful text and thus derive combined image-text surrogates. Our algorithm first recognizes which documents consist essentially of informative and multimedia content. Then, the algorithm recognizes the informative sub-trees within each such document, discards advertisements and navigation, and extracts images with contextual descriptions. Experimental results demonstrate the algorithm's efficacy. An implementation of the algorithm is provided in combinFormation, a creativity support tool for collection authoring. The enhanced image-text surrogates enhance the experiences of users finding and collecting information as part of developing new ideas.

In order to evaluate the performance of information retrieval and extraction algorithms, we need test collections. A test collection consists of a set of documents, a clearly formed problem that an algorithm is supposed to provide solutions to, and the answers that the algorithm should produce when executed on the documents. Defining the association between elements in the test collection and answers is known as labeling. For mainstream information retrieval problems, there are publicly available test collections which have been maintained for years. However, the scope of these problems, and thus the associated test collections, is limited. In other cases, researchers need to build, label, and manage their own test collections, which can be a tedious and error-prone task. We were building test collections of HTML documents, for problems in which the answers that the algorithm supplies is a sub-tree of the DOM (Document Object Model). To lighten the burden of this task, we developed a test collection management and labeling system (TCMLS), to facilitate usability in the process of building test collections, applying them to validate algorithms, and potentially sharing them across the research community.

We take the work practices of fire emergency responders as the basis for developing simulations to teach team coordination. We introduce non-mimetic simulation: economic operational environments that represent human-centered components of practice, such as team structures and information flows, without mimicking concrete aspects of an environment. Emergent team coordination phenomena validate the non-mimetic simulation of fire emergency response.

As information environments grow in complexity, we yearn for simple interfaces that streamline human cognition and effort. Users need to perform complex operations on thousands of objects. Human attention and available screen real estate are constrained. We develop a new fluid interface component for the visualization and adjustment of values while authoring, the In-Context Slider, which reduces physical effort and demand on attention by using fluid mouse gestures and in-context interaction. We hypothesize that such an interface will make adjusting values easier for the user. We evaluated the In-Context Slider as an affordance for adjusting values of interest in text and images, compared with a more typical interface. Participants performed faster with the In-Context Slider. They found the new interface easier to use and more natural for expressing interest. We then integrated the In-Context Slider in the information composition platform, combinFormation. Participants experienced the In-Context Slider as easier to use while developing collections to answer open-ended information discovery questions. This research is relevant for many applications in which users provide ratings, such as recommender systems, as well as for others in which users' adjustment of values on concurrently displayed objects is integrated with extensive interactive functionality.

Semantic web researchers tend to assume that XML Schema and OWL-S are the correct means for representing the types, structure, and semantics of XML data used for documents and interchange between programs and services. These technologies separate information representation from implementation. The separation may seem like a benefit, because it is platform-agnostic. The problem is that the separation interferes with writing correct programs for practical document engineering, because it violates a primary principle of object-oriented programming: integration of data structures and algorithms. We develop an XML binding framework that connects Java object declarations with serialized XML representation. A basis of the framework is a metalanguage, embedded in Java object and field declarations, designed to be particularly concise, to facilitate the authoring and maintenance of programs that generate and manipulate XML documents. The framework serves as the foundation for a layered software architecture that includes meta-metadata descriptions for multimedia information extraction, modeling, and visualization; Lightweight Semantic Distributed Computing Services; interaction logging services; and a user studies framework.

Historically, the ACM Multimedia Conference is split into a "technical" program and an "arts" program. These programs sometimes seem completely separate from one another, victims of a "semantic gap" between disciplines. The goal of this panel is to create a space in which scientists learn from artists, and arts from science. We need to discover new connections between modalities of research. In order to create the most exciting and powerful future forms of interactive multimedia systems, the ones that will create the most beneficial broader impact on humanity, we need to foster new collaborations between artists and scientists. This panel seeks to bridge the great divide of language and communities that has fragmented us, creating a new space for developing connections between the arts and sciences of multimedia research, as embodied through the artists and scientists of ACM Multimedia. The goal is to make this conference a premier site for catalyzing emergent connections.

Although sometimes the task that motivates searching, browsing, and collecting information resources is finding a particular fact, humans often use information resources in intellectual and creative tasks that can include comparison, understanding, and discovery. Information discovery tasks involve not only finding relevant information but also seeing relationships among collected information resources and developing new ideas. The hypothesis presented here is that how information is represented impacts the magnitude of human creativity in information discovery tasks. How can we measure this creative cognition? Studies of search have focused on time and accuracy, metrics of limited value for measuring creative discovery. A new experimental method is developed, which measures the emergence of new ideas in information discovery, to evaluate the efficacy of representations. The efficacy of the typical textual list representation for information collections is compared with an alternative representation, combinFormation's composition of image and text surrogates. Representing collections with such compositions increases emergence in information discovery.

combinFormation is a mixed-initiative creativity support tool for searching, browsing, organizing, and integrating information. Images and text are connected to represent surrogates (enhanced bookmarks), optimizing the use of human cognitive facilities. Composition, an alternative to lists and spatial hypertext, is used to represent a collection of surrogates as a connected whole, using principles from art and design. This facilitates the creative process ofinformation discovery, in which humans develop new ideas while finding and collecting information. To provoke the user to think about the large space of potentially relevant information resources, a generative agent proactively engages in collecting information resources, forming image and text surrogates, and composing them visually. The agent develops the collection and its visual representation over time, enabling the user to see ideas and relationships. To keep the human in control, we develop interactive mechanisms for authoring the composition and directing the agent. In a field study in an interdisciplinary course on The Design Process, over a hundred students alternated using combinFormation and Google+Word to collect prior work on information discovery invention assignments. The students that used combinFormation's mixed-initiative composition of image and text surrogates performed better.

While sometimes the task that motivates searching, browsing, and collecting information resources is finding a particular fact, humans often engage in intellectual and creative tasks, such as comparison, understanding, and discovery. Information discovery tasks involve not only finding relevant information, but also seeing relationships among collected information resources, and developing new ideas. Prior studies of search have focused on time and accuracy, metrics of limited value for measuring creativity. We develop new experimental methods to evaluate the efficacy of representational systems for information discovery by measuring the emergence of new ideas. We also measure the variety of web sites that participants visit when engaging in a creative task, and gather experience report data. We compare the efficacy of the typical format for collections, the textual list with a new format, the composition of image and text surrogates. We conduct an experiment that establishes that representing collections with composition of image and text surrogates promotes emergence in information discovery.

Fire emergency response requires rapidly processing and communicating information to coordinate teams that protect lives and property. Students studying to become fire emergency responders must learn to communicate, process, and integrate information during dangerous, stressful, and time-sensitive work. We are performing an ethnographic investigation that includes interviews with experienced fire emergency responders and observations of team burn training exercises with students. We distill salient components of firefighting practice, which are relevant to the design of fire emergency response education systems. We derive design implications for systems that teach fire emergency responders to deal with issues surrounding the communication and integration of fireground information: the mixing of communication modalities, the distribution of information acquisition sources to create information differential and uncertainty, and audible clues.

A surrogate is an object that stands for a document and enables navigation to that document. Hypermedia is often represented with textual surrogates, even though studies have shown that image and text surrogates facilitate the formation of mental models and overall understanding. Surrogates may be formed by breaking a document down into a set of smaller elements, each of which is a surrogate candidate. While processing these surrogate candidates from an HTML document, relevant information may appear together with less useful junk material, such as navigation bars and advertisements.

combinFormation's mixed-initiative composition space enables system agents and humans to engage in processes of finding relevant information, and forming and authoring collections. Previously, the system was developed and utilized to support information discovery. The efficacy of the system for supporting creativity has been established in some contexts.

We are developing support for creativity in learning through information discovery and exploratory search. Users engage in creative tasks, such as inventing new products and services. The system supports evolving information needs. It gathers and presents relevant information visually using images and text. Users are able to search, browse, and explore results from multiple queries and interact with information elements by manipulating design and expressing interest. A field study was conducted to evaluate the system in an undergraduate class. The results demonstrated the efficacy of our system for developing creative ideas. Exposure to diverse information in visual and interactive forms is shown to support students engaged in invention tasks.

Access to devices that integrate Global Positioning data with image and sound acquisition becomes more common, enabling people to build large collections of locative multimedia. As the size and number of these locative media collections grow, so too does the importance of systems that support collection sensemaking. Media semantics, which include automatically acquired location data, as well as user-supplied annotations, play a key role in these user-centered processes of collection utilization. This demo presents a Zoomable Map Image Collection Sensemaking System that enables the collection, organization, browsing, and annotation of locative images. The Zoomable Map Perspective is supplemented by event-based clustering. Dynamic views are generated automatically from captured media. The system is currently being used to document the location and condition of homes and neighborhoods in the aftermath of Hurricane Katrina.

We used human movement as the basis for designing a collaborative aesthetic design environment. Our intention was to promote social interaction and creative expression. We employed off-the-shelf computer vision technology. Movement became the basis for the choreography of gestures, the development of gesture recognition, and the development of imagery and visualization. We discovered that the design of clear affordances is no less important in movement-based than in mouse-based systems. Through an integrated and iterative design process, we developed a new type of affordance, the choreographic button, which integrates choreography, gesture recognition, and visual feedback. Jumping, a quick movement, and crouching, a sustained gesture, were choreographed to form a vocabulary that is personally expressive, and which also facilitates automatic recognition.How can we evaluate socially motivated interactive systems? To create a context for evaluation, we held an integrated exhibition, party, and user study event. This mixing of events produced an engaging environment in which participants could choose to interact with each other, as well as with the design environment. We prepared a mouse-based version of the design environment, and compared how people experienced it with the movementbased system. Our study demonstrates that movement-based affordances promote social interaction.

This paper presents an iterative method for generative semantic clustering of related information elements in spatial hypertext documents. The goal is to automatically organize them in ways that are meaningful to the user. We consider a process in which elements are gradually added to a spatial hypertext. The method for generating meaningful layout is based on a quantitative model that measures and represents the mutual relatedness between each new element and those already in the document. The measurement is based on attributes such as metadata, term vectors, user interest expressions, and document locations. We call this model relatedness potential, because it represents how much the new element is related and thus attracted to existing elements as a vector field across the space. Using this field as a gradient potential, the new element will be placed near the most attracted elements, forming clusters of related elements. The relative magnitude of contribution of attributes to relatedness potential can be controlled through an interactive interface.Unlike prior clustering methods such as k-means and self-organizing-maps, relatedness potential works well in iterative systems, in which the collection of elements is not defined a priori. Further, users can invoke relatedness potential to re-cluster elements, as they engage in on-the-fly provisional acts of direct manipulation reorganization and latching of a few most significant elements. A preliminary study indicates that users find this method generates spatial hypertext documents that are easier to read.

In this paper, we describe Censor Chair, an art installation that creates a shared experience addressing forms of censorship including self-censorship, censorship of a group upon an individual, visual and auditory censorship in digital media, and censorship in society. We are taking a playful position in considering relationships between censorship and sensors that monitor physiology. Censor Chair makes use of a galvanic skin response (GSR) sensor, live video feeds, and a barcode reader to drive the presentation of a digital media library.

This paper describes an interactive installation that addresses issues of presence and absence by creating a virtualized representation of the abandoned town, Playas, New Mexico. This town is slated for conversion into an anti-terrorism training facility by New Mexico Tech University in conjunction with the United States Department of Homeland Security. Using the metaphor of the mirage, the work functions as a critique of our understanding of "reality.

This paper continues the movement from technology centered to human centered approaches in the study of tasks that involve finding, understanding, and using information, and tools that support these tasks. The iterative role of information as a stimulus to cognition is considered. The information discovery framework consists of a flowchart of connected human cognitive and digital computer states and processes. The purpose of the framework is to inform the design of tools for finding and using information. Divergent thinking laboratory tasks serve as an evaluation method.

combinFormation is a tool that enables browsing and collecting information elements in a generative space. By generative, we mean that the tool is an agent that automatically retrieves information elements and visually composes them. A combinFormation session presents a dynamic, evolving recombination of information elements from different sources. The elements are manipulable in the information space. Recombination is the process of taking previously unconnected elements, and combining them to create new configurations.One purpose of this space is to support the formation of ideas, through more and less focused processes of foraging. While ideas are forming, the criteria that underlie information foraging activities may not be well defined. Collecting the specific subset of related information elements is challenging. Cognitive scientists have established that combinations of images and textual elements are examples of preinventive structures that can lead to the emergence of new ideas. These preinventive structures often combine existing representations.Our program generates recombinant visualizations that develop interrelationships between the information elements. The generative visualization is based on a procedural model of the information, and the user's interests. The user model reflects interactions in which s/he explicitly expresses interest. The agent retrieves information based on the evolving model. The visual composition is also developed to emphasize the user's evolving sense of what is important. This involves solving problems in the dynamic visualization of dynamic, heterogeneous collections. In our novel interaction model, the human being shares control of the evolving information space with the agent. The user can express interest in information elements as they stream in, and design the visual space, using interactive tools. Expressions feed back through the model to drive the program's retrieval and visual composition decisions.

Existing CHI interaction models are focused on understanding the needs of users. They begin with user tasks and user feedback. While the involvement of users is critical for human computer interaction development, so is the imagination the artist-scientist-designer-developer. We constitute a legitimate source of impetus in the definition and development of interactive artifacts, as well as functioning as interpreters, measurers, and respondents. The development of interactivity is a fundamentally creative process. This paper distills a creative model for the development of interactivity as a residue of the CollageMachine development experience. Both the model and the artifact are components that contribute to the integrated approach of interface ecology.