As end users interact with increasing numbers of autonomous or semi-autonomous systems, collaboration and supervision become more complicated. People must simultaneously manage multiple systems, and most current interfaces do not scale. Augmented Reality (AR) offers a promising solution by placing information over the real world, allowing users to concurrently track the scene and robot(s) – potentially improving scaling for these devices. In this work, we leverage user gaze - a powerful indicator of attention suitable for reactive systems - to lower cognitive burden and improve performance such that scaling to multiple agents is possible. Gaze is probed in two modalities. In active mode, the user looks at a menu and presses a button to request additional information. The passive mode provides more information when the user’s gaze dwells on the menu. We performed two studies: 1) participants complete a visual search task with increasing numbers of virtual robotic agents and 2) participants must track the dynamic status of a team with physical agents. Results from the first study show that the passive and active interfaces provide better scaling compared to a non-interactive interface as the number of robots increases. In both studies, users preferred the passive mode, citing a lower mental demand, effort, and frustration.

Invasion of locust swarms has affected the crops in many countries in Africa and Asia, which is a significant threat to food security. Therefore, different approaches are adopted to monitor and control the locust swarms to save the crops. Furthermore, it has been proved in various studies that technology can help in agriculture through drones, real-time data monitoring, or teaching the farmers with the latest tools. Following the UN sustainability goals for food security, this research has presented a Virtual Reality(VR) based educational application to teach sustainable locust management strategies. Using hand tracking technology in the Oculus Quest lets users learn how farmers can deal with locusts without pesticides. Based on a storytelling approach, the methods presented are profitable for the farmers and free of any harm to crops regarding food security. This application can help motivate the adoption of these sustainable locust control strategies in broader interventions for environmental recovery.

People spend a significant amount of time trying to make sense of the internet, collecting content from a variety of sources and organizing it to make decisions and achieve their goals. While humans are able to fluidly iterate on collecting and organizing information in their minds, existing tools and approaches introduce significant friction into the process. We introduce Fuse, a browser extension that externalizes users’ working memory by combining low-cost collection with lightweight organization of content in a compact card-based sidebar that is always available. Fuse helps users simultaneously extract key web content and structure it in a lightweight and visual way. We discuss how these affordances help users externalize more of their mental model into the system (e.g., saving, annotating, and structuring items) and support fast reviewing and resumption of task contexts. Our 22-month public deployment and follow-up interviews provide longitudinal insights into the structuring behaviors of real-world users conducting information foraging tasks.

Tabs have become integral to browsing the Web yet have changed little since their introduction nearly 20 years ago. In contrast, the internet has gone through dramatic changes, with users increasingly moving from navigating to websites to exploring information across many sources to support online sensemaking. This paper investigates how tabs today are overloaded with a diverse set of functionalities and issues users face when managing them. We interviewed ten information workers asking about their tab management strategies and walk through each open tab on their work computers four times over two weeks. We uncovered competing pressures pushing for keeping tabs open (ranging from interaction to emotional costs) versus pushing for closing them (such as limited attention and resources). We then surveyed 103 participants to estimate the frequencies of these pressures at scale. Finally, we developed design implications for future browser interfaces that can better support managing these pressures.

While there is an enormous amount of information online for making decisions such as choosing a product, restaurant, or school, it can be costly for users to synthesize that information into confident decisions. Information for users' many different criteria needs to be gathered from many different sources into a structure where they can be compared and contrasted. The usefulness of each criterion for differentiating potential options can be opaque to users, and evidence such as reviews may be subjective and conflicting, requiring users to interpret each under their personal context. We introduce Mesh, which scaffolds users to iteratively build up a better understanding of both their criteria and options by evaluating evidence gathered across sources in the context of consumer decision-making. Mesh bridges the gap between decision support systems that typically have rigid structures and the fluid and dynamic process of exploratory search, changing the cost structure to provide increasing payoffs with greater user investment. Our lab and field deployment studies found evidence that Mesh significantly reduces the costs of gathering and evaluating evidence and scaffolds decision-making through personalized criteria enabling users to gain deeper insights from data.

Microtasks enable people with limited time and context to contribute to a larger task. In this paper we explore casual microtasking, where microtasks are embedded into other primary activities so that they are available to be completed when convenient. We present a casual microtasking experience that inserts writing microtasks from an existing microwriting tool into the user's Facebook feed. From a two-week deployment of the system with nine people, we observe that casual microtasking enabled participants to get things done during their breaks, and that they tended to do so only after first engaging with Facebook's social content. Participants were most likely to complete the writing microtasks during periods of the day associated with low focus, and would occasionally use them as a springboard to open the original document in Word. These findings suggest casual microtasking can help people leverage spare micromoments to achieve meaningful micro-goals, and even encourage them to return to work.

Whether figuring out where to eat in an unfamiliar city or deciding which apartment to live in, consumer generated data (i.e. reviews and forum posts) are often an important influence in online decision making. To make sense of these rich repositories of diverse opinions, searchers need to sift through a large number of reviews to characterize each item based on aspects that they care about. We introduce a novel system, SearchLens, where searchers build up a collection of "Lenses" that reflect their different latent interests, and compose the Lenses to find relevant items across different contexts. Based on the Lenses, SearchLens generates personalized interfaces with visual explanations that promotes transparency and enables deeper exploration. While prior work found searchers may not wish to put in effort specifying their goals without immediate and sufficient benefits, results from a controlled lab study suggest that our approach incentivized participants to express their interests more richly than in a baseline condition, and a field study showed that participants found benefits in SearchLens while conducting their own tasks.

Developers spend a significant portion of their time searching for solutions and methods online. While numerous tools have been developed to support this exploratory process, in many cases the answers to developers' questions involve trade-offs among multiple valid options and not just a single solution. Through interviews, we discovered that developers express a desire for help with decision-making and understanding trade-offs. Through an analysis of Stack Overflow posts, we observed that many answers describe such trade-offs. These findings suggest that tools designed to help a developer capture information and make decisions about trade-offs can provide crucial benefits for both the developers and others who want to understand their design rationale. In this work, we probe this hypothesis with a prototype system named Unakite that collects, organizes, and keeps track of information about trade-offs and builds a comparison table, which can be saved as a design rationale for later use. Our evaluation results show that Unakite reduces the cost of capturing tradeoff-related information by 45%, and that the resulting comparison table speeds up a subsequent developer's ability to understand the trade-offs by about a factor of three.

People engaged in complex searches such as planning a vacation or understanding their medical symptoms are often overwhelmed by opening and managing many tabs. These challenges are exacerbated as search moves to smartphones and mobile devices where screen real-estate is limited and tasks are frequently suspended, resumed, and interleaved. Rather than continue to utilize tab-based browsing for complex search, we introduce a new way of browsing through a scaffolded interface. The list of search results serves as a mutable workspace, where a user can track progress on a specific information query. The search query serves as a gateway into this workspace, accessed through a task-subtask hierarchy. We instantiate this in the Bento mobile search system and investigate its effectiveness in three studies. We find converging evidence that users were able to make progress on their complex searching tasks with this structure, and find it more organized and easier to revisit.

Crowdsourced clustering approaches present a promising way to harness deep semantic knowledge for clustering complex information. However, existing approaches have difficulties supporting the global context needed for workers to generate meaningful categories, and are costly because all items require human judgments. We introduce Alloy, a hybrid approach that combines the richness of human judgments with the power of machine algorithms. Alloy supports greater global context through a new "sample and search" crowd pattern which changes the crowd's task from classifying a fixed subset of items to actively sampling and querying the entire dataset. It also improves efficiency through a two phase process in which crowds provide examples to help a machine cluster the head of the distribution, then classify low-confidence examples in the tail. To accomplish this, Alloy introduces a modular "cast and gather" approach which leverages a machine learning backbone to stitch together different types of judgment tasks.

Crowdsourcing offers a powerful new paradigm for online work. However, real world tasks are often interdependent, requiring a big picture view of the difference pieces involved. Existing crowdsourcing approaches that support such tasks -- ranging from Wikipedia to flash teams -- are bottlenecked by relying on a small number of individuals to maintain the big picture. In this paper, we explore the idea that a computational system can scaffold an emerging interdependent, big picture view entirely through the small contributions of individuals, each of whom sees only a part of the whole. To investigate the viability, strengths, and weaknesses of this approach we instantiate the idea in a prototype system for accomplishing distributed information synthesis and evaluate its output across a variety of topics. We also contribute a set of design patterns that may be informative for other systems aimed at supporting big picture thinking in small pieces.

Patients researching medical diagnoses, scientist exploring new fields of literature, and students learning about new domains are all faced with the challenge of capturing information they find for later use. However, saving information is challenging on mobile devices, where the small screen and font sizes combined with the inaccuracy of finger based touch screens makes it time consuming and stressful for people to select and save text for future use. Furthermore, beyond the challenge of simply selecting a region of bounded text on a mobile device, in many learning and data exploration tasks the boundaries of what text may be relevant and useful later are themselves uncertain for the user. In contrast to previous approaches which focused on speeding up the selection process by making the identification of hard boundaries faster, we introduce the idea of intentionally supporting uncertain input in the context of saving information during complex reading and information exploration. We embody this idea in a system that uses force touch and fuzzy bounding boxes along with posthoc expandable context to support identifying and saving information in an intentionally uncertain way on mobile devices. In a two part user study we find that this approach reduced selection time and was preferred by participants over the default system text selection method.

Learning about a new area of knowledge is challenging for novices partly because they are not yet aware of which topics are most important. The Internet contains a wealth of information for learning the underlying structure of a domain, but relevant sources often have diverse structures and emphases, making it hard to discern what is widely considered essential knowledge vs. what is idiosyncratic. Crowdsourcing offers a potential solution because humans are skilled at evaluating high-level structure, but most crowd micro-tasks provide limited context and time. To address these challenges, we present Crowdlines, a system that uses crowdsourcing to help people synthesize diverse online information. Crowdworkers make connections across sources to produce a rich outline that surfaces diverse perspectives within important topics. We evaluate Crowdlines with two experiments. The first experiment shows that a high context, low structure interface helps crowdworkers perform faster, higher quality synthesis, while the second experiment shows that a tournament-style (parallelized) crowd workflow produces faster, higher quality, more diverse outlines than a linear (serial/iterative) workflow.