Human feedback plays a critical role in learning and refining reward models for text-to-image generation, but the optimal form the feedback should take for learning an accurate reward function has not been conclusively established. This paper investigates the effectiveness of fine-grained feedback which captures nuanced distinctions in image quality and prompt-alignment, compared to traditional coarse-grained feedback (for example, thumbs up/down or ranking between a set of options). While fine-grained feedback holds promise, particularly for systems catering to diverse societal preferences, we show that demonstrating its superiority to coarse-grained feedback is not automatic. Through experiments on real and synthetic preference data, we surface the complexities of building effective models due to the interplay of model choice, feedback type, and the alignment between human judgment and computational interpretation. We identify key challenges in eliciting and utilizing fine-grained feedback, prompting a reassessment of its assumed benefits and practicality. Our findings -- e.g., that fine-grained feedback can lead to worse models for a fixed budget, in some settings; however, in controlled settings with known attributes, fine grained rewards can indeed be more helpful -- call for careful consideration of feedback attributes and potentially beckon novel modeling approaches to appropriately unlock the potential value of fine-grained feedback in-the-wild.

In our era of rapid technological advancement, the research landscape for writing assistants has become increasingly fragmented across various research communities. We seek to address this challenge by proposing a design space as a structured way to examine and explore the multidimensional space of intelligent and interactive writing assistants. Through community collaboration, we explore five aspects of writing assistants: task, user, technology, interaction, and ecosystem. Within each aspect, we define dimensions and codes by systematically reviewing 115 papers, while leveraging the expertise of researchers in various disciplines. Our design space aims to offer researchers and designers a practical tool to navigate, comprehend, and compare the various possibilities of writing assistants, and aid in the design of new writing assistants.

There is a growing interest in extending crowdwork beyond traditional desktop-centric design to include mobile devices (e.g., smartphones). However, mobilizing crowdwork remains significantly tedious due to a lack of understanding about the mobile usability requirements of human intelligence tasks (HITs). We present a taxonomy of characteristics that defines the mobile usability of HITs for smartphone devices. The taxonomy is developed based on findings from a study of three consecutive steps. In Step 1, we establish an initial design of our taxonomy through a targeted literature analysis. In Step 2, we verify and extend the taxonomy through an online survey with Amazon Mechanical Turk crowdworkers. Finally, in Step 3 we demonstrate the taxonomy’s utility by applying it to analyze the mobile usability of a dataset of scraped HITs. In this paper, we present the iterative development of the taxonomy, highlighting the observed practices and preferences around mobile crowdwork. We conclude with the implications of our taxonomy for accessibly and ethically mobilizing crowdwork not only within the context of smartphone devices, but beyond them.

We present a design fiction, which is set in the near future as significant Mars habitation begins. Our goal in creating this fiction is to address current work-life issues on Earth and Mars in the future. With shelter-in-place measures, established norms of productivity and relaxation have been shaken. The fiction creates an opportunity to explore boundaries between work and life, which are changing with shelter-in-place and will continue to change. Our work includes two primary artifacts: (1) a propaganda recruitment poster and (2) a fictional narrative account. The former paints the work-life on Mars as heroic, fulfilling, and fun. The latter provides a contrast that depicts the lived experience of early Mars inhabitants. Our statement draws from our design fiction in order to reflect on the structure of work, stress identification and management, family and work-family communication, and the role of automation.

Crowdsourcing enables users (task requesters) to outsource complex tasks to an unspecified crowd of workers. To guarantee the quality of crowdsourcing service, it is necessary to select the most appropriate task workers to complete the tasks. To this end, the crowdsourcing platform (broker) must conduct the mutual matching between tasks and workers based on the task requirements and worker preferences. However, both task requirements and worker preferences may contain sensitive information (e.g., time, location of the task, etc.), which should not be revealed to the broker and other adversaries. In this paper, we propose a secure and efficient task matching scheme to enable the broker to conduct the mutual matching between tasks and workers, according to task requirements and worker preferences with multiple keywords, while preserving the privacy of keywords contained in task requirements and worker preferences. Specifically, we design a new multi-reader and multi-writer searchable encryption primitive that can support the batch matching of multiple keywords. The security proof shows that our proposed task matching scheme is provably secure in the random oracle model under the Bilinear Diffie-Hellman (BDH) assumption. The performance evaluation shows that our multi-keyword batch matching can significantly reduce the computation cost compared to existing methods.