Human-Computer Interaction practitioners have been proposing best practices in user interface design for decades. However, generative Artificial Intelligence (GenAI) brings additional design considerations and currently lacks sufficient user guidance regarding affordances, inputs, and outputs. In this context, we developed a recommender system to promote responsible AI (RAI) practices while people prompt GenAI systems. We detail 10 interviews with IT professionals, the resulting recommender system developed, and 20 user sessions with IT professionals interacting with our prompt recommendations. Results indicate that responsible prompting recommendations have the potential to support novice prompt engineers and raise awareness about RAI in prompting-time. They also suggest that recommendations should simultaneously maximize both a prompt’s similarity to a user’s input as well as a diversity of associated social values provided. These findings contribute to RAI by offering practical ways to provide user guidance and enrich human-GenAI interaction via prompt recommendations.

This talk invites reflection on essential human factors to consider when designing multimodal conversational user interfaces for positive social impact. Responsible AI has been a popular topic in academic and industry settings with the advent of conversational AI based on generative models in the last two years. Despite the growing scientific research in this field, what should be considered when designing for responsibility and social positive impact in conversational AI interactions is still being investigated. In this talk, I will revisit some of my projects to discuss the human values identified as essential to design responsible CUIs. Furthermore, we will discuss how bias emerged as a criterion when interacting with CUIs in museum settings, how we investigated accountability and trust embedded into financial advisors? chatbots, and our recent work into elucidating values such as creditworthiness with micro businesswomen in underrepresented communities using conversational systems. This talk can serve as the basis for further discussions during the conference on promoting social impact and mitigating harms when designing conversational multimodal systems with responsibility.

The increased popularity of CUIs has motivated HCI work around specific approaches to research, design, and implementation, while also reflecting on these topics. However, current research is highly fragmented and lacks critical mass around topics such as theory, methods and design. Building this critical mass is a fundamentally multidisciplinary endeavour. CUIs involve language based interaction, either through speech or text, with another agent(s) or device(s). This type of interaction not only needs to engage with traditional HCI approaches, but also to embrace methods from communicative and social sciences. This is crucial for making progress towards human-centred conversational interfaces. Along with the recent ACM SIGCHI Conversational User Interfaces conference (ACM CUI), this special issue showcases research to further solidify the foundations of the field in these areas. Below we outline some key challenges faced by the field, describe the papers in this special issue, and then outline areas for future research.

Conversational interfaces require two types of curation: data curation by data science workers and content curation by domain experts. Recent years have seen the possibilities for content curators to instruct conversational machines in the customer service domain (i.e., Machine Teaching). The activities of curating specialized data are time-consuming. These activities have a learning curve for the domain expert, and they rely on collaborators beyond the domain experts, including product owners, technology expert curators, management, marketing, and communication employees. However, recent research has looked at making this task easier for domain experts with a lack of knowledge in the Machine Learning system, and few papers have investigated the work practices and collaborations involved in this role. This paper aims to fill this gap, presenting and unveiling practices extracted from eleven semi-structured interviews and four design workshops with experts in Banking, Technical support, Humans Resources, Telecommunications, and Automotive sectors. First, we investigate the articulation work of the content curators and tech curators in training conversational machines. Second, we inspect the curatorial and collaboration strategies they use, which are not afforded by current conversational platforms. Third, we draw the design implications and possibilities to support individual and collaboration curating practices. We reflect on how those practices rely on self and collaboration with others for curation, trust, and data tracking and ownership.

Claudio Pinhanez, Paulo Cavalin, Victor Henrique Alves Ribeiro, Ana Appel, Heloisa Candello, Julio Nogima, Mauro Pichiliani, Melina Guerra, Maira de Bayser, Gabriel Malfatti, Henrique Ferreira. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

This paper describes how machine learning training data and symbolic knowledge from curators of conversational systems can be used together to improve the accuracy of those systems and to enable better curatorial tools. This is done in the context of a real-world practice of curators of conversational systems who often embed taxonomically-structured meta-knowledge into their documentation. The paper provides evidence that the practice is quite common among curators, that is used as part of their collaborative practices, and that the embedded knowledge can be mined by algorithms. Further, this meta-knowledge can be integrated, using neuro-symbolic algorithms, to the machine learning-based conversational system, to improve its run-time accuracy and to enable tools to support curatorial tasks. Those results point towards new ways of designing development tools which explore an integrated use of code and documentation by machines.

How does the presence of an audience influence the social interaction with a conversational system in a physical space? To answer this question, we analyzed data from an art exhibit where visitors interacted in natural language with three chatbots representing characters from a book. We performed two studies to explore the influence of audiences. In Study 1, we did fieldwork cross-analyzing the reported perception of the social interaction, the audience conditions (visitor is alone, visitor is observed by acquaintances and/or strangers), and control variables such as the visitor's familiarity with the book and gender. In Study 2, we analyzed over 5,000 conversation logs and video recordings, identifying dialogue patterns and how they correlated with the audience conditions. Some significant effects were found, suggesting that conversational systems in physical spaces should be designed based on whether other people observe the user or not.

How much do visual aspects influence the perception of users about whether they are conversing with a human being or a machine in a mobile-chat environment? This paper describes a study on the influence of typefaces using a blind Turing test-inspired approach. The study consisted of two user experiments. First, three different typefaces (OCR, Georgia, Helvetica) and three neutral dialogues between a human and a financial adviser were shown to participants. The second experiment applied the same study design but OCR font was substituted by Bradley font. For each of our two independent experiments, participants were shown three dialogue transcriptions and three typefaces counterbalanced. For each dialogue typeface pair, participants had to classify adviser conversations as human or chatbot-like. The results showed that machine-like typefaces biased users towards perceiving the adviser as machines but, unexpectedly, handwritten-like typefaces had not the opposite effect. Those effects were, however, influenced by the familiarity of the user to artificial intelligence and other participants' characteristics.

This study aims to develop design principles for outdoor mobile multimedia guides in cultural heritage.