Representations of AI agents in user interfaces and robotics are predominantly White, not only in terms of facial and skin features, but also in the synthetic voices they use. In this paper we explore some unexpected challenges in the representation of race we found in the process of developing an U.S. English Text-to-Speech (TTS) system aimed to sound like an educated, professional, regional accent-free African American woman. The paper starts by presenting the results of focus groups with African American IT professionals where guidelines and challenges for the creation of a representative and appropriate TTS system were discussed and gathered, followed by a discussion about some of the technical difficulties faced by the TTS system developers. We then describe two studies with U.S. English speakers where the participants were not able to attribute the correct race to the African American TTS voice while overwhelmingly correctly recognizing the race of a White TTS system of similar quality. A focus group with African American IT workers not only confirmed the representativeness of the African American voice we built, but also suggested that the surprising recognition results may have been caused by the inability or the latent prejudice from non-African Americans to associate educated, non-vernacular, professionally-sounding voices to African American people.

Paulo Cavalin, Pedro Henrique Domingues, Claudio Pinhanez, Julio Nogima. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.

In this paper we discuss how AI can contribute to support the documentation and vitalization of Indigenous languages and how that involves a delicate balancing of ensuring social impact, exploring technical opportunities, and dealing with ethical constraints. We start by surveying previous work on using AI and NLP to support critical activities of strengthening Indigenous and endangered languages and discussing key limitations of current technologies. After presenting basic ethical constraints of working with Indigenous languages and communities, we propose that creating and deploying language technology ethically with and for Indigenous communities forces AI researchers and engineers to address some of the main shortcomings and criticisms of current technologies. Those ideas are also explored in the discussion of a real case of development of large language models for Brazilian Indigenous languages.

Claudio Pinhanez, Paulo Cavalin, Victor Henrique Alves Ribeiro, Ana Appel, Heloisa Candello, Julio Nogima, Mauro Pichiliani, Melina Guerra, Maira de Bayser, Gabriel Malfatti, Henrique Ferreira. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

This paper describes how machine learning training data and symbolic knowledge from curators of conversational systems can be used together to improve the accuracy of those systems and to enable better curatorial tools. This is done in the context of a real-world practice of curators of conversational systems who often embed taxonomically-structured meta-knowledge into their documentation. The paper provides evidence that the practice is quite common among curators, that is used as part of their collaborative practices, and that the embedded knowledge can be mined by algorithms. Further, this meta-knowledge can be integrated, using neuro-symbolic algorithms, to the machine learning-based conversational system, to improve its run-time accuracy and to enable tools to support curatorial tasks. Those results point towards new ways of designing development tools which explore an integrated use of code and documentation by machines.

Chatbots have received significant attention in the last years. These systems have improved operational efficiency by reducing the cost of customer service and more and more are used in customer service channels. More recently, with the addition of speech capabilities added to those systems, bot language modifications have to be done in order to improve user experience. In this paper, we analyze the language used in those systems using datasets collected from real chatbots. For that, we first propose models that are able to identify the language style (writing, speech, and computer-mediated) using as linguist features syntactic (Biber's dimensions) and sentence embeddings as well as state-of-art datasets. Our results show that our models were able to distinguish among the three classes with an accuracy of up to 66%. Finally, we evaluate real chatbots systems, either speech- and text-based ones, using our proposed models. We found that these real chatbots are generally using a computer-mediated style, but there is indication that their developers tend to adapt the language style to be more speech-like when text-to-speech is used.

Video transmission in multiparty video conferencing is challenging due to the demanding bandwidth usage and stringent latency requirement. In this paper, we systematically analyze the problem of collaborative stream routing using one-hop forwarding assistance in a bandwidth constraint environment. We model the problem as a multi-source degree-constrained multicast tree construction problem, and investigate heuristic algorithms to construct bandwidth-feasible shared multicast trees. The contribution of this work is primarily two-fold: (1) we study the solution space of finding a feasible bandwidth configuration for stream routing in a peer-to-peer (P2P) setting, and propose two heuristic algorithms that can quickly produce a bandwidth-feasible solution, making them suitable for large-scale conference sessions, (2) we conduct an empirical study using a realistic dataset and show the effectiveness of our heuristic algorithms. Various QoS metrics are taken into account to evaluate the performance of our algorithms. Finally, we discuss open issues for further exploration. The feasibility study presented in this paper will shed light on the design and implementation of practical P2P multiparty video conferencing applications.

Motivated by the problem of limited bandwidth in peer-to-peer (P2P) multi-party video conferencing systems, in this paper we propose a non-intrusive adaptive multi-media routing algorithm that effectively calculates stream routing to achieve a maximum number of receiving streams. The technique is non-intrusive in that it makes use of current streaming status to infer link bottlenecks rather than sending active probing packets, which would seriously interfere with the latency-sensitive video conferencing application and waste bandwidth. When link bottlenecks are detected, the method will adaptively calculate streaming routes, allowing bandwidth abundant peers to act as relays. To test the performance, we use real data from a world wide bandwidth distribution archive and investigate the algorithm convergence rate and distribution fairness through simulation. Results show that the technique works well to achieve effective multi-media routing for latency-sensitive video conferencing applications.