The use of gamified crowdsourcing mechanisms through serious games and games with a purpose (GWAPs) has emerged as an effective motivational strategy for enhancing performance in human intelligence tasks (HITs). In this systematic literature review, we examine the underlying characteristics of competitive and cooperative player-oriented GWAPs and how they can be leveraged to optimize crowdsourcing performance in completing batches of HITs. By exploring gamified crowdsourcing elements in GWAPs, we can evaluate the impact of these two types of player behaviors (i.e., competition and cooperation) on motivation and performance. We reviewed 27 publications and grouped them into five categories: player orientation, game elements and motivation, crowd work optimization, gamified knowledge collection, and comparative studies and best practices. Our research pinpoints the significance of intuitive task instructions, alignment of game elements with player motivations, and the role of competitive and cooperative dynamics in enhancing engagement and performance.

Online reviews are a crucial asset for e-commerce platforms as they provide consumers with valuable insights into products. It is important to note that these reviews are subjective and may contain biases. Therefore, it is essential to approach them with a critical eye. Despite this, online reviews remain a valuable tool for consumers when making purchasing decisions. This study focuses on developing web-based mini-games that target cognitive biases. The games are specifically designed to enhance the perception of e-commerce online reviews. A pilot study involving 85 participants was conducted to explore the potential of integrating these cognitive bias games into web platforms. The findings indicate promising avenues for leveraging these games to enhance cognitive personalization and improve the quality of e-commerce online reviews.

Large Language Models (LLMs) are widely used today in virtual assistants and content generation. However, there are suspicions that LLMs present confirmation bias, responding in a way that reinforces beliefs or assumptions embedded in users' questions, which can lead to erroneous decision-making, especially in sensitive areas such as healthcare. The objective of this research is to determine how often and under what conditions LLMs present confirmation bias and to identify the causes of this effect. The methodology involves conducting an experiment in which 52 biased healthcare questions are presented to 10 of the most popular models and analyzing whether their responses were biased. This work proves with statistical power the behavior of confirmation bias. We show that confirmation bias in LLMs occurs in all LLMs with a frequency of 20% to 60% of the occasions. The evidence suggests that the bias arises from the training database, the Transformer architecture itself, and the instructions in the fine-tuning phase by the companies behind the LLMs. This research explores pathways for the development of trustworthy LLMs.

Demand for ocean-based high-quality and sustainable fish protein soared in the last decade. Unlike precision agriculture, aquaculture remains an under-equipped farming activity. The aquaculture industry has provided remarkable contributions to the Sustainable Development Goal of zero hunger based on providing animal-based protein for human consumption worldwide. The success of the aquaculture industry hinges on appropriate monitoring of key water quality indicators to ensure both animal health and optimal productivity. In this context, the present work presents a cloud-based LoRaWAN system for quasi-real-time tracking of essential water quality parameters by integrating Internet of Things (IoT) sensor devices. The proposed approach harnesses the power of Long Range (LoRa) technology - especially the LoRa Wide Area Network (LoRaWAN) protocol - to facilitate efficient, large-scale monitoring focusing on data security and scalability. With practical insights drawn from IoT system deployment at an industrially relevant aquaculture farm in Brazil, this research provides a comprehensive look into the system's capabilities, drawbacks, and end-user feedback, offering a blueprint for future aquaculture innovations.

Determining the relatedness of publications by detecting similarities and connections between researchers and their outputs can help science stakeholders worldwide to find areas of common interest and potential collaboration. To this end, many studies have tried to explore authorship attribution and research similarity detection through the use of automatic approaches. Nonetheless, inferring author research relatedness from imperfect data containing errors and multiple references to the same entities is a long-standing challenge. In a previous study, we conducted an experiment where a homogeneous crowd of volunteers contributed to a set of author name disambiguation tasks. The results demonstrated an overall accuracy higher than 75% and we also found important effects tied to the confidence level indicated by participants in correct answers. However, this study left many open questions regarding the comparative accuracy of a large heterogeneous crowd with monetary rewards involved. This paper seeks to address some of these unanswered questions by repeating the experiment with a crowd of 140 online paid workers recruited via MTurk’s microtask crowdsourcing platform. Our replication study shows high accuracy for name disambiguation tasks based on authorship-level information and content features. These findings can be of greater informative value since they also explore hints of crowd behavior activity in terms of time duration and mean proportion of clicks per worker with implications for interface and interaction design.

When crowd workers provide their contributions in a shared working environment, they may be influenced by the inputs of other contributors in implicit ways. Stigmergy in crowdsourcing consists of tracking changes in work activities to guide crowd workers based on the digital traces left by other workers. In such scenarios, there is no direct communication between the contributors. Still, the traceable changes they left during their actions act as a mediating element that clearly affects the final work product. From a behavior analysis perspective, the properties recorded in event logs can be of practical value in observing the behavioral traces produced by crowd workers when performing microtasks. This form of task fingerprinting has been explored for over a decade to better understand performance-related data and user navigational behavior in crowdsourcing markets. In line with this, the goal of this paper is to study the feasibility of task fingerprinting alongside the stigmergic effect occurring in a crowdsourcing setting through a user event logger. To this end, a case study was conducted using a real-world scenario of extreme weather phenomena represented on interactive maps. Each user could observe the traces of other crowd members while providing annotations. Twelve experts in weather forecasting were recruited to participate in this study to annotate extreme weather events. The results indicate that it is possible to use task fingerprinting for tracking the stigmergic effect in such activities with gains in terms of implicit coordination. Furthermore, the task fingerprinting allowed to map participants with similar behavioral traces, suggesting an increase in the accuracy of annotation clusters.

Visualizing and examining the intellectual landscape and evolution of scientific communities to support collaboration is crucial for multiple research purposes. In some cases, measuring similarities and matching patterns between research publication document sets can help to identify people with similar interests for building research collaboration networks and university–industry linkages. The premise of this work is assessing feasibility for resolving ambiguous cases in similarity detection to determine authorship with natural language processing (NLP) techniques so that crowdsourcing is applied only in instances that require human judgment. Using an NLP-crowdsourcing convergence strategy, we can reduce the costs of microtask crowdsourcing while saving time and maintaining disambiguation accuracy over large datasets. This article contributes a next-gen crowd-artificial intelligence framework that used an ensemble of term frequency-inverse document frequency and bidirectional encoder representation from transformers to obtain similarity rankings for pairs of scientific documents. A sequence of content-based similarity tasks was created using a crowd-powered interface for solving disambiguation problems. Our experimental results suggest that an adaptive NLP-crowdsourcing hybrid framework has advantages for inter-researcher similarity detection tasks where fully automatic algorithms provide unsatisfactory results, with the goal of helping researchers discover potential collaborators using data-driven approaches.

Crowdsourcing has received considerable attention over the last fifteen years and has been the subject of several experiments that demonstrate its large potential for use in real-world situations. With the rapid growth of and access to crowd work environments, there is a need for new ways to ensure more equitable access for all people. Task design is one of the core aspects of the crowdsourcing process and its optimization is a priority for many requesters that want to have their tasks solved in short times and with high levels of accuracy. Aligned with this goal, a cognitive personalization framework can make it feasible to assess the information processing preferences of crowd workers in order to provide a useful user interface (UI) adaptation. In an effort to address this issue, this study recruited a total of 64 crowd workers to take cognitive style tests and perform prototypical tasks. The results indicate that it is possible to apply short tests and then obtain some useful indicators for better matching tasks to workers with implications for improving the general outcomes and acceptance rates in crowdsourcing.

Despite decades of research and development in named entity resolution, dealing with name ambiguity is still a challenging issue for many bibliometric-enhanced information retrieval (IR) tasks. As new bibliographic datasets are created as a result of the upward growth of publication records worldwide, more problems arise when considering the effects of errors resulting from missing data fields, duplicate entities, misspellings, extra characters, etc. As these concerns tend to be of large-scale, both the general consistency and the quality of electronic data are largely affected. This paper presents an approach to handle these name ambiguity problems through the use of crowdsourcing as a complementary means to traditional unsupervised approaches. To this end, we present “AuthCrowd”, a crowdsourcing system with the ability to decompose named entity disambiguation and entity matching tasks. Experimental results on a real-world dataset of publicly available papers published in peer-reviewed venues demonstrate the potential of our proposed approach for improving author name disambiguation. The findings further highlight the importance of adopting hybrid crowd-algorithm collaboration strategies, especially for handling complexity and quantifying bias when working with large amounts of data.

A large-scale urban sensing platform, composed of multiple Data Collection Units (DCUs) equipped with sensor hardware scattered accross the city, allows pervasive monitoring of environmental parameters. Gathering sensor data from a number of disparate locations at a backend server can be supported by delay-tolerant services provided by existing vehicular networks. Our real-world sensing platform takes advantage of an existing vehicular network with more than 400 vehicles equipped with On-Board Units (OBUs). A purposely-developed implementation of a delay tolerant service is installed in all elements involved in the communication flow, from DCUs to the backend server. In this demo, we showcase the full end-to-end data flow with the actual equipment being used in our real-world deployment. Data produced at a DCU is collected by an OBU installed in a vehicle and delivered to a Road-Side Unit (RSU), which then forwards the data to the backend server.