Determining the relatedness of publications by detecting similarities and connections between researchers and their outputs can help science stakeholders worldwide to find areas of common interest and potential collaboration. To this end, many studies have tried to explore authorship attribution and research similarity detection through the use of automatic approaches. Nonetheless, inferring author research relatedness from imperfect data containing errors and multiple references to the same entities is a long-standing challenge. In a previous study, we conducted an experiment where a homogeneous crowd of volunteers contributed to a set of author name disambiguation tasks. The results demonstrated an overall accuracy higher than 75% and we also found important effects tied to the confidence level indicated by participants in correct answers. However, this study left many open questions regarding the comparative accuracy of a large heterogeneous crowd with monetary rewards involved. This paper seeks to address some of these unanswered questions by repeating the experiment with a crowd of 140 online paid workers recruited via MTurk’s microtask crowdsourcing platform. Our replication study shows high accuracy for name disambiguation tasks based on authorship-level information and content features. These findings can be of greater informative value since they also explore hints of crowd behavior activity in terms of time duration and mean proportion of clicks per worker with implications for interface and interaction design.

Visualizing and examining the intellectual landscape and evolution of scientific communities to support collaboration is crucial for multiple research purposes. In some cases, measuring similarities and matching patterns between research publication document sets can help to identify people with similar interests for building research collaboration networks and university–industry linkages. The premise of this work is assessing feasibility for resolving ambiguous cases in similarity detection to determine authorship with natural language processing (NLP) techniques so that crowdsourcing is applied only in instances that require human judgment. Using an NLP-crowdsourcing convergence strategy, we can reduce the costs of microtask crowdsourcing while saving time and maintaining disambiguation accuracy over large datasets. This article contributes a next-gen crowd-artificial intelligence framework that used an ensemble of term frequency-inverse document frequency and bidirectional encoder representation from transformers to obtain similarity rankings for pairs of scientific documents. A sequence of content-based similarity tasks was created using a crowd-powered interface for solving disambiguation problems. Our experimental results suggest that an adaptive NLP-crowdsourcing hybrid framework has advantages for inter-researcher similarity detection tasks where fully automatic algorithms provide unsatisfactory results, with the goal of helping researchers discover potential collaborators using data-driven approaches.

With the increasing development of human-AI teaming structures within and across geographies, the time is ripe for a continuous and objective look at the predictors, barriers, and facilitators of human-AI scientific collaboration from a multidisciplinary point of view. This paper aims at contributing to this end by exploiting a set of factors affecting attitudes towards the adoption of human-AI interaction into scientific work settings. In particular, we are interested in identifying the determinants of trust and acceptability when considering the combination of hybrid human-AI approaches for improving research practices. This includes the way as researchers assume human-centered artificial intelligence (AI) and crowdsourcing as valid mechanisms for aiding their tasks. Through the lens of a unified theory of acceptance and use of technology (UTAUT) combined with an extended technology acceptance model (TAM), we pursue insights on the perceived usefulness, potential blockers, and adoption drivers that may be representative of the intention to use hybrid intelligence systems as a way of unveiling unknown patterns from large amounts of data and thus enabling novel scientific discoveries.

Despite decades of research and development in named entity resolution, dealing with name ambiguity is still a challenging issue for many bibliometric-enhanced information retrieval (IR) tasks. As new bibliographic datasets are created as a result of the upward growth of publication records worldwide, more problems arise when considering the effects of errors resulting from missing data fields, duplicate entities, misspellings, extra characters, etc. As these concerns tend to be of large-scale, both the general consistency and the quality of electronic data are largely affected. This paper presents an approach to handle these name ambiguity problems through the use of crowdsourcing as a complementary means to traditional unsupervised approaches. To this end, we present “AuthCrowd”, a crowdsourcing system with the ability to decompose named entity disambiguation and entity matching tasks. Experimental results on a real-world dataset of publicly available papers published in peer-reviewed venues demonstrate the potential of our proposed approach for improving author name disambiguation. The findings further highlight the importance of adopting hybrid crowd-algorithm collaboration strategies, especially for handling complexity and quantifying bias when working with large amounts of data.

It has been a quarter of a century since the publication of the first edition of the IEEE International Conference on Computer Supported Cooperative Work in Design (CSCWD) held in 1996 in Beijing, China. Despite some attempts to empirically examine the evolution and identity of the field of CSCW and its related communities and disciplines, the scarcity of scientometric studies on the IEEE CSCWD research productivity is noteworthy. To fill this gap, this study reports on an exploratory quantitative analysis of the literature published in the IEEE CSCWD conference proceedings with the purpose of visualizing and understanding its structure and evolution for the 2001-2019 period. The findings offer valuable insights into the paper and author distribution, country and citation-level productivity indicators, degree of collaboration, and collaboration index. Through this analysis we also expect to get an initial overview of the IEEE CSCWD conference concerning the main topics being presented, most cited papers, and variances in the number of keywords, full-text views, and references.

With cutting edge scientific breakthroughs, human-centred algorithmic approaches have proliferated in recent years and information technology (IT) has begun to redesign socio-technical systems in the context of human-AI collaboration. As a result, distinct forms of interaction have emerged in tandem with the proliferation of infrastructures aiding interdisciplinary work practices and research teams. Concomitantly, large volumes of heterogeneous datasets are produced and consumed at a rapid pace across many scientific domains. This results in difficulties in the reliable analysis of scientific production since current tools and algorithms are not necessarily able to provide acceptable levels of accuracy when analyzing the content and impact of publication records from large continuous scientific data streams. On the other hand, humans cannot consider all the information available and may be adversely influenced by extraneous factors. Using this rationale, we propose an initial design of a human-AI enabled pipeline for performing scientometric analyses that exploits the intersection between human behavior and machine intelligence. The contribution is a model for incorporating central principles of human-machine symbiosis (HMS) into scientometric workflows, demonstrating how hybrid intelligence systems can drive and encapsulate the future of research evaluation.

The structure and evolution of a scientific research community can be quantitatively assessed taking into account the interactions between scientific agents dispersed geographically. In the recent years, CSCW has stabilized as a cross-disciplinary field suffering significant changes in its core structure, and there is limited understanding about the factors influencing the nature and progress of collaborative computing research. In this paper, we measure the correlation between a set of features related to the influence of collaboration types on the number of citations as well as the geographical distribution of the accumulated contribution to the CSCW literature. Overall, our work can represent a starting point to demonstrate how the study of scientific collaboration can partly explain the variations in the number of citations, frequency of papers, and topics addressed.

A substantial amount of work is often overlooked due to the exponential rate of growth in global scientific output across all disciplines. Current approaches for addressing this issue are usually limited in scope and often restrict the possibility of obtaining multidisciplinary views in practice. To tackle this problem, researchers can now leverage an ecosystem of citizens, volunteers and crowd workers to perform complex tasks that are either difficult for humans and machines to solve alone. Motivated by the idea that human crowds and computer algorithms have complementary strengths, we present an approach where the machine will learn from crowd behavior in an iterative way. This approach is embodied in the architecture of SciCrowd, a crowd-powered human-machine hybrid system designed to improve the analysis and processing of large amounts of publication records. To validate the proposal's feasibility, a prototype was developed and an initial evaluation was conducted to measure its robustness and reliability. We conclude this paper with a set of implications for design.

Crowdsourcing has shown to be a valuable problem-solving approach to handle the increasing complexity and scale of tasks for which the current AI algorithms are still struggling. Crowd intelligence can be particularly useful to train and supervise AI systems in a symbiotic, co-evolutionary relationship that raises long-term research challenges to the hybrid, crowd-computing design space. With the increase in the scale of mixed-initiative approaches, we need to gain a better understanding of the implications of crowd-powered systems as a scaffold for AI through the study of massive crowd-machine interactions. In this paper, we identify some open challenges and design implications for future crowd-AI hybrid systems. A framework is also proposed based on the practical challenges of addressing human-centered AI methods and processes.

Gestural interaction devices emerged and originated various studies on multimodal human–computer interaction to improve user experience (UX). However, there is a knowledge gap regarding the use of these devices to enhance learning. We present an exploratory study which analysed the UX with a multimodal immersive videogame prototype, based on a Portuguese historical/cultural episode. Evaluation tests took place in high school environments and public videogaming events. Two users would be present simultaneously in the same virtual reality (VR) environment: one as the helmsman aboard Vasco da Gama’s fifteenth-century Portuguese ship and the other as the mythical Adamastor stone giant at the Cape of Good Hope. The helmsman player wore a VR headset to explore the environment, whereas the giant player used body motion to control the giant, and observed results on a screen, with no headset. This allowed a preliminary characterisation of UX, identifying challenges and potential use of these devices in multi-user virtual learning contexts. We also discuss the combined use of such devices, towards future development of similar systems, and its implications on learning improvement through multimodal human–computer interaction.

Three dimensional virtual worlds (3DVW) have experienced a large growth in number of users, and are being used for collaboration activities. In parallel, the research field of Computer Supported Cooperative Work (CSCW) has developed taxonomies to classify systems that support collaboration. However, the CSCW perspective presents a bias towards traditional user interface paradigms, whose affordances are quite distinct from those of 3DVW, which include features such as the spatial environment, embodiment, and their dynamics. These are features which are regarded as significant factors in the research field of Presence, and yet, in our opinion, are not well appreciated from the perspective of CSCW analysis. Because of this, we question of the ability of CSCW taxonomies to properly describe the collaboration characteristics of 3DVW. By “properly”, we mean to say that 3DVW bring to fore collaboration characteristics that are in fact distinctive of them as collaboration tools, impacting collaboration in ways that are seldom found in usual groupware, and yet CSCW taxonomies do not distinguish them. We posit that these features should be contemplated in CSCW taxonomies and their usefulness taken into account in the development of future systems that aim to support collaboration.

In this paper we propose a model for assessing the quality of educational collaborative virtual environments. The objective is to establish a theoretical model that highlights a number of relevant sets of requirements in relation to the quality in educational collaborative virtual environments. It is intended to apply the model during the lifecycle of product development and in the selection of the environment in order to support the learning/teaching process.

With the goal of lessening barriers to the learning of advanced programming techniques, we put into place a trial which required students to get involved with online communities of programmers. Using a course assignment on software architecture styles, students had study a problem, find basis for a tentative approach, and discuss it online with programmers. The expectation was that students would find motivation for their studies from both the contact with communities of programmers, and from having to study and reflect upon their problem well enough to be able to draw the interest of members of those communities. We present the strategy we used, the developments and outcomes, and ideas for further application of this approach.

The paper presents a proposal high-level model for the development of Educational Collaborative virtual Environments based on engineering software and quality concepts for software development. A life cycle was identified, which was detailed the phases of development, taking into account different techniques and methods and related documentation. The main goal of this research is to demonstrate how we can use the model to develop our applications by virtual world's platforms. The model was conceived based on research carried out in area the development of models that integrate all phases of process development software and models for assessing collaborative virtual environments. The model used contains a set of diagrams to support developer teams in their tasks, mainly in the creation of educational collaborative virtual environments to be used in an e-learning context.

This paper presents two contributions: (i) a system architecture capable of staging platform-independent choreographies within different virtual worlds, and (ii) an ontology-based solution for capturing and representing multi-user choreographies with reduced time/effort. We argue that choreographies for virtual worlds should be clearly separated from the technical characteristics of their execution in virtual world technological platforms. Due to the heterogeneity of the various virtual worlds and their domain requirements, we propose exploiting the modularity, generality, and granularity dimensions of ontologies to simplify and empower the choreography modeling capabilities. Instead of a unique ontology, several ontologies with different levels of generality and granularity can be progressively combined to support the modeling requirements of a given choreography. Because these ontologies are aligned with the ontology of each specific virtual world platform, the mapping and transformation between the core ontology is simplified and automated, thus reducing the development and time-to-market.

In this demo, we present an application for mobile phones, which can allow communication between deaf and emergency medical services using an iconographic touch interface. This application can be useful especially for deaf but also for persons without disabilities that face sudden situations where speech is hard to articulate.

The production of scientific publications is a cooperative activity that usually involves several authors. Current solutions of cooperative editors do not contemplate features such as references management and publications deposit. In this context, the authors have previously developed the PaperFlow platform that integrates the three main functionalities required to produce scientific publications: a cooperative text editor, a cooperative reference manager and a connector to scientific digital repositories. However, this approach did not included tools for a previous important stage: the conception of the scientific publications. This paper proposes an evolution of the original platform to accommodate the conception stage. For this purpose, the specification of PaperFlow/R includes new functionalities such as whiteboards, communication tools and versioning control that improve the cooperation abilities of the platform.

Groupware specification and development has always been a complex task, requiring special attention to issues such as notification of cooperative actions and ensuring consistency of shared data. Some years ago SAGA was developed as a framework to build groupware applications based on a set of core Web services that provide the most common cooperative functionalities. Despite its potential, the last few years brought some technological developments that placed new challenges. This paper presents a new generation of the SAGA platform that adds to the original framework features that emerged recently, namely the regulation of social interaction, incorporation of new communication technologies, connectors to several external services and interaction environments, and the addition of contextual information.

Recent advances in wireless networks and mobile devices have brought about new scenes for the provision of services to end-users. Besides traditional services, new ones may be provided that transparently adjust and adapt to the user context. The user would have more choice and flexibility if, besides using the services, he could also compose his own services in an ad-hoc way. This paper presents iCas, an architecture to create context-aware services on the fly and discusses its main components. Also an application scenario is briefly described.