Scientometrics is undergoing a methodological transformation driven by the increasing availability of large-scale scientific data and advances in artificial intelligence (AI). Traditional approaches centered on citation analysis and bibliographic coupling are now complemented by methods that leverage semantic representations, structured knowledge, and natural language understanding. However, current generative AI systems pose inherent challenges such as hallucinations, lack of transparency in decision-making and explainability, and issues with source reliability. In this article, we seek to mitigate these challenges by outlining a framework that integrates knowledge graphs (KGs) and large language models (LLMs) for scientometric inquiry. Taking a socio-technical perspective, the article sets out to explore the intersectional space of computing and information science from a human-centered approach. The framework is designed to support both established scientometric tasks, such as trend analysis, topic detection, and collaboration mapping, and more exploratory, insight-generating applications, including question answering, knowledge discovery, and contextual enrichment of scientific content. Drawing on recent developments in the use of KGs and LLMs in scientific domains, we provide a comparative overview of existing work, identify key design principles, and discuss the advantages and limitations of such a framework. Our goal is to chart a pathway toward more interactive, transparent, and generative approaches in information science and cross-disciplinary research. • Multidimensional mapping from large bibliographic datasets remains challenging. • A KG-LLM framework is proposed to support interactive scientometric inquiry. • HITL validation addresses the black-box limitations of KG- and LLM-enabled solutions. • Implications for knowledge-based question answering are derived.

In this paper we present findings from a bibliometric evaluation of scientific publications on human-AI systems, indexed in the Dimensions database over the past five years (2018 to 2022). The study maps the research landscape in this burgeoning area, as it relates to the topic of collaboration. To this end, we assessed publication and citation counts over time, authorship-level indicators, and keyword occurrence frequency. We also examined funding information as an indicator of research priorities, alongside usage-based statistics and alternative metrics such as social media mentions, recommendations, and reads. Our preliminary findings highlight a significant focus on aspects like trust, explainability, transparency, and autonomy in highly complex scenarios through the use of generative models and hybrid interaction techniques. The results also reveal a growth in the number of publications and funding grants, although a certain lack of maturity is observable in terms of citation patterns and coherence of thematic clusters.

With the increasing development of human-AI teaming structures within and across geographies, the time is ripe for a continuous and objective look at the predictors, barriers, and facilitators of human-AI scientific collaboration from a multidisciplinary point of view. This paper aims at contributing to this end by exploiting a set of factors affecting attitudes towards the adoption of human-AI interaction into scientific work settings. In particular, we are interested in identifying the determinants of trust and acceptability when considering the combination of hybrid human-AI approaches for improving research practices. This includes the way as researchers assume human-centered artificial intelligence (AI) and crowdsourcing as valid mechanisms for aiding their tasks. Through the lens of a unified theory of acceptance and use of technology (UTAUT) combined with an extended technology acceptance model (TAM), we pursue insights on the perceived usefulness, potential blockers, and adoption drivers that may be representative of the intention to use hybrid intelligence systems as a way of unveiling unknown patterns from large amounts of data and thus enabling novel scientific discoveries.

With cutting edge scientific breakthroughs, human-centred algorithmic approaches have proliferated in recent years and information technology (IT) has begun to redesign socio-technical systems in the context of human-AI collaboration. As a result, distinct forms of interaction have emerged in tandem with the proliferation of infrastructures aiding interdisciplinary work practices and research teams. Concomitantly, large volumes of heterogeneous datasets are produced and consumed at a rapid pace across many scientific domains. This results in difficulties in the reliable analysis of scientific production since current tools and algorithms are not necessarily able to provide acceptable levels of accuracy when analyzing the content and impact of publication records from large continuous scientific data streams. On the other hand, humans cannot consider all the information available and may be adversely influenced by extraneous factors. Using this rationale, we propose an initial design of a human-AI enabled pipeline for performing scientometric analyses that exploits the intersection between human behavior and machine intelligence. The contribution is a model for incorporating central principles of human-machine symbiosis (HMS) into scientometric workflows, demonstrating how hybrid intelligence systems can drive and encapsulate the future of research evaluation.