Abstract This special issue of the “Journal of the Association for Information Science and Technology” examines conceptual models as products of, and tools for, critical inquiry in Information Science (IS). The papers included in this issue present diverse perspectives on how conceptual models impact sociotechnical systems, spanning topics such as knowledge organization, representation, and information system design. Key themes include the intersection of model development with ethical considerations, the historical and future implications of conceptual modeling decisions, and the potential for conceptual models to address issues of power, representation, and justice in emerging technologies. This introduction situates the contributions within broader discussions of conceptual modeling in IS and highlights the field's unique approach to reflexive critique and sociotechnical analysis.

Abstract The increasing variety of research strategies and data collection techniques in information science, the access to large secondary data sets, and the ubiquity of information visualization call for expanding the classification of research methods and exploring how research is communicated visually. This study examined the relationship between types of data used in empirical research, visualizations, and research methods applied in information science studies. It analyzed 751 research articles published in the Journal of the Association for Information Science and Technology (JASIST) using content analysis and machine learning techniques. The study finds that most empirical studies adopted a quantitative design with data mining, bibliometrics, experiments, and surveys as dominant strategies. The substantial use of secondary data points to the shift in how data are collected in empirical research. The JASIST articles used a variety of visualizations to present research designs and findings, with quantitative and mixed methods studies employing primarily tables and charts and qualitative studies relying more on tables and diagrams. This study uniquely explores the relationship between research methods and visualization. It contributes to the classification of the methods in information science by expanding the range of strategies within the quantitative, qualitative, and mixed methods designs.

Abstract The emergence of large multi‐institutional digital libraries has opened the door to aggregate‐level examinations of the published word. Such large‐scale analysis offers a new way to pursue traditional problems in the humanities and social sciences, using digital methods to ask routine questions of large corpora. However, inquiry into multiple centuries of books is constrained by the burdens of scale, where statistical inference is technically complex and limited by hurdles to access and flexibility. This work examines the role that exploratory data analysis and visualization tools may play in understanding large bibliographic datasets. We present one such tool, HathiTrust+Bookworm, which allows multifaceted exploration of the multimillion work HathiTrust Digital Library, and center it in the broader space of scholarly tools for exploratory data analysis.

Music Information Retrieval (MIR) evaluation has traditionally focused on system‐centered approaches where components of MIR systems are evaluated against predefined data sets and golden answers (i.e., ground truth). There are two major limitations of such system‐centered evaluation approaches: (a) The evaluation focuses on subtasks in music information retrieval, but not on entire systems and (b) users and their interactions with MIR systems are largely excluded. This article describes the first implementation of a holistic user‐experience evaluation in MIR, the MIREX Grand Challenge, where complete MIR systems are evaluated, with user experience being the single overarching goal. It is the first time that complete MIR systems have been evaluated with end users in a realistic scenario. We present the design of the evaluation task, the evaluation criteria and a novel evaluation interface, and the data‐collection platform. This is followed by an analysis of the results, reflection on the experience and lessons learned, and plans for future directions.

Personalization is a way for computers to support people’s diverse interests and needs by providing content tailored to the individual. While strides have been made in algorithmic approaches to personalization, most require access to a significant amount of data. However, even when data is limited online crowds can be used to infer an individual’s personal preferences. Aided by the diversity of tastes among online crowds and their ability to understand others, we show that crowdsourcing is an effective on-demand tool for personalization. Unlike typical crowdsourcing approaches that seek a ground truth, we present and evaluate two crowdsourcing approaches designed to capture personal preferences. The first, taste-matching, identifies workers with similar taste to the requester and uses their taste to infer the requester’s taste. The second, taste-grokking, asks workers to explicitly predict the requester’s taste based on training examples. These techniques are evaluated on two subjective tasks, personalized image recommendation and tailored textual summaries. Taste-matching and taste-grokking both show improvement over the use of generic workers, and have different benefits and drawbacks depending on the complexity of the task and the variability of the taste space.

Crowdsourcing systems leverage short bursts of focused attention from many contributors to achieve a goal. By requiring people’s full attention, existing crowdsourcing systems fail to leverage people’s cognitive surplus in the many settings for which they may be distracted, performing or waiting to perform another task, or barely paying attention. In this paper, we study opportunities for low-effort crowdsourcing that enable people to contribute to problem solving in such settings. We discuss the design space for low-effort crowdsourcing, and through a series of prototypes, demonstrate interaction techniques, mechanisms, and emerging principles for enabling low-effort crowdsourcing.

Collections containing a large number of short documents are becoming increasingly common. As these collections grow in number and size, providing effective retrieval of brief texts presents a significant research problem. We propose a novel approach to improving information retrieval (IR) for short texts based on aggressive document expansion. Starting from the hypothesis that short documents tend to be about a single topic, we submit documents as pseudo-queries and analyze the results to learn about the documents themselves. Document expansion helps in this context because short documents yield little in the way of term frequency information. However, as we show, the proposed technique helps us model not only lexical properties, but also temporal properties of documents. We present experimental results using a corpus of microblog (Twitter) data and a corpus of metadata records from a federated digital library. With respect to established baselines, results of these experiments show that applying our proposed document expansion method yields significant improvements in effectiveness. Specifically, our method improves the lexical representation of documents and the ability to let time influence retrieval.