The traditional Information Retrieval (IR) evaluation framework—anchored in topical relevance and relevance‑based metrics—reflects a system-centred perspective. Yet for specific user groups, relevance alone is insufficient; benchmarking that relies exclusively on conventional metrics overlooks qualities intrinsic to the users IR approaches are meant to serve. Here, we draw attention to Children IR and examine the value of extending traditional evaluation with a human-centred perspective that accounts for how children interpret and evaluate information to more authentically capture performance and better reflect how well an approach truly meets children’s needs. Our empirical exploration using a child‑focused dataset, multiple ranking strategies, and traditional and extended frameworks reveals not only the limitations of relevance-based assessments but also the advantages of employing frameworks that are tailored to reflect the needs of child users, paving the way for more inclusive and effective evaluation frameworks.

Human input is pivotal in building reliable and robust artificial intelligence systems. By providing a means to gather diverse, high-quality, representative, and cost-effective human input on demand, microtask crowdsourcing marketplaces have thrived. Despite the unmistakable benefits available from online crowd work, the lack of health provisions and safeguards, along with existing work practices threatens the sustainability of this paradigm. Prior work has investigated worker engagement and mental health, yet no such investigations into the effects of crowd work on the physical health of workers have been undertaken. Crowd workers complete their work in various sub-optimal work environments, often using a conventional input modality of a mouse and keyboard. The repetitive nature of microtask crowdsourcing can lead to stress-related injuries, such as the well-documented carpal tunnel syndrome. It is known that stretching exercises can help reduce injuries and discomfort in office workers. Gestures, the act of using the body intentionally to affect the behavior of an intelligent system, can serve as both stretches and an alternative form of input for microtasks. To better understand the usefulness of the dual-purpose input modality of ergonomically-informed gestures across different crowdsourced microtasks, we carried out a controlled 2 x 3 between-subjects study (N=294). Considering the potential benefits of gestures as an input modality, our results suggest a real trade-off between worker accuracy in exchange for potential short to long-term health benefits.

In this paper, we take a step towards understanding how to design search engine results pages (SERP) that encourage children’s engagement as they seek for online resources. For this, we conducted a participatory design session to enable us to elicit children’s preferences and determine what children (ages 6–12) find lacking in more traditional SERP. We learned that children want more dynamic means of navigating results and additional ways to interact with results via icons. We use these findings to inform the design of a new SERP interface, which we denoted CHIRP. To gauge the type of engagement that a SERP incorporating interactive elements–CHIRP–can foster among children, we conducted a user study at a public school. Analysis of children’s interactions with CHIRP, in addition to responses to a post-task survey, reveals that adding additional interaction points results in a SERP interface that children prefer, but one that does not necessarily change engagement levels through clicks or time spent on SERP.

Music is an important part of childhood development, with online music listening platforms being a significant channel by which children consume music. Children’s offline music listening behavior has been heavily researched, yet relatively few studies explore how their behavior manifests online. In this paper, we use data from LastFM 1 Billion and the Spotify API to explore online music listening behavior of children, ages 6–17, using education levels as lenses for our analysis. Understanding the music listening behavior of children can be used to inform the future design of recommender systems.

We present the preliminary exploration we conducted to identify traits that can influence children’s preferences in books. Findings offer insights for the design of recommender algorithms that would look beyond patterns inferred from traditional user-system interactions (e.g., ratings) for recommendation purposes, since when it comes to children such data is rarely, if at all, available.