The adoption of large language models (LLMs) as rerankers in multi-stage retrieval systems has gained significant traction in academia and industry. These models refine a candidate list of retrieved documents, often through carefully designed prompts, and are typically used in applications built on retrieval-augmented generation (RAG). This paper introduces RankLLM, an open-source Python package for reranking that is modular, highly configurable, and supports both proprietary and open-source LLMs in customized reranking workflows. To improve usability, RankLLM features optional integration with Pyserini for retrieval and provides integrated evaluation for multi-stage pipelines. Additionally, RankLLM includes a module for detailed analysis of input prompts and LLM responses, addressing reliability concerns with LLM APIs and non-deterministic behavior in Mixture-of-Experts (MoE) models. This paper presents the architecture of RankLLM, along with a detailed step-by-step guide and sample code. We reproduce results from RankGPT, LRL, RankVicuna, RankZephyr, and other recent models. RankLLM integrates with common inference frameworks and a wide range of LLMs. This compatibility allows for quick reproduction of reported results, helping to speed up both research and real-world applications. The complete repository is available at rankllm.ai, and the package can be installed via PyPI.

The rapid development and deployment of “smart space” technology in commercial and educational buildings has brought efficiencies and comfort to users. Yet, the “smart” is largely limited to knowledge gained from occupancy and activity data. These limited data explain what is happening in a space but nothing about how effectively the space is being used. In this work, we explore how to understand effective space use and how it may impact future smart spaces and their design. As part of this investigation, we design generalizable heuristics derived from observable human behaviors and space usage to detect group configurations in public spaces. The resultant system models similar qualities of a space that end-users will judge and measure themselves before use. The effectiveness of our system is validated against a crowdsourced study. Our system serves as a foundation for applications building upon improving space use efficacy.

This paper considers the scenario in which there are multiple institutions, each with a limited capacity for candidates, and candidates, each with preferences over the institutions. A central entity evaluates the utility of each candidate to the institutions, and the goal is to select candidates for each institution in a way that maximizes utility while also considering the candidates’ preferences. The paper focuses on the setting in which candidates are divided into multiple groups and the observed utilities of candidates in some groups are biased–systematically lower than their true utilities. The first result is that, in these biased settings, prior algorithms can lead to selections with sub-optimal true utility and significant discrepancies in the fraction of candidates from each group that get their preferred choices. Subsequently, an algorithm is presented along with proof that it produces selections that achieve near-optimal group fairness with respect to preferences while also nearly maximizing the true utility under distributional assumptions. Further, extensive empirical validation of these results in real-world and synthetic settings, in which the distributional assumptions may not hold, are presented.

A fundamental paradigm shift for in-home sensing is apparent. Special-purpose sensing, where there is a one-to-one relationship between sensors and applications, is evolving into general-purpose sensing, where there is a many-to-many relationship between sensors and applications. This new shift may impact how individuals think about in-home sensing, where utility and comfort are often linked to applications rather than sensed data. We explore the evolution of individuals' perceptions as they become increasingly and contextually aware of sensor capabilities and data characteristics. Through a multi-phase study where 12 participants were progressively led through six exposure conditions across laboratory and home environments, we find that exposure changes represent inflection points for perceptions of utility and comfort with data collection. These changes define opportunities for increasing trust in sensing infrastructures via data- and context-aware interventions, managing over-reliance on awareness notifications, and providing data-enabled "what if" analyses to balance comfort and utility within an individual's unique context and environment.

Low-cost sensors have enabled a wide array of data-driven applications and insights. As a result, encountering spaces with pervasive sensing has become all but unavoidable. This creates a fundamental tension: the success of smart environments will become increasingly dependent on equity of access to data-driven insights and consideration of the privacy expectations of sensed individuals. These concerns highlight the need to bring equity to all stakeholders of smart environments, which in turn would preserve public trust in these smart spaces. In this work, we explored several approaches to identity-obscuring visual representations through a progressive series of experiments. We designed and validated a series of visual representations through stakeholder interactions and tested the ability of these visual representations to limit identification via a crowdsourced study. An evaluation across three months of data gathered within our organization also showed that the identity-obscured data could still be leveraged to accurately count group size. Our contributions lay the groundwork for sensing frameworks that bring utility to all stakeholders of shared spaces while being cognizant of their diverse privacy expectations.