More-than-human design asks us to attend to non-human lifeworlds, yet human perception is overwhelmingly visual, making it difficult to engage with sensory realities unlike our own. Inspired by the star-nosed mole—a creature that navigates entirely through touch—we designed FeltSight, a mixed-reality wearable system that shifts the user's perceptual priority from vision to touch. The system pairs a visually-reduced MR headset with custom vibrotactile haptic gloves, suppressing visual dominance while extending tactile sensitivity beyond the skin, so that users encounter their surroundings through active, pre-contact probing rather than passive observation. This sensory inversion induces somatic defamiliarization, drawing on Haraway's “tentacular thinking” to invite users to approximate a tactile-first umwelt. We conceptualize this novel form of interaction as Touch Beyond Reach. Previously exhibited as an interactive artwork, this pictorial details the design process and implementation of the system.

Drawing on infrastructure studies in HCI and CSCW, this paper introduces Protocol Futuring, a methodological framework that extends design futuring by foregrounding protocols—rules, standards, and coordination mechanisms—as the primary material of speculative inquiry. Rather than imagining discrete future artifacts, Protocol Futuring examines how protocol rules accumulate drift, jam, and other second-order effects over long temporal horizons. We demonstrate the method through a case study of Knowledge Futurama, a multi-team participatory workshop exploring millennial-scale knowledge preservation. Using a relay format in which teams inherited and reinterpreted partially formed designs, the workshop revealed how ambiguous handovers, adversarial reinterpretations, shifting cultural norms, and crisis dynamics transform protocols as they move across communities and epochs. The case shows how Protocol Futuring makes infrastructural politics and long-run consequences analytically visible. We discuss the method’s strengths, limitations, and implications for researchers investigating emergent sociotechnical systems whose impacts unfold over extended timescales.

Mixed Reality (MR) increasingly explores how virtual elements can shape physical behavior, yet how MR objects guide group movement remains underexplored. We address this gap by examining how virtual objects can nudge collective, co-located movement without relying on explicit instructions or choreography. We developed GravField, a research-through-design, co-located MR performance system where an “object jockey” live-configures virtual objects (e.g., ropes, springs, magnetic fields) with real-time, parameterized “digital physics” (e.g., weight, elasticity, force) to influence headset-wearing participants’ movement, made perceptible through augmented visual and audio feedback serving as cognitive-somatic cues. Our bricolage analysis of the performances, based on video, interviews, soma trajectories, and field notes, indicates that these live nudges support emergent intercorporeal coordination and that ambiguity and real-time configuration sustain open-ended, exploratory engagement. Ultimately, our work offers empirical insights and design principles for MR systems that can guide group movement through embodied, felt dynamics while preserving participants’ sense of agency.

As see-through Mixed Reality Head-Mounted Displays (MRHMDs) proliferate, their usage is gradually shifting from controlled, private settings to spontaneous, public contexts. While location-based augmented reality mobile games such as Pokémon GO have been successful, the embodied interaction afforded by MRHMDs moves play beyond phone-based screen-tapping toward co-located, bodily, movement-based play. In anticipation of widespread MRHMD adoption, major technology companies have teased concept videos envisioning urban streets as vast mixed reality playgrounds-imagine Harry Potter-style wizard duels in city streets-which we term Immersive Mixed Reality Street Play (IMRSP). However, few real-world studies examine such scenarios. Through empirical, in-the-wild studies of our research-through-design game probe, Multiplayer Omnipresent Fighting Arena (MOFA), deployed across diverse public venues, we offer initial insights into the social implications, challenges, opportunities, and design recommendations of IMRSP. The MOFA framework, which includes three gameplay modes-''The Training'', ''The Duel'', and ''The Dragon''-is open-sourced at https://github.com/realitydeslab/mofa.

Remote sensing for observing and recording our surroundings is becoming mainstream. Technologies, such as light, detection, and ranging (LiDAR), are now part of consumer mobile devices and provide a variety of novel interaction opportunities with the environment. Mobile remote sensing also provides affordances for crowdsourcing through location-based applications such as games and gamified systems. While such use cases today are technologically feasible, there is a lack of understanding of how and what kinds of interactions and applications would be both (1) engaging and motivating for users and also (2) maximize the volume and quality of the data being gathered. In this study, we investigate these challenges by developing and testing four gamified augmented reality prototypes that use LiDAR for collecting point cloud data during location-based gaming. Through field testing, interviews, and surveys with 21 participants, followed by reflexive thematic analysis, we identified five themes of dynamics, which exemplify tensions and challenges to designing gamified AR crowdsourcing. The findings primarily point to hazards in design that may undermine user motivation as well as constraints of the environments themselves in facilitating and affording meaningful and rich (gameful) interaction.

Most competitive exertion games are highly reliant on visual cues, presenting a certain risk for visually impaired players. These individuals not only need to exert more effort and courage to participate in these games but also face a higher risk of injury. Additionally, during competition with sighted players, concerns about injuries may prevent both parties from fully enjoying the game, diminishing the fun for everyone involved. Although many sports games have been adapted for visually impaired players, these games often fail to engage sighted individuals or might be perceived as dull by them. This study introduces an archery exertion game called “Hearing the Bullseye", designed to provide a harmonious gaming environment for visually impaired players and their sighted family and friends. Utilizing bows equipped with infrared sensors, the game enables players to locate the invisible target through sound rather than sight. Our empirical research, involving 18 visually impaired and sighted participants, indicates that visually impaired players can quickly and safely master the game, ensuring a pleasant and friendly experience for all players.

Mobile phones equipped with a monocular camera and an inertial measurement unit (IMU) are ideal platforms for augmented reality (AR) applications, but the lack of direct metric distance measurement and the existence of aggressive motions pose significant challenges on the localization of the AR device. In this work, we propose a tightly-coupled, optimization-based, monocular visual-inertial state estimation for robust camera localization in complex indoor and outdoor environments. Our approach does not require any artificial markers, and is able to recover the metric scale using the monocular camera setup. The whole system is capable of online initialization without relying on any assumptions about the environment. Our tightly-coupled formulation makes it naturally robust to aggressive motions. We develop a lightweight loop closure module that is tightly integrated with the state estimator to eliminate drift. The performance of our proposed method is demonstrated via comparison against state-of-the-art visual-inertial state estimators on public datasets and real-time AR applications on mobile devices. We release our implementation on mobile devices as open source software1.

Click modeling aims to interpret the users' search click data in order to predict their clicking behavior. Existing models can well characterize the position bias of documents and snippets in relation to users' mainstream click behavior. Yet, current advances depict users' search actions only in a general setting by implicitly assuming that all users act in the same way, regardless of the fact that anyone, motivated with some individual interest, is more likely to click on a link than others. It is in light of this that we put forward a novel personalized click model to describe the user-oriented click preferences, which applies and extends matrix / tensor factorization from the view of collaborative filtering to connect users, queries and documents together. Our model serves as a generalized personalization framework that can be incorporated to the previously proposed click models and, in many cases, to their future extensions. Despite the sparsity of search click data, our personalized model demonstrates its advantage over the best click models previously discussed in the Web-search literature, supported by our large-scale experiments on a real dataset. A delightful bonus is the model's ability to gain insights into queries and documents through latent feature vectors, and hence to handle rare and even new query-document pairs much better than previous click models.

Modeling a user's click-through behavior in click logs is a challenging task due to the well-known position bias problem. Recent advances in click models have adopted the examination hypothesis which distinguishes document relevance from position bias. In this paper, we revisit the examination hypothesis and observe that user clicks cannot be completely explained by relevance and position bias. Specifically, users with different search intents may submit the same query to the search engine but expect different search results. Thus, there might be a bias between user search intent and the query formulated by the user, which can lead to the diversity in user clicks. This bias has not been considered in previous works such as UBM, DBN and CCM. In this paper, we propose a new intent hypothesis as a complement to the examination hypothesis. This hypothesis is used to characterize the bias between the user search intent and the query in each search session. This hypothesis is very general and can be applied to most of the existing click models to improve their capacities in learning unbiased relevance. Experimental results demonstrate that after adopting the intent hypothesis, click models can better interpret user clicks and achieve a significant NDCG improvement.

Recent advances in click model have positioned it as an effective approach to estimate document relevance based on user behavior in web search. Yet, few works have been conducted to explore the use of click model to help web search ranking. In this paper, we focus on learning a ranking function by taking the results from a click model into account. Thus, besides the editorial relevance data arising from the explicit manually labeled search result by experts, we also have the estimated relevance data that is automatically inferred from click models based on user search behavior. We carry out extensive experiments on large-scale commercial datasets and demonstrate the effectiveness of the proposed methods.

Recent advances in click models have positioned them as an effective approach to the improvement of interpreting click data, and some typical works include UBM, DBN, CCM, etc. After formulating the knowledge of user search behavior into a set of model assumptions, each click model developed an inference method to estimate its parameters. The inference method plays a critical role in terms of accuracy in interpreting clicks, and we observe that different inference methods for a click model can lead to significant accuracy differences. In this paper, we propose a novel Bayesian inference approach for click models. This approach regards click model under a unified framework, which has the following characteristics and advantages: