In this pictorial, we explore alternative ways of knowing urban trees through a more-than-human lens. Using a municipal tree dataset, we focus on “unknown” trees—entries unclassified due to error, decay, or absence—highlighting the limits of quantification and fixed knowledge systems. Urban trees, while critical for ecosystems, are often shaped by technological interventions (e.g., GIS, IoT sensors, AI diagnostics) that prioritize their utility over other expressions. We engage in knitting as a material inquiry to foreground nonhuman agencies and relational entanglements. Through reflective shifts and compromises, this project questions normative design practices, seeking to amplify nonhuman participation. We make two contributions. Firstly, we offer insights into fostering alternative, relational engagements with urban ecologies. Secondly, we reflect on our process of surfacing and working with agentic capacities, articulating guidance for other design researchers. Through this, we advocate for fragmented approaches that embrace complicity and complexity in more-than-human design.

Textbook reading has long dominated classical poetry education in Chinese-speaking communities. However, research has shown that extensive text-based learning can lead to learner disengagement and a less pleasant experience. This paper aims to improve the experience of classical Chinese poetry learning by introducing LivePoem—a system that generates musical storyboards (storyboards with background music) as audiovisual aids to support poetry comprehension. We employ a pre-trained diffusion model for storyboard generation and train a prosody-based poem-to-melody generator using a Transformer model, both validated by standard objective metrics to ensure generation quality. Through a within-subjects study involving 25 non-native Chinese learners, we compared learning outcomes from textbook reading and musical storyboard viewing through standardised reading comprehension tests. Additionally, the learning experience was assessed by the Self-Assessment Manikin (SAM) and an inductive thematic analysis of learners' open-ended feedback. Experimental results show that musical storyboards retained the learning outcomes of textbooks, while more effectively engaging learners and providing a more pleasant learning experience.

We present a unified framework for automatic multitrack music arrangement that enables a single pre-trained symbolic music model to handle diverse arrangement scenarios, including reinterpretation, simplification, and additive generation. At its core is a segment-level reconstruction objective operating on token-level disentangled content and style, allowing for flexible any-to-any instrumentation transformations at inference time. To support track-wise modeling, we introduce REMI-z, a structured tokenization scheme for multitrack symbolic music that enhances modeling efficiency and effectiveness for both arrangement tasks and unconditional generation. Our method outperforms task-specific state-of-the-art models on representative tasks in different arrangement scenarios---band arrangement, piano reduction, and drum arrangement, in both objective metrics and perceptual evaluations. Taken together, our framework demonstrates strong generality and suggests broader applicability in symbolic music-to-music transformation.

In-person human interaction relies on our spatial perception of each other and our surroundings. Current remote communication tools partially address each of these aspects. Video calls convey real user representations but without spatial interactions. Augmented and Virtual Reality (AR/VR) experiences are immersive and spatial but often use virtual environments and characters instead of real-life representations. Bridging these gaps, we introduce DualStream, a system for synchronous mobile AR remote communication that captures, streams, and displays spatial representations of users and their surroundings. DualStream supports transitions between user and environment representations with different levels of visuospatial fidelity, as well as the creation of persistent shared spaces using environment snapshots. We demonstrate how DualStream can enable spatial communication in real-world contexts, and support the creation of blended spaces for collaboration. A formative evaluation of DualStream revealed that users valued the ability to interact spatially and move between representations, and could see DualStream fitting into their own remote communication practices in the near future. Drawing from these findings, we discuss new opportunities for designing more widely accessible spatial communication tools, centered around the mobile phone.

Augmented Reality (AR), with its ability to make people feel like they are in the same space as friends from across the world, is an ideal medium for the purpose of Networked Musical Collaboration. Most conventional systems that enable networked musical collaboration minimize network latency by focusing on the transfer of auditory information at the expense of visual feedback. Studies into human perception have shown that sensory integration of audio and visual stimuli can take place even when there is a slight delay between the two signals. We studied the way changes in network latency effect participants’ auditory and visual perception in latency detection, latency tolerance and attention focus; in this paper, we explore the trade-off between the presence of AR visuals and the minimization of latency. Twenty-four participants were asked to play a hand drum and collaborate with a prerecorded remote musician rendered as an avatar in AR. Multiple trials involving different levels of audio-visual latency were conducted. We then analyzed the subjective responses of the participants together with the recorded musical information from each session. Results indicate a minimum noticeable delay value–defined as the highest amount of delay that can be experienced before two stimulated senses are perceived as separate events–between 160 milliseconds (ms) and 320 ms. Players also reported that a delay between sound and an accompanying avatar animation became less tolerable at 320 ms of delay, but was never completely intolerable, even up to 1200 ms of delay. We conclude that players begin to notice delay at about 320 ms and most players can tolerate large delays between sound and animation.

The Jam Station is a collaborative musical experience (CME), with embedded sensors to detect, evaluate, and reinforce collaboration among players of all skill levels. The system is also gamified, consisting of four instruments that face a centrally located, vertically oriented, display that visualizes various game states. Playing any of the instruments affect the game as musical collaboration is assessed and visualized on the display. The display also includes a “progress bar’’ which fills up the more players collaborate musically, and to “win’’ the game, the progress bar must be full, initiating a multi-color light show for the players. By creating an environment focused primarily on collaboration, we aim to support collaborative music-making for musicians of all skill levels.