Jaechang Kim, Jinmin Goh, Inseok Hwang, Jaewoong Cho, Jungseul Ok. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

In social VR, bodily motions are a major nonverbal channel for expressing intent or emotion. However, freely making bodily motions is not always possible due to unaffordability of rich tracking devices, physical disabilities, or social/spatial constraints. While current social VR platforms provide methods like emotes, expressions are limited to a finite preset. To facilitate open-ended and socially-aligned motion in constrained environments, our insight is peer-relativity found in everyday interaction. Specifically, we propose ArithMotion, an end-to-end system to generate peer-relative motions by combining generative models with arithmetic-inspired interaction. We fully implemented and iteratively refined the system. User studies show participants experienced novel, open-ended expressions closely tied to social context.

Children acquire language by interacting with their surroundings. Due to the different language environments each child is exposed to, the words they encounter and need in their life vary. Despite the standard tools for assessment and intervention as per predefined vocabulary sets, speech-language pathologists and parents struggle with the absence of systematic tools for child-specific custom vocabulary, i.e., out-of-standard but personally more important. We propose “Open Sesame? Open Salami! (OSOS)”, a personalized vocabulary assessment and intervention system with pervasive language profiling and targeted storybook generation, collaboratively developed with speech-language pathologists. Melded into a child’s daily life and powered by large language models (LLM), OSOS profiles the child’s language environment, extracts priority words therein, and generates bespoke storybooks naturally incorporating those words. We evaluated OSOS through 4-week-long deployments to 9 families. We report their experiences with OSOS, and its implications in supporting personalization outside standards.

The prevalence of counterfeit infant formulas worldwide poses serious threats to infant health and safety, a concern highlighted by the notorious Melamine Milk Scandal that affected hundreds of thousands of children. The primary challenge in detecting counterfeit formulas lies in their sophisticated adulteration and substitution techniques. Such detection is feasible only in laboratory settings, making it nearly impossible for average consumers to test the formula before feeding their infants. To address this problem, we propose PowDew, a novel and practical system for detecting counterfeit infant formula that utilizes only a commodity smartphone. PowDew operates by capturing and analyzing the interaction of a water droplet with the powdered formula, focusing on the droplet motion, namely its spreading and penetration. Our insight is that the droplet motions are governed by powder-specific properties such as wettability and porosity. PowDew analyzes the subtle differences in droplet motions, and infers the formula's authenticity. To demonstrate PowDew's effectiveness, we implement PowDew and conduct comprehensive real-world experiments under varying conditions with different brands of powdered infant formula and adulterants. Our experiments result in a total of 12,000 minutes of video recordings of the droplet motions on various infant formulas, including authentic and altered. Our experiments demonstrate that PowDew yields an overall detection accuracy of up to 96.1%.

The rise of counterfeit powdered food products, exemplified by notorious incidents such as the Melamine Milk Scandal, poses significant risks to consumers. The primary challenge in identifying these counterfeit products comes from their intricate adulteration and substitution techniques. Currently, such identification methods are only viable in laboratory settings, making average consumers nearly impossible to authenticate their products. To address this limitation, we propose PowDew, a novel system that employs a smartphone to detect counterfeit powdered food products. PowDew utilizes the powder's physical property, namely droplet motion, as a basis for verification. Through real-world experiments, PowDew demonstrate a practicality with achieving an overall detection accuracy of up to 96.1%.

We present LiLoc , a system for precise 3D localization and tracking of mobile IoT devices (e.g., robots) in indoor environments using multi-perspective LiDAR sensing. LiLoc stands out with two key differentiators. First, unlike traditional localization approaches, our method remains robust in dynamically changing environments, adeptly handling varying crowd levels and object layout changes. Second, LiLoc is independent of pre-built static maps, employing dynamically updated point clouds from infrastructural-mounted LiDARs and LiDARs on individual IoT devices. For fine-grained, near real-time tracking, LiLoc intermittently utilizes complex 3D “global” registration between point clouds for robust spot location estimates. It further complements this with simpler “local” registrations, continuously updating IoT device trajectories. We demonstrate that LiLoc can (a) support accurate location tracking with location and pose estimation error being ≦7.4 cm and ≦3.2°, respectively, for 84% of the time and the median error increasing only marginally (8%), for correctly estimated trajectories, when the ambient environment is dynamic; (b) achieve a 36% reduction in median location estimation error compared to an approach that uses only quasi-static global point cloud; and (c) obtain spot location estimates with a latency of only 973 msec. We also demonstrate how LiLoc efficiently integrates low-power inertial sensing, using a novel integration of inertial-based displacement to accelerate the local registration process, to enhance localization energy efficiency and latency.

This work demonstrates the VGGlass system, which simultaneously interprets human instructions for a target acquisition task and determines the precise 3D positions of both user and the target object. This is achieved by utilizing LiDARs mounted in the infrastructure and a smart glass device worn by the user. Key to our system is the union of LiDAR-based localization termed LiLOC and a multi-modal visual grounding approach termed RealG(2)In-Lite. To demonstrate the system, we use Intel RealSense L515 cameras and a Microsoft HoloLens 2, as the user devices. VGGlass is able to: a) track the user in real-time in a global coordinate system, and b) locate target objects referred by natural language and pointing gestures.

Public spaces1 are a basis of urban lives. The mode and culture of sharing could be an indicator of the quality of life in the cities. For example, the buses and restaurants should comfort and bring satisfaction to individual visitors, including the vulnerable with special needs. However, their operation mostly occurs in rather a closed and exclusive manner [2]. An inherent limitation to such an inclusive sharing lies in the exclusive modes of traditional device interfaces; a variety of devices, called public devices hereafter, are installed in the public space and determine operational details of the space. As such, the space itself is shared, however, the public devices are controlled in exclusive ways. Could the sharing of the public space be operated in a democratic way?

Widely-accepted sleep guidelines advise regular bedtimes and sleep hygiene. An individual’s adherence is often viewed as a matter of self-regulation and anti-procrastination. We pose a question from a different perspective: What if it comes to a matter of one’s social or professional duty that mandates irregular daily life, making it incompatible with the premise of standard guidelines? We propose SleepGuru, an individually actionable sleep planning system featuring one’s real-life compatibility and extended forecast. Adopting theories on sleep physiology, SleepGuru builds a personalized predictor on the progression of the user’s sleep pressure over a course of upcoming schedules and past activities sourced from her online calendar and wearable fitness tracker. Then, SleepGuru service provides individually actionable multi-day sleep schedules which respect the user’s inevitable real-life irregularities while regulating her week-long sleep pressure. We elaborate on the underlying physiological principles and mathematical models, followed by a 3-stage study and deployment. We develop a mobile user interface providing individual predictions and adjustability backed by cloud-side optimization. We deploy SleepGuru in-the-wild to 20 users for 8 weeks, where we found positive effects of SleepGuru in sleep quality, compliance rate, sleep efficiency, alertness, long-term followability, and so on.

Aging often comes with declining social interaction, a known adversarial factor impacting the life satisfaction of senior population. Such decline appears even in family–a permanent social circle, as their adult children eventually go independent. We present MomentMeld, an AI-powered, cloud-backed mobile application that blends with everyday routine and naturally encourages rich and frequent inter-generational interactions in a family, especially those between the senior generation and their adult children. Firstly, we design a photographic interaction aid called mutually stimulatory memento, which is a cross-generational juxtaposition of semantically related photos to bring natural arousal of context-specific inter-generational empathy and reminiscence. Secondly, we build comprehensive ensemble AI models consisting of various deep neural networks and a runtime system that automates the creation of mutually stimulatory memento on top of the user’s usual photo-taking routines. We deploy MomentMeld in-the-wild with six families for an eight-week period, and discuss the key findings and further implications.

Public spaces are equipped with 'public actuators', e.g., HVAC, lighting fixtures, speakers, or streaming TV channels to ensure their visitors' comfort. However, many public actuators rarely allow the visitors to adjust their operation, limiting their utility and fairness across the visitors. Also, the social bar is often too high to speak up one's preference and attempt to change an actuator's operation. Social control and use of IoT devices is an underexplored new direction of research even with its huge potential and implication, but comes with high complexity and scale. This paper proposes a novel architecture, namely, Social Control-and-Use Architecture for IoT Devices, which provides a systematic view and an effective tool to handle the complication and intricacy in system design. It also proposes Hivemind, a first-of-a-kind system developed, upon the architecture, for sharing IoT-enabled actuators in a public space. It transforms an exclusively-controlled actuator in a public space into a true public actuator, supporting visitors to instantly participate in the democratic collective control. Also, a myriad of off-the-shelf actuators are easily incorporated without modification to their implementation. The field deployment of Hivemind shows its comprehensive service coverage as well as the users' approval on the democratic collective control of public actuators.

Public spaces, where we gather, commune, and take a rest, are the essential parts of a modern urban landscape, enriching citizen's everyday life [3]. How we share these spaces are considered an indicator of the quality of life. Public spaces thus have a responsibility to provide comfort and satisfaction to any visitors. However, in most times, the operations of the spaces are managed in rather an exclusive manner.

We propose FlexReduce, an efficient and flexible all-reduce algorithm for distributed deep learning under irregular network hierarchies. With ever-growing deep neural networks, distributed learning over multiple nodes is becoming imperative for expedited training. There are several approaches leveraging the symmetric network structure to optimize the performance over different hierarchy levels of the network. However, the assumption of symmetric network does not always hold, especially in shared cloud environments. By allocating an uneven portion of gradients to each learner (GPU), FlexReduce outperforms conventional algorithms on asymmetric network structures, and still performs even or better on symmetric networks.

We present ERICA, a digital personal trainer for users performing free weights exercises, with two key differentiators: (a) First, unlike prior approaches that either require multiple on-body wearables or specialized infrastructural sensing, ERICA uses a single in-ear "earable" device (piggybacking on a form factor routinely used by millions of gym-goers) and a simple inertial sensor mounted on each weight equipment; (b) Second, unlike prior work that focuses primarily on quantifying a workout, ERICA additionally identifies a variety of fine-grained exercising mistakes and delivers real-time, in-situ corrective instructions. To achieve this, we (a) design a robust approach for user-equipment association that can handle multiple (even 15) concurrently exercising users; (b) develop a suite of statistical models to detect several commonplace repetition-level mistakes; and (c) experimentally study the efficacy of multiple in-situ corrective feedback strategies. Via an end-to-end evaluation of ERICA with 33 participants naturally performing 3 dumbbell-based exercises, we show that (a) ERICA identifies over 94% of mistakes during the first 5 repetitions of a set, (b) the resulting feedback is viewed favorably by 78% of users, and (c) the feedback is effective, reducing mistakes by 10+% during subsequent repetitions.

Today's smartphone application (hereinafter `app') markets do not provide information on power consumption of apps, which is essential for users. Continuous sensing apps make this problem more severe because significant power is consumed without the users' awareness. We propose PowerForecaster to break through such an exhaustive cycle. It provides users with personalized estimation of sensing apps' power cost at pre-installation time. It is challenging to provide such estimation in advance because the actual power cost of a sensing app varies depending on user behavior such as physical activities and phone use patterns. To address this, we develop a novel power emulator as a core component of PowerForecaster. It achieves accurate, personalized power estimation by reproducing users' behaviors and emulating the target app's power use. We optimize the system to make the power emulation fast and its trace collection energy efficient. We further address the problem of dealing with large-scale emulation requests from worldwide deployment. We develop a novel selective emulation approach to minimize the server-side resource cost. We performed extensive experiments and the experimental results show that PowerForecaster achieves the power estimation accuracy of 93.4 percent and saves on 60 percent of the emulator instance usage.

As personal interactive devices become more ingrained into our daily lives, it becomes more important to understand how seamless interaction with those devices can be fostered. A typical mechanism to interface with a personal device is via a touch screen, in which users use their fingertip or stylus to scroll, type, select, or otherwise control device usage. Touch-based techniques, however, can become restrictive or inconvenient under a variety of scenarios. For example, personal devices such as phones or tablets are continuously increasing in size, making one-handed interaction difficult because one cannot easily hold the phone and touch the screen (with the thumb) at the same time with one hand. Therefore, in this demo, we present a new technique to interact with personal devices in which the screen and touch screen interactions can adapt to a user's grip or current touch constraints.

Real-time remote interaction has become easier and richer powered by recent advances in mobile computing and communication. A number of research have been explored on enriching family interaction by augmenting an interaction channel with asynchronous communication [6] or additional sensory stimuli [5]. However, it is still far from achieving a sense of living together for family members involuntarily living apart, especially in context-aware impromptu interaction. For families living together, it is trivial to naturally perceive behavioral and situational contexts of the other and initiate a relevant interaction intuitively. For example, a wife starts a casual chat with asking her husband what he is going to cook when she sees him going to the kitchen or hears a simmering sound.

People in work-separated families have been heavily relying on cutting-edge face-to-face communication services. Despite their ease of use and ubiquitous availability, experiences in living together are still far incomparable to those through remote face-to-face communication. We envision that enabling a remote person to be spatially superposed in one's living space would be a breakthrough to catalyze pseudo living-together interactivity. We propose HomeMeld, a zero-hassle self-mobile robotic system serving as a co-present avatar to create a persistent illusion of living together for those who are involuntarily living apart. The key challenges are 1) continuous spatial mapping between two heterogeneous floor plans and 2) navigating the robotic avatar to reflect the other's presence in real time under the limited maneuverability of the robot. We devise a notion of functionally equivalent location and orientation to translate a person's presence into another in a heterogeneous floor plan. We also develop predictive path warping to seamlessly synchronize the presence of the other. We conducted extensive experiments and deployment studies with real participants.

No abstract available.

We introduce Zaturi, a system enabling parents to create an audio book for their babies by utilizing micro spare time at work. We define micro spare time at work as tiny fragments of time with low cognitive loads that frequently occur at work, such as waiting for an elevator. We show that putting together micro spare time at work helps a working parent (1) build a tangible symbol conveying his/her thoughts to the beloved baby and (2) develop his/her own feelings of parental achievement without compromising regular working hours. Zaturi lets the parent immediately be aware of micro spare time and provides a crafted interface to seamlessly record the book piece by piece, so that the baby can enjoy listening to the book recorded in the parent's own voice. Through an extensive design process, we characterize the notion of micro spare time and build a working prototype of Zaturi. We also report parents' perceptions and family reactions after a two-week deployment.

We propose Sci-Fii, a speculative conversational interface framework for incremental inference on modularized services. To build one's own conversational interface with existing business logic, cloud-based modularized services offer a suite of ready-to-use components to ease development, ensure cross-platform flexibility, and encapsulate computational complexities. However developing with the modularized services often results in a chain of discrete modules with limited inter-module data sharing, which yields unnecessarily long response times of the conversational interface, aggravates user experiences, and eventually harms user retention. Sci-Fii offers a uniform framework that enables existing serviced modules to benefit from intermediate data and early parallel execution. Transparent to developers, Sci-Fii helps the end-to-end conversational interface system work fluidly and exhibit faster and more natural response times.

Imagine a densely packed crowd that gathers to convey a common message, such as people in a candlelight vigil or a protest. We envision an innovation through mobile computing technologies to empower such a crowd by enabling them simply to hold their phones up and create a massive collective visualization on top of them. We propose Card-stunt as a Service (CaaS). CaaS is a service enabling a densely packed crowd to instantly visualize symbols using their mobile devices and a server-side service. The key challenge toward realizing an instant collective visualization is how to achieve instant, infrastructure-free, decimeter-level localization of individuals in a massively packed crowd, while maintaining low latency. CaaS addresses the challenges by mobile visible-light angle-of-arrival (AoA) sensing and scalable constrained optimization. It reconstructs relative locations of all individuals and dispatches individualized timed pixels to each one so that they can do their part in the overall visualization. We evaluate CaaS with extensive experiments under diverse reality settings as well as under synthetic workloads scaling up to tens of thousands of people. We deploy CaaS to 49 participants so that they successfully perform a collective visualization cheering up MobiSys.

Consider a massive crowd who gathered together to convey their common voice to public, e.g., supporters of a team sitting together in a stadium, people doing a candlelight vigil in a public square, and so on. Imagine that they hold up their smartphone displays which collectively compose a huge public screen; the crowd's messages are now shown big on the top of them. We present CaaS [3], a mobile service to realize such an instant, massive, collective visualization with commodity smartphones and cloud services. In this demo, we demonstrate the collective localization feature of CaaS so that the audience can watch a given pattern or symbol collectively displayed on top of arbitrarily positioned phones (See the video demo, https://goo.gl/GfsORc).

Driven to create intuitive computing interfaces throughout our everyday space, various state-of-the-art technologies have been proposed for near-surface localization of a user's finger input such as hover or touch. However, these works require specialized hardware not commonly available, limiting the adoption of such technologies. We present SymmetriSense, a technology enabling near-surface 3-dimensional fingertip localization above arbitrary glossy surfaces using a single commodity camera device such as a smartphone. SymmetriSense addresses the localization challenges in using a single regular camera by a novel technique utilizing the principle of reflection symmetry and the fingertip's natural reflection casted upon surfaces like mirrors, granite countertops, or televisions. SymmetriSense achieves typical accuracies at sub-centimeter levels in our localization tests with dozens of volunteers and remains accurate under various environmental conditions. We hope SymmetriSense provides a technical foundation on which various everyday near-surface interactivity can be designed.

Continuous mobile sensing applications are emerging. Despite their usefulness, their real-world adoption has been slow. Many users are turned away by the drastic battery drain caused by continuous sensing and processing. In this paper, we propose CoMon+, a novel cooperative context monitoring system, which addresses the energy problem through opportunistic cooperation among nearby users. For effective cooperation, we develop a benefit-aware negotiation method to maximize the energy benefit of context sharing. CoMon+ employs heuristics to detect cooperators who are likely to remain in the vicinity for a long period of time, and the negotiation method automatically devises a cooperation plan that provides mutual benefit to cooperators, while considering running applications, available devices, and user policies. Especially, CoMon+ improves the negotiation method proposed in our earlier work, CoMon [30], to exploit multiple processing plans enabled by various personal sensing devices; each plan can be alternatively used for cooperation, which in turn will maximize overall power saving. We implement a CoMon+ prototype and show that it provides significant benefit for mobile sensing applications, e.g., saving 27-71 percent of smartphone power consumption depending on cooperation cases. Also, our deployment study shows that CoMon+ saves an average 19.7 percent of battery under daily use of a prototype application compared to the case without CoMon+ running.

The unique aquatic nature of swimming makes it difficult to use social or technical strategies to mitigate the tediousness of monotonous exercises. In this study, we propose the use of a smartphone-based multiplayer exergame named MobyDick . MobyDick is designed to be played while swimming, where a team of swimmers collaborate to hunt down a virtual monster. To this end, we take into account both human factors and technical challenges under swimming contexts. First, we perform a comparative analysis of a variety of wireless networking technologies in the aquatic environment and identify various technical constraints on wireless networking. Second, we develop a swimming activity recognition system to enable precise and real-time game inputs. Third, we devise a multiplayer game design by employing the unique interaction mode viable in an underwater environment, where the abilities of human communication are highly limited. Finally, we prototype MobyDick on waterproof off-the-shelf Android phones, and we deploy it in real swimming pool environments ( n = 8). Our qualitative analysis of user interview data reveals certain unique aspects of multiplayer swimming games.

Knowing individuals' relational orientation is imperative for effective offline, as well as online, interactions and collaborations. We use attachment theory to examine the link between Facebook users' relational orientation (in terms of attachment styles: anxiety and avoidance) and their relational activities. Our research examines whether and how the two key relational processes identified in offline social relationships (self-expression and responsiveness) are manifested on online social networks and related to attachment styles. We describe our dataset of 640 Facebook users, their attachment scale survey results, and their 525,334 posts. We define four features that map onto relational activities on Facebook: status updates and status updates with emotional words (self-expression); comments and likes (responsiveness). We find significant relationships between the users' attachment styles and their self-expression and responsiveness activities on Facebook. A key takeaway of our research is that without relying on self-reported surveys, a computational analysis of a Facebook user's self-expressing and responding activities alone can reveal the user's underlying relational orientation (i.e., attachment style).

Emerging continuous sensing apps introduce new major factors governing phones' overall battery consumption behaviors: (1) added nontrivial persistent battery drain, and more importantly (2) different battery drain rate depending on the user's different mobility condition. In this paper, we address the new battery impacting factors significant enough to outdate users' existing battery model in real life. We explore an initial approach to help users understand the cause and effect between their physical activity and phones' battery life. To this end, we present Sandra, a novel mobility-aware smartphone battery information advisor, and study its potential to help users redevelop their battery model. We perform an extensive explorative study and deployment for 30 days with 24 users. Our findings reveal what they essentially learned, and in which situations they found Sandra very helpful. We share the lessons learned to help in the design of future mobility-aware battery advisors.

Today's smartphone application (hereinafter 'app') markets miss a key piece of information, power consumption of apps. This causes a severe problem for continuous sensing apps as they consume significant power without users' awareness. Users have no choice but to repeatedly install one app after another and experience their power use. To break such an exhaustive cycle, we propose PowerForecaster, a system that provides users with power use of sensing apps at pre-installation time. Such advanced power estimation is extremely challenging since the power cost of a sensing app largely varies with users' physical activities and phone use patterns. We observe that the time for active sensing and processing of an app can vary up to three times with 27 people's sensor traces collected over three weeks. PowerForecaster adopts a novel power emulator that emulates the power use of a sensing app while reproducing users' physical activities and phone use patterns, achieving accurate, personalized power estimation. Our experiments with three commercial apps and two research prototypes show that PowerForecaster achieves 93.4% accuracy under 20 use cases. Also, we optimize the system to accelerate emulation speed and reduce overheads, and show the effectiveness of such optimization techniques.

Recently, a number of continuous sensing applications have been actively proposed in research communities and commercially released in the market. However, due to their unique power characteristics, user behavior-dependent battery drain, they bring new challenges for users' power management on these applications. In this demonstration, we present a comprehensive approach to support users' power management for continuous sensing applications. First, at pre-installation time, we provide an instant, personalized power estimation of a continuous sensing application. Without exhaustive trial and error, users can decide judiciously to install a certain application or not. Second, at runtime, we provide mobility-aware battery information. With this information, users can better estimate the phone's remaining battery life based on their imminent mobility conditions and take necessary actions in advance such as carrying an additional battery or minimizing the use of applications.

Exergame controllers are intended to add fun to monotonous exercise. However, studies on exergame controllers mostly focus on designing new controllers and exploring specific application domains without analyzing human factors, such as performance, comfort, and effort. In this paper, we examine the characteristics of a speed-based exergame controller that bear on human factors related to body movement and exercise. Users performed tasks such as changing and maintaining exercise speed for avatar control while their performance was measured. The exergame controller follows Fitts' law, but requires longer movement time than a gamepad and Wiimote. As well, resistance force and target speed affect performance. User experience data confirm that the comfort and mental effort are adequate as practical game controllers. The paper concludes with discussion on applying our findings to practical exergame design.

Language delay is a developmental problem of children who do not acquire language as expected for their chronological ages. Without timely intervention, language delay can act as a lifelong risk factor. Speech-language pathologists highlight that effective parent participation in everyday parent-child conversation is important to treat children's language delay. For effective roles, however, parents need to alter their own lifelong-established conversation habits, requiring extensive period of conscious effort and staying alert. In this paper, we present new opportunities for mobile and social computing to reinforce everyday parent-child conversation with therapeutic implications for children with language delays. Specifically, we propose TalkBetter, a mobile in-situ intervention service to help parents in daily parent-child conversation through real-time meta-linguistic analysis of ongoing conversations. Through extensive field studies with speech-language pathologists and parents, we report the multilateral motivations and implications of TalkBetter. We present our development of TalkBetter prototype and report its performance evaluation.

Interpersonal touch is our most primitive social language strongly governing our emotional well-being. Despite the positive implications of touch in many facets of our daily social interactions, we find wide-spread caution and taboo limiting touch-based interactions in workplace relationships that constitute a significant part of our daily social life. In this paper, we explore new opportunities for ubicomp technology to promote a new meme of casual and cheerful interpersonal touch such as high-fives towards facilitating vibrant workplace culture. Specifically, we propose High5, a mobile service with a smartwatch-style system to promote high-fives in everyday workplace interactions. We first present initial user motivation from semi-structured interviews regarding the potentially controversial idea of High5. We then present our smartwatch-style prototype to detect high-fives based on sensing electric skin potential levels. We demonstrate its key technical observation and performance evaluation.

The unique aquatic nature of swimming makes it very difficult to use social or technical strategies to mitigate the tediousness of monotonous exercises. In this study, we propose MobyDick, a smartphone-based multi-player exergame designed to be used while swimming, in which a team of swimmers collaborate to hunt down a virtual monster. In this paper, we present a novel, holistic game design that takes into account both human factors and technical challenges. Firstly, we perform a comparative analysis of a variety of wireless networking technologies in the aquatic environment and identify various technical constraints on wireless networking. Secondly, we develop a single phone-based inertial and barometric stroke activity recognition system to enable precise, real-time game inputs. Thirdly, we carefully devise a multi-player interaction mode viable in the underwater environment highly limiting the abilities of human communication. Finally, we prototype MobyDick on waterproof off-the-shelf Android phones, and deploy it to real swimming pool environments (n = 8). Our qualitative analysis of user interview data reveals certain unique aspects of multi-player swimming games.

In this paper, we propose SocioPhone, a novel initiative to build a mobile platform for face-to-face interaction monitoring. Face-to-face interaction, especially conversation, is a fundamental part of everyday life. Interaction-aware applications aimed at facilitating group conversations have been proposed, but have not proliferated yet. Useful contexts to capture and support face-to-face interactions need to be explored more deeply. More important, recognizing delicate conversational contexts with commodity mobile devices requires solving a number of technical challenges. As a first step to address such challenges, we identify useful meta-linguistic contexts of conversation, such as turn-takings, prosodic features, a dominant participant, and pace. These serve as cornerstones for building a variety of interaction-aware applications. SocioPhone abstracts such useful meta-linguistic contexts as a set of intuitive APIs. Its runtime efficiently monitors registered contexts during in-progress conversations and notifies applications on-the-fly. Importantly, we have noticed that online turn monitoring is the basic building block for extracting diverse meta-linguistic contexts, and have devised a novel volume-topography-based method. We show the usefulness of SocioPhone with several interesting applications: SocioTherapist, SocioDigest, and Tug-of-War. Also, we show that our turn-monitoring technique is highly accurate and energy-efficient under diverse real-life situations.

No abstract available.

On school field trips, chaperoning teachers' foremost concern is the safety of the children, particularly ensuring that none of them go missing. However, they have limited attention resources and face many challenges in keeping track of their charges. We present RubberBand, an assistive application that helps alleviate the teacher's burden. Our approach adapts to diverse field trip environmental and child behavioral dynamicity, utilizing observations of the relative dispersion of children and their tendency to form sub-groups.

Mobile applications that sense continuously, such as location monitoring, are emerging. Despite their usefulness, their adoption in real-world deployment situations has been extremely slow. Many smartphone users are turned away by the drastic battery drain caused by continuous sensing and processing. Also, the extractable contexts from the phone are quite limited due to its position and sensing modalities. In this paper, we propose CoMon, a novel cooperative ambience monitoring platform, which newly addresses the energy problem through opportunistic cooperation among nearby mobile users. To maximize the benefit of cooperation, we develop two key techniques, (1) continuity-aware cooperator detection and (2) benefit-aware negotiation. The former employs heuristics to detect cooperators who will remain in the vicinity for a long period of time, while the latter automatically devises a cooperation plan that provides mutual benefit to cooperators, while considering running applications, available devices, and user policies. Through continuity- and benefit-aware operation, CoMon enables applications to monitor the environment at much lower energy consumption. We implement and deploy a CoMon prototype and show that it provides significant benefit for mobile sensing applications.

No abstract available.

No abstract available.

We envision that diverse social exercising games, or exergames, will emerge, featuring much richer interactivity with immersive game play experiences. Further, the recent advances of mobile devices and wireless networking will make such social engagement more pervasive - people carry portable exergame devices (e.g., jump ropes) and interact with remote users anytime, anywhere. Towards this goal, we explore the potential of using heterogeneous exercise devices as game controllers for a multi-player social exergame; e.g., playing a boat paddling game with two remote exercisers (one with a jump rope, and the other with a treadmill). In this paper, we propose a novel platform called ExerLink that converts exercise intensity to game inputs and intelligently balances intensity/delay variations for fair game play experiences. We report the design considerations and guidelines obtained from the design and development processes of game controllers. We validate the efficacy of game controllers and demonstrate the feasibility of social exergames with heterogeneous exercise devices via extensive human subject studies.

We demonstrate a pervasive social exergame platform called ExerLink that converts exercise intensity to game inputs and intelligently balances intensity/delay variations for fair game play experiences. Also, we show the potential of using multiple exercise devices as game controllers and incorporating multiple heterogeneous controllers into a game. Specifically, we consider a class of exercise equipment used for repetitive, individual, and aerobic (RIA) exercises such as treadmill running, stationary cycling, hula hooping, and jump roping.

Field trips in kindergarten imply excellent chances to attain a wide spectrum of educational clues for the children. However, in-depth observation on their exploratory behaviors is uniquely challenging. Teachers mostly take all possible precautions against any incidents, sparing little time and attention for observation. We collaborated with kindergarten teachers to develop a system for delegated observation of the children's exploratory behaviors by using smartphones and sensor technologies.

We demonstrate E-Gesture, a collaborative architecture for energy-efficient gesture recognition on a hand-worn sensor device and an off-the-shelf smartphone that greatly reduces energy consumption while achieving high accuracy recognition under dynamic mobile situations. E-gesture employs a novel gesture segmentation and classification architecture carefully crafted by studying sporadic occurrence patterns of gestures in continuous sensor data streams and analyzing energy consumption characteristics in both sensor and smartphone.

Gesture is a promising mobile User Interface modality that enables eyes-free interaction without stopping or impeding movement. In this paper, we present the design, implementation, and evaluation of E-Gesture, an energy-efficient gesture recognition system using a hand-worn sensor device and a smartphone. E-gesture employs a novel gesture recognition architecture carefully crafted by studying sporadic occurrence patterns of gestures in continuous sensor data streams and analyzing the energy consumption characteristics of both sensors and smartphones. We developed a closed-loop collaborative segmentation architecture, that can (1) be implemented in resource-scarce sensor devices, (2) adaptively turn off power-hungry motion sensors without compromising recognition accuracy, and (3) reduce false segmentations generated from dynamic changes of body movement. We also developed a mobile gesture classification architecture for smartphones that enables HMM-based classification models to better fit multiple mobility situations.

A kindergarten is an interesting community of young children. The children continuously share their interactions and experiences, and grow along similar developmental stages. In this setting, studying relative differences among them can be an interesting approach to investigating how to help their individual and social development. In this study, we present our intuition on inter-child behavioral relativity and apply it to a real kindergarten environment. We conduct a close user study necessitating the monitoring of the children's behavior. Then, utilizing wearable sensor technologies, we perform a field study to explore various interesting aspects of behavioral relativity in an automatic and quantitative fashion. We consulted the kindergarten teachers with our results obtained from our field study in order to validate the practical benefits in the kindergarten environment. We further discuss the potential, limitations, and opportunities of our approach.