Scientific problem-solving involves synthesizing information while applying expert knowledge. We introduce CURIE, a scientific long-Context Understanding, Reasoning, and Information Extraction benchmark to measure the potential of Large Language Models (LLMs) in scientific problem-solving and assisting scientists in realistic workflows. This benchmark introduces ten challenging tasks with a total of 580 problems and solution pairs curated by experts in six disciplines - materials science, condensed matter physics, quantum computing, geo-spatial analysis, biodiversity, and proteins - covering both experimental and theoretical work-flows in science. We evaluate a range of closed and open LLMs on tasks in CURIE which requires domain expertise, comprehension of long in-context information,and multi-step reasoning. While Gemini Flash 2.0 and Claude-3 show consistent high comprehension across domains, the popular GPT-4o and command-R+ fail dramatically on protein sequencing tasks. With the best performance at 32% there is much room for improvement for all models. We hope that insights gained from CURIE can guide the future development of LLMs in sciences. Links to the data and evaluation code are in https://github.com/google/curie

Protecting and restoring forest ecosystems is critical for bio-diversity conservation and climate change mitigation. Forest monitoring on a global scale is essential for prioritizing and assessing conservation efforts. Satellite-based remote sensing is the only viable solution for providing global coverage, but to date, large-scale forest monitoring is limited mostly to single modalities or single time points. In this paper, we present a dataset consisting of data from five public satellites for recognizing forest plantations and planted tree species across the globe. Each satellite modality consists of a multi-year time series. The dataset, named "Planted" includes over 2M examples of 64 tree label classes (46 genera and 40 distinctive species), distributed among 41 countries. This dataset is released to foster research in forest monitoring using multimodal, multi-scale, multi-temporal data sources. Additionally, we present initial baseline results and evaluate modality fusion and data augmentation approaches for this dataset.

Abstract Camera traps facilitate non‐invasive wildlife monitoring, but their widespread adoption has created a data processing bottleneck: a camera trap survey can create millions of images, and the labour required to review those images strains the resources of conservation organisations. AI is a promising approach for accelerating image review, but AI tools for camera trap data are imperfect; in particular, classifying small animals remains difficult, and accuracy falls off outside the ecosystems in which a model was trained. It has been proposed that incorporating an object detector into an image analysis pipeline may help address these challenges, but the benefit of object detection has not been systematically evaluated in the literature. In this work, the authors assess the hypothesis that classifying animals cropped from camera trap images using a species‐agnostic detector yields better accuracy than classifying whole images. We find that incorporating an object detection stage into an image classification pipeline yields a macro‐average F1 improvement of around 25% on a large, long‐tailed dataset; this improvement is reproducible on a large public dataset and a smaller public benchmark dataset. The authors describe a classification architecture that performs well for both whole and detector‐cropped images, and demonstrate that this architecture yields state‐of‐the‐art benchmark accuracy.

OBJECTIVE: While non-invasive, cuffless blood pressure (BP) measurement has demonstrated relevancy in controlled environments, ambulatory measurement is important for hypertension diagnosis and control. We present both in-lab and ambulatory BP estimation results from a diverse cohort of participants. METHODS: Participants (N=1125, aged 21-85, 49.2% female, multiple hypertensive categories) had BP measured in-lab over a 24-hour period with a subset also receiving ambulatory measurements. Radial tonometry, photoplethysmography (PPG), electrocardiography (ECG), and accelerometry signals were collected simultaneously with auscultatory or oscillometric references for systolic (SBP) and diastolic blood pressure (DBP). Predictive models to estimate BP using a variety of sensor-based feature groups were evaluated against challenging baselines. RESULTS: Despite limited availability, tonometry-derived features showed superior performance compared to other feature groups and baselines, yieldingprediction errors of 0.32 ±9.8 mmHg SBP and 0.54 ±7.7 mmHg DBP in-lab, and 0.86 ±8.7 mmHg SBP and 0.75 ±5.9 mmHg DBP for 24-hour averages. SBP error standard deviation (SD) was reduced in normotensive (in-lab: 8.1 mmHg, 24-hr: 7.2 mmHg) and younger (in-lab: 7.8 mmHg, 24-hr: 6.7 mmHg) subpopulations. SBP SD was further reduced 15-20% when constrained to the calibration posture alone. CONCLUSION: Performance for normotensive and younger participants was superior to the general population across all feature groups. Reference type, posture relative to calibration, and controlled vs. ambulatory setting all impacted BP errors. SIGNIFICANCE: Results highlight the need for demographically diverse populations and challenging evaluation settings for BP estimation studies. We present the first public dataset of ambulatory tonometry and cuffless BP over a 24-hour period to aid in future cardiovascular research.

Predicting all applicable labels for a given image is known as multi-label classification. Compared to the standard multi-class case (where each image has only one label), it is considerably more challenging to annotate training data for multi-label classification. When the number of potential labels is large, human annotators find it difficult to mention all applicable labels for each training image. Furthermore, in some settings detection is intrinsically difficult e.g. finding small object instances in high resolution images. As a result, multi-label training data is often plagued by false negatives. We consider the hardest version of this problem, where annotators provide only one relevant label for each image. As a result, training sets will have only one positive label per image and no confirmed negatives. We explore this special case of learning from missing labels across four different multi-label image classification datasets for both linear classifiers and end-to-end fine-tuned deep networks. We extend existing multi-label losses to this setting and propose novel variants that constrain the number of expected positive labels during training. Surprisingly, we show that in some cases it is possible to approach the performance of fully labeled classifiers despite training with significantly fewer confirmed labels.

Multi-modal program synthesis refers to the task of synthesizing programs (code) from their specification given in different forms, such as a combination of natural language and examples. Examples provide a precise but incomplete specification, and natural language provides an ambiguous but more "complete" task description. Machine-learned pre-trained models (PTMs) are adept at handling ambiguous natural language, but struggle with generating syntactically and semantically precise code. Program synthesis techniques can generate correct code, often even from incomplete but precise specifications, such as examples, but they are unable to work with the ambiguity of natural languages. We present an approach that combines PTMs with component-based synthesis (CBS): PTMs are used to generate candidates programs from the natural language description of the task, which are then used to guide the CBS procedure to find the program that matches the precise examples-based specification. We use our combination approach to instantiate multi-modal synthesis systems for two programming domains: the domain of regular expressions and the domain of CSS selectors. Our evaluation demonstrates the effectiveness of our domain-agnostic approach in comparison to a state-of-the-art specialized system, and the generality of our approach in providing multi-modal program synthesis from natural language and examples in different programming domains.

We propose incorporating human labelers in a model fine-tuning system that provides immediate user feedback. In our framework, human labelers can interactively query model predictions on unlabeled data, choose which data to label, and see the resulting effect on the model's predictions. This bi-directional feedback loop allows humans to learn how the model responds to new data. We implement this framework for fine-tuning high-resolution land cover segmentation models and compare human-selected points to points selected using standard active learning methods. Specifically, we fine-tune a deep neural network – trained to segment high-resolution aerial imagery into different land cover classes in Maryland, USA – to a new spatial area in New York, USA using both our human-in-the-loop method and traditional active learning methods. The tight loop in our proposed system turns the algorithm and the human operator into a hybrid system that can produce land cover maps of large areas more efficiently than the traditional workflows. Our framework has applications in machine learning settings where there is a practically limitless supply of unlabeled data, of which only a small fraction can feasibly be labeled through human efforts, such as geospatial and medical image-based applications.

Normalization layers have been shown to improve convergence in deep neural networks, and even add useful inductive biases. In many vision applications the local spatial context of the features is important, but most common normalization schemes including Group Normalization (GN), Instance Normalization (IN), and Layer Normalization (LN) normalize over the entire spatial dimension of a feature. This can wash out important signals and degrade performance. For example, in applications that use satellite imagery, input images can be arbitrarily large; consequently, it is nonsensical to normalize over the entire area. Positional Normalization (PN), on the other hand, only normalizes over a single spatial position at a time. A natural compromise is to normalize features by local context, while also taking into account group level information. In this pa- per, we propose Local Context Normalization (LCN): a normalization layer where every feature is normalized based on a window around it and the filters in its group. We propose an algorithmic solution to make LCN efficient for arbitrary window sizes, even if every point in the image has a unique window. LCN outperforms its Batch Normalization (BN), GN, IN, and LN counterparts for object detection, semantic segmentation, and instance segmentation applications in several benchmark datasets, while keeping performance in- dependent of the batch size and facilitating transfer learning.

The ability to detect and classify rare occurrences in images has important applications - for example, counting rare and endangered species when studying biodiversity, or detecting infrequent traffic scenarios that pose a danger to self-driving cars. Few-shot learning is an open problem: current computer vision systems struggle to categorize objects they have seen only rarely during training, and collecting a sufficient number of training examples of rare events is often challenging and expensive, and sometimes outright impossible. We explore in depth an approach to this problem: complementing the few available training images with ad-hoc simulated data.Our testbed is animal species classification, which has a real-world long-tailed distribution. We present two natural world simulators, and analyze the effect of different axes of variation in simulation, such as pose, lighting, model, and simulation method, and we prescribe best practices for efficiently incorporating simulated data for real-world performance gain. Our experiments reveal that synthetic data can considerably reduce error rates for classes that are rare, that as the amount of simulated data is increased, accuracy on the target class improves, and that high variation of simulated data provides maximum performance gain.

Blood pressure (BP) is typically captured at irregular intervals, mostly in clinic environments. This approach treats BP as a static snapshot for health classification and largely ignores its value as a continuously fluctuat-ing measure. Recognizing that consumers are increasing-ly capturing health metrics through wearable devices, we explored BP measurement in relation to everyday living through a two-week field study with 34 adults. Based on questionnaires, measurement logs, and interviews, we examined participants' perceptions and attitudes to-wards BP variability and their associations of BP with aspects of their lives. We found that participants modi-fied their use of BP devices in response to BP variabil-ity, made associations with stress, food, and daily rou-tines, and revealed challenges with the design of current BP devices for personal use. We present design recom-mendations for BP use in everyday contexts and de-scribe strategies for re-framing BP capture and reporting.

Emerging uses of imaging technology for consumers cover a wide range of application areas from health to interaction techniques; however, typical cameras primarily transduce light from the visible spectrum into only three overlapping components of the spectrum: red, blue, and green. In contrast, hyperspectral imaging breaks down the electromagnetic spectrum into more narrow components and expands coverage beyond the visible spectrum. While hyperspectral imaging has proven useful as an industrial technology, its use as a sensing approach has been fragmented and largely neglected by the UbiComp community. We explore an approach to make hyperspectral imaging easier and bring it closer to the end-users. HyperCam provides a low-cost implementation of a multispectral camera and a software approach that automatically analyzes the scene and provides a user with an optimal set of images that try to capture the salient information of the scene. We present a number of use-cases that demonstrate HyperCam's usefulness and effectiveness.

Logging food and calorie intake has been shown to facilitate weight management. Unfortunately, current food logging methods are time-consuming and cumbersome, which limits their effectiveness. To address this limitation, we present an automated computer vision system for logging food and calorie intake using images. We focus on the "restaurant" scenario, which is often a challenging aspect of diet management. We introduce a key insight that addresses this problem specifically: restaurant plates are often both nutritionally and visually consistent across many servings. This insight provides a path to robust calorie estimation from a single RGB photograph: using a database of known food items together with restaurant-specific classifiers, calorie estimation can be achieved through identification followed by calorie lookup. As demonstrated on a challenging Menu-Match dataset and an existing third party dataset, our approach outperforms previous computer vision methods and a commercial calorie estimation app. Our Menu-Match dataset of realistic restaurant meals is made publicly available.

Four characteristic points of pulse pressure waves-the systolic peak, the anacrotic notch, the dicrotic notch, and the diastolic foot-are used to estimate various aspects of cardiovascular function, such as heart rate and augmentation index. We propose a novel approach to extracting these characteristic points using a topological signal processing framework. We characterize the topology of the signals using a collection of persistence intervals, which are encapsulated in a persistence diagram. The characteristic points are identified based on their time of occurrence and their distance from the identity line in the persistence diagram. We validate this approach by collecting radial pulse pressure data from twenty-eight participants using a wearable tonometer, and computing the peripheral augmentation index using a traditional derivative-based method and our novel persistence-based method. The augmentation index values computed using the two methods are statistically indistinguishable, suggesting that this representation merits further exploration as a tool for analyzing pulse pressure waves.

Movie directors and cinematographers impart style onto video using techniques that are learned through years of experience: camera movement, framing, color, lighting, etc. Without this experience and expensive equipment, it is very difficult to control stylistic aspects of a video. We introduce a novel approach for post-hoc editing of one specific aspect of cinematography -- camera motion style -- via an equalizer-like set of controls that manipulates the power spectra of a video's apparent motion path. We explore free manipulation of apparent camera motion as well as the transfer of motion styles from an example video to a new video to create a wide range of stylistic variations. We report on a user study confirming the ability of non-expert users to create motion styles.

Although numerous devices exist to track and share exercise routines based on running and walking, these devices offer limited functionality for strength-training exercises. We introduce RecoFit, a system for automatically tracking repetitive exercises - such as weight training and calisthenics - via an arm-worn inertial sensor. Our goal is to provide real-time and post-workout feedback, with no user-specific training and no intervention during a workout. Toward this end, we address three challenges: (1) segmenting exercise from intermittent non-exercise periods, (2) recognizing which exercise is being performed, and (3) counting repetitions. We present cross-validation results on our training data and results from a study assessing the final system, totaling 114 participants over 146 sessions. We achieve precision and recall greater than 95% in identifying exercise periods, recognition of 99%, 98%, and 96% on circuits of 4, 7, and 13 exercises respectively, and counting that is accurate to ±1 repetition 93% of the time. These results suggest that our approach enables a new category of fitness tracking devices.

Input modalities such as speech and gesture allow users to interact with computers without holding or touching a physical device, thus enabling at-a-distance interaction. It remains an open problem, however, to incorporate haptic feedback into such interaction. In this work, we explore the use of air vortex rings for this purpose. Unlike standard jets of air, which are turbulent and dissipate quickly, vortex rings can be focused to travel several meters and impart perceptible feedback. In this paper, we review vortex formation theory and explore specific design parameters that allow us to generate vortices capable of imparting haptic feedback. Applying this theory, we developed a prototype system called AirWave. We show through objective meas urements that AirWave can achieve spatial resolution of less than 10 cm at a distance of 2.5 meters. We further demonstrate through a user study that this can be used to direct tactile stimuli to different regions of the human body.

The weight scale is perhaps the most ubiquitous health sensor of all and is important to many health and lifestyle decisions, but its fundamental interface--a single numerical estimate of a person's current weight--has remained largely unchanged for 100 years. An opportunity exists to impact public health by re-considering this pervasive interface. Toward that end, we investigated the correspondence between consumers' perceptions of weight data and the realities of weight fluctuation. Through an analysis of online product reviews, a journaling study on weight fluctuations, expert interviews, and a large-scale survey of scale users, we found that consumers' perception of weight scale behavior is often disconnected from scales' capabilities and from clinical relevance, and that accurate understanding of weight fluctuation is associated with greater trust in the scale itself. We propose significant changes to how weight data should be presented and discuss broader implications for the design of other ubiquitous health sensing devices.

Haptic devices that provide robust and realistic force feedback are generally grounded to counterweight the applied force, prohibiting their use in mobile devices. Many ungrounded force-feedback devices rely on the gyro effect to produce torques on the human body, but their active control systems render them extremely bulky for implementation in small mobile devices. We present GyroTab, a relatively flat handheld system that utilizes the gyro effect to provide torque feedback. GyroTab relies on the user to produce an input torque and provides feedback by opposing that torque, making its feedback reactive to the user's motion. We describe the implementation of GyroTab, discuss the kinds of feedback it generates, and explore some of the psychophysical results we obtained from a study with the device.

Computer vision and inertial measurement have made it possible for people to interact with computers using whole-body gestures. Although there has been rapid growth in the uses and applications of these systems, their ubiquity has been limited by the high cost of heavily instrumenting either the environment or the user. In this paper, we use the human body as an antenna for sensing whole-body gestures. Such an approach requires no instrumentation to the environment, and only minimal instrumentation to the user, and thus enables truly mobile applications. We show robust gesture recognition with an average accuracy of 93% across 12 whole-body gestures, and promising results for robust location classification within a building. In addition, we demonstrate a real-time interactive system which allows a user to interact with a computer using whole-body gestures

Gesture is becoming an increasingly popular means of interacting with computers. However, it is still relatively costly to deploy robust gesture recognition sensors in existing mobile platforms. We present SoundWave, a technique that leverages the speaker and microphone already embedded in most commodity devices to sense in-air gestures around the device. To do this, we generate an inaudible tone, which gets frequency-shifted when it reflects off moving objects like the hand. We measure this shift with the microphone to infer various gestures. In this note, we describe the phenomena and detection algorithm, demonstrate a variety of gestures, and present an informal evaluation on the robustness of this approach across different devices and people.

Gestural input can greatly improve computing experiences away from the desktop, and has the potential to provide always-available access to computing. Specifically, accelerometers and gyroscopes worn on the arm (e.g., in a wristwatch) can sense arm gestures, enabling natural input in untethered scenarios. Two core components of any gesture recognition system are detecting when a gesture is occurring and classifying which gesture a person has performed. In previous work, accurate detection has required significant computation, and high-accuracy classification has come at the cost of training the system on a per-user basis. In this note, we present a gesture detection method whose computational complexity does not depend on the duration of the gesture, and describe a novel method for recognizing gestures with only a single example from a new user.

The awareness that hospital patients have of the people and events surrounding their care has a dramatic impact on satisfaction and clinical outcomes. However, patients are often under-informed about even basic aspects of their care. In this work, we hypothesize that mobile devices - which are increasingly available to patients - can be used as real-time information conduits to improve patient awareness and consequently improve patient care. To better understand the unique affordances that mobile devices offer in the hospital setting, we provided twenty-five patients with mobile phones that presented a dynamic, interactive report on their progress, care plan, and care team throughout their emergency department stay. Through interviews with these patients, their visitors, and hospital staff, we explore the benefits and challenges of using the mobile phone as an information display, finding overall that this is a promising approach to improving patient awareness. Furthermore, we demonstrate that only a small number of technology challenges remain before such a system could be deployed without researcher intervention.

Wearable sensor systems have been used in the ubiquitous computing community and elsewhere for applications such as activity and gesture recognition, health and wellness monitoring, and elder care. Although the power consumption of accelerometers has already been highly optimized, this work introduces a novel sensing approach which lowers the power requirement for motion sensing by orders of magnitude. We present an ultra-low-power method for passively sensing body motion using static electric fields by measuring the voltage at any single location on the body. We present the feasibility of using this sensing approach to infer the amount and type of body motion anywhere on the body and demonstrate an ultra-low-power motion detector used to wake up more power-hungry sensors. The sensing hardware consumes only 3.3 μW, and wake-up detection is done using an additional 3.3 μW (6.6 μW total).

Touch sensing and computer vision have made human-computer interaction possible in environments where keyboards, mice, or other handheld implements are not available or desirable. However, the high cost of instrumenting environments limits the ubiquity of these technologies, particularly in home scenarios where cost constraints dominate installation decisions. Fortunately, home environments frequently offer a signal that is unique to locations and objects within the home: electromagnetic noise. In this work, we use the body as a receiving antenna and leverage this noise for gestural interaction. We demonstrate that it is possible to robustly recognize touched locations on an uninstrumented home wall using no specialized sensors. We conduct a series of experiments to explore the capabilities that this new sensing modality may offer. Specifically, we show robust classification of gestures such as the position of discrete touches around light switches, the particular light switch being touched, which appliances are touched, differentiation between hands, as well as continuous proximity of hand to the switch, among others. We close by discussing opportunities, limitations, and future work.

We explore the use of social learning - improving knowledge skills by observing peer behavior - in the domain of Web search skill acquisition, focusing specifically on co-located classroom scenarios. Through a series of interviews, pilot studies, and classroom deployments, we conclude that a peripheral display of Web search activity within a classroom facilitates both social learning and teacher-led discourse. We present the ClassSearch system for shared awareness of Web search activity, which embodies principles gleaned from our iterative design process, and show results from a ClassSearch deployment in twelve middle-school classroom sessions. Finally, we highlight design suggestions and opportunities for future work while taxonomizing the space of co-located search pedagogies.

Patients' basic understanding of clinical events has been shown to dramatically improve patient care. We propose that the automatic generation of very short micro-explanations, suitable for real-time delivery in clinical settings, can transform patient care by giving patients greater awareness of key events in their electronic medical record. We present results of a survey study indicating that it may be possible to automatically generate such explanations by extracting individual sentences from consumer-facing Web pages. We further inform future work by characterizing physician and non-physician responses to a variety of Web-extracted explanations of medical lab tests.

We present Skinput, a technology that appropriates the human body for acoustic transmission, allowing the skin to be used as an input surface. In particular, we resolve the location of finger taps on the arm and hand by analyzing mechanical vibrations that propagate through the body. We collect these signals using a novel array of sensors worn as an armband. This approach provides an always available, naturally portable, and on-body finger input system. We assess the capabilities, accuracy and limitations of our technique through a two-part, twenty-participant user study. To further illustrate the utility of our approach, we conclude with several proof-of-concept applications we developed.

Recent work in muscle sensing has demonstrated the poten-tial of human-computer interfaces based on finger gestures sensed from electrodes on the upper forearm. While this approach holds much potential, previous work has given little attention to sensing finger gestures in the context of three important real-world requirements: sensing hardware suitable for mobile and off-desktop environments, elec-trodes that can be put on quickly without adhesives or gel, and gesture recognition techniques that require no new training or calibration after re-donning a muscle-sensing armband. In this note, we describe our approach to over-coming these challenges, and we demonstrate average clas-sification accuracies as high as 86 % for pinching with one of three fingers in a two-session, eight-person experiment.

Electronic medical records are increasingly comprehensive, and this vast repository of information has already contributed to medical efficiency and hospital procedure. However, this information is not typically accessible to patients, who are frequently under-informed and unclear about their own hospital courses. In this paper, we propose a design for in-room, patient-centric information displays, based on iterative design with physicians. We use this as the basis for a Wizard-of-Oz study in an emergency department, to assess patient and provider responses to in-room information displays. 18 patients were presented with real-time information displays based on their medical records. Semi-structured interviews with patients, family members, and hospital staff reveal that subjective response to in-room displays was overwhelmingly positive, and through these interviews we elicited guidelines regarding specific information types, privacy, use cases, and information presentation techniques. We describe these findings, and we discuss the feasibility of a fully-automatic implementation of our design.

Multi-touch interactions are a promising means of control for interactive tabletops. However, a lack of precision and tactile feedback makes multi-touch controls a poor fit for tasks where precision and feedback are crucial. We present an approach that offers precise control and tactile feedback for tabletop systems through the integration of dynamically re-mappable physical controllers with the multi-touch environment, and we demonstrate this approach in our collaborative tabletop audio editing environment. An observational user study demonstrates that our approach can provide needed precision and feedback, while preserving the collaborative benefits of a shared direct-manipulation surface. Our observations also suggest that direct touch and physical controllers can offer complementary benefits, and that providing both allows users to adjust their control strategy based on considerations including precision, convenience, visibility, and user role.

Previous research has explored how technology can motivate healthy living in social groups such as friends and coworkers. However, little research has focused on the implications of collecting, sharing, and reflecting upon health information within families. To explore this domain, we conducted a study that consisted of a week-long journaling activity followed by semi-structured interviews and formative design activities with 15 families (66 people). We identified four areas in which these practices are unique in a family context. Based on these findings we propose preliminary considerations for technologies that effectively support family reflections on health data.

We present data-driven methods for supporting musical creativity by capturing the statistics of a musical database. Specifically, we introduce a system that supports users in exploring the high-dimensional space of musical chord sequences by parameterizing the variation among chord sequences in popular music. We provide a novel user interface that exposes these learned parameters as control axes, and we propose two automatic approaches for defining these axes. One approach is based on a novel clustering procedure, the other on principal components analysis. A user study compares our approaches for defining control axes both to each other and to an approach based on manually-assigned genre labels. Results show that our automatic methods for defining control axes provide a subjectively better user experience than axes based on manual genre labeling.

Previous work has demonstrated the viability of applying offline analysis to interpret forearm electromyography (EMG) and classify finger gestures on a physical surface. We extend those results to bring us closer to using muscle-computer interfaces for always-available input in real-world applications. We leverage existing taxonomies of natural human grips to develop a gesture set covering interaction in free space even when hands are busy with other objects. We present a system that classifies these gestures in real-time and we introduce a bi-manual paradigm that enables use in interactive systems. We report experimental results demonstrating four-finger classification accuracies averaging 79% for pinching, 85% while holding a travel mug, and 88% when carrying a weighted bag. We further show generalizability across different arm postures and explore the tradeoffs of providing real-time visual feedback.

Repetitive strain injuries and ergonomics concerns have become increasingly significant health issues as a growing number of individuals frequently use computers for long periods of time. Currently, limited software mechanisms exist for managing ergonomics; the most well-known are packages that schedule and recommend typing breaks. Yet despite the proven benefits of taking breaks, such packages are rarely adopted due to the over-head of introducing periodic interruptions into a user's workflow. In this paper, we describe SuperBreak, a break-reminder package that provides hands-free interactions during breaks, with the goal of encouraging users to take more breaks and enhancing the benefits of those breaks. In a field study of 26 knowledge workers, 85% preferred SuperBreak over a traditional break-reminder system, and on average participants took a higher percentage of the interactive breaks suggested to them. Our results highlight the value of interactivity for improving the adoption and retention of ergonomic break practices.

Current user interfaces for Web search, including browsers and search engine sites, typically treat search as a transient activity. However, people often conduct complex, multi-query investigations that may span long durations and may be interrupted by other tasks. In this paper, we first present the results of a survey of users' search habits, which show that many search tasks span long periods of time. We then introduce SearchBar, a system for proactively and persistently storing query histories, browsing histories, and users' notes and ratings in an interrelated fashion. SearchBar supports multi-session investigations by assisting with task context resumption and information re-finding. We describe a user study comparing use of SearchBar to status-quo tools such as browser histories, and discuss our findings, which show that users find SearchBar valuable for task reacquisition. Our study also reveals the strategies employed by users of status-quo tools for handling multi-query, multi-session search tasks.

We explore the feasibility of muscle-computer interfaces (muCIs): an interaction methodology that directly senses and decodes human muscular activity rather than relying on physical device actuation or user actions that are externally visible or audible. As a first step towards realizing the mu-CI concept, we conducted an experiment to explore the potential of exploiting muscular sensing and processing technologies for muCIs. We present results demonstrating accurate gesture classification with an off-the-shelf electromyography (EMG) device. Specifically, using 10 sensors worn in a narrow band around the upper forearm, we were able to differentiate position and pressure of finger presses, as well as classify tapping and lifting gestures across all five fingers. We conclude with discussion of the implications of our results for future muCI designs.

We introduce MySong, a system that automatically chooses chords to accompany a vocal melody. A user with no musical experience can create a song with instrumental accompaniment just by singing into a microphone, and can experiment with different styles and chord patterns using interactions designed to be intuitive to non-musicians.

One way to help all users of commercial Web search engines be more successful in their searches is to better understand what those users with greater search expertise are doing, and use this knowledge to benefit everyone. In this paper we study the interaction logs of advanced search engine users (and those not so advanced) to better understand how these user groups search. The results show that there are marked differences in the queries, result clicks, post-query browsing, and search success of users we classify as advanced (based on their use of query operators), relative to those classified as non-advanced. Our findings have implications for how advanced users should be supported during their searches, and how their interactions could be used to help searchers of all experience levels find more relevant information and learn improved searching strategies.

A novel method based on deformable length of elastic element control (DLEEC) to realize the stiffness display for perception of virtual soft object is proposed, and the stiffness display interface device has been developed and is presented. The stiffness display interface device is composed of a thin elastic beam and an actuator to adjust the length of the beam. The deformation of the beam under a force is proportional to the third power of the beam length. By controlling the beam length, the stiffness display device can reproduce the stiffness of the virtual object from very soft to hard, so that the human fingertip can feel it as if he directly touches with the virtual object by interacting with the device. A real time position control algorithm is employed to guarantee the real time stiffness display.

We introduce a system that allows four users to each receive sound from a private audio channel while using a shared tabletop display. In order to explore how private audio channels affect a collaborative work environment, we conducted a user study with this system. The results reveal differences in work strategies when groups are presented with individual versus public audio, and suggest that the use of private audio does not impede group communication and may positively impact group dynamics. We discuss the findings, as well as their implications for the design of future audio-based "single display privacyware" systems.