Web agents, like OpenAI's Operator and Google's Project Mariner, are powerful agentic systems pushing the boundaries of Large Language Models (LLM). They can autonomously interact with the internet at the user's behest, such as navigating websites, filling search masks, and comparing price lists. Though web agent research is thriving, induced sustainability issues remain largely unexplored. To highlight the urgency of this issue, we provide an initial exploration of the energy and CO₂ cost associated with web agents from both a theoretical —via estimation— and an empirical perspective —by benchmarking. Our results show how different philosophies in web agent creation can severely impact the associated expended energy, and that more energy consumed does not necessarily equate to better results. We highlight a lack of transparency regarding disclosing model parameters and processes used for some web agents as a limiting factor when estimating energy consumption. Our work contributes towards a change in thinking of how we evaluate web agents, advocating for dedicated metrics measuring energy consumption in benchmarks.

Effective communication and teamwork are vital in high-stakes environments such as the operating room, where timely and accurate information exchange directly affects patient safety and surgical outcomes. Among intraoperative interactions, the collaboration between the surgeon and anesthesiologist is especially critical for maintaining smooth workflows and preventing adverse events. Despite its importance, little HCI research has explicitly examined the unique needs of this dyad or how AI-driven supportive systems might be designed to address them. In this work, we present a qualitative study of surgeon–anesthesiologist collaboration, drawing on focus groups with both specialties and in-situ observations of 45 surgeries spanning open, laparoscopic, and robotic procedures. Our findings uncover key challenges, unmet needs, and coordination breakdowns that shape this relationship. Based on these insights, we conceptualize a systems design to better support intraoperative collaboration.

Real-time sensor-based applications in pervasive computing require edge-deployable models to ensure low latency, privacy, and efficient interaction. A prime example is sensor-based human activity recognition (HAR), where models must balance accuracy with stringent resource constraints. Yet many deep learning approaches treat temporal sensor signals as black-box sequences, overlooking spectral–temporal structure while demanding excessive computation. We present SPECTRA, a deployment-first, co-designed spectral–temporal architecture that integrates short-time Fourier transform (STFT) feature extraction, depthwise separable convolutions, and channel-wise self-attention to capture spectral–temporal dependencies under real edge runtime and memory constraints. A compact bidirectional GRU with attention pooling summarizes within-window dynamics at low cost, reducing downstream model burden while preserving accuracy. Across five public HAR datasets, SPECTRA matches or approaches larger CNN/LSTM/Transformer baselines while substantially reducing parameters, latency, and energy. Deployments on a Google Pixel 9 smartphone and an STM32L4 microcontroller further demonstrate end-to-end deployable real-time, private, and efficient HAR.

E-textiles are rapidly emerging as an important area of electronic circuit applications. It also facilitates many socially important applications such as personalized health, elderly care, and smart agriculture. However, the environmental impact and sustainability of e-textiles remain very problematic. STELEC, short for Sustainable Textile ELECtronics, is an interdisciplinary research project funded by the European Innovation Council (EIC) under the Pathfinder programme on the responsible elec-tronics topic seeking cutting-edge innovation. STELEC started in September 2024 and is in its initial stage. The project is a multinational collaboration of research institutes, universities and companies across Europe. It aims at developing next-generation textile-based electronics in applications from sensing, processing to AI, with a commitment to full lifecycle sustainability.

Understanding human-to-human interactions, especially in contexts like public security surveillance, is critical for monitoring and maintaining safety. Traditional activity recognition systems are limited by fixed vocabularies, predefined labels, and rigid interaction categories that often rely on choreographed videos and overlook concurrent interactive groups. These limitations make such systems less adaptable to real-world scenarios, where interactions are diverse and unpredictable. In this paper, we propose an open vocabulary human-to-human interaction recognition (OV-HHIR) framework that leverages large language models to generate open-ended textual descriptions of both seen and unseen human interactions in open-world settings without being confined to a fixed vocabulary. Additionally, we create a comprehensive, large-scale human-to-human interaction dataset by standardizing and combining existing public human interaction datasets into a unified benchmark. Extensive experiments demonstrate that our method outperforms traditional fixed-vocabulary classification systems and existing cross-modal language models for video understanding, setting the stage for more intelligent and adaptable visual understanding systems in surveillance and beyond.

Human-AI coevolution, defined as a process in which humans and AI algorithms continuously influence each other, increasingly characterises our society, but is understudied in artificial intelligence and complexity science literature. Recommender systems and assistants play a prominent role in human-AI coevolution, as they permeate many facets of daily life and influence human choices through online platforms. The interaction between users and AI results in a potentially endless feedback loop, wherein users' choices generate data to train AI models, which, in turn, shape subsequent user preferences. This human-AI feedback loop has peculiar characteristics compared to traditional human-machine interaction and gives rise to complex and often “unintended” systemic outcomes. This paper introduces human-AI coevolution as the cornerstone for a new field of study at the intersection between AI and complexity science focused on the theoretical, empirical, and mathematical investigation of the human-AI feedback loop. In doing so, we: (i) outline the pros and cons of existing methodologies and highlight shortcomings and potential ways for capturing feedback loop mechanisms; (ii) propose a reflection at the intersection between complexity science, AI and society; (iii) provide real-world examples for different human-AI ecosystems; and (iv) illustrate challenges to the creation of such a field of study, conceptualising them at increasing levels of abstraction, i.e., scientific, legal and socio-political.

Disordered Quantum many-body Systems (DQS) and Quantum Neural Networks (QNN) have many structural features in common. However, a DQS is essentially an initialized QNN with random weights, often leading to non-random outcomes. In this work, we emphasize the possibilities of random processes being a deceptive quantum-generating model effectively hidden in a QNN. When we choose weights in a QNN randomly the unitarity property of quantum gates is unchanged. As we show, this can lead to memory effects with multiple consequences on the learnability and trainability of QNN one would not expect from a classical neural network with random weights. This phenomenon may lead to a fundamental misunderstanding of the capabilities of common quantum generative models, where the generation of new samples is essentially averaging over random outputs. While we suggest that DQS can be effectively used for tasks like image augmentation, we draw the attention that overly simple datasets are often used to show the generative capabilities of quantum models, potentially leading to overestimation of their effectiveness.

Human activity recognition (HAR) is a long-standing problem in artificial intelligence with applications in a broad range of areas, including healthcare, sports and fitness, security, and more. The performance of HAR in real-world settings is strongly dependent on the type and quality of the input signal that can be acquired. Given an unobstructed, high-quality camera view of a scene, computer vision systems, in particular in conjunction with foundation models, can today fairly reliably distinguish complex activities. On the other hand, recognition using modalities such as wearable sensors (which are often more broadly available, e.g., in mobile phones and smartwatches) is a more difficult problem, as the signals often contain less information and labeled training data is more difficult to acquire. To alleviate the need for labeled data, we introduce our comprehensive Fitness Multimodal Activity Dataset (FiMAD) in this work, which can be used with the proposed pre-training method MuJo (Multimodal Joint Feature Space Learning) to enhance HAR performance across various modalities. FiMAD was created using YouTube fitness videos and contains parallel video, language, pose, and simulated IMU sensor data. MuJo utilizes this dataset to learn a joint feature space for these modalities. We show that classifiers pre-trained on FiMAD can increase the performance on real HAR datasets such as MM-Fit, MyoGym, MotionSense, and MHEALTH. For instance, on MM-Fit, we achieve a Macro F1-Score of up to 0.855 when fine-tuning on only 2% of the training data and 0.942 when utilizing the complete training set for classification tasks. We compare our approach with other self-supervised ones and show that, unlike them, ours consistently improves compared to the baseline network performance while also providing better data efficiency.

Head-mounted displays (HMDs) are increasingly used in safety-critical fields such as surgery, aviation, and industrial manufacturing. As major manufacturers shift toward video-see through (VST) designs to deliver unified AR and VR experiences, they also replace direct visual access to the real world with a video feed. This design choice raises concerns about its impact on user performance. This study investigates the isolated impacts of VST and optical-see through HMDs on user real-world perceptual–motor performance by comparing two leading HMDs, Apple Vision Pro (AVP) and HoloLens 2 (MHL2) against unencumbered vision using the Purdue Pegboard Test (PPT), a standard assessment of manual dexterity. Twenty participants completed tasks across three conditions (AVP, MHL2, and Baseline), while we recorded dexterity scores, cognitive load, system usability, VR sickness, and subjective feedback. Movement data were also collected via Apple Watches. Study results with 20 participants revealed that dexterity scores significantly declined under the AVP condition across all subtests. This was accompanied by significantly higher cognitive load and a notable drop in RMS acceleration values (observed in the RMS analysis of a subset of 13 participants). The analysis on dexterity score yielded a significant difference between MHL2 and Baseline only for a single subtest of the PPT (Left Hand). Post-task interviews revealed greater discomfort, visual fatigue, and reduced task confidence with AVP. These findings suggest that current VST HMDs impose a hidden ergonomic cost undermining user performance in tasks where precision, and comfort are essential. For AR applications designed to enhance user performance, such as assistive tools, training systems, or task guidance interfaces, designers must account for and mitigate this performance degradation through counterbalancing strategies to offset the visual and cognitive burden introduced by VST HMDs.

Human-AI coevolution, defined as a process in which humans and AI algorithms continuously influence each other, increasingly characterises our society, but is understudied in artificial intelligence and complexity science literature. Recommender systems and assistants play a prominent role in human-AI coevolution, as they permeate many facets of daily life and influence human choices through online platforms. The interaction between users and AI results in a potentially endless feedback loop, wherein users' choices generate data to train AI models, which, in turn, shape subsequent user preferences. This human-AI feedback loop has peculiar characteristics compared to traditional human-machine interaction and gives rise to complex and often “unintended” systemic outcomes. This paper introduces human-AI coevolution as the cornerstone for a new field of study at the intersection between AI and complexity science focused on the theoretical, empirical, and mathematical investigation of the human-AI feedback loop. In doing so, we: (i) outline the pros and cons of existing methodologies and highlight shortcomings and potential ways for capturing feedback loop mechanisms; (ii) propose a reflection at the intersection between complexity science, AI and society; (iii) provide real-world examples for different human-AI ecosystems; and (iv) illustrate challenges to the creation of such a field of study, conceptualising them at increasing levels of abstraction, i.e., scientific, legal and socio-political.

Text entry tasks for emerging wireless Augmented Reality (AR) and Virtual Reality (VR) devices can be realized in many ways, one of the most promising methods is based on an Inertial Measurement Unit (IMU) sensor, which resembles a human writing style and is called air-writing. An existing air-writing Deep Neural Network (DNN) based algorithm called FAirWrite achieves state-of-the-art accuracy. However, this algorithm is optimized only for accuracy without considering the implementation constraints. State-of-the-art implementation executes the algorithm in a cloud, which is associated with large communication latency; privacy issues with respect to data transfer; and a need for reliable internet connection. A solution that tackles all three challenges is executing the algorithm locally at the edge in the closest proximity to the sensor. However, inference at the edge is challenging due to limited memory and computing resources, which can impact accuracy and increase latency. Additionally, battery-powered edge devices must adhere to strict power and energy consumption limits. All these constraints collectively restrict the model size and computational complexity that can be deployed near the sensor. In this work, we explore various optimizations required to enable state-of-the-art FAirWrite algorithm for a real-world deployment scenario, i.e. executing on an edge device. We perform a multi-layer design-space exploration considering multiple levels of design hierarchy spanning from optimizations applied on algorithmic level down to hardware level, considering various deployment run-times and edge platforms including embedded micro-controller, embedded Central Processing Unit (CPU), embedded Graphics Processing Unit (GPU), Neural Processing Unit (NPU), and Field-Programmable Gate Array (FPGA). The complexity reduction optimizations result in a smaller eFAirWrite model without any accuracy degradation. We propose and implement a custom hardware architecture of the algorithm on an FPGA by utilizing custom data-types and memory hierarchy. We demonstrate that the FPGA implementation achieves , , , and higher energy efficiency as compared to NPU, embedded GPU, embedded CPU, and micro-controller, respectively, which makes it a suitable edge platform for portable and real-time air-writing text entry.

Breathing is a spontaneous but controllable body function that can be used for hands-free interaction. Our work introduces “iBreath”, a novel system to detect breathing gestures similar to clicks using bio-impedance. We evaluated iBreath’s accuracy and user experience using two lab studies (n=34). Our results show high detection accuracy (F1-scores > 95.2%). Furthermore, the users found the gestures easy to use and comfortable. Thus, we developed eight practical guidelines for the future development of breathing gestures. For example, designers can train users on new gestures within just 50 seconds (five trials), and achieve robust performance with both user-dependent and user-independent models trained on data from 21 participants, each yielding accuracies above 90%. Users preferred single clicks and disliked triple clicks. The median gesture duration is 3.5-5.3 seconds. Our work provides solid ground for researchers to experiment with creating breathing gestures and interactions.

We present a quantum inspired image augmentation protocol which is applicable to classical images and, in principle, due to its known quantum formulation applicable to quantum systems and quantum machine learning in the future. The augmentation technique relies on the phenomenon Anderson localization. As we will illustrate by numerical examples the technique changes classical wave properties by interference effects resulting from scatterings at impurities in the material. We explain that the augmentation can be understood as multiplicative noise, which counter-intuitively averages out, by sampling over disorder realizations. Furthermore, we show how the augmentation can be implemented in arrays of disordered waveguides with direct implications for an efficient optical image transfer.

We present a novel local-global feature fusion framework for body-weight exercise recognition with floor-based dynamic pressure maps. One step further from the existing studies using deep neural networks mainly focusing on global feature extraction, the proposed framework aims to combine local and global features using image processing techniques and the YOLO object detection to localize pressure profiles from different body parts and consider physical constraints. The proposed local feature extraction method generates two sets of high-level local features consisting of cropped pressure mapping and numerical features such as angular orientation, location on the mat, and pressure area. In addition, we adopt a knowledge distillation for regularization to preserve the knowledge of the global feature extraction and improve the performance of the exercise recognition. Our experimental results demonstrate a notable 11 percent improvement in F1 score compared to the baseline 3DCNN for exercise recognition while preserving label-specific features.

The integration of Augmented Reality (AR) technology into surgical procedures offers significant potential to enhance clinical outcomes. While there are plenty of lab-proven prototypes, systems employed in actual clinical settings require specialized design and rigorous clinical evaluation of these AR-based solutions to meet the high demands of complex medical fields. Our research exposes these complex requirements emerging from clinical environments, such as operation theaters. To address the challenges, we introduce ARAS, an operational AR assistance system for live open pancreatic surgery. Employing a user-centric design methodology, we designed and refined ARAS through several iterations, ensuring its practical applicability and effectiveness in a real-world surgical setting during clinical trials. ARAS enables in situ and precise visualization of the patient’s 3D reconstructed vascular system and tumor during the surgical procedure. We evaluated ARAS through clinical trials (N=7) involving patients diagnosed with pancreatic cancer. Our interviews with the surgeons underscored the utility of ARAS for open pancreatic surgery, especially in critical and highly time-pressured phases of the surgery, as it proved to be exceptionally beneficial in aiding surgeons during in their decision-making process. In a post-surgery evaluation, the surgeons also certified the precise visualization accuracy of ARAS during those critical phases. Our findings showcased the heightened requirements for AR-based solutions in operational clinical use and proved that ARAS met the challenges emerging during live surgeries. Consequently, surgical AR assistance systems do have a transformative potential to revolutionize traditional practices, but their applicability is subject to high design constraints during critical medical procedures.

The increased presence of large language models (LLMs) in educational settings has ignited debates concerning negative repercussions, including overreliance and inadequate task reflection.Our work advocates moderated usage of such models, designed in a way that supports students and encourages critical thinking.We developed two moderated interaction methods with ChatGPT: hintbased assistance and presenting multiple answer choices.In a study with students (N=40) answering physics questions, we compared the effects of our moderated models against two baseline settings: unmoderated ChatGPT access and internet searches.We analyzed the interaction strategies and found that the moderated versions exhibited less unreflected usage (e.g., copy & paste) compared to the unmoderated condition.However, neither ChatGPT-supported condition could match the ratio of reflected usage present in internet searches.Our research highlights the potential benefits of

Automatic and precise fitness activity recognition can be beneficial in aspects from promoting a healthy lifestyle to personalized preventative healthcare. While IMUs are currently the prominent fitness tracking modality, through iMove, we show bio-impedence can help improve IMU-based fitness tracking through sensor fusion and contrastive learning. To evaluate our methods, we conducted an experiment including six upper body fitness activities performed by ten subjects over five days to collect synchronized data from bio-impedance across two wrists and IMU on the left wrist. The contrastive learning framework uses the two modalities to train a better IMU-only classification model, where bio-impedance is only required at the training phase, by which the average Macro F1 score with the input of a single IMU was improved by 3.22 % reaching 84.71 % compared to the 81.49 % of the IMU baseline model. We have also shown how bio-impedance can improve human activity recognition (HAR) directly through sensor fusion, reaching an average Macro F1 score of 89.57 % (two modalities required for both training and inference) even if Bio-impedance alone has an average macro F1 score of 75.36 %, which is outperformed by IMU alone. In addition, similar results were obtained in an extended study on lower body fitness activity classification, demonstrating the generalisability of our approach.Our findings underscore the potential of sensor fusion and contrastive learning as valuable tools for advancing fitness activity recognition, with bio-impedance playing a pivotal role in augmenting the capabilities of IMU-based systems.

Manufacturing industries strive to improve production efficiency and product quality by deploying advanced sensing and control systems. Wearable sensors are emerging as a promising solution for achieving this goal, as they can provide continuous and unobtrusive monitoring of workers’ activities in the manufacturing line. This article presents a novel wearable sensing prototype that combines IMU and body capacitance sensing modules to recognize worker activities in the manufacturing line. To handle these multimodal sensor data, we propose and compare early, and late sensor data fusion approaches for multichannel time-series convolutional neural networks and deep convolutional LSTM. We evaluate the proposed hardware and neural network model by collecting and annotating sensor data using the proposed sensing prototype and Apple Watches in the testbed of the manufacturing line. Experimental results demonstrate that our proposed methods achieve superior performance compared to the baseline methods, indicating the potential of the proposed approach for real-world applications in manufacturing industries. Furthermore, the proposed sensing prototype with a body capacitive sensor (BCS) and feature fusion method improves by 6.35%, yielding a 9.38% higher macro F1 score than the proposed sensing prototype without a BCS and Apple Watch data, respectively.

Distractions caused by digital devices are increasingly causing dangerous situations on the road, particularly for more vulnerable road users like cyclists. While researchers have been exploring ways to enable richer interaction scenarios on the bike, safety concerns are frequently neglected and compromised. In this work, we propose Head 'n Shoulder, a gesture-driven approach to bike interaction without affecting bike control, based on a wearable garment that allows hands- and eyes-free interaction with digital devices through integrated capacitive sensors. It achieves an average accuracy of 97% in the final iteration, evaluated on 14 participants. Head 'n Shoulder does not rely on direct pressure sensing, allowing users to wear their everyday garments on top or underneath, not affecting recognition accuracy. Our work introduces a promising research direction: easily deployable smart garments with a minimal set of gestures suited for most bike interaction scenarios, sustaining the rider's comfort and safety.

Password sharing is a convenient means to access shared resources, save on subscription costs, provide emergency access, and avoid forgetting vital account details. However, it also raises significant privacy concerns, especially in digital communication contexts where content may be inadvertently exposed to unintended recipients. In this paper, we investigate this duality, using a survey of 86 Egyptian women to understand their sharing behavior and the design and evaluation of a chat application used by 60 participants. This application issues warnings based on content sensitivity, leading to increased user awareness about privacy risks. Our findings indicate that, while many participants initially shared passwords, they were surprised to discover others doing the same. Furthermore, our application effectively reduced password sharing, reflecting improved awareness of associated risks. This research acknowledges the cultural aspects of password sharing while striving to enhance the experience, enabling participants to make informed choices that enhance their information control.

In this work, we propose an open-source scalable end-to-end RTL framework FieldHAR, for complex human activ-ity recognition (HAR) from heterogeneous sensors using artificial neural networks (ANN) optimized for FPGA or ASIC integration. FieldHAR aims to address the lack of apparatus to transform complex HAR methodologies often limited to offline evaluation to efficient runtime edge applications. The framework uses parallel sensor interfaces and integer-based multi-branch convolutional neural networks (CNNs) to support flexible modality extensions with synchronous sampling at the maximum rate of each sensor. To validate the framework, we used a sensor-rich kitchen scenario HAR application which was demonstrated in a previous offline study. Through resource-aware optimizations, with FieldHAR the entire RTL solution was created from data acquisition to ANN inference taking as low as 25% logic elements and 2% memory bits of a low-end Cyclone IV FPGA and less than 1% accuracy loss from the original FP32 precision offline study. The RTL implementation also shows advantages over MCU-based solutions, including superior data acquisition performance and virtually eliminating ANN inference bottleneck.

Online clothing shopping has become increasingly popular, but the high rate of returns due to size and fit issues has remained a major challenge. To address this problem, virtual try-on systems have been developed to provide customers with a more realistic and personalized way to try on clothing. In this paper, we propose a novel virtual try-on method called ClothFit, which can predict the draping shape of a garment on a target body based on the actual size of the garment and human attributes. Unlike existing try-on models, ClothFit considers the actual body proportions of the person and available cloth sizes for clothing virtualization, making it more appropriate for current online apparel outlets. The proposed method utilizes a U-Net-based network architecture that incorporates cloth and human attributes to guide the realistic virtual try-on synthesis. Specifically, we extract features from a cloth image using an auto-encoder and combine them with features from the user’s height, weight, and cloth size. The features are concatenated with the features from the U-Net encoder, and the U-Net decoder synthesizes the final virtual try-on image. Our experimental results demonstrate that ClothFit can significantly improve the existing state-of-the-art methods in terms of photo-realistic virtual try-on results.

Early prediction of remaining useful life (RUL) is crucial for effective battery management across various industries, ranging from household appliances to large-scale applications. Accurate RUL prediction improves the reliability and maintainability of battery technology. However, existing methods have limitations, including assumptions of data from the same sensors or distribution, foreknowledge of the end of life (EOL), and neglect to determine the first prediction cycle (FPC) to identify the start of the unhealthy stage. This paper proposes a novel method for RUL prediction of Lithium-ion batteries. The proposed framework comprises two stages: determining the FPC using a neural network-based model to divide the degradation data into distinct health states and predicting the degradation pattern after the FPC to estimate the remaining useful life as a percentage. Experimental results demonstrate that the proposed method outperforms conventional approaches in terms of RUL prediction. Furthermore, the proposed method shows promise for real-world scenarios, providing improved accuracy and applicability for battery management.

This work presents an active software instrument allowing deep learning architects to interactively inspect neural network models' output behavior from user-manipulated values in any latent layer. Latent Inspector offers multiple dimension reduction techniques to visualize the model's high dimensional latent layer output in human-perceptible, two-dimensional plots. The system is implemented with Node.js front end for interactive user input and Python back end for interacting with the model. By utilizing a general and modular architecture, our proposed solution dynamically adapts to a versatile range of models and data structures. Compared to already existing tools, our asynchronous approach of separating the training process from the inspection offers additional possibilities, such as interactive data generation, by actively working with the model instead of visualizing training logs. Overall, Latent Inspector demonstrates the possibilities as well as the appearing limits for providing a generalized, tool-based concept for enhancing model insight in terms of explainable and transparent AI.

This paper presents a novel and generic method to employ deep neural networks for segmenting in-air performed gestures to detect writing activity. We consider various factors such as temporal, geometric, and frequency constraints to define the parameters and fine-tune the deep-learning methods. The proposed method is benchmarked on air-gesture data from 50 participants, which included air-writing gestures followed and preceded by non-writing gestures. The reported results establish the potential of deep-learning methods to segment air-writing activity. The proposed novel approach provides a foundation to develop sophisticated systems for recognizing air gestures to enhance interaction in virtual and augmented reality environments.

The estimation of 3D human body shape and clothing measurements is crucial for virtual try-on and size recommendation problems in the fashion industry but has always been a challenging problem due to several conditions, such as lack of publicly available realistic datasets, ambiguity in multiple camera resolutions, and the undefinable human shape space. Existing works proposed various solutions to these problems but could not succeed in the industry adaptation because of complexity and restrictions. To solve the complexity and challenges, in this paper, we propose a simple yet effective architecture to estimate both shape and measures from frontal- and side-view images. We utilize silhouette segmentation from the two multi-view images and implement an auto-encoder network to learn low-dimensional features from segmented silhouettes. Then, we adopt a kernel-based regularized regression module to estimate the body shape and measurements. The experimental results show that the proposed method provides competitive results on the synthetic dataset, NOMO-3d-400-scans Dataset, and RGB Images of humans captured in different cameras.

User dependence remains one of the most difficult general problems in Human Activity Recognition (HAR), in particular when using wearable sensors. This is due to the huge variability of the way different people execute even the simplest actions. In addition, detailed sensor fixtures and placement will be different for different people or even at different times for the same users. In theory, the problem can be solved by a large enough data set. However, recording data sets that capture the entire diversity of complex activity sets is seldom practicable. Instead, models are needed that focus on features that are invariant across users. To this end, we present an adversarial subject-independent feature extraction method with the maximum mean discrepancy (MMD) regularization for human activity recognition. The proposed model is capable of learning a subject-independent embedding feature representation from multiple subjects datasets and generalizing it to unseen target subjects. The proposed network is based on the adversarial encoder-decoder structure with the MMD to realign the data distribution over multiple subjects. Experimental results show that the proposed method not only outperforms state-of-the-art methods over the four real-world datasets but also improves the subject generalization effectively. We evaluate the method on well-known public data sets showing that it significantly improves user-independent performance and reduces variance in results.

Current research in eSports lacks the tools for proper game practising and performance analytics. The majority of prior work relied only on in-game data for advising the players on how to perform better. However, in-game mechanics and trends are frequently changed by new patches limiting the lifespan of the models trained exclusively on the in-game logs. In this article, we propose the methods based on the sensor data analysis for predicting whether a player will win the future encounter. The sensor data were collected from ten participants in 22 matches in the League of Legends video game. We have trained machine learning models, including the transformer and gated recurrent unit, to predict whether the player wins the encounter taking place after some fixed time in the future. For 10-s forecasting horizon, the transformer neural network architecture achieves the ROC AUC score of 0.706. This model is further developed into the detector capable of predicting that a player will lose the encounter occurring in 10 s in 88.3% of cases with 73.5% accuracy. This might be used as a players’ burnout or fatigue detector, advising players to retreat. We have also investigated which physiological features affect the chance to win or lose the next in-game encounter.

The demands of automated shipping address recognition and verification have increased to handle a large number of packages and to save costs associated with misdelivery. A previous study proposed a deep learning system where the shipping address is recognized and verified based on a camera image capturing the shipping address and barcode area. Because the system performance depends on the input image quality, inspection of input image quality is necessary for image preprocessing. In this paper, we propose an input image quality verification method combining global and local features. Object detection and scale-invariant feature transform in different feature spaces are developed to extract global and local features from several independent convolutional neural networks. The conditions of shipping label images are classified by fully connected fusion layers with concatenated global and local features. The experimental results regarding real captured and generated images show that the proposed method achieves better performance than other methods. These results are expected to improve the shipping address recognition and verification system by applying different image preprocessing steps based on the classified conditions.

Multilingualism is a cultural cornerstone of Europe and firmly anchored in the European treaties including full language equality. However, language barriers impacting business, cross-lingual and cross-cultural communication are still omnipresent. Language Technologies (LTs) are a powerful means to break down these barriers. While the last decade has seen various initiatives that created a multitude of approaches and technologies tailored to Europe’s specific needs, there is still an immense level of fragmentation. At the same time, AI has become an increasingly important concept in the European Information and Communication Technology area. For a few years now, AI – including many opportunities, synergies but also misconceptions – has been overshadowing every other topic. We present an overview of the European LT landscape, describing funding programmes, activities, actions and challenges in the different countries with regard to LT, including the current state of play in industry and the LT market. We present a brief overview of the main LT-related activities on the EU level in the last ten years and develop strategic guidance with regard to four key dimensions.

We investigate if providing shoppers with Augmented reality (AR) as a shopping tool can lead to an increase in purchase rate compared with conventional shopping applications. In a ”simulated shopping” study with two groups with a total number of 20 participants, a test group used a wearable AR device (HoloLens) as a primary shopping method while a control group used a tablet as a conventional 2D shopping device. According to the surveys collected from participants, 19 participants (95%) commented that the AR method was more joyful than conventional 2D method. Furthermore, all participants said that their experience with AR technology was more realistic. 16 participants (80%) believed that the AR method was more influential than the conventional method, but the other four participants (20%) said that neither of the methods had any impact on their buying intentions. Although the mean number of products added to the basket by each person in HoloLens experience and tablet experience was not significantly different (p=0.675), the mean number of bought items was significantly higher in HoloLens experience compared to the tablet experience (p=0.004). In conclusion, AR increased customer’s interest in shopping. While it had no significant impact on adding products to the shopping cart, it affected the rate of purchase.

In this paper, we present and evaluate a method for trajectory reconstruction from IMU signals generated when a person ”air writes” text with a finger worn IMU to make the resulting text as human-readable as possible. The vision is to provide a virtual ”sticky note” allowing people to digitally attach simple texts to locations. Thus, for example, we envision a person walking by someone’s locked office door and simply air writing, ”let me know when you are back”. The other person would then have, for example, their phone vibrate when they come into the office and would see the message on their screen. The problem that we address is how to extract from such ”air writing”, performed without visual feedback or a real surface to write, de-noised 2D trajectories that can be later displayed on a screen in a way that is well readable to humans. We describe the sensor and its signals, the trajectory extraction algorithm, and a user study that shows that we can achieve a high degree of readability.

We present the system CoRSA to incorporate integrated sensors in millimeter-scale packages for continuous cardiorespiratory (CR) evaluation in sports activities. CoRSA retrofits trending sports apparel to add on CR sensing capability. The system uses an air pressure sensor inside a vented mask to approximate a spirometer, and an earlobe pulse-oximeter (PO) to monitor heart rate (HR) and oxygen saturation (SpO2). CoRSA also includes an inertial measurement unit for tracking activity and future study on motion artifact correction on the CR signals in active sports. An aerobic exercise evaluation is also performed which shows results similar to sports studies using bulkier conventional medical equipment in the CR signals' characteristics.

Being able to reliably predict muscle contractions is important for athletes and rehabilitation patients alike. Numerous techniques and surrogates exist for this task. However, they are in general not well suited for everyday use and not able to extract information of muscles located in deeper body layers. To address this shortcoming, we present an approach to classify muscle contractions with raw ultrasound radio-frequency data (A-Scans) collected with a wearable system. It consists of a single element ultrasound transducer connected to custom-built acquisition hardware and an Android app to receive, store and analyze the data. We rely on data from the lower legs of healthy volunteers performing squats as sample exercises and use machine learning methods, ranging from sequence similarity measurement techniques to artificial neural networks, to classify the radio-frequency data. Results of our preliminary experimental setup prove its feasibility to classify muscle contractions based on ultrasound measurements.

Shipping labels are widely used in logistics. It is important to ensure the quality of printing label and to verify contents of the shipping label on the package. We developed a verification and recognition method for various types of shipping labels by using deep neural networks. The experimental results showed 96% recognition accuracy in rotation-invariant conditions. Also, we introduce Google Maps API for validating the address which can reduce the cost of returning packages due to the invalid address. To train and evaluate the method, we have generated and collected 25 different types of shipping label dataset. We plan to release the dataset on our website1.

In this work, we present the design and implementation of a micro watt level power consumption, human body capacitance based sensor for recognizing and counting gym workouts. The concept also works when the device is attached to a body part which is not directly involved in the activity's movement. In contrast, most of the widely used motion sensing based approaches require placing the sensor on the moving body part (e.g. for analyzing leg based gym exercises the sensor needs to be placed on the leg). We described the physical principle behind the ubiquitous electric coupling between human body and environment, and explored the capability of this sensing modality in gym workouts. We evaluated our sensor with 11 subjects, performing 7 popular gym workouts each day over 5 days with our sensor being placed at 3 different body positions, including a non-contact position, where the sensor is placed in the subject's pocket. Results showed that our sensing approach achieved an average counting accuracy of 91%, which is highly competitive with commercial devices on the market. The mean leave one user out workout recognition f-scores obtained were of 63%, 56%, 45% for sensors located on wrist, on calf and in pocket, respectively. As every subject performed activities over multiple days changing shoe height, shoe and clothes type, we demonstrate that full body activity counting and to some extent recognition is feasible, regardless of personal habit of movement speed and scale.

Monitoring of human activities is an essential capability of many smart systems. In recent years much progress has been achieved. One of the key remaining challenges is the availability of labeled training data, in particular taking into account the degree of variability in human activities. A possible solution is to leverage large scale online data repositories. This has been previously attempted with image and sound data, as both microphones and cameras are widely used sensing modalities. In this paper, we describe a first step towards the use of online, text-based activity descriptions to support general sensor-based activity recognition systems. The idea is to extract semantic information from online texts about the way complex activities are composed of simple ones that have to be performed (e.g. a manual for assembling a furniture piece) and use such a semantic description in conjunction with sensor based, statistical classifiers of basic actions to recognize the complex activities and compose them into semantic trees. Extraction of domain relevant information evaluated in 11 different text-based manuals from different domains reached an average recall of 77%, and precision of 88%. Actual structural error-rate in the construction of respective trees was around 1%.

We present LYRA, a modular in-flight system that enhances service and assists flight attendants during their work. LYRA enables passengers to browse and order services from their smartphones. Smart glasses and a smart shoe-clip with RFID reader module provides flight attendants with situated information. We gained first insights into how flight attendants and passengers use of the system during a long distance flight from Frankfurt to Houston.

We present a study comparing the effect of real-time wearable feedback with traditional training methods for cardiopulmonary resuscitation (CPR). The aim is to ensure that the students can deliver CPR with the right compression speed and depth. On the wearable side, we test two systems: one based on a combination of visual feedback and tactile information on a smart-watch and one based on visual feedback and audio information on a Google Glass. In a trial with 50 subjects (23 trainee nurses and 27 novices,) we compare those modalities to standard human teaching that is used in nurse training. While a single traditional teaching session tends to improve only the percentage of correct depth, it has less effect on the percentage of effective CPR (depth and speed correct at the same time). By contrast, in a training session with the wearable feedback device, the average percentage of time when CPR is effective improves by up to almost 25%.

B This paper presents a novel approach, named D-StaR, for stamp segmentation from scanned document images. The presented approach is generic (applicable to stamps of any color, shape, size, and orientation) and based on deep learning. In particular, it uses Fully Convolutional networks for semantic analysis of documents to extract stamps. The presented approach is evaluated on a publicly available stamp dataset. Evaluation results show that the presented approach outperforms the state-of-the-art methods for stamp segmentation and achieves pixel based precision and recall of 87% and 84%, respectively. Deeper analysis of the evaluation reveals that the presented approach can segment both overlapping and non-overlapping stamps, which was always a problem for existing systems in the literature.

Convolutional Neural Networks (CNNs) have become the state-of-the-art in various computer vision tasks, but they are still premature for most sensor data, especially in pervasive and wearable computing. A major reason for this is the limited amount of annotated training data. In this paper, we propose the idea of leveraging the discriminative power of pre-trained deep CNNs on 2-dimensional sensor data by transforming the sensor modality to the visual domain. By three proposed strategies, 2D sensor output is converted into pressure distribution imageries. Then we utilize a pre-trained CNN for transfer learning on the converted imagery data. We evaluate our method on a gait dataset of floor surface pressure mapping. We obtain a classification accuracy of 87.66%, which outperforms the conventional machine learning methods by over 10%.

We present a wearable textile sensor system for monitoring muscle activity, leveraging surface pressure changes between the skin and an elastic sport support band. The sensor is based on an 8×16 element fabric resistive pressure sensing matrix of 1cm spatial resolution, which can be read out with 50fps refresh rate. We evaluate the system by monitoring leg muscles during leg workouts in a gym out of the lab. The sensor covers the lower part of quadriceps of the user. The shape and movement of the two major muscles (vastus lateralis and medialis) are visible from the data during various exercises. The system registers the activity of the user for every second, including which machine he/she is using, walking, relaxing and adjusting the machines; it also counts the repetitions from each set and evaluate the force consistency which is related to the workout quality. 6 people participated in the experiment of overall 24 leg workout sessions. Each session includes cross-trainer warm-up and cool-down, 3 different leg machines, 4 sets on each machine. Plus relaxing, adjusting machines, and walking, we perform activity recognition and quality evaluation through 2-dimensional mapping and the time sequence of the average force. We have reached 81.7% average recognition accuracy on a 2s sliding window basis, 93.3% on an event basis, and 85.6% spotting F1-score. We further demonstrate how to evaluate the workout quality through counting, force pattern variation and consistency.

We present a novel sensor system for the support of nutrition monitoring. The system is based on smart table cloth equipped with a fine grained pressure textile matrix and a weight sensitive tablet. Unlike many other nutrition monitoring approaches, our system is unobtrusive, non privacy invasive and easily deployable in every day life. It allows the spotting and recognition of food intake related actions, such as cutting, scooping, stirring, etc., the identification of the plate/container on which the action is executed, and the tracking of the weight change in the containers. In other words, we can determine how many pieces are cut on the main dish plate, how many are taken from the side dish, how many sips are taken from the drink, how fast the food is being consumed and how much weight is taken overall. In addition, the distinction between different eating actions, such as cutting, scooping, poking, provides clues to the type of food taken and the way the meal is consumed. We have evaluated our system on 40 meals (5 subjects) in a real life living environment: for seven eating related actions (cutting, scooping, stirring, etc.), resulting in above 90% average recognition rate for person dependent cases, and spotting each action out of continuous data streams (average F1 score 87%).

Today's health care is difficult to imagine without the possibility to objectively measure various physiological parameters related to patients' symptoms (from temperature through blood pressure to complex tomographic procedures). Psychiatric care remains a notable exception that heavily relies on patient interviews and self-assessment. This is due to the fact that mental illnesses manifest themselves mainly in the way patients behave throughout their daily life and, until recently there were no "behavior measurement devices." This is now changing with the progress in wearable activity recognition and sensor enabled smartphones. In this paper, we introduce a system, which, based on smartphone-sensing is able to recognize depressive and manic states and detect state changes of patients suffering from bipolar disorder. Drawing upon a real-life dataset of ten patients, recorded over a time period of 12 weeks (in total over 800 days of data tracing 17 state changes) by four different sensing modalities, we could extract features corresponding to all disease-relevant aspects in behavior. Using these features, we gain recognition accuracies of 76% by fusing all sensor modalities and state change detection precision and recall of over 97%. This paper furthermore outlines the applicability of this system in the physician-patient relations in order to facilitate the life and treatment of bipolar patients.

We demonstrate that a cheap (30USD) small, low power 8x8 thermal sensor array can by itself provide a broad range of information relevant for human activity monitoring in home and office environments. In particular the sensor can track people with an accuracy in the range of 1m (which is sufficient to recognize activity relevant regions), detect the operation mode of various appliances such as toaster, water cooker or egg cooker and actions such as opening a refrigerator, the oven or taking a shower. While there are sensing modalities for each of the above types of information (e.g. current sensors for appliances) the fact that they can all be detected by such a simple sensor is highly relevant for practical activity recognition systems. Compared to vision (or thermal imaging systems) the system has the advantage is being less privacy invasive allowing it for example to monitor bathroom activities (as shown in one of our evaluation scenarios). The paper describes the sensor, the methods used for activity detection and the evaluation.

There is a large class of routine physical exercises that are performed on the ground, often on dedicated "mats" (e.g. push-ups, crunches, bridge). Such exercises involve coordinated motions of different body parts and are difficult to recognize with a single body worn motion sensors (like a step counter). Instead a network of sensors on different body parts would be needed, which is not always practicable. As an alternative we describe a cheap, simple textile pressure sensor matrix, that can be unobtrusively integrated into exercise mats to recognize and count such exercises. We evaluate the system on a set of 10 standard exercises. In an experiment with 7 subjects, each repeating each exercise 20 times, we achieve a user independent recognition rate of 82.5% and a user independent counting accuracy of 89.9%. The paper describes the sensor system, the recognition methods and the experimental results.

We demonstrate through a pressure sensor matrix, that weight distribution on feet is influenced by body posture. A small cheap carpet equipped with low precision pressure sensor matrix is already sufficient to detect subtle activities and identity of the person on the carpet. By a 0.4 m2 matrix of 32 × 32, 12 bit pressure sensors, we achieve 78.7% accuracy for 11 test subjects performing 7 subtle activities (open 7 different drawers or cabinet doors) and 88.6% accuracy in recognizing who has performed the activities. We thus see the potential of using a single carpet as a unified approach in houses to detect how inhabitants interact with the furniture without attaching different sensors onto each single furniture.

In this paper we use ubiquitous smart-chairs to evaluate live presentations. We validate the hypothesis that the audiences' activities can be recognized with pressure sensors under chairs' legs (74.6% accuracy rate from 8 typical activities in 8 live presentations, each with 6 chairs seated), and ce

The output delivered by body-wide inertial sensing systems has proven to contain sufficient information to distinguish between a large number of complex physical activities. The bottlenecks in these systems are in particular the parts of such systems that calculate and select features, as the high dimensionality of the raw sensor signals with the large set of possible features tends to increase rapidly. This paper presents a novel method using a hierarchical clustering method on raw trajectory and angular segments from inertial data to detect and analyze the data from such a distributed set of inertial sensors. We illustrate on a public dataset, how this novel way of modeling can be of assistance in the process of designing a fitting activity recognition system. We show that our method is capable of highlighting class-representative modalities in such high-dimensional data and can be applied to pinpoint target classes that might be problematic to classify at an early stage.

No abstract available.

Mobile computing is changing the landscape of clinical monitoring and self-monitoring. One of the major impacts will be in healthcare, where increase in number of sensing modalities is providing more and more information on the state of overall wellbeing, behaviour and health. There are numerous applications of mobile computing that range from wellbeing applications, such as physical fitness, stress or burnout up to applications that target mental disorders including bipolar disorder. Use of information provided by mobile computing devices can track the state of the subjects and also allow for experience sampling in order to gather subjective information. This paper reports on the results obtained from a medical trial with monitoring of bipolar disorder patients and how the episodes of the diseases correlate to the analysis of the data sampled from mobile phone acting as a monitoring device.

We describe a method for user tracking and localization based on textile capacitive sensor arrays placed under the floor. The sensor array is a commercial product (SensFloor®) that can be installed under any standard floor type (from carpet to stone) and is able to detect objects (including the user's foot) being placed on it. The challenges addressed in this paper are (1) how to map sequences of such signals onto user trajectories and (2) how to correlate the steps detected by the SensFloor system with the step detection based on a wearable accelerometer as means of user identification. Footstep detection is performed online on the devices, which are seamlessly integrated with the floor's wireless sensor network. Initial experiments performed over a week in a real life office environment show the ability to track multiple humans and to identify up to three users walking in a narrow corridor at the same time.

We present a technique for estimating crowd density by using a mobile phone to scan the environment for Bluetooth devices. The paper builds on previous work directed to use Bluetooth scans to analyze social context and extends it with more advanced features, leveraging collaboration between close by devices, and the use of relative features that do not directly depend on the absolute number of devices in the environment. The method is evaluated on a data set from an experiment at the public viewing event in Kaiserslautern during the European soccer championship showing over 75% recognition accuracy on seven discrete classes.

We describe the design, implementation, and evaluation of an indoor positioning system based on resonant magnetic coupling. The system has an accuracy of less than 1 m2 and, because of the underlying physical principle, is robust with respect to disturbances such as people moving around or changes in room configuration. It consists of 16x16x16 cm transmitter coils, each able to cover an area of up to 50 m2, and provides location information to an arbitrary number of mobile receivers with an update rate of up to 30Hz. We evaluate the actual accuracy of the positioning with a robotic arm and show quantitatively that even large metallic objects have little effect on the signal. We then present an elaborate study of the performance of our system for the recognition of abstract locations such as "at the table", "in front of a cabinet". It comprises four different sites with a total of 100 individual locations some as little as 50 cm apart.

Summary form only given. With the advance of sensor enabled smart phones simple context awareness has become a mainstream feature. Commercial apps routinely use location knowledge for the delivery of customized information or fostering social interaction. There are also scores of apps that analyse modes of locomotion for purposes such as calories calorie expenditure assessment or exercise support. On the other hand, more detailed recognition of human activities and complex situations has so far had very little impact on real-life applications. That talk will look at factors that prevent wide spread use of complex activity recognition and discuss research that works to mitigate those factors. Topics will include working with dynamic, opportunistic sensor configurations, collaborative recognition and new sensing modalities.

The paper describes a year-long experiment dedicated to the development, deployment and evaluation of a system for coarse assessment of the state of dementia patients in a real world nursing home. The system is based on the analysis of users' motion patterns between broadly defined, semantically meaningful areas of the living space. On a data set recorded from six inhabitants in different stages of dementia during a year (a total of about 900 individual daily traces organized in groups of 14 days) a recognition accuracy of 92% is demonstrated for a two state problem (positive, negative)with 3 subjects having perfect (100%) recognition. For the 3 state problem (positive, normal, negative) we get 80% with one case of perfect recognition. We present an elaborate evaluation of the methods' performance and describe relevant practical issues involved in the design and deployment in the nursing home.

In this article, we introduce and evaluate a comprehensive set of performance metrics and visualisations for continuous activity recognition (AR). We demonstrate how standard evaluation methods, often borrowed from related pattern recognition problems, fail to capture common artefacts found in continuous AR—specifically event fragmentation, event merging and timing offsets. We support our assertion with an analysis on a set of recently published AR papers. Building on an earlier initial work on the topic, we develop a frame-based visualisation and corresponding set of class-skew invariant metrics for the one class versus all evaluation. These are complemented by a new complete set of event-based metrics that allow a quick graphical representation of system performance—showing events that are correct, inserted, deleted, fragmented, merged and those which are both fragmented and merged. We evaluate the utility of our approach through comparison with standard metrics on data from three different published experiments. This shows that where event- and frame-based precision and recall lead to an ambiguous interpretation of results in some cases, the proposed metrics provide a consistently unambiguous explanation.

We present an elaborate experimental study of the influence of device placement on the GPS location accuracy in three different mobile appliances: iphone 3gs, nokia n95 and nokia 810. On a total of some 52 km of walking traces we shaw that there are statistically significant difference in the errors between 5 on body locations: hand, front pocket of trousers, inner pocket of the jacket, inside a backpack. While the fact that such difference exists was well known before, this paper is the first to systematically study the effect.

In order to develop complexity science based modeling, prediction and simulation methods for large scale socio-technical systems in an Ambient Intelligence (AmI) based smart environment, we propose a scenario based modeling approach. With a case study on AmI technology to support the evacuation from emergency scenarios, i.e. the Life Belt, a wearable computing systems for vibro-tactile directional guidance, we introduce the concept of model scaling from a micro to a macro level. Aligned with the scenario, we present how crowd simulation strategies encoded into a small scale simulation setup can be extended to a mixed-level simulation based on combining model aspects also coming from the large scale model. The experimental results of a real evacuation trail at a local railway station are incorporated to compare the evacuation efficiency for three strategies: (i) Potential Map, (ii) Evacuees familiarity of the exits and (iii) Exits usage optimization. A comparison with the earlier results from small scale simulation suggest that a real large scale simulation results may not be similar to that of small scale simulation due to dynamics of crowd built up and complexity of building structure.

Opportunistic sensing allows to efficiently collect information about the physical world and the persons behaving in it. This may mainstream human context and activity recognition in wearable and pervasive computing by removing requirements for a specific deployed infrastructure. In this paper we introduce the newly started European research project OPPORTUNITY within which we develop mobile opportunistic activity and context recognition systems. We outline the project's objective, the approach we follow along opportunistic sensing, data processing and interpretation, and autonomous adaptation and evolution to environmental and user changes, and we outline preliminary results.

We present a set of heuristics that significantly increase the robustness of motion sensor-based activity recognition with respect to sensor displacement. In this paper placement refers to the position within a single body part (e.g, lower arm). We show how, within certain limits and with modest quality degradation, motion sensorbased activity recognition can be implemented in a displacement tolerant way. We first describe the physical principles that lead to our heuristic. We then evaluate them first on a set of synthetic lower arm motions which are well suited to illustrate the strengths and limits of our approach, then on an extended modes of locomotion problem (sensors on the upper leg) and finally on a set of exercises performed on various gym machines (sensors placed on the lower arm). In this example our heuristic raises the displaced recognition rate from 24% for a displaced accelerometer, which had 96% recognition when not displaced, to 82%.

In order to provide relevant information to mobile users, such as workers engaging in the manual tasks of maintenance and assembly, a wearable computer requires information about the user's specific activities. This work focuses on the recognition of activities that are characterized by a hand motion and an accompanying sound. Suitable activities can be found in assembly and maintenance work. Here, we provide an initial exploration into the problem domain of continuous activity recognition using on-body sensing. We use a mock "wood workshop" assembly task to ground our investigation. We describe a method for the continuous recognition of activities (sawing, hammering, filing, drilling, grinding, sanding, opening a drawer, tightening a vise, and turning a screwdriver) using microphones and three-axis accelerometers mounted at two positions on the user's arms. Potentially "interesting" activities are segmented from continuous streams of data using an analysis of the sound intensity detected at the two different locations. Activity classification is then performed on these detected segments using linear discriminant analysis (LDA) on the sound channel and hidden Markov models (HMMs) on the acceleration data. Four different methods at classifier fusion are compared for improving these classifications. Using user-dependent training, we obtain continuous average recall and precision rates (for positive activities) of 78 percent and 74 percent, respectively. Using user-independent training (leave-one-out across five users), we obtain recall rates of 66 percent and precision rates of 63 percent. In isolation, these activities were recognized with accuracies of 98 percent, 87 percent, and 95 percent for the user-dependent, user-independent, and user-adapted cases, respectively.

We describe our initial efforts to learn high-level human behaviors from low-level gestures observed using on-body sensors. Such an activity discovery system could be used to index captured journals of a person's life automatically. In a medical context, an annotated journal could assist therapists in helping to describe and treat symptoms characteristic to behavioral syndromes such as autism. We review our current work on user-independent activity recognition from continuous data where we identify "interesting" user gestures through a combination of acceleration and audio sensors placed on the user's wrists and elbows. We examine an algorithm that can take advantage of such a sensor framework to automatically discover and label recurring behaviors, and we suggest future work where correlations of these low-level gestures may indicate higher-level activities

Wearable computing has recently gained much popularity as an ambitious vision for future personalized mobile systems. Its aim is intelligent, environment aware systems unobtrusively embedded into the mobile environments of their users. With the combination of complex processing requirements, the necessity of placing sensors and input/output modules at different locations on the user's body, and stringent limits on size, weight, and battery capacity, the design of such systems is an inherently challenging problem. We demonstrate how systematic design and quantitative analysis can be applied to wearable architectures. We first present a model that allows various factors influencing the design of a wearable system to be incorporated into formal cost metrics. In particular, we show how to consistently incorporate specific wearable factors such as device placement requirements, ergonomics, and dynamic workload profiles into the model. We then discuss how efficient estimation algorithms can be extended and applied to the evaluation of different architectures with respect to our cost metrics. Finally, we discuss quantitative results from a proof-of-concept case study showing the trade offs between different architectures for a given wearable scenario. Summarized, we demonstrate how the description and the design of wearable systems can be put on a systematic, formal basis allowing us to treat them similar as conventional embedded systems.

This paper describes an advanced care and alert portable telemedical monitor (AMON), a wearable medical monitoring and alert system targeting high-risk cardiac/respiratory patients. The system includes continuous collection and evaluation of multiple vital signs, intelligent multiparameter medical emergency detection, and a cellular connection to a medical center. By integrating the whole system in an unobtrusive, wrist-worn enclosure and applying aggressive low-power design techniques, continuous long-term monitoring can be performed without interfering with the patients' everyday activities and without restricting their mobility. In the first two and a half years of this EU IST sponsored project, the AMON consortium has designed, implemented, and tested the described wrist-worn device, a communication link, and a comprehensive medical center software package. The performance of the system has been validated by a medical study with a set of 33 subjects. The paper describes the main concepts behind the AMON system and presents details of the individual subsystems and solutions as well as the results of the medical validation.