Abstract Depending on the degree of disability, simple tasks of daily living can be challenging for people with physical disabilities, such as picking up and placing objects, eating, or reaching for a cup to drink independently. Pervasive technologies such as robotic arms can be used to assist with these daily tasks, allowing patients to regain independence while reducing the need for care. Specialized devices, such as assistive forks or spoons, can facilitate these tasks. Image datasets of everyday objects such as MS COCO do not contain assistive devices, which tend to look different from their non-assistive counterparts. We present the dataset WLRI-AD (Work-Life Robotics Institute–Assistive Devices) to enable a robot to interact with devices in assisted living homes. The benefits of including assistive devices are demonstrated by comparing versions of the dataset with each other and to a baseline. Initial results show an improvement in the detection of assistive devices by training a YOLOv8 model on the assistive devices.

Eye-tracking-driven controls for assistive robotic arms provide people with severe physical disabilities with intuitive interaction opportunities.In this context, shared control can improve acceptance of the robot by partially automating task execution. Based on recent literature, we present a Wizard of Oz design for shared control driven by eye-tracking. This approach allows for the rapid exploration of user expectations to inform future design iterations. Two studies were conducted to assess user experience, identify design challenges, and determine ways to improve usability and accessibility. The first study involved people with severe motor disabilities (PSMD) and consisted of an online survey. The second study aimed to gain technical insights through direct robot interaction to provide a comprehensive overview of the findings.

Social robots are increasingly recognized as valuable supporters in the field of well-being coaching. They can function as independent coaches or provide support alongside human coaches, and healthcare professionals. In coaching interactions, these robots often handle sensitive information shared by users, making privacy a relevant issue. Despite this, little is known about the factors that shape users’ privacy perceptions. This research aims to examine three key factors systematically: (1) the transparency about information usage, (2) the level of specific user control over how the robot uses their information, and (3) the robot’s behavioral approach – whether it acts proactively or only responds on demand. Our results from an online study (N = 200) show that even when users grant the robot general access to personal data, they additionally expect the ability to explicitly control how that information is interpreted and shared during sessions. Experimental conditions that provided such control received significantly higher ratings for perceived privacy appropriateness and trust. Compared to user control, the effects of transparency and proactivity on privacy appropriateness perception were low, and we found no significant impact. The results suggest that merely informing users or proactive sharing is insufficient without accompanying user control. These insights underscore the need for further research on mechanisms that allow users to manage robots’ information processing and sharing, especially when social robots take on more proactive roles alongside humans.

Wearable devices have become efficient tools for sports performance analysis. Professional systems heavily rely on the high-tech setup, which are expensive and privacy-invasive for amateur players. This paper addresses the gap between advanced professional systems and limited consumer options by proposing a low-cost, privacy-preserving approach for basketball shot detection and outcome prediction. We leverage accelerome-ter data from wrist-worn smartwatches, combined with audio recordings, to develop a system capable of identifying shot movements and predicting shot outcomes. The shot detection was achieved by a ID CNN model through accelerometer data and outcome classification was achieved by an audio classification model. We evaluated the system on 6 participants, and the macro F1 score for shot outcome classification in data streams are 81.53% and 78.07% on dominant hand and non-dominant hand, respectively. Our system opens up explorations in other domains, including medical or industrial activity recognition, where similar approaches can be applied.

Research in sign languages, in general, is still a relatively new topic of study when compared to research into spoken languages. Most of African sign languages are endangered and severely under-studied [11]. In an attempt to (lexically) document as many endangered sign languages in Africa as possible, we have developed a low-barrier, online crowd-sourcing platform (AfricaSign) that enables the African deaf communities to document their sign languages. AfricaSign offers to users multiple input modes, accommodates regional variation in multiple sign languages and allows the use of avatar technology to describe signs. It is likely that this research will uncover typological features exhibited by African sign languages. Documentation of STEM vocabulary will also help facilitate access to education for the Deaf community.

Hardware accelerators for compute intensive algorithms such as convolutional neural networks benefit from number representations with reduced precision. In this paper, we evaluate and extend a number representation based on power-of-two quantization enabling bit-shift-based processing of multiplications. We found that weights of a neural network can either be represented by a single 4 bit power-of-two value or with two 4 bit values depending on accuracy requirements. We evaluate the classification accuracy of VGG-16 and ResNet50 on the ImageNet dataset with weights represented in our novel number format. To include a more complex task, we additionally evaluate the format on two networks for semantic segmentation. In addition, we design a novel processing element based on bit-shifts which is configurable in terms of throughput (4 bit mode) and accuracy (8 bit mode). We evaluate this processing element in an FPGA implementation of a dedicated accelerator for neural networks incorporating a 32-by-64 processing array running at 250 MHz with 1 TOp/s peak throughput in 8 bit mode. The accelerator is capable of processing regular convolutional layers and dilated convolutions in combination with pooling and upsampling. For a semantic segmentation network with 108.5 GOp/frame, our FPGA implementation achieves a throughput of 7.0 FPS in the 8 bit accurate mode and upto 11.2 FPS in the 4 bit mode corresponding to 760.1 GOp/s and 1,218 GOp/s effective throughput, respectively. Finally, we compare the novel design to classical multiplier-based approaches in terms of FPGA utilization and power consumption. Our novel multiply-accumulate engines designed for the optimized number representation uses 9 % less logical elements while allowing double throughput compared to a classical implementation. Moreover, a measurement shows 25 % reduction of power consumption at same throughput. Therefore, our flexible design offers a solution to the trade-off between energy efficiency, accuracy, and high throughput.

Affective computing aims to detect a person's affective state (e.g. emotion) based on observables. The link between affective states and biophysical data, collected in lab settings, has been established successfully. However, the number of realistic studies targeting affect detection in the wild is still limited. In this paper we present an exploratory field study, using physiological data of 11 healthy subjects. We aim to classify arousal, State-Trait Anxiety Inventory (STAI), stress, and valence self-reports, utilizing feature-based and convolutional neural network (CNN) methods. In addition, we extend the CNNs to multi-task CNNs, classifying all labels of interest simultaneously. Comparing the F1 score averaged over the different tasks and classifiers the CNNs reach an 1.8% higher score than the classical methods. However, the F1 scores barely exceed 45%. In the light of these results, we discuss pitfalls and challenges for physiology-based affective computing in the wild.

In this study, we consider the problem of detecting process-related anomalies for machine tools. The similar shape of successive sensor signals, which arises due to the same process step sequence applied to each workpiece, suggests extracting shape-related features. In recent years, shapelets dominated the field of shape-related features. Unfortunately, they involve a high computational burden due to hyperparameter optimization.We introduce alternative shape-related features relying on abrupt signal changes (changepoints) reflecting the changes of process steps. During normal operation, changepoints follow a highly recurrent pattern, i.e., appear at similar locations. Thus, being able to distinguish regular, recurrent from abnormal, non-recurrent changepoints allows detecting process anomalies.For changepoint recurrence estimation, we extend the Bayesian Online Changepoint Detection (BOCPD) method. The extension allows distinguishing normal and abnormal changepoints relying on empirical estimates of the changepoint recurrence distribution. Subsequently, changepoint-related features are introduced and compared to shapelets and wavelet-based features in a case study comprising real-world machine tool data.Qualitative results verify changepoint locations being comparable to shapelet locations found by the FLAG shapelet approach. Furthermore, quantitative results suggest superior classification performance both to shapelets and wavelet-based features.

Affect recognition aims to detect a person's affective state based on observables, with the goal to e.g. improve human-computer interaction. Long-term stress is known to have severe implications on wellbeing, which call for continuous and automated stress monitoring systems. However, the affective computing community lacks commonly used standard datasets for wearable stress detection which a) provide multimodal high-quality data, and b) include multiple affective states. Therefore, we introduce WESAD, a new publicly available dataset for wearable stress and affect detection. This multimodal dataset features physiological and motion data, recorded from both a wrist- and a chest-worn device, of 15 subjects during a lab study. The following sensor modalities are included: blood volume pulse, electrocardiogram, electrodermal activity, electromyogram, respiration, body temperature, and three-axis acceleration. Moreover, the dataset bridges the gap between previous lab studies on stress and emotions, by containing three different affective states (neutral, stress, amusement). In addition, self-reports of the subjects, which were obtained using several established questionnaires, are contained in the dataset. Furthermore, a benchmark is created on the dataset, using well-known features and standard machine learning methods. Considering the three-class classification problem ( baseline vs. stress vs. amusement ), we achieved classification accuracies of up to 80%,. In the binary case ( stress vs. non-stress ), accuracies of up to 93%, were reached. Finally, we provide a detailed analysis and comparison of the two device locations ( chest vs. wrist ) as well as the different sensor modalities.

A searchable database of multi-media recordings provides a way to augment one's memory. This database might contain video, audio and motion data, which is indexed to allow for quick searches. Ultimately, queries for similarity on each recorded modality would be supported. For example video sequencing showing similar objects or comparable sequences of gestures can be retrieved. An important aspect of this challenge is how to encode such multi-modal data, and how to make it searchable. One approach, based on a multi-media container format, is proposed in this paper together with an architecture to allow for similarity queries on multiple modalities.

Wet Laboratories are highly dynamic, shared environments full of tubes, racks, compounds, and dedicated machinery. The recording of experiments, despite the fact that several ubiquitous computing systems have been suggested in the past decades, still relies predominantly on hand-written notes. Similarly, the information retrieval capabilities inside a laboratory are limited to traditional computing interfaces, which due to safety regulations are sometimes not usable at all. In this paper, Google Glass is combined with a wrist-worn gesture sensor to support Wetlab experimenters. Taking "in-situ" documentation while an experiment is performed, as well as contextualizing the protocol at hand can be implemented on top of the proposed system. After an analysis of current practices and needs through a series of explorative deployments in wet labs, we motivate the need for a wearable hands-free system, and introduce our specific design to guide experimenters. Finally, using a study with 22 participants evaluating the system on a benchmark DNA extraction experiment, we explore the use of gesture recognition for enabling the system to track where the user might be in the experiment.

The physical activities we perform throughout our daily lives tell a great deal about our goals, routines, and behavior, and as such, have been known for a while to be a key indicator for psychiatric disorders. This paper focuses on the use of a wrist-watch with integrated inertial sensors. The algorithms that deal with the data from these sensors can automatically detect the activities that the patient performed from characteristic motion patterns. Such a system can be deployed for several weeks continuously and can thus provide the consulting psychiatrist an insight in their patient's behavior and changes thereof. Since these algorithms will never be flawless, however, a remaining question is how we can support the psychiatrist in assigning confidence to these automatic detections. To this end, we present a study where visualizations at three levels from a detection algorithm are used as feedback, and examine which of these are the most helpful in conveying what activities the patient has performed. Results show that just visualizing the classifier's output performs the best, but that user's confidence in these automated predictions can be boosted significantly by visualizing earlier pre-processing steps.

In the participatory sensing model, humans may serve as opportunistic sensors and flexible actuators while also consuming sensing services. Integrating humans into sensing systems has the potential to increase scale and reduce costs. However, contemporary participatory sensing software provides poor consideration of user dynamism, which includes: mobility across networks, mobility across devices and context-awareness. To address these limitations we propose the User Component and User Bindings. The former represents the user as a first class reconfigurable element of evolving and shared participatory sensing platforms. The latter allows the middleware to support multiple communications channels including Online Social Networks (OSN) to connect users with sensing applications. Our approach increases user participation, reduces out-of-context interactions and only consumes a limited amount of energy by sharing context information between applications. We support these claims by evaluating our approach on a two weeks experiment in which three participants take part in three concurrent participatory applications.

An ever-growing range of wireless sensors for medical monitoring has shown that there is significant interest in monitoring patients in their everyday surroundings. It however remains a challenge to merge information from several wireless sensors and applications are commonly built from scratch. This paper presents a middleware targeted for medical applications on smartphone-like platforms that relies on an event-based design to enable flexible coupling with changing sets of wireless sensor units, while posing only a minor overhead on the resources and battery capacity of the interconnected devices. We illustrate the requirements for such middleware with three different healthcare applications that were deployed with our middleware solution, and characterize the performance with energy consumption, overhead caused for the smartphone, and processing time under real-world circumstances. Results show that with sensing-intensive applications, our solution only minimally impacts the phone's resources, with an added CPU utilization of 3% and a memory usage under 7 MB. Furthermore, for a minimum message delivery ratio of 99.9%, up to 12 sensor readings per second are guaranteed to be handled, regardless of the number of applications using our middleware.

The output delivered by body-wide inertial sensing systems has proven to contain sufficient information to distinguish between a large number of complex physical activities. The bottlenecks in these systems are in particular the parts of such systems that calculate and select features, as the high dimensionality of the raw sensor signals with the large set of possible features tends to increase rapidly. This paper presents a novel method using a hierarchical clustering method on raw trajectory and angular segments from inertial data to detect and analyze the data from such a distributed set of inertial sensors. We illustrate on a public dataset, how this novel way of modeling can be of assistance in the process of designing a fitting activity recognition system. We show that our method is capable of highlighting class-representative modalities in such high-dimensional data and can be applied to pinpoint target classes that might be problematic to classify at an early stage.

We report in this paper on a wireless sensor network deployment at railway tracks to monitor and analyze the vibration patterns caused by trains passing by. We investigate in particular a system that relies on having a distributed network of sensor nodes that individually contain efficient feature extraction algorithms and classifiers that fit the restricted hardware resources, rather than using few complex and specialized sensors. A feasibility study is described on the raw data obtained from a real-world deployment on one of Europe's busiest railroad sections, which was annotated with the help of video footage and contains vibration patterns of 186 trains. These trains were classified in 6 types by various methods, the best performing at an accuracy of 97%. The trains' length in wagons was estimated with a mean-squared error of 3.98. Visual inspection of the data shows further opportunities in the estimation of train speed and detection of worn-out cargo wheels.

Having mobile devices that are capable of finding out what activity the user is doing, has been suggested as an attractive way to alleviate interaction with these platforms, and has been identified as a promising instrument in for instance medical monitoring. Although results of preliminary studies are promising, researchers tend to use high sampling rates in order to obtain adequate recognition rates with a variety of sensors. What is not fully examined yet, are ways to integrate into this the information that does not come from sensors, but lies in vast data bases such as time use surveys. We examine using such statistical information combined with mobile acceleration data to determine 11 activities. We show how sensor and time survey information can be merged, and we evaluate our approach on continuous day-and-night activity data from 17 different users over 14 days each, resulting in a data set of 228 days. We conclude with a series of observations, including the types of activities for which the use of statistical data has particular benefits.

Comparing experimental results obtained on different wireless sensor network deployments is typically very cumbersome and in most cases unfeasible. This is due to the lack of a methodology to describe the properties of network deployments and the experimental conditions under which experiments have been run. Our work focuses on the design and development of a site properties assessment framework, called SiteWork, that aims at providing the means to quickly, automatically and accurately quantify their properties. This poster abstract describes the preliminary design and evaluation of the basic site properties assessment mechanisms provided by SiteWork.

This paper proposes an activity inference system that has been designed for deployment in mood disorder research, which aims at accurately and efficiently recognizing selected leisure activities in week-long continuous data. The approach to achieve this relies on an unobtrusive and wrist-worn data logger, in combination with a custom data mining tool that performs early data abstraction and dense motif discovery to collect evidence for activities. After presenting the system design, a feasibility study on weeks of continuous inertial data from 6 participants investigates both accuracy and execution speed of each of the abstraction and detection steps. Results show that our method is able to detect target activities in a large data set with a comparable precision and recall to more conventional approaches, in approximately the time it takes to download and visualize the logs from the sensor.

Many sensor network applications observe trends over an area by regularly sampling slow-moving values such as humidity or air pressure (for example in habitat monitoring). Another well-published type of application aims at spotting sporadic events, such as sudden rises in temperature or the presence of methane, which are tackled by detection on the individual nodes. This paper focuses on a zone between these two types of applications, where phenomena that cannot be detected on the nodes need to be observed by relatively long sequences of sensor samples. An algorithm that stems from data mining is proposed that abstracts the raw sensor data on the node into smaller packet sizes, thereby minimizing the network traffic and keeping the essence of the information embedded in the data. Experiments show that, at the cost of slightly more processing power on the node, our algorithm performs a shape abstraction of the sensed time series which, depending on the nature of the data, can extensively reduce network traffic and nodes' power consumption.

The use of a wrist-worn sensor that is able to read nearby RFID tags and the wearer's gestures has been suggested frequently as a way to both detect the objects we interact with and to identify the interaction. Making such a prototype feasible for longer-term deployments is far from solved however, as plenty of challenges remain in the hardware, embedded algorithms, and the overall design of such a bracelet-like device. This paper presents several of the challenges that emerged during the development of a functioning prototype that is able to sense interaction data for several days. We focus in particular on RFID tag reading range optimization, efficient data logging methods, meaningful evaluation techniques, and long-term deployments.

Long-term activity recognition relies on wearable sensors that log the physical actions of the wearer, so that these can be analyzed afterwards. Recent progress in this field has made it feasible to log high-resolution inertial data, resulting in increasingly large data sets. We propose the use of piecewise linear approximation techniques to facilitate this analysis. This paper presents a modified version of SWAB to approximate human inertial data as efficiently as possible, together with a matching algorithm to query for similar subsequences in large activity logs. We show that our proposed algorithms are faster on human acceleration streams than the traditional ones while being comparable in accuracy to spot similar actions, benefitting post-analysis of human activity data.

This paper illustrates both challenges and benefits found in expanding a traditional game concept to a situated environment with a distributed set of wireless sensing modules. Our pervasive game equivalent of the Whac-A-Mole game, Whac-A-Bee, retains the find-and-seek aspects of the original game while extending the location, the number of players, and the time-span in which it can be played. We discuss the obstacles met during this work, and specifically address challenges in making the game robust and flexible enough for large and long-term deployments in unknown territory.

This paper introduces a fuzzy inference system, based on the Takagi-Sugeno-Kang model, to achieve efficient and reliable classification in the domain of ubiquitous computing, and in particular for smart or context-aware, sensor-augmented devices. As these are typically deployed in unpredictable environments and have a large amount of correlated sensor data, we propose to use a Gath-Geva clustering specification as well as a genetic algorithm approach to improve the model's robustness. Experiments on data from such a sensor-augmented device show that accuracy is boosted from 83% to 97% with these optimizations under normal conditions, and for more. challenging data from 54% to 79%.

This paper presents a low-cost and practical approach to achieve basic input using a tactile cube-shaped object, augmented with a set of sensors, processor, batteries and wireless communication. The algorithm we propose combines a finite state machine model incorporating prior knowledge about the symmetrical structure of the cube, with maximum likelihood estimation using multivariate Gaussians. The claim that the presented solution is cheap, fast and requires few resources, is demonstrated by implementation in a small-sized, microcontroller-driven hardware configuration with inexpensive sensors. We conclude with a few prototyped applications that aim at characterizing how the familiar and elementary shape of the cube allows it to be used as an interaction device.