Ambient intelligence demands collaboration schemes for distributed constrained devices which are not only highly energy efficient in distributed sensing, processing and communication, but which also respect data privacy. Traditional algorithms for distributed processing suffer in Ambient intelligence domains either from limited data privacy, or from their excessive processing demands for constrained distributed devices. In this paper, we present Camouflage learning, a distributed machine learning scheme that obscures the trained model via probabilistic collaboration using physical-layer computation offloading and demonstrate the feasibility of the approach on backscatter communication prototypes and in comparison with Federated learning. We show that Camouflage learning is more energy efficient than traditional schemes and that it requires less communication overhead while reducing the computation load through physical-layer computation offloading. The scheme is synchronization-agnostic and thus appropriate for sharply constrained, synchronization-incapable devices. We demonstrate model training and inference on four distinct datasets and investigate the performance of the scheme with respect to communication range, impact of challenging communication environments, power consumption, and the backscatter hardware prototype.

We describe a methodology to measure ballistocardiography (BCG) signals from the body surface, using body-worn digital accelerometers to extract medically relevant information for Pervasive Healthcare. We are able to measure measuring heart rate with an 95% accuracy as well as other cardiac metrics, such as the S1-S2 interval, deviating from ECG by only 1.3%. Our results show that BCG can be a viable alternative to an electrocardiogram to provide complementary information on the heart's condition in mobile and pervasive use cases. We further show that BCG information can be detected from arm as reliably as from chest, which is especially convenient for measuring from supine positions in Pervasive healthcare applications.

Voluntary fire departments have limited human and material resources. Machine learning aided prediction of fire department operation details can benefit their resource planning and distribution. While there is previous work on predicting certain aspects of operations within a given operation category, operation categories themselves have not been predicted yet. In this paper we propose an approach to fire department operation category prediction based on location, time, and weather information, and compare the performance of multiple machine learning models with cross validation. To evaluate our approach, we use two years of fire department data from Upper Austria, featuring 16.827 individual operations, and predict its major three operation categories. Preliminary results indicate a prediction accuracy of 61%. While this performance is already noticeably better than uninformed prediction (34% accuracy), we intend to further reduce the prediction error utilizing more sophisticated features and models.

Automatic tennis stroke recognition can help tennis players improve their training experience. Previous work has used sensors positions on both wrist and tennis racket, of which different physiological aspects bring different sensing capabilities. However, no comparison of the performance of both positions has been done yet. In this paper we comparatively assess wrist and racket sensor positions for tennis stroke detection and classification. We investigate detection and classification rates with 8 well-known stroke types and visualize their differences in 3D acceleration and angular velocity. Our stroke detection utilizes a peak detection with thresholding and windowing on the derivative of sensed acceleration, while for our stroke recognition we evaluate different feature sets and classification models. Despite the different physiological aspects of wrist and racket as sensor position, for a controlled environment results indicate similar performance in both stroke detection (98.5%-99.5%) and user-dependent and independent classification (89%-99%).

Smart glasses allow for gaze gesture passwords as a hands-free form of mobile authentication. However, pupil movements for password input are easily observed by attackers, who thereby can derive the password. In this paper we investigate closed-eye gaze gesture passwords with EOG sensors in smart glasses. We propose an approach to detect and recognize closed-eye gaze gestures, together with a 7 and 9 character gaze gesture alphabet. Our evaluation indicates good gaze gesture detection rates. However, recognition is challenging specifically for vertical eye movements with 71.2%-86.5% accuracy and better results for opened than closed eyes. We further find that closed-eye gaze gesture passwords are difficult to attack from observations with 0% success rate in our evaluation, while attacks on open eye passwords succeed with 61%. This indicates that closed-eye gaze gesture passwords protect the authentication secret significantly better than their open eye counterparts.

Contemporary personal mobile devices support a variety of authentication approaches, featuring different levels of security and usability. With cameras embedded in smart glasses, seamless, hands-free mobile authentication based on gaze is possible. Gaze authentication relies on knowledge as a secret, and gaze passwords are composed from a series of gaze points or gaze gestures. This paper investigates the concept of free-form mobile gaze passwords. Instead of relying on gaze gestures or points, free-form gaze gestures exploit the trajectory of the gaze over time. We collect and investigate a set of 29 different free-form gaze passwords from 19 subjects. In addition, the practical security of the approach is investigated in a study with 6 attackers observing eye movements during password input to subsequently perform spoofing. Our investigation indicates that most free-form gaze passwords can be expressed as a set of common geometrical shapes. Further, our free-form gaze authentication yields a true positive rate of 81% and a false positive rate with other gaze passwords of 12%, while targeted observation and spoofing is successful in 17.5% of all cases. Our usability study reveals that further work on the usability of gaze input is required as subjects reported that they felt uncomfortable creating and performing free-form passwords.

This paper presents the design and open source implementation of Cormorant, an Android authentication framework able to increase usability and security of mobile authentication. It uses transparent behavioral and physiological biometrics like gait, face, voice, and keystrokes dynamics to continuously evaluate the user's identity without explicit interaction. Using signals like location, time of day, and nearby devices to assess the risk of unauthorized access, the required level of confidence in the user's identity is dynamically adjusted. Authentication results are shared securely, end-to-end encrypted using the Signal messaging protocol, with trusted devices to facilitate cross-device authentication for co-located devices, detected using Bluetooth low energy beacons. Cormorant is able to reduce the authentication overhead by up to 97% compared to conventional knowledge-based authentication whilst increasing security at the same time. We share our perspective on some of the successes and shortcomings we encountered implementing and evaluating Cormorant to hope to inform others working on similar projects.

Biometrics have become important for mobile authentication, e.g., to unlock devices before using them. One way to protect biometric information stored on mobile devices from disclosure is using embedded smart cards (SCs) with biometric match-on-card (MOC) approaches. However, computational restrictions of SCs also limit biometric matching procedures. We present a mobile MOC approach that uses offline training to obtain authentication models with a simplistic internal representation in the final trained state, where we adapt features and model representation to enable their usage on SCs. The pre-trained model can be shipped with SCs on mobile devices without requiring retraining to enroll users. We apply our approach to acceleration based mobile gait authentication as well as face authentication and compare authentication accuracy and computation time of 16 and 32 bit Java Card SCs. Using 16 instead of 32 bit SCs has little impact on authentication performance and is faster due to less data transfer and computations on the SC. Results indicate 11.4 and 2.4-5.4 percent EER for gait respectively face authentication, with transmission and computation durations on SCs in the range of 2 s respectively 1 s. To the best of our knowledge, this work represents the first practical approach towards acceleration based gait MOC authentication.

As users start carrying multiple mobile devices, we propose a novel, token based mobile device unlocking approach. Mobile devices are conjointly shaken to transfer the authentication state from an unlocked token device to another device to unlock it. A common use case features a wrist watch as token device, which remains unlocked as long as it is strapped to the user's wrist, and a locked mobile phone, which is unlocked if both devices are shaken conjointly. Shaking can be done single-handedly, requires little user attention (users don't have to look at the device for unlocking it), and does not cause additional cognitive load on users. In case attackers gain control over the locked phone, forging shaking is difficult, which impedes malicious unlocks. We evaluate our approach using acceleration records from our 29 people sized ShakeUnlock database and discuss influence of its constituent parts on the system performance. We further present a performance study using an Android implementation and live data, which shows the true negative rate of observational attacks to be in the range of 0.8-if an attacker manages to gain control over the locked device and shake it in parallel to the device owner shaking the token device.

Mobile devices, ubiquitous in modern lifestyle, embody and provide convenient access to our digital lives. Being small and mobile, they are easily lost or stole, therefore require strong authentication to mitigate the risk of unauthorized access. Common knowledge-based mechanism like PIN or pattern, however, fail to scale with the high frequency but short duration of device interactions and ever increasing number of mobile devices carried simultaneously. To overcome these limitations, we present CORMORANT, an extensible framework for risk-aware multi-modal biometric authentication across multiple mobile devices that offers increased security and requires less user interaction.

Users usually authenticate to mobile devices before using them (e.g. PIN, password), but devices do not do the same to users. Revealing the authentication secret to a non-authenticated device potentially enables attackers to obtain the secret, by replacing the device with an identical-looking malicious device. The revealed authentication secret could be transmitted to the attackers immediately, who then conveniently authenticate to the real device. Addressing this attack scenario, we analyze different approaches towards mobile device-to-user (D2U) authentication, for which we provide an overview of advantages/drawbacks, potential risks and device authentication data bandwidth estimations. We further analyze vibration as one D2U feedback channel that is unobtrusive and hard to eavesdrop, including a user study to estimate vibration pattern recognition using a setup of ~7 bits per second (b/s). Study findings indicate that users are able to distinguish vibration patterns with median correctness of 97.5% (without taking training effects into account) -- which indicates that vibration could act as authentication feedback channel and should be investigated further in future research.

Detecting if two or multiple devices are moved together is an interesting problem for different applications. However, these devices may be aligned arbitrarily with regards to each other, and the three dimensions sampled by their respective local accelerometers can therefore not be directly compared. The typical approach is to ignore all angular components and only compare overall acceleration magnitudes --- with the obvious disadvantage of discarding potentially useful information. In this paper, we contribute a method to analytically determine relative spatial alignment of two devices based on their acceleration time series. Our method uses quaternions to compute the optimal rotation with regards to minimizing the mean squared error. The implication is that the reference system of one device can be (locally and independently) aligned with the other, and thus that all three dimensions can consequently be compared for more accurate classification. Based on real-world experimental data from smart phones and smart watches shaken together, we demonstrate the effectiveness of our method with a magnitude squared coherence metric, for which we show an improved EER of 0.16 (when using derotation) over an EER of 0.18 (when not using derotation).

The inherent weakness of typical mobile device unlocking approaches (PIN, password, graphic pattern) is that they demand time and attention, leading a majority of end users to disable them, effectively lowering device security.

Smartphones and tablets are an indispensable part of modern communication and people spend considerable time interacting with their devices every day. While substantial research has been conducted concerning smartphone usage, little is known about how tablets are used. This paper studies mobile device usage characteristics like session length, interaction frequency, and daily usage in locked and unlocked state with respect to location context. Based on logs from 1,585 Android devices (470 years of total usage time), we derive and analyze 23 million usage sessions. We found that devices remain locked for 60% of the interactions and usage at home occurs twice as frequent as at work. With an average of 58 interactions per day, smartphones are used twice as often as tablets, while tablet sessions are 2.5 times longer, resulting in almost equal aggregated daily usage. We conclude that usage session characteristics differ considerably between tablets and smartphones.

Face detection (finding faces of different perspectives in images) is an important task as prerequisite to face recognition. This is especially difficult in the mobile domain, as bad image quality and illumination conditions lead to overall reduced face detection rates. Background information still present in segmented faces and unequally normalized faces further decrease face recognition rates. We present a novel approach to robust single upright face detection and segmentation from different perspectives based on range information (pixel values corresponding to the camera-object distance). We use range template matching for finding the face's coarse position and gradient vector flow (GVF) snakes for precisely segmenting faces. We further evaluate our approach on range faces from the u'smile face database, then perform face recognition using the segmented faces to evaluate and compare our approach with previous research. Results indicate that range template matching might be a good approach to finding a single face; in our tests we achieved an error free detection rate and average recognition rates above 98%/96% for color/range images.

Currently, reliable face detection and recognition are becoming more important on mobile devices -- e.g. to unlock the screen. However, using only frontal face images for authentication purposes can no longer be considered secure under the assumption of easy availability of frontal snapshots of the respective device owners from social networks or other media. In most current implementations, a sufficiently high-resolution face image displayed on another mobile device will be enough to circumvent security measures. In this paper, we analyze current methods to face detection and recognition regarding their usability in the mobile domain, and then propose an approach to a Face Unlock system on a smart phone intended to be more secure than current approaches while still being convenient to use: we use both frontal and profile face information available during a pan shot around the user's head, by combining camera images and movement sensor data. Current results to face detection are promising, but reliable face recognition needs further research.