Benchmarking and establishing proper statistical validation metrics for reinforcement learning (RL) remain ongoing challenges, where no consensus has been established yet. The emergence of quantum computing and its potential applications in quantum reinforcement learning (QRL) further complicate benchmarking efforts. To enable valid performance comparisons and to streamline current research in this area, we propose a novel benchmarking methodology, which is based on a statistical estimator for sample complexity and a definition of statistical outperformance. Furthermore, considering QRL, our methodology casts doubt on some previous claims regarding its superiority. We conducted experiments on a novel benchmarking environment with flexible levels of complexity. While we still identify possible advantages, our findings are more nuanced overall. We discuss the potential limitations of these results and explore their implications for empirical research on quantum advantage in QRL.

Infrared Local Positioning Systems (IRLPS) offer a cost-effective and accurate alternative for indoor localization, where GNSS signals are typically not available. Despite their potential, IRLPS face significant challenges such as noise, multipath effects, multiple access interference, and calibration requirements, which limit their performance. In response, this work explores the integration of machine learning by proposing a Feedforward Neural Network (FNN) trained on energy measurements collected from a quadrant photodiode. We conducted a thorough analysis of hyperparameters and an ablation study across five neural network topologies and identified configurations that balance accuracy and model complexity. Experimental evaluations in a controlled indoor environment (2.4×2.4×3.4 m3) demonstrate that even simple FNN architectures can generalize well and achieve a high accuracy, with 90% of the positioning errors being below 0.05 m.

Fingerprint-based passive localization enables high localization accuracy using low-cost UWB IoT radio sensors. However, fingerprinting demands extensive effort for data acquisition. The concept of channel charting reduces this effort by modeling and projecting the manifold of channel state information (CSI) onto a 2D coordinate space. So far, researchers have only applied this concept to active radio localization, where a mobile device intentionally and actively emits a specific signal.In this paper, we apply channel charting to passive localization. We use a pedestrian dead reckoning (PDR) system to estimate a target's velocity and derive a distance matrix from it. We then use this matrix to learn a distance-preserving embedding in 2D space, which serves as a fingerprinting model. In our experiments, we deploy six nodes in a fully connected ultra-wideband (UWB) mesh network to show that our method achieves high localization accuracy, with an average error of just 0.24 m, even when we train and test on different targets.

The rapid growth of 5G and future 6G networks requires efficient and scalable radio access network (RAN) densification, especially in dense urban and industrial areas. Traditional planning uses manual surveys and simple propagation models, but these lack spatial accuracy and adaptability. Stochastic RF simulation tools often fail to model real-world conditions, such as material properties and geometry. This leads to poor site selection, higher costs, and rollout delays.This paper proposes an AI-based framework that combines high-resolution 3D modeling, Digital Twin technology, and deterministic ray tracing. It uses aerial and ground imagery to build detailed 3D models, enhanced with object detection and material classification through segmentation models. These models enable automatic feature extraction for RF simulation and planning. The system uses open-source 3D tools, vision transformers for segmentation, and a simulation engine with antenna radiation patterns and material-aware propagation. Tests in urban and campus settings show better prediction accuracy, less manual work, and lower costs than traditional methods. Results show that AI and Digital Twins improve and automate network deployment.

Federated learning (FL) enables multiple devices to collaboratively train a global model while maintaining data on local servers. Each device trains the model on its local server and shares only the model updates (i.e., gradient weights) during the aggregation step. A significant challenge in FL is managing the feature distribution of novel and unbalanced data across devices. In this paper, we propose an FL approach using few-shot learning and aggregation of the model weights on a global server. We introduce a dynamic early stopping method to balance out-of-distribution classes based on representation learning, specifically utilizing the maximum mean discrepancy of feature embeddings between local and global models. An exemplary application of FL is to orchestrate machine learning models along highways for interference classification based on snapshots from global navigation satellite system (GNSS) receivers. Extensive experiments on four GNSS datasets from two real-world highways and controlled environments demonstrate that our FL method surpasses state-of-the-art techniques in adapting to both novel interference classes and multipath scenarios. https://gitlab.cc-asp.fraunhofer.de/darcy_gnss/federated_learning

On-device training of deep neural networks (DNNs) allows models to adapt and fine tune to newly collected data or changing domains while deployed on microcontroller units (MCUs). However, DNN training is a resource-intensive task, making the implementation and execution of DNN training algorithms on MCUs challenging due to low processor speeds, constrained throughput, limited floating-point support, and memory constraints. In this work, we explore on-device training DNNs for different sized Cortex-M MCUs (Cortex-M0+, Cortex-M4, and Cortex-M7). We present a method that enables efficient training of DNNs completely in place on the MCU using fully quantized training (FQT) and dynamic partial gradient updates. We demonstrate the feasibility of our approach on multiple vision and time-series datasets and provide insights into the tradeoff between training accuracy, memory overhead, energy, and latency on real hardware. The results show that compared to related work, our approach requires 34.8% less memory and has a 49.0% lower latency per training sample, with dynamic partial gradient updates allowing a speedup of up to 8.7 compared to fully updating all weights.

Deploying deep neural networks (DNNs) on microcontrollers (TinyML) is a common trend to process the increasing amount of sensor data generated at the edge, but in practice, resource and latency constraints make it difficult to find optimal DNN candidates. Neural architecture search (NAS) is an excellent approach to automate this search and can easily be combined with DNN compression techniques commonly used in TinyML. However, many NAS techniques are not only computationally expensive, especially hyperparameter optimization (HPO), but also often focus on optimizing only a single objective, e.g., maximizing accuracy, without considering additional objectives such as memory requirements or computational complexity of a DNN, which are key to making deployment at the edge feasible. In this article, we propose a novel NAS strategy for TinyML based on multi-objective Bayesian optimization (MOBOpt) and an ensemble of competing parametric policies trained using augmented random search (ARS) reinforcement learning (RL) agents. Our methodology aims at efficiently finding tradeoffs between a DNN’s predictive accuracy, memory requirements on a given target system, and computational complexity. Our experiments show that we consistently outperform existing MOBOpt approaches on different datasets and architectures such as ResNet-18 and MobileNetv3.

Deep reinforcement learning (DRL) often requires a large number of data and environment interactions, making the training process time-consuming. This challenge is further exacerbated in the case of batch RL, where the agent is trained solely on a pre-collected dataset without environment interactions. Recent advancements in quantum computing suggest that quantum models might require less data for training compared to classical methods. In this paper, we investigate this potential advantage by proposing a batch RL algorithm that utilizes variational quantum circuits (VQCs) as function approximators within the discrete batch-constraint deep Q-learning (BCQ) algorithm. Additionally, we introduce a novel data re-uploading scheme by cyclically shifting the order of input variables in the data encoding layers. We evaluate the efficiency of our algorithm on the OpenAI CartPole environment and compare its performance to the classical neural network-based discrete BCQ.

Localization based on channel impulse responses (CIRs) of radio frequency (RF) signals yields centimeter-accurate positions under optimal line-of-sight (LOS) propagation conditions. However, in real indoor environments, e.g., in car manufacturing, non-line-of-sight (NLOS) situations dominate. Here, multipath propagation affects the time-of-arrival (ToA) estimation and downstream multilateration and localization accuracy. The detection and subsequent mitigation of NLOS per transceiver line compensates for these effects. To detect NLOS, the state-of-the-art employs either supervised or unsupervised learning methods that require the acquisition of expensive reference data or do not generalize to changes or unknown environments. This is due to, among other things, the fact that they cannot exploit spatial and temporal information from CIR signal streams.Thus, we propose a generative deep state space model (SSM) for NLOS detection on CIRs that exploits time and space. Our ultra-wideband (UWB) experiments show that our dynamical variational autoencoder (DVAE) detects NLOS signals from sequences of CIRs more accurately than the state-of-the-art and is robust to unknown environments.

Artificial Intelligence (AI)-based radio fingerprinting (FP) outperforms classic localization methods in propagation environments with strong multipath effects. However, the model and data orchestration of FP are time-consuming and costly, as it requires many reference positions and extensive measurement campaigns for each environment. Instead, modern unsupervised and self-supervised learning schemes require less reference data for localization, but either their accuracy is low or they require additional sensor information, rendering them impractical.In this paper we propose a self-supervised learning framework that pre-trains a general transformer (TF) neural network on 5G channel measurements that we collect on-the-fly without expensive equipment. Our novel pretext task randomly masks and drops input information to learn to reconstruct it. So, it implicitly learns the spatiotemporal patterns and information of the propagation environment that enable FP-based localization. Most interestingly, when we optimize this pre-trained model for localization in a given environment, it achieves the accuracy of state-of-the-art methods but requires ten times less reference data and significantly reduces the time from training to operation.

The localization of objects is essential in many applications, such as robotics, virtual and augmented reality, and warehouse logistics. Recent advancements in deep learning have enabled localization using monocular cameras. Traditionally, structure from motion (SfM) techniques predict an object’s absolute position from a point cloud, while absolute pose regression (APR) methods use neural networks to understand the environment semantically. However, both approaches face challenges from environmental factors like motion blur, lighting changes, repetitive patterns, and featureless areas. This study addresses these challenges by incorporating additional information and refining absolute pose estimates with relative pose regression (RPR) methods. RPR also struggles with issues like motion blur. To overcome this, we compute the optical flow between consecutive images using the Lucas–Kanade algorithm and use a small recurrent convolutional network to predict relative poses. Combining absolute and relative poses is difficult due to differences between global and local coordinate systems. Current methods use pose graph optimization (PGO) to align these poses. In this work, we propose recurrent fusion networks to better integrate absolute and relative pose predictions, enhancing the accuracy of absolute pose estimates. We evaluate eight different recurrent units and create a simulation environment to pre-train the APR and RPR networks for improved generalization. Additionally, we record a large dataset of various scenarios in a challenging indoor environment resembling a warehouse with transportation robots. Through hyperparameter searches and experiments, we demonstrate that our recurrent fusion method outperforms PGO in effectiveness.

Quantum machine learning implemented by variational quantum circuits (VQCs) is considered a promising concept for the noisy intermediate-scale quantum computing era. Focusing on applications in quantum reinforcement learning, we propose an action decoding procedure for a quantum policy gradient approach. We introduce a quality measure that enables us to optimize the classical post-processing required for action selection, inspired by local and global quantum measurements. The resulting algorithm demonstrates a significant performance improvement in several benchmark environments. With this technique, we successfully execute a full training routine on a 5-qubit hardware device. Our method introduces only negligible classical overhead and has the potential to improve VQC-based algorithms beyond the field of quantum reinforcement learning.

The widespread success of convolutional neural networks may largely be attributed to their intrinsic property of translation equivariance. However, convolutions are not equivariant to variations in scale and fail to generalize to objects of different sizes. Despite recent advances in this field, it remains unclear how well current methods generalize to unobserved scales on real-world data and to what extent scale equivariance plays a role. To address this, we propose the novel Scaled and Translated Image Recognition (STIR) benchmark based on four different domains. Additionally, we introduce a new family of models that applies many re-scaled kernels with shared weights in parallel and then selects the most appropriate one. Our experimental results on STIR show that both the existing and proposed approaches can improve generalization across scales compared to standard convolutions. We also demonstrate that our family of models is able to generalize well towards larger scales and improve scale equivariance. Moreover, due to their unique design we can validate that kernel selection is consistent with input scale. Even so, none of the evaluated models maintain their performance for large differences in scale, demonstrating that a general understanding of how scale equivariance can improve generalization and robustness is still lacking.

Modern radio frequency based positioning systems exploit multipath propagation to achieve accurate and robust positioning at a minimum effort in infrastructure. A key concept is exploitation of multipath component (MPC) delays from channel measurements, which have a direct relation to the geometry of the environment. This is a challenging task given complex multipath-rich environments and limited bandwidths. However, downstream tasks suffer from false or missed detections, which is why reliable MPC detection and delay estimation is crucial. We propose an MPC delay estimation pipeline based on a Transformer (TF) neural network, which implicitly estimates the number and delays of the MPCs. We achieve subsample accuracy without using computational expensive super-resolution techniques. Our approach outperforms state-of-the art on detection and delay estimation of MPCs on different bandwidths. We also show that our approach can easily be fine-tuned on real world data with very few labeled data samples, making it a well-suited candidate for real world deployments.

Indoor radio environments often consist of areas with mixed propagation conditions. In LoS-dominated areas, classic ToF methods reliably return optimal (accurate) positions, while in NLoS-dominated areas (AI-based) fingerprinting methods are required. However, these fingerprinting methods are only cost-efficient if they are used exclusively in NLoS-dominated areas due to an expensive life cycle management. Systems that are both accurate and cost-efficient in LoS- and NLoS-dominated areas require an identification of those areas to select the optimal localization method. In this paper we propose methods for uncertainty estimation of AI-based fingerprinting to determine its validity. Our experiments show that we can implicitly switch between classic and fingerprinting-based approaches to reliably estimate accurate positions, even in NLoS-dominated radio environments. Our approach works even if the AI models are only trained on radio data in certain areas of the environment. In contrast to the state-of-the-art, our approach intrinsically identifies the spatial boundaries of the AI model, and thus does not require prior area identification.

Beamforming-capable antenna arrays overcome the high free-space path loss at higher carrier frequencies. However, the beams must be properly aligned to ensure that the highest power is radiated towards (and received by) the user equipment (UE). While there are methods that improve upon an exhaustive search for optimal beams by some form of hierarchical search, they can be prone to return only locally optimal solutions with small beam gains. Other approaches address this problem by exploiting contextual information, e.g., the position of the UE or information from neighboring base stations (BS), but the burden of computing and communicating this additional information can be high. Methods based on machine learning so far suffer from the accompanying training, performance monitoring and deployment complexity that hinders their application at scale.This paper proposes a novel method for solving the initial beam-discovery problem. It is scalable, and easy to tune and to implement. Our algorithm is based on a recommender system that associates groups (i.e., UEs) and preferences (i.e., beams from a codebook) based on a training data set. Whenever a new UE needs to be served our algorithm returns the best beams in this user cluster. Our simulation results demonstrate the efficiency and robustness of our approach, not only in single BS setups but also in setups that require a coordination among several BSs. Our method consistently outperforms standard baseline algorithms in the given task.

The performance of a machine learning model degrades when it is applied to data from a similar but different domain than the data it has initially been trained on. To mitigate this domain shift problem, domain adaptation (DA) techniques search for an optimal transformation that converts the (current) input data from a source domain to a target domain to learn a domain-invariant representation that reduces domain discrepancy. This paper proposes a novel supervised DA based on two steps. First, we search for an optimal class-dependent transformation from the source to the target domain from a few samples. We consider optimal transport methods such as the earth mover's distance, Sinkhorn transport and correlation alignment. Second, we use embedding similarity techniques to select the corresponding transformation at inference. We use correlation metrics and higher-order moment matching techniques. We conduct an extensive evaluation on time-series datasets with domain shift including simulated and various online handwriting datasets to demonstrate the performance.

Fingerprint-based indoor positioning has attracted a lot of interest due to its potential to meet a positional accuracy that enables many location-based 5G indoor services. However, the accuracy of fingerprinting decreases with changes in the environment which prevents positioning in new scenarios. On the other hand, naively acquiring up-to-date training data from the changed environment to retrain the model is often time-consuming. It is unclear whether after a change in the environment, a fingerprint model can be (data-)efficiently updated.This paper examines the generalizability (with respect to accuracy, robustness, and effort in recording data) of state-of-the-art fingerprint models based on a convolutional neural network (CNN) in realistic setups with changes in the environment. We propose a transfer learning (TL) method that exploits realistic synthetic Channel State Information (CSI) obtained with the Quasi Deterministic Radio channel Generator (QuaDRiGa), used to pre-train the CNN-based fingerprint model so that it can be adapted to any real (NLoS) propagation scenario with a low number of real training samples. Our experiments show that the positioning accuracy using fine-tuning improves by 37% in changed and by 19% in new environments.

Multivariate Time Series (MTS) classification is important in various applications such as signature verification, person identification, and motion recognition. In deep learning these classification tasks are usually learned using the cross-entropy loss. A related yet different task is predicting trajectories observed as MTS. Important use cases include handwriting reconstruction, shape analysis, and human pose estimation. The goal is to align an arbitrary dimensional time series with its ground truth as accurately as possible while reducing the error in the prediction with a distance loss and the variance with a similarity loss. Although learning both losses with Multi-Task Learning (MTL) helps to improve trajectory alignment, learning often remains difficult as both tasks are contradictory. We propose a novel neural network architecture for MTL that notably improves the MTS classification and trajectory regression performance in online handwriting (OnHW) recognition. We achieve this by jointly learning the cross-entropy loss in combination with distance and similarity losses. On an OnHW task of handwritten characters with multivariate inertial and visual data inputs we are able to achieve crucial improvements (lower error with less variance) of trajectory prediction while still improving the character classification accuracy in comparison to models trained on the individual tasks.

Channel measurements at sufficiently high bandwidth in multipath-rich environments include a variety of delay information, which, if accurately extracted, can be exploited for accurate positioning. While previous methods are limited in practice as they rely on iteratively extracting a fixed number of delays, we instead formulate the delay extraction problem as a time series segmentation task. For this, we propose a pipeline built upon the U-Net convolutional neural network architecture. Unlike the state of the art our pipeline extracts an arbitrary number of delays without prior knowledge, includes a threshold for weighting between detection rate and false alarms, and does not rely on computationally demanding operations such as eigenvalue decomposition. We evaluate the presented method with synthetic data of different noise configurations and signal bandwidths and a publicly available dataset, achieving considerable performance gains w.r.t. detection performance and tracking accuracy. Furthermore, we show that the proposed method is far less computationally demanding in inference.

Abstract Handwriting is one of the most frequently occurring patterns in everyday life and with it comes challenging applications such as handwriting recognition, writer identification and signature verification. In contrast to offline HWR that only uses spatial information (i.e., images), online HWR uses richer spatio-temporal information (i.e., trajectory data or inertial data). While there exist many offline HWR datasets, there are only little data available for the development of OnHWR methods on paper as it requires hardware-integrated pens. This paper presents data and benchmark models for real-time sequence-to-sequence learning and single character-based recognition. Our data are recorded by a sensor-enhanced ballpoint pen, yielding sensor data streams from triaxial accelerometers, a gyroscope, a magnetometer and a force sensor at 100 Hz. We propose a variety of datasets including equations and words for both the writer-dependent and writer-independent tasks. Our datasets allow a comparison between classical OnHWR on tablets and on paper with sensor-enhanced pens. We provide an evaluation benchmark for seq2seq and single character-based HWR using recurrent and temporal convolutional networks and transformers combined with a connectionist temporal classification (CTC) loss and cross-entropy (CE) losses. Our convolutional network combined with BiLSTMs outperforms transformer-based architectures, is on par with InceptionTime for sequence-based classification tasks and yields better results compared to 28 state-of-the-art techniques. Time-series augmentation methods improve the sequence-based task, and we show that CE variants can improve the single classification task. Our implementations together with the large benchmark of state-of-the-art techniques of novel OnHWR datasets serve as a baseline for future research in the area of OnHWR on paper.

Active learning prioritizes the labeling of the most informative data samples. However, the performance of active learning heuristics depends on both the structure of the underlying model architecture and the data. We propose IALE, an imitation learning scheme that imitates the selection of the best-performing expert heuristic at each stage of the learning cycle in a batch-mode pool-based setting. We use Dagger to train a transferable policy on a dataset and later apply it to different datasets and deep classifier architectures. The policy reflects on the best choices from multiple expert heuristics given the current state of the active learning process, and learns to select samples in a complementary way that unifies the expert strategies. Our experiments on well-known image datasets show that we outperform state of the art imitation learners and heuristics.

This work proposes metric learning for fast similarity-based scene retrieval of unstructured ensembles of trajectory data from large databases. We present a novel representation learning approach using Siamese Metric Learning that approximates a distance preserving low-dimensional representation and that learns to estimate reasonable solutions to the assignment problem. To this end, we employ a Temporal Convolutional Network architecture that we extend with a gating mechanism to enable learning from sparse data, leading to solutions to the assignment problem exhibiting varying degrees of sparsity. Our experimental results on professional soccer tracking data provides insights on learned features and embeddings, as well as on generalization, sensitivity, and network architectural considerations. Our low approximation errors for learned representations and the interactive performance with retrieval times several magnitudes smaller shows that we outperform previous state of the art.

Ultra-wideband (UWB) systems based on Channel State Information (CSI) estimate the position of mobile nodes within an environment by using Channel Impulse Responses (CIRs) of multiple stationary nodes. These contain spatial information caused by environment interactions such as reflections and scattering. To estimate positions from CSI of stationary nodes, we must transmit them to a centralized node. This introduces considerable communication overhead.We present a large-scale study to determine whether CSI can be compressed into a small set of underlying latent variables that describe the most valuable information. We evaluate multiple neural network architectures containing encoding (compressing) and decoding (reconstructing) components and compare them to the state-of-the-art compression techniques Discrete Cosine Transform (DCT) and Discrete Wavelet Transform (DWT). We show that fully connected autoencoders achieve the lowest error, outperforming both DCT and DWT. Further experiments prove that the reconstructed CSI can be used for positioning with only mild performance deterioration at a compression of >97% and even when trained on a different environment.

Many radio-based positioning systems use time-of-arrival (ToA). We obtain it from the first and direct path of arrival (FDPoA) in a corresponding set of multipath components (MPC) of the underlying channel state information (CSI). While detection of the FDPoA under Line-of-Sight (LoS) is simple, it is prone to errors in environments with specular and diffuse reflections, as well as nonlinear diffraction, absorption, and transmission of a signal. Such Obstructed- or Non-Line-of-Sight (OLoS, NLoS) situations lead to incorrect FDPoA and consequently to incorrect ToA estimates and inaccurate positions. State-of-the-art estimators are computationally expensive and usually fail with O/NLoS at low signal-to-noise ratios (SNRs).We propose a deep learning (DL) approach to identify optimal FDPoAs as ToA directly from the raw CSI. Our 1D Convolutional Neural Network (CNN) learns the spatial distribution of MPCs of the CSI to predict correct estimates of the ToA. To train our DL model, we use QuaDRiGa to generate datasets with CIRs and ground truth ToAs for realistic 5G channel models. We found that Delay Spread (DS), k-Factor (kF), and SNR are appropriate metrics to cover most LoS-NLoS constellations in realistic datasets. We compare our DL model with state-of-the-art estimators such as threshold (PEAK), inflection point (IFP), and MUSIC and show that we consistently outperform them by about 17% for SNRs below -10 dB.

Digital Contact Tracing (CT) protocols based on Bluetooth are best implemented at the system level to save resources and preserve security aspects. Combined with a government-monitored software platform, these CT-protocols can then be used to support controlling pandemics such as COVID-19. However, it is unclear how these protocols have to be parameterized to ensure the most accurate and reliable CT.This paper describes how we derived optimal parameters for a decentralized CT from extensive measurement campaigns that we carried out together with Deutsche Telekom (DT) and SAP under the supervision of the Robert Koch Institut (RKI). We examined the Google/Apple Exposure Notification Framework (ENF), which in combination with the front-end, i.e., the German Corona-Warn-App (CWA), enables digital CT in Germany. With centimeter accurate optical reference systems we show that optimal parameters are application-specific. However, they cause impractical high resource costs. In contrast, optimized general parameters offer an everyday compromise between energy costs, applicability, accuracy, and reliability of the ENF.

Safe and efficient behavior are the key guiding principles for autonomous vehicles. Manually designed rule-based systems need to act very conservatively to ensure a safe operation. This limits their applicability to real-world systems. On the other hand, more advanced behaviors, i.e., policies, learned through means of reinforcement learning (RL) suffer from non-interpretability as they are usually expressed by deep neural networks that are hard to explain. Even worse, there are no formal safety guarantees for their operation. In this paper we introduce a novel pipeline that builds on recent advances in imitation learning and that can generate safe and efficient behavior policies. We combine a reinforcement learning step that solves for safe behavior through the introduction of safety distances with a subsequent innovative safe extraction of decision tree policies. The resulting decision tree is not only easy to interpret, it is also safer than the neural network policy trained for safety. Additionally, we formally prove the safety of trained RL agents for linearized system dynamics, showing that the learned and extracted policy successfully avoids all catastrophic events.

The main challenge in estimating human velocity from noisy Inertial Measurement Units (IMUs) are the errors that accumulate by integrating noisy accelerometer signals over a long time. Known approaches that work on step length estimation are optimized for a specific application, sensor position, and movement type, require an exhaustive (manual) parameter tuning, and can thus not be applied to other movement types or to a broader range of applications. Moreover, varying dynamics (as they are present for instance in sports applications) cause abrupt and unpredictable changes in step frequency or step length and hence result in erroneous velocity estimates. We use machine learning (ML) and deep learning (DL) to estimate a human's velocity. Our approach is robust to varying motion states and orientation changes in dynamic situations. On data from a single un-calibrated IMU, our novel recurrent model not only outperforms the state-of-the-art on instantaneous velocity (≤0.10 m/s) and on traveled distance (≤29 m/km). It can also generalize to different and varying rates of motion and provides accurate and precise velocity estimates.

We explore motion parameters, more specifically gait parameters, as an objective indicator to assess simulator sickness in Virtual Reality (VR). We discuss the potential relationships between simulator sickness, immersion, and presence. We used two different camera pose (position and orientation) estimation methods for the evaluation of motion tasks in a large-scale VR environment: a simple model and an optimized model that allows for a more accurate and natural mapping of human senses. Participants performed multiple motion tasks (walking, balancing, running) in three conditions: a physical reality baseline condition, a VR condition with the simple model, and a VR condition with the optimized model. We compared these conditions with regard to the resulting sickness and gait, as well as the perceived presence in the VR conditions. The subjective measures confirmed that the optimized pose estimation model reduces simulator sickness and increases the perceived presence. The results further show that both models affect the gait parameters and simulator sickness, which is why we further investigated a classification approach that deals with non-linear correlation dependencies between gait parameters and simulator sickness. We argue that our approach could be used to assess and predict simulator sickness based on human gait parameters and we provide implications for future research.

With free movement and multi-user capabilities, there is demand to open up Virtual Reality (VR) for large spaces. However, the cost of accurate camera-based tracking grows with the size of the space and the number of users. No-pose (NP) tracking is cheaper, but so far it cannot accurately and stably estimate the yaw orientation of the user's head in the long-run. Our novel yaw orientation estimation combines a single inertial sensor located at the human's head with inaccurate positional tracking. We exploit that humans tend to walk in their viewing direction and that they also tolerate some orientation drift. We classify head and body motion and estimate heading drift to enable low-cost long-time stable head orientation in NP tracking on 100 m×100 m. Our evaluation shows that we estimate heading reasonably well.

Kalman filters (KFs) are popular methods to estimate position information from a set of time-of-flight (ToF) values in radio frequency (RF)-based locating systems. Such filters are proven to be optimal under zero-mean Gaussian error distributions. In presence of multipath propagation ToF measurement errors drift due to small-scale motion. This results in changing phases of the multipath components (MPCs) which cause a drift on the ToF measurements. Thus, on a short-term scale the ToF measurements have a non-constant bias that changes while moving. KFs cannot distinguish between the drifting measurement errors and the true motion of the tracked object. Hence, very rigid motion models have to be used for the KF which commonly causes the filters to diverge. Therefore, the KF cannot resolve the short-term errors of consecutive measurements and the long-term motion of the tracked object. This paper presents a data-driven approach that uses training sequences to derive a near-optimal position estimator. A Long Short-Term Memory (LSTM) Recurrent Neural Network (RNN) learns to interpret drifting errors in ToF measurements of a tracked dynamic object directly from raw ToF data. Our evaluation shows that our approach outperforms state-of-the-art KFs on both synthetically generated and real-world dynamic motion trajectories that include drifting ToF measurement errors.

Precise indoor positioning is essential to support emerging applications of location aware mobile computing. Current positioning techniques that use signals transmitted in the Gigahertz region of radio-frequency spectrum do not provide highly accurate position estimates due to multipath propagation of signals. This paper proposes a novel positioning system which uses the entities of visible light communication (VLC) system as anchors and tags. Our technique scales with the number of tags and the VLC network provides both positioning and communication capabilities. The anchor points transmit the OFDM-based VLC signals synchronously, and we estimate the time differences of arrival between anchor points and tags using positioning reference signals embedded into the air-interface of the VLC system. Simulation results show that positioning accuracy of 10 cm or better is possible for over 95% of users if the sampling clock offset is better that 10 ppm, clock jitter is below 1 ps, and a bit resolution of at least 16 bits is available.

Real-time tracking allows to trace goods and enables the optimization of logistics processes in many application areas. Camera-based inside-out tracking that uses an infrastructure of fixed and known markers is costly as the markers need to be installed and maintained in the environment. Instead, systems that use natural markers suffer from changes in the physical environment. Recently a number of approaches based on machine learning (ML) aim to address such issues. This paper proposes evaluation criteria that consider algorithmic properties of ML-based positioning schemes and introduces a dataset from an indoor warehouse scenario to evaluate for them. Our dataset consists of images labeled with millimeter precise positions that allows for a better development and performance evaluation of learning algorithms. This allows an evaluation of machine learning algorithms for monocular optical positioning in a realistic indoor position application for the first time. We also show the feasibility of ML-based positioning schemes for an industrial deployment.

Object localization and tracking is essential for many applications including logistics and industry. Many local Time-of-Flight (ToF)-based locating systems use synchronized antennas to receive radio signals emitted by mobile tags. They detect the Time-of-Arrival (TOA) of the signal at each antenna and trilaterate the position from the Time Difference-of-Arrival (TDoA) between antennas. However, in multipath scenarios it is difficult to extract the correct ToA. This causes wrong positions. This paper proposes a signal processing method that uses deep learning to estimate the absolute tag position directly from the raw channel impulse response (CIR) data. We use the CIR together with ground truth positional data to train a convolutional neural network (CNN) that not only estimates non-linearities in the signal propagation space but also analyzes the signal for multipath effects. Our evaluation shows that our position estimation works in multipath environments and also outperforms classical signal processing in line-of-sight situations.

For the evolving Internet-of-Things ubiquitous positioning is a core feature. Hence, energy- and location-awareness are essential properties of wireless sensor networks (WSNs). In terms low power consumption received signal strength (RSS)-based localization techniques outperform timing-based localization approaches. Therefore, RSS-based direction finding is prospective approach to location-aware, low-power sensor nodes. However, RSS-based direction-of-arrival (DOA) estimation is prone to multipath propagation. In this paper, a subspace-based approach to frequency-domain multipath resolution is presented. Resolution of multipath components allows for a RSS-based DOA estimation considering the power of the line of sight (LOS) component only. The impact of the multipath channel is considerably reduced with our approach. In contrast to common broadband DOA estimation techniques, the presented approach does not need phase-coherent receive channels or a synchronized sensor network. Hence, the proposed super-resolution technique is applicable to low-power sensor networks and brings accurate positioning to small-sized and energy-efficient sensor nodes.

No-Pose (NP) tracking systems rely on a single sensor located at the user's head to determine the position of the head. They estimate the head orientation with inertial sensors and analyze the body motion to compensate their drift. However with orientation drift, VR users implicitly lean their heads and bodies sidewards. Hence, to determine the sensor drift and to explicitly adjust the orientation of the VR display there is a need to understand and consider both the user's head and body orientations. This paper studies the effects of head orientation drift around the yaw axis on the user's absolute head and body orientations when walking naturally in the VR. We study how much drift accumulates over time, how a user experiences and tolerates it, and how a user applies strategies to compensate for larger drifts.

Pose tracking does not yet reliably work in large-scale interactive multi-user VR. Our novel head orientation estimation combines a single inertial sensor located at the user's head with inaccurate positional tracking. We exploit that users tend to walk in their viewing direction and classify head and body motion to estimate heading drift. This enables low-cost long-time stable head orientation. We evaluate our method and show that we sustain immersion.

This paper presents a novel approach for the augmentation of social behaviors in virtual reality (VR). We designed three visual transformations for behavioral phenomena crucial to everyday social interactions: eye contact, joint attention, and grouping. To evaluate the approach, we let users interact socially in a virtual museum using a large-scale multi-user tracking environment. Using a between-subject design (N = 125) we formed groups of five participants. Participants were represented as simplified avatars and experienced the virtual museum simultaneously, either with or without the augmentations. Our results indicate that our approach can significantly increase social presence in multi-user environments and that the augmented experience appears more thought-provoking. Furthermore, the augmentations seem also to affect the actual behavior of participants with regard to more eye contact and more focus on avatars/objects in the scene. We interpret these findings as first indicators for the potential of social augmentations to impact social perception and behavior in VR.

Redirected Walking (RDW) manipulates a scene that is displayed to VR users so that they unknowingly compensate for scene motion and can thus explore a large virtual world on a limited space. So far, mostly visual manipulation techniques have been studied.

Localization of wireless nodes within the IoT received much attention lately. However, strong constraints on power consumption, scalability, and complexity of the nodes pose a big challenge for localization techniques. This paper presents a concept for energy-efficient low-complexity localization based on Phase Difference of Arrival (PDoA). Besides a novel method for reference transmitter selection we propose a waveform, well-suited for PDoA measurements, and evaluate its ranging performance. We compare multiple signal classification (MUSIC), linear fitting, and mean phase difference and compare their estimation variance to the Cramer Rao Lower Bound (CRLB). Our system concept allows for the mitigation of near-far effects for reference and tag signals at the receiver nodes, and an efficient implementation of a wideband frequency hopping scheme.

Distributed event-based systems are used to detect meaningful events with low latency in high data-rate event streams that occur in surveillance, sports, finances, etc. However, both known approaches to dealing with the predominant out-of-order event arrival at the distributed detectors have their shortcomings: buffering approaches introduce latencies for event ordering, and stream revision approaches may result in system overloads due to unbounded retraction cascades. This article presents an adaptive speculative processing technique for out-of-order event streams that enhances typical buffering approaches. In contrast to other stream revision approaches developed so far, our novel technique encapsulates the event detector, uses the buffering technique to delay events but also speculatively processes a portion of it, and adapts the degree of speculation at runtime to fit the available system resources so that detection latency becomes minimal. Our technique outperforms known approaches on both synthetical data and real sensor data from a realtime locating system (RTLS) with several thousands of out-of-order sensor events per second. Speculative buffering exploits system resources and reduces latency by 40% on average.

Event-based Systems (EBS) are used to detect and analyze meaningful events in surveillance, sports, finances and many other areas. With rising data and event rates and with correlations among these events, sequential event processing becomes infeasible and needs to be distributed. Existing approaches cannot deal with the ubiquity of out-of-order event arrival that is introduced by network delays when distributing EBS. Order-less event processing may result in a system failure. We present a low-latency approach based on K-slack that achieves ordered event processing on high data rate sensor and event streams without a-priori knowledge. Slack buffers are dynamically adjusted to fit the disorder in the streams without using local or global clocks. The middleware transparently reorders the event input streams so that events can still be aggregated and processed to a granularity that satisfies the demands of the application. On a Realtime Locating System (RTLS) our system performs accurate low-latency event detection under the predominance of out-of-order event arrival and with a close to linear performance scale-up when the system is distributed over several threads and machines.