With the rapid proliferation of edge devices, such as those in the Internet of Things (IoT), which generate critical data for machine learning applications, it is essential to enable their participation in privacy-preserving Federated Learning (FL) systems. Given their limited computational resources, an effective approach is to adapt and reduce their training workload to align with their capabilities. Previous FL research has focused primarily on workload reduction through lightweight models at the edge, with limited attention given to optimizing on-device training efficiency by reducing the amount of data required for training. In this work, we propose FedFT-EDS, a novel approach that combines Fine-Tuning of partial client models with Entropy-based Data Selection to reduce training workloads on edge devices. By actively selecting the most informative local instances for learning, FedFT-EDS significantly reduces the training data in FL and demonstrates that not all user data are equally beneficial across training rounds. We show that FedFT-EDS uses only 50% of the available training data while improving the global model performance compared to the baseline methods, FedAvg and FedProx. Importantly, FedFT-EDS improves the learning efficiency of client models by up to 3×, to achieve a similar performance to the baselines in only one third of their training time. This work underscores the critical role of data selection in Federated Learning and offers a promising direction for achieving scalable and efficient FL systems.

Graphics Processing Units (GPUs) are notoriously hard to optimize for manually. What is needed are good automatic code generators and optimizers. Accelerate, Futhark and Lift demonstrated that a functional approach is well suited for this challenge. Lift, for instance, uses a system of rewrite rules with a multi-stage approach. Algorithmic optimizations are first explored, followed by hardware-specific optimizations such as using shared memory and mapping parallelism.

Indoor positioning has been widely researched in recent years due to its high demand for developing localization services and its complexity in GPS-denied environments. However, the diversity of indoor spaces and temporal variation of local conditions impose the need for building specific and periodic calibrations at high cost for deployment and maintenance of these localization systems. A robust positioning solution that overcomes these challenges is yet to be available. Previous systems achieve good performance when specializing their solution to the unique characteristics of the deployment site. The drive is now to automatically model these localization solutions on the sensor data from each site with the least amount of effort. We propose to accelerate the model adaptation to new deployment sites by using transfer learning of a multimodal deep neural network architecture. We demonstrate that the required training data is drastically reduced compared to training the model from scratch, while also boosting its accuracy, due to the additional knowledge from pretraining on other sites. The resulting model is also fault-tolerant, showing good performance in missing modalities experiment. Our research opens the way toward scalable and cost efficient localization systems.

Much like how sentient beings use multiple senses, Multimodal Machine Learning (MML) uses multiple input signals to form an estimation about the environment. An advantage MML has over standard Unimodal Learning is that if one input becomes corrupt or unavailable, a multimodal recognition model can rely on the other inputs by zeroing out the corrupt or missing one. As such, MML can become more robust and fault-tolerant. Unfortunately, it is not always known when an input has become corrupt or not. Corrupt inputs are anomalous in comparison to normal and expected data. We use an immune inspired algorithm, the Negative Selection Algorithm (NSA) as a resilient anomaly detection algorithm. Using the multimodal Smart Gym “MM-FIT” dataset, we evaluate how corrupt inputs affect the accuracy of MML classification. We implement the NSA specifically for a multimodal pipeline of activity recognition with anomaly detection. The results indicate that our recognition pipeline is effective in introducing fault-tolerance to MML.

Maintaining smartphone indoor localization systems operational and accurate is challenging. Landmark and WiFi based systems rely on up-to-date reference points and radio maps. These are expensive to refresh for most current systems because of the exhaustive site surveys these rely on. Any physical change in the environment may cause unpredictable transformation in the radio propagation medium or it may affect the typical navigation paths inside the building. Active Learning (AL) involves human intervention in annotating sensor samples collected from the target environment. However, the burden on users to perform this manual task should be minimized to make the approach more user-friendly. We propose a solution that improves AL by involving the users only when sensor data deviates from what is expected at a location. Our solution analyses the confidence of location estimations in the environment. We propose a robust data distribution shift detector that looks at several levels in our location estimation neural network to determine the confidence of predictions. During the training stage, the model builds high confidence for estimations in the areas where data is available. We show that data from outside the training distribution is detected as anomaly, which triggers the AL to prompt the user for her current location as label. These annotated samples and other augmented data are then used to update the location estimation model.

Indoor positioning systems have been explored for decades to facilitate universal location-based services. However, complex environment conditions and sensing imperfections continue to be limiting factors to their large scale adoption. Rather than customising more ingenious solutions to handle corner cases in complex environments, we believe that a more efficient solution is to learn entirely from data with minimal engineering effort. We develop neural network based solutions for two positioning approaches, modelling Dead Reckoning with recurrent neural networks and WiFi Fingerprinting with deep neural networks. We propose a multimodal deep neural network architecture (MM-Loc) that bridges the features extracted by the modality-specific complements (sensor based and WiFi based) to join the two perspectives. We observe that this multimodal approach is better than single-modality models, and elegantly trains directly from raw data with minimal intervention.

Convolutional neural networks (CNNs) are used in many embedded applications, from industrial robotics and automation systems to biometric identification on mobile devices. State-of-the-art classification is typically achieved by large networks, which are prohibitively expensive to run on mobile and embedded devices with tightly constrained memory and energy budgets. We propose an approach for ahead-of-time domain specific optimization of CNN models, based on an integer linear programming (ILP) for selecting primitive operations to implement convolutional layers. We optimize the trade-off between execution time and memory consumption by: 1) attempting to minimize execution time across the whole network by selecting data layouts and primitive operations to implement each layer; and 2) allocating an appropriate work space that reflects the upper bound of memory footprint per layer. These two optimization strategies can be used to run any CNN on any platform with a C compiler. Our evaluation with a range of popular ImageNet neural architectures (GoogleNet, AlexNet, VGG, ResNetand SqueezeNet) on the ARM Cortex-A15 yields speedups of 8× compared to a greedy algorithm based primitive selection, reduces memory requirement by 2.2× while sacrificing only 15% of inference time compared to a solver that considers inference time only. In addition, our optimization approach exposes a range of optimal points for different configurations across the Pareto frontier of memory and latency trade-off, which can be used under arbitrary system constraints.

Convolutional neural networks (CNNs) are used in many applications, from industrial robotics to biometric identification on mobile devices. But they can be too resource-hungry for mobile and embedded devices with tightly constrained memory and energy budgets. We propose an ahead-of-time primitive selection for CNNs, based on integer linear programming (ILP). Under a tight memory budget, our ILP solver selects the optimal primitive for each layer such that the entire network is optimized for execution time subject to a memory budget, or vice versa. Our method yields significant speedup and memory reduction compared to existing methods.

Advances in hardware and algorithms are driving the exponential growth of Internet of Things (IoT), with increasingly more pervasive computations being performed near the data generation sources. With this wave of technology, a range of intelligent devices can perform local inferences (activity recognition, fitness monitoring, etc.), which have obvious advantages: reduced inference latency for interactive (real-time) applications and better data privacy by processing user data locally. Video processing can benefit many applications and data labelling systems, although performing this efficiently at the edge of the Internet is not trivial. In this paper, we show that accurate pedestrian location estimation is achievable using deep neural networks on fixed cameras with limited computing resources. Our approach, CamLoc, uses pose estimation from key body points detection to extend pedestrian skeleton when the entire body is not in view (occluded by obstacles or partially outside the frame). Our evaluation dataset contains over 2100 frames from surveillance cameras (including two cameras simultaneously pointing at the same scene from different angles), in 42 different scenarios of activity and occlusion. We make this dataset available together with annotations indicating the exact 2D position of person in frame as ground-truth information. CamLoc achieves good location estimation accuracy in these complex scenarios with high levels of occlusion, matching the performance of state-of-the-art solutions, but using less computing resources and attaining a higher inference throughput.

Although research in the field of indoor localization has been fruitful and abundant, the research space is currently disorganised, which hinders progress. With many indoor localization solutions proposed over the years, only few have their code and datasets publicly available. Many researchers are not publishing the source code for their localization systems due to aspiring commercial ambitions inspired by recent successes in this direction or to avoid their system being evaluated in less than favourable conditions. This creates a barrier in terms of accessing the work already performed in this field. It is difficult to compare new solutions to the existing ones, or to improve upon previous systems. Replicating a localization system is not easy. It requires a significant effort and, many times, essential details are missing from the associated paper. As a solution to this problem, we want to initiate a community effort to collect many localization systems' implementations (original and community replication attempts) and datasets. As a mechanism for this, we introduce the Localization Systems Repository (LSR), which is a platform for localization systems source code, datasets and transparent performance ranking. The platform is open to everyone, accepting contributions to improve existing implementations or to add new systems and datasets. Initially, the focus of the LSR platform is on indoor localization systems, which use widely available infrastructure such as WiFi networks and inertial sensors within smartphones and wearable devices.

The need for location tracking in many mobile services has given rise to the broad research topic of indoor positioning we see today. However, the majority of proposed systems in this space is based on traditional approaches of signal processing and simple machine learning solutions. In the age of big data, it is imperative to evolve our techniques to learn the complexity of indoor environments directly from data with modern machine learning approaches inspired from deep learning. We model location tracking from smartphone inertial sensor data with recurrent neural networks. Through our broad experimentation we provide an empirical study of the best model configuration, data preprocessing and training process to achieve improved inference accuracy. Our explored solutions are lightweight to run efficiently under limited computing resources available on mobile devices, while also achieving accurate estimations, within 5 meters median error from inertial sensors alone.

The environmental context of a mobile device determines where/how it is used, which can be exploited for efficient operation and better usability. In this work we describe a general method using only the lightweight sensors on a smartphone to detect if a device is indoor or outdoor. Using semi-supervised machine learning techniques, our method automatically learns characteristics of new environments and devices, thereby achieves detection accuracy of over 90% even in unfamiliar circumstances. Therefore, it easily outperforms existing indoor-outdoor detection techniques based on static algorithms, or relying on energy hungry and unreliable GPS.

The environmental context of a mobile device determines how it is used and how the device can optimize operations for greater efficiency and usability. We consider the problem of detecting if a device is indoor or outdoor. Towards this end, we present a general method employing semi-supervised machine learning and using only the lightweight sensors on a smartphone. We find that a particular semi-supervised learning method called co-training, when suitably engineered, is most effective. It is able to automatically learn characteristics of new environments and devices, and thereby provides a detection accuracy exceeding 90% even in unfamiliar circumstances. It can learn and adapt online, in real time, at modest computational costs. Thus the method is suitable for on-device learning. Implementation of the indoor-outdoor detection service based on our method is lightweight in energy use -- it can sleep when not in use and does not need to track the device state continuously. It is shown to outperform existing indoor-outdoor detection techniques that rely on static algorithms or GPS, in terms of both accuracy and energy-efficiency.

WiFi in indoor environments exhibits spatio-temporal variations in terms of coverage and interference in typical WLAN deployments with multiple APs, motivating the need for automated monitoring to aid network administrators to adapt the WLAN deployment in order to match the user expectations. We develop Pazl, a mobile crowdsensing based indoor WiFi monitoring system that is enabled by a novel hybrid localization mechanism to locate individual measurements taken from participant phones. The localization mechanism in Pazl integrates the best aspects of two well known localization techniques, pedestrian dead reckoning and WiFi fingerprinting; it also relies on crowdsourcing for constructing the WiFi fingerprint database. Compared to existing WiFi monitoring systems based on static sniffers, Pazl is low cost and provides a user-side perspective. Pazl is significantly more automated than wireless site survey tools such as Ekahau Mobile Survey tool by drastically reducing the manual point-and-click based measurement location determination. We implement Pazl through a combination of Android mobile app and cloud backend application on the Google App Engine. Experimental evaluation of Pazl with a trial set of users shows that it yields similar results to manual site surveys but without the tedium.

The large number of applications that rely on indoor positioning encourages more advancement in this field. Smartphones are becoming a common presence in our daily life, so taking advantage of their sensors can help to provide ubiquitous positioning solution. We propose HiMLoc, a novel solution that synergistically uses Pedestrian Dead Reckoning (PDR) and WiFi fingerprinting to exploit their positive aspects and limit the impact of their negative aspects. Specifically, HiMLoc combines location tracking and activity recognition using inertial sensors on mobile devices with location-specific weighted assistance from a crowd-sourced WiFi fingerprinting system via a particle filter. By using just the most common sensors available on the large majority of smartphones (accelerometer, compass, and WiFi card) and offering an easily deployable method (requiring just the locations of stairs, elevators, corners and entrances), HiMLoc is shown to achieve median accuracies lower than 3 meters in most cases.

We consider the mobile phone location tracking problem in indoor environments. We propose a hybrid approach for energy-efficient indoor mobile phone localization by combining two well-known techniques namely, pedestrian dead reckoning and WiFi fingerprinting. Our preliminary evaluation of the proposed approach shows that it achieves better location accuracy (median error around 2m) than either of the two techniques on which it is based.