Multi-user Augmented Reality (AR) holds great promise for facilitating collaboration among individuals by allowing them to perceive and interact with virtual objects. To achieve multi-user AR, it is important to keep different users' coordinates well-synchronized so that one user can share the created virtual object with others. However, most existing synchronization methods rely on resource-intensive object detection algorithms or mobile SLAM, which leads to high energy consumption and is not suitable for mobile devices. In this paper, we propose to achieve lightweight synchronization for multi-user AR by leveraging multiple embedded sensors on mobile devices such as IMU sensor, ultra-wideband (UWB) sensor, and camera sensors. Specifically, we propose a novel ARlocalization algorithm for tracking the relative positions of AR users. It utilizes particle filters to fuse the IMU and UWB sensing data, and employs an adaptive resampling method to improve the localization accuracy. To further enhance the positional accuracy of virtual objects displayed in AR, we design a feature-based position modification algorithm. Extensive evaluations demonstrate that our synchronization method significantly improves accuracy and reduces energy consumption compared to existing baselines.

The detection of malicious social bots has become a crucial task, as bots can be easily deployed and manipulated to spread disinformation, promote conspiracy messages, and more. Most existing approaches utilize graph neural networks (GNNs) to capture both user profile and structural features, achieving promising progress. However, they still face limitations including the expensive training on large underlying graph, the performance degradation when “similar neighborhood patterns” assumption preferred by GNNs is not satisfied, and the distinguishable features of bots in a highly adversarial context. Motivated by these limitations, this paper proposes a method named BSG4Bot with an intuition that GNNs training on Biased SubGraphs can improve both performance and time/space efficiency in bot detection. Specifically, BSG4Bot first pre-trains a classifier on node features efficiently to define the node similarities, and constructs biased subgraphs by combining the similarities computed by the pre-trained classifier and the node importances computed by Personalized PageRank (PPR scores). BSG4Bot then introduces a heterogeneous GNN over the constructed subgraphs to detect bots effectively and efficiently. The relatively stable features, including the content category and temporal activity features, are explored and incorporated into BSG4Bot after preliminary verification on sample data. The extensive experimental studies show that BSG4Bot outperforms the state-of-the-art bot detection methods, while only needing nearly 1/4 training time.

Sleep apnea is a serious sleep disorder where patients have multiple extended pauses in breath during sleep. Although some portable or contactless sleep apnea detection systems have been proposed, none of them can achieve fine-grained sleep apnea detection without strict requirements on the device or environmental settings. To address this problem, we present DeepApnea, a deep learning based sleep apnea detection system that leverages patients' wrist movement data collected by smartwatches to identify different types of sleep apnea events (i.e., central apneas, obstructive apneas, and hypopneas). Through a clinical study, we identify some special characteristics associated with different types of sleep apnea captured by smartwatch. However, there are many technical challenges such as how to extract informative apnea features from the noisy data and how to leverage features extracted from the multi-axis sensing data. To address these challenges, we first propose signal pre-processing methods to filter the raw accelerometer (ACC) data, smoothing away noise while preserving the respiratory signal and potential features for identifying sleep apnea. Then, we design a deep learning architecture to extract features from three ACC axes collaboratively, where self attention and cross-axis correlation techniques are leveraged to improve the classification accuracy. We have implemented DeepApnea on smartwatches and performed a clinical study. Evaluation results demonstrate that DeepApnea can significantly outperform existing work on identifying different types of sleep apnea.

The ubiquity of mobile devices allows mobile users to participate in crowdsourcing anywhere, anytime. One potential application is to crowdsource photos/videos on demand to search for interested targets. Crowdsourced photos/videos have much better coverage compared to surveillance cameras, and thus help improve the effectiveness of target search. However, broadcasting the crowdsourcing task to all mobile users can significantly increase the cost in terms of resource and incentive budget. To reduce cost, the crowdsourcing server selects a subset of participating workers, and there are many challenges on worker selection. For example, due to occlusions in the photo/video scene, each worker only covers part of the area with certain probability. Due to the non-deterministic nature of this problem, we study two kinds of optimization problems: max-coverage which maximizes the probability of finding the target given a cost, and min-selection which minimizes the number of workers given the required probability of finding the target. Considering that workers may report exact locations or coarse-grained locations, we formalize four probability-based optimization problems for worker selection, and develop optimal or efficient approximation algorithms to solve them. The effectiveness of the proposed algorithms is evaluated and validated via extensive trace-driven simulations and a real-world demo.

Mobile Augmented Reality (AR), which overlays digital content on the real-world scenes surrounding a user, is bringing immersive interactive experiences where the real and virtual worlds are tightly coupled. To enable seamless and precise AR experiences, an image recognition system that can accurately recognize the object in the camera view with low system latency is required. However, due to the pervasiveness and severity of image distortions, an effective and robust image recognition solution for “in the wild” mobile AR is still elusive. In this article, we present CollabAR, an edge-assisted system that provides distortion-tolerant image recognition for mobile AR with imperceptible system latency . CollabAR incorporates both distortion-tolerant and collaborative image recognition modules in its design. The former enables distortion-adaptive image recognition to improve the robustness against image distortions, while the latter exploits the spatial-temporal correlation among mobile AR users to improve recognition accuracy. Moreover, as it is difficult to collect a large-scale image distortion dataset, we propose a Cycle-Consistent Generative Adversarial Network-based data augmentation method to synthesize realistic image distortion. Our evaluation demonstrates that CollabAR achieves over 85% recognition accuracy for “in the wild” images with severe distortions, while reducing the end-to-end system latency to as low as 18.2 ms.

Conventional radio-frequency (RF) sensing systems rely on either frequency diversity or spatial diversity to ensure high sensing accuracy. Such reliance introduces several practical limitations that hinder the pervasive deployment of existing solutions. To circumvent this prevalent reliance, we present MetaSense, a system that leverages antenna pattern diversity for fine-grained RF sensing. MetaSense incorporates the dynamic metasurface antenna (DMA) and the auxiliary-assisted ensemble multimask learning (AEMML) framework in its design. The DMA is a novel type of antenna that can provide a diverse set of uncorrelated radiation patterns in a low-cost and low-complexity manner. The AEMML is a quality-aware learning framework that can dynamically assess and aggregate the heterogeneous channel measurements from different antenna patterns to ensure high sensing accuracy. It also incorporates a transfer learning model that allows it to generalize to new sensing conditions with few training instances required. We prototype MetaSense and demonstrate its effectiveness on a writing motion recognition task using a custom-designed 2-D DMA. The results show that MetaSense achieves 92% to 98% accuracy in classifying ten miniature writing motions, outperforming a nontunable antenna by 20% in all scenarios. Moreover, when deployed in new sensing positions where limited training instances are available, MetaSense requires as few as five training instances per class to achieve over 90% accuracy.

Mobile Augmented Reality (AR), which overlays digital content on the real-world scenes surrounding a user, is bringing immersive interactive experiences where the real and virtual worlds are tightly coupled. To enable seamless and precise AR experiences, an image recognition system that can accurately recognize the object in the camera view with low system latency is required. However, due to the pervasiveness and severity of image distortions, an effective and robust image recognition solution for mobile AR is still elusive. In this paper, we present CollabAR, an edge-assisted system that provides distortion-tolerant image recognition for mobile AR with imperceptible system latency. CollabAR incorporates both distortion-tolerant and collaborative image recognition modules in its design. The former enables distortion-adaptive image recognition to improve the robustness against image distortions, while the latter exploits the ‘spatial-temporal’ correlation among mobile AR users to improve recognition accuracy. We implement CollabAR on four different commodity devices, and evaluate its performance on two multi-view image datasets. Our evaluation demonstrates that CollabAR achieves over 96% recognition accuracy for images with severe distortions, while reducing the end-to-end system latency to as low as 17.8ms for commodity mobile devices.

Mobile Augmented Reality (AR), which overlays digital information with real-world scenes surrounding a user, provides an enhanced mode of interaction with the ambient world. Contextual AR applications rely on image recognition to identify objects in the view of the mobile device. In practice, due to image distortions and device resource constraints, achieving high performance image recognition for AR is challenging. Recent advances in edge computing offer opportunities for designing collaborative image recognition frameworks for AR. In this demonstration, we present CollabAR, an edge-assisted collaborative image recognition framework. CollabAR allows AR devices that are facing the same scene to collaborate on the recognition task. Demo participants develop an intuition for different image distortions and their impact on image recognition accuracy. We showcase how heterogeneous images taken by different users can be aggregated to improve recognition accuracy and provide a better user experience in AR.